[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] condor_submit long argument list

One way to do this is to use the new submit queue syntax, this is especially useful if you want to submit more than one set of arguments at a time.   This works in HTCondor 8.4 and later

create a submit file WITHOUT A QUEUE STATEMENT and then

condor_submit  foo.sub -queue arguments from <argsfile>

where <argsfile> is a file with one line for each set of arguments  this will queue one process for each line

of the file,  so if your <argsfile> has

-process filename.01.json -x 1 -y 1 outfile.0101.json

-process filename.02.json -x 2 -y 2 outfile.0102.json

then you will end up with 2 HTCondor jobs (one cluster id, and 2 procids)

if <argsfile> is -,  then stdin is read.


From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Lee Gramling <leegramling@xxxxxxxxx>
Sent: Friday, February 26, 2016 7:11 AM
To: htcondor-users@xxxxxxxxxxx
Subject: [HTCondor-users] condor_submit long argument list
What are ways to pass a executable arguments to my submit description file?

I have a executable that takes a lot of parameters so I have tried to do something like this

condor_submit process.sub arglist="-process filename.01.json -x 1 -y 1 outfile.0101.json"
arguments = $arglist

I am having issues because I have put this in a python script that builds the command string and then calls os.system(cmd) and the quotes are getting lost even if I try to escape them

The number of arguments also are variable so I cannot just hard code in the submit 7 arguments,
but is there another way to just specify the arguments to condor_submit in the process.sub file
with $1 $2 or $args?
arguments = $(args)

I have looked through the examples and they all seem very simple but I am sure this is more common in the real world.