[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] program arguments conditional on filesystem domain?



Jason,

Something like the following should work.

Arguments = "-i '$$([ifThenElse(MY.FileSystemDomain==TARGET.FileSystemDomain,""$(inputdir)/"","""")])$(video).mp4' -o ."

The $$([ expr ]) construct allows you to evaluate a ClassAd expression that refers to both job and machine attributes. The result is inserted into the string in which it is embedded.

If the input directory contains only files needed by the job and not lots of other unneeded stuff, you could also just transfer the input directory rather than transferring the video file within it. This would result in an identical relative path to the file in the two cases of file transfer vs. no file transfer. Currently, there is no way to transfer the input directory and only some of the files within it. Directory transfer is all or nothing.

--Dan

On 1/22/13 3:43 PM, Jason Ferrara wrote:
Is it possible in a submit file to make program arguments conditional on whether or not the job is running in the same filesystem domain from which it was submitted?

For running in the same filesystem domain my submit files typically look like...

executable = /bindir/mycommand
transfer_executable = false
universe = vanilla
should_transfer_files = NO
initialdir = $(outputdir)/$(video)
Arguments = -i $(inputdir)/$(video).mp4 -o .
Queue

where $(outputdir) and $(inputdir) are absolute paths.

For a different filesystem domain they need to look like....

executable = /bindir/mycommand
transfer_executable = false
universe = vanilla
should_transfer_files = YES
initialdir = $(outputdir)/$(video)
transfer_input_files = $(inputdir)/$(video).mp4
Arguments = -i $(video).mp4 -o .
Queue

Is there a way to combine these into one submit file that could run both in the same filesystem domain and outside of it? Something like

executable = /bindir/mycommand
transfer_executable = false
universe = vanilla
should_transfer_files = IN_NEEDED
initialdir = $(outputdir)/$(video)
transfer_input_files = $(inputdir)/$(video).mp4
if (same filesystem domain)
    Arguments = -i $(inputdir)/$(video).mp4 -o .
else
    Arguments = -i $(video).mp4 -o .
Queue

or some other way to handle input files from outside the working directory that doesn't require an extra copy when the job runs in the same filesystem domain?

Thanks

- Jason