[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Non-sequential input files?



Hello,

Is there any way to work with a batch of jobs that use input files that are not numbered sequentially? I have a set of input files that are numbered with an ID; some IDs are not present so there are gaps in the sequence (1, 5, 6, 10, etc, not 1, 2, 3). So I can't call those input files with $(Process) in the submit file because they are not in sequence.

Is there any way to refer to a specific sequence of numbers for a batch - e.g. instead of using $(Process), can I call the numbers from a text file that lists which IDs are actually present? Or, can I somehow tell HTCondor to simply skip the jobs where the input file is not present, rather than aborting the job submission as it normally does?

I'd rather avoid a workaround like renaming all the input files sequentially, which would take some time as I'm working with tens of thousands of files.

Thanks,

Emily



Emily L. Weiser, PhD
2630 Fanta Reed Rd
La Crosse, WI 54603
(608) 781-6193