[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Putting output and error files in output-transferred subdirectory?

Hi folks,

We're working through a conversion of legacy Grid Engine job submission scripts which had originally had essentially a s/qsub/condor_submit/g done to them, to take advantage of the full range of HTCondor capabilities. The new 8.4 features with "queue from" allow us to massively simplify the submissions, eliminating nearly all submit-side scripting.

This leaves us with some legacy considerations, however. The output of each run in a batch goes into a numbered subdirectory, and the stdout of the executable has always been found in that subdirectory as a "screen_out.txt" file.

However, we can't do this:

output = dir_$(Process)/screen_out.txt

... because condor_submit wants the directory where the output file is going to go to exist already, and I'd rather allow the output transfer to create the directory rather than having to stage all the output subdirectories before submitting the job - that's way too "Grid Engine" for my tastes.

Any suggestions? Maybe an output transfer remap? Though output and error aren't really under that umbrella, are they? Or a PostCmd to move the scratch directory's "_condor_stdout" into the subdirectory which would then be covered by the output transfer list?

Thanks for anything you can suggest.

        -Michael Pelletier.