[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Remap output directories

Dear Todd,
Thank you very much it works perfectly !!
Best regards,
On sept. 11 2020, at 9:30 pm, Todd Tannenbaum <tannenba@xxxxxxxxxxx> wrote:
On 9/11/2020 1:45 PM, romain.bouquet04@xxxxxxxxx wrote:
Dear all,

I would like to remap a directory at the end of condor jobs.
Let say I have two jobs (job 1 and 2) each producing a directory results/ but producing different subdirectories.
  • job1 produce the subdirectory results/A/
  • job2 produces the subdirectory results/B/
I would like to remap the results/ directory of each job to a common path different than where the jobs have been submitted such that in the end I obtain both subdirectories A and B to be in : other_path/results/.
i.e other_path/results/A and other_path/results/B

For the directory to be copied in the condor submit file I use
transfer_output_files = results

If I understand what you desire correctly, I think you can accomplish it via a submit file like the below.  Note the use of "initialdir" to specify a location for your results other than the directory where you ran condor_submit (see condor_submit man page for initialdir).  Also note that by setting initialdir, your input data will be relative to that directory as well, so maybe you want to use a full qualified path to your input.  So a complete example would look like:

  executable = foo.exe
  should_transfer_files = YES
  transfer_input_files = /my/input/data
  transfer_output_files = results
  initialdir = /location/to/my/other_path

Hope the above helps,