Re: [HTCondor-users] Dag generated files to specific directory

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 5/17/2022 8:40 AM, Takis Angelides wrote:

Hello,

Is there a way to specify in the .dag file which directory it should put the .nodes.log, .lock, .lib.out, .lib.err etc?

I guess one can do it manually for every dag submitted by forcing the .dag.condor.sub not to be immediately submitted when the condor_submit_dag is used and go edit the .dag.condor.sub file on the lines

output = SW_MPS.dag.lib.out
error = SW_MPS.dag.lib.err
log = SW_MPS.dag.dagman.log

but this is a bit impractical to do manually and also the other files like .rescue, .lock etc are not there to specify their directory.

Thanks in advance for your help!

Kind regards,

Takis

    Hi Takis,
    
    I believe that DAGMan is currently hard-wired to place these files
    in the same directory where your primary .dag file lives.  There are
    some good reasons for this behavior, but I admit I am curious about
    your use case that made you ask the above question.
    
    One thing you can do today is simply copy your .dag file to the
    directory you wish to contain the all these other files and submit
    from there.
    
    For example, let say you have the DAGMan file
    "/usr/local/examples/foo.dag", but you want all the DAGMan generated
    output files to live in /home/takis/myanalysis.  You could do the
    following:
    
       % cd /home/takis/myanalysis
         % cp /usr/local/examples/foo.dag .
         % condor_submit_dag foo.dag
    
    Hope this helps,
    regards,
    Todd