Re: [Condor-users] Setting environment variable to hold remote working dir

["Tomas Lidén" <tomas.liden@xxxxxxxxxxxx>]

Brain - that was a quick respons!!

> 
> On Thu, Aug 09, 2012 at 11:44:00AM +0200, Tomas Lid n wrote:
> >    I want to store the remote working directory in an environment variable
> >    (for later access in a sub-program).
> 
> Do you mean a sub-program spawned as a child of the first program? 

Yes!
> Or a separate job submitted by condor_dagman?  Or something else?
> 
> I'm a condor newbie, but I believe that when a job completes on a
> remote node, its temporary directory is deleted. So you are only
> guaranteed that it exists for as long as the first job is running.
> 

That's fine for me.

> >    universe = vanilla
> >    should_transfer_files = yes
> >    run_as_owner = true
> >    environment = "MY_ENV_VAR=$$(RemoteIwd)"
> >    queue
> 
> http://research.cs.wisc.edu/condor/manual/v7.8/11_Appendix_A.html#85394
> 
> RemoteIwd is an attribute of the Job ClassAd, whereas $$() accesses the
> Machine ClassAd.
> 
> A quick test suggests you can set RemoteIwd to set the directory where
> you
> *want* the job to excute:
> 
> cmd = /bin/pwd
> output = bc.out.$(Process)
> error = bc.err.$(Process)
> log = bc.log
> RemoteIwd = /tmp
> queue 6
> 
> However, I don't know if it's also updated when the job runs, so that
> it reflects the directory where it *is* running.
> 

Tried that. But that only sets the variable RemoteIwd. The program is still run in the normal place (condor/execute/dir_*).

> >    I can see that Condor sets a number of environment variables, such as
> >    TMP, TEMP, TMPDIR, _CONDOR_JOB_IWD etc. But how do I transfer these to
> >    MY_ENV_VAR?
> 
> Testing with a slightly different submit file:
> 
> cmd = /usr/bin/printenv
> output = bc.out.$(Process)
> error = bc.err.$(Process)
> log = bc.log
> queue 6
> 
> then I can see that the job itself receives these environment
> variables, at the time it runs:
> 
> $ grep IWD bc.out.*
> bc.out.0:_CONDOR_JOB_IWD=/var/lib/condor/execute/dir_2310
> bc.out.1:_CONDOR_JOB_IWD=/var/lib/condor/execute/dir_38759
> bc.out.2:_CONDOR_JOB_IWD=/var/lib/condor/execute/dir_2313
> bc.out.3:_CONDOR_JOB_IWD=/var/lib/condor/execute/dir_38763
> bc.out.4:_CONDOR_JOB_IWD=/var/lib/condor/execute/dir_2318
> bc.out.5:_CONDOR_JOB_IWD=/var/lib/condor/execute/dir_38766
> 
> So if you want to pass this information to a sub-process of your job,
> it's already in that job's environment.  Of course, the job could just
> do a "pwd"
> to find out what directory it is running in.
> 

Well, that would need to be in the first program - which I do not have control over (it's a testing framework).

Thanks for your effort, Brian. But problem is still not solved. (And all would be much simpler under Unix..)

If I can't solve it this way, we have to move the files to a network directory accessible from both the submit and execute machine (and setting the environment variable to that location).

/Tomas