Re: [condor-users] globus universe question

[Mark Calleja <mcal00@xxxxxxxxxxxxx>]

Hi Sebastian,

I also came up against the problem of retrieving output files from
condor-g jobs (see a previous thread in last month's mailing list), and
the only way I got round it was to amend the condor.pm file in globus so
that a new process is forked by the jobmanager which is then exec'd with
a monitoring process. This process waits for the condor job to finish
before using gsiftp to return all output files back to the submitting
machine. Not pretty, but it works. If you're interested in this route
then drop me a line and I'll give you what I've done.

Cheers,

Mark Calleja
-- 
Department of Earth Sciences, University of Cambridge
Downing Street, Cambridge CB2 3EQ, UK
Tel. (+44/0) 1223 333408, Fax  (+44/0) 1223 333450
http://www.esc.cam.ac.uk/~mcal00
 
On Mon, 2003-12-08 at 20:25, Jaime Frey wrote:
> On Mon, 8 Dec 2003, Sebastian Grinstein wrote:
> 
> > Hello Condor users and experts,
> >
> > I'm starting to use condor. I have simple question:
> >
> > I submit a job in the globus universe, the executable generates
> > an output file. How do I retrieve this file (or files)?
> >
> > This is the submission file I'm using:
> > > cat tio_globus.sdf
> > # Condor submit description file
> > Executable           = tio
> > Universe             = globus
> > globusscheduler      = remote.machine.edu
> > transfer_input       = true
> > transfer_input_files = arg_tio.i
> > transfer_files       = ONEXIT
> > transfer_output      = true
> > Input = arg_tio.i
> > Arguments = arg_tio.i arg_gtio.o
> > Output = /tmp/gtio.out
> > Error  = /tmp/gtio.err
> > Log    = /tmp/gtio.log
> > Queue
> >
> > "tio in out" copies the contents of the first file (in) into the
> > second (out).
> >
> > When I condor_submit, in the remote.machine, I see the job being done
> > in .globus/.gass_cache/...
> >
> > However, when the job is done the .globus/.gass_cache/ files are
> > removed, and condor gives me the gtio.* files, but how do I get
> > arg_gtio.o ?
> >
> > Of course there were no errors reported. If I try a
> > "globus-job-get-output" (with the id from the logs) I get a "Invalid
> > job id." (while "globus-job-status" says "DONE").
> 
> You need to specify "arg_gtio.o" with the transfer_output_files parameter
> in your submit description file. In most universes in Condor, if you omit
> transfer_output_files, Condor automatically transfers back all
> created/modified files in the job's execute directory. This doesn't work
> in the globus universe, as Globus doesn't an appropriate mechanism.
> 
> I see that this exception isn't well documented in the Condor manual. I'll
> see that we fix it.
> 
> +------------------------------------+-------------------------------+
> |             Jaime Frey             |There are 10 types of people in|
> |         jfrey@xxxxxxxxxxx          |the world: Those who understand|
> |   http://www.cs.wisc.edu/~jfrey/   |  binary, and those who don't  |
> +------------------------------------+-------------------------------+
> Condor Support Information:
> http://www.cs.wisc.edu/condor/condor-support/
> To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
> unsubscribe condor-users <your_email_address>



Condor Support Information:
http://www.cs.wisc.edu/condor/condor-support/
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>