[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Want to keep and see executable, input and output files transferred/generated on condor execute machine



Here are a couple things you can do to examine the output of individual jobs on the execute machine to test that the output is being transferred back appropriately:

1) Wrap a job in a shell script that copies everything in the execute directory to some other location after the real program exits. Then, you can examine those files after the job completes.

2) While the job is running, run condor_ssh_to_job. You will get a terminal session on the execute machine in the execute directory of the job. You can examine the files there interactively. Once the job finishes and you exit from the terminal session, the output files will be transferred back as normal and the job will be marked as completed.

 -- Jaime Frey

On Feb 19, 2013, at 10:16 PM, Harshad Prajapati <harshad.b.prajapati@xxxxxxxxx> wrote:

Jaime Frey:

Thank you.

I want to just make sure that whatever output I am getting is really what is present on execute machine. Otherwise, there is no specific reason.

With regards,
Harshad Prajapati

On Tue, Feb 19, 2013 at 11:20 PM, Jaime Frey <jfrey@xxxxxxxxxxx> wrote:
On Feb 14, 2013, at 5:37 AM, Harshad Prajapati <harshad.b.prajapati@xxxxxxxxx> wrote:

> My question is related to submitting a job to condor using condor-g submission.
> I want to see where transferred executable and input files are stored on execute machine for job submitted using condor-g. Similarly, what about output file?
> Moreover, I want to keep those files on execute machines, what configuration do I need to do?
> Do I need to configure gsi-gatekeeper of globus or Startd of condor?


On the execute machine, HTCondor creates a temporary directory for each job to run in. All files for the job are stored there while the job runs and the directory is deleted when the job completes. There are no configuration parameters to disable the cleanup of this directory.

On the gatekeeper machine, Globus GRAM keeps job-related files in directories under the user's home directory. These directories are deleted when the GRAM client (e.g. Condor-G) tells it to do so.

Why do you want to keep these files on the execute machine after they've been transferred back to the job submitter?

Thanks and regards,
Jaime Frey
UW-Madison HTCondor Project