[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Conditional output file transfer

On Thu, 2 May 2013, Brian Candler wrote:

Is it possible to set a condition on which output files are transferred back
to the submitting host?

The two conditions I'm specifically interested in are:

(1) Don't return the error file if it is zero-bytes long


(2) Don't return the error file if the job exited normally (rc=0)

That is, if I submit 10,000 jobs I'd like to get back only the error files
for those jobs which failed.

Any ways to achieve this?

I don't know of a way to actually make the file transfer conditional. But if you're running your jobs inside a DAG, you could use a POST script to get rid of the undesired files -- so you wouldn't save file transfer bandwidth, but you at least wouldn't have to deal with the files manually. A POST script can know the return code of the job, so it would be easy to delete the error file if the job succeeded...

Kent Wenger