Re: [Condor-users] Job completion hooks

[Tyler Strickland <tyler@xxxxxxxxxx>]

On 07/14/2011 08:17 AM, Matthew Farrellee wrote:
> On 07/07/2011 05:36 PM, Tyler Strickland wrote:
> > All,
> >
> > Here's my issue: on my cluster, I'm handling user notification of job
> > completion instead of letting condor send out those messages - I've
> > automated the process of job submission and file retrieval via a Django
> > website. This process tends to work well - the programs I run are
> > actually scripts, where the first and last lines of the scripts
> > interface with my software and let me know when the job has actually
> > started and stopped. The problem comes in when the generated output
> > files are large and take several minutes to transfer back to the
> > submitting computer; technically the user could receive a completion
> > email and visit the website to download their files before those files
> > have finished transferring.
> >
> > I've looked into hooks, but the one that seems to run at the latest
> > point in the processing chain - HOOK_JOB_EXIT - runs prior to file
> > transfer. I thought about HOOK_JOB_CLEANUP, but the part in the
> > documentation about running the specified cleanup script instead of
> > Condor's cleanup scripts causes a bit of trepidation regarding that
> > route. Is there any way to run a program once _all_ the steps have
> > completed processing without interfering with Condor's inner workings?
> >
> > Thanks,
> > Tyler
>
> The JOB_CLEANUP hook is for the JobRouter and would not help you anyway.
> There isn't a hook that will help you. I imagine it would be a submit
> side hook, maybe in the Shadow. I believe people monitor job event logs
> in cases like this.
>
> Best,
>
>
> matt

I'm sorry to hear that, Matt.  Knowing when a job is fully complete - 
file transfers included - could be a very useful thing, especially when 
files could take up to several minutes to transfer.  I'll look further 
into watching the log files, but my preliminary look into that didn't 
turn up much that would be of direct assistance.

On a side note, does Condor's email notification system send out emails 
before or after the file transfer occurs?  If after, I might be able to 
redirect those emails into my administration software and use them as 
"hook" themselves...

Thanks,
Tyler