Re: [HTCondor-users] finalize feature

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

Todd, I think what he's getting at is running something when all processes in a cluster are finished.

What I've done in the past is set up a final scheduler-universe job in the submit description with a final "queue" command that sets up a "/bin/sh" command line or a script which does a condor_wait on the $(LOG) file for the first set of jobs and then run the post-completion tasks. Something along these lines:

Blah blah blah
Task_Log = task.$(Cluster).log
Log = $(Task_Log)
Queue 100

Universe = scheduler
Executable = post_completion.sh
Arguments = $(Task_Log)
Log = UNDEFINED
Queue 1

.. if you see what I'm getting at. The script does a condor_wait on the $1 argument to watch the log file of the first 100 jobs, then proceeds with the tasks-completed processing after condor_wait is finished.


Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Todd L Miller
Sent: Wednesday, July 24, 2019 2:04 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [External] Re: [HTCondor-users] finalize feature

> Without DAGMan, is there a way to run POST after all jobs have been 
> completed? I would like to atleast send notification after all jobs 
> are completed.

 	If you don't want to manually add a node whose parents are the set of all otherwise-terminal nodes, you can use a FINAL node and the
$(FAILED_COUNT) macro.  See the following for details:

https://htcondor.readthedocs.io/en/latest/users-manual/dagman-applications.html#final-node

- ToddM
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/