[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Determine when all jobs in a cluster have finished?

If you know the cluster ID, you do something like this (taken directly from the manual: http://research.cs.wisc.edu/htcondor/manual/current/condor_wait.html)
condor_wait logfile 40
This command waits for all jobs that exist in logfile with a job ClassAd attribute ClusterId of 40 to complete.

I have code that does something similar but i just wait for any job to get back to post process it.  I utilize "condor_wait -num" (also described int he manual).  In that case, i have a loop that is something like this (in psuedo-ish c++)

submitAllJobs(); // use MyCondorLog.txt as the log
for(int i=0;i<numJobs;i++)
    string command = "condor_wait " + intToString(i) + " MyCondorLog.txt";

All jobs will be running, when the first one comes back, it post processes it and continues in that fashion.  If 5 jobs have finished and your code calls condor_wait -num 2, it does not "wait" at all since 5 jobs have already come back and you are telling it to wait until 2 jobs are done.

On Thu, Jan 31, 2013 at 11:09 AM, Brian Pipa <brianpipa@xxxxxxxxx> wrote:
On Thu, Jan 31, 2013 at 1:01 PM, Brian Pipa <brianpipa@xxxxxxxxx> wrote:
> The only tricky part I can think of is QueryDB knowing when each job
> is finished and to know when to stop looking for results (so it
> doesn't wait forever and knows when it's done). I guess I'll just have
> to periodically run condor_q and condor_query to figure out which
> worker jobs are done and once I know they are done, postprocess them.
Edit: that should have said condor_history not condor_query. But...I
just saw the condor_wait part of the reply - that should do nicely to
tell which workers are done... is there a way to know when all of the
workers in a cluster are done? I guess that's where I would use
condor_q <clusterid> instead.

HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at: