[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] How to check whther jobs have finished executing when submitted from a program

Sameer Bahl <sameer.bahl@xxxxxxxxx> wrote:
> I  want the function submitting the jobs in the program to
> return only when the jobs have finshed executing.

I would recommend either calling the program condor_wait, or one
of the solutions for trackign a job below.

> Is there a way to retrieve the status of the jobs sumitted i.e.
> whether they are idle, running or finished from within the
> program?  I mean something like condor_q but i need to capture
> the job status in the program.

There are several ways we recommend to programmatically watch
Condor jobs.

First, you can poll condor_q.  The "-format" makes it easy to get
the data in an easy to parse format.  On the down side, calling
this frequently can overload the schedd.  Also, jobs that have
completed lead the schedd's queue, so condor_q can't tell you
anything about them.  (You can work around this by using the
very, very slow condor_history, or using on_exit_hold or
on_exit_remove to keep completed jobs in the queue until you're
done with them.)

Second, you can use SOAP.  Condor's SOAP interface does suffer
the same weaknesses of calling condor_q.

Finally, you can watch the job's user log (the log created when
you add "log=/path/to/my/log" to your jobs).  The file was
specifically designed to be reasonably easy to parse.
condor_dagman and condor_wait use this interface.  Condor
provides a C++ interface (see include/user_log.README in your
Condor installation), although rolling your own for simple
purposes isn't too hard.  For example, the Grid Exerciser
(http://www.cs.wisc.edu/condor//tools/exerciser/) includes
"summarize_condor_log", which includes a user log parser written
in Perl.

Alan De Smet                              Condor Project Research
adesmet@xxxxxxxxxxx                 http://www.condorproject.org/