Re: [HTCondor-users] job status from python

[Jason Patton <jpatton@xxxxxxxxxxx>]

For looking up jobs that have finished, there is
htcondor.Schedd().history(expression, projection), where you could
query for something like history(expression = 'ClusterId == 123',
projection = []) to get the ClassAd of job 123 *if* it has cleared the
queue. However, querying the history is ***very slow***.

Two better options:
1) Parse and watch the job log file, if your script can get to it. The
job log file will update when the job has started running, update
occasionally with resource usage, and update with the exit code when
it has finished.

2) Leave the job in the queue when it's completed and have your script
remove it:

In your Submit objects, set { 'leave_in_queue': '(JobStatus == 4)' },
which means when a job has completed, leave it in the queue. See
https://research.cs.wisc.edu/htcondor/manual/current/12_Appendix_A.html
to see what the value of JobStatus means.

Query for jobs' ClassAds using htcondor.Schedd().xquery().

If JobStatus == 4 and/or if ExitCode is defined, then you know the job
is done. Remove it from the queue by sending a
htcondor.Schedd().act(htcondor.JobAction.Remove, str(ClusterId)). (You
can also send a  list of ClusterIds.)


Constantly querying the Schedd probably won't scale well with the size
of the queue, so if you can use the job log, that's probably the
better of the two solutions.

Jason

On Jan 8, 2018 5:24 PM, "Larry Martell" <larry.martell@xxxxxxxxx> wrote:
>
> Is there any python API for checking the status of jobs?
>
> On Sun, Jan 7, 2018 at 3:47 PM Larry Martell <larry.martell@xxxxxxxxx> wrote:
>>
>> I am submitting jobs like this:
>>
>>     sub = htcondor.Submit(submit_dict)
>>     with schedd.transaction() as txn:
>>         id = sub.queue(txn)
>>
>> Now I want to be able to tell if the job has completed or not, and
>> when it has completed, if it succeeded or failed. How can I do that?
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/