[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] collecting job statistics



On 6/10/2016 10:21 AM, Thomas Hartmann wrote:
Hi all,


Hi Thomas, some quick thoughts below...

Since this is a bit cumbersome, to go upstream to query job statistics,
I wonder if there is a more direct way to get a job's ClassAds (run as
on an exit hook, so in a job's context I suppose)?


So in the job's context, there will be an environment variable _CONDOR_JOB_AD that contains the path to a file in the job's scratch directory which contains the job ad for the currently running job. The job ad is current as of the start of the job, but is not updated during the running of the job. The job may read attributes and their values out of this file as it runs, but any changes will not be acted on in any way by HTCondor. The format is the same as the output of the condor_q -l command. See the Manual at https://is.gd/A3GYLr

If you also want to see job attributes that HTCondor is updating while the job runs, such as RemoteUserCpu.... since it sounds like you are using a Job Exit hook, isn't the full job classad with all the runtime statistics included passed to your hook via stdin ? At least that is what the Manual implies in section https://is.gd/KcVpRN. Can your Job Exit hook simply read in everything from stdin and call it good?


ps: on the way I noticed, that the Local{User,Sys}Cpu actually stays
zero for a running job at us - while the remote values change. Is there
something missing at us, that the statistics are not recorded/updated?


I think the Local* values are only updated for Standard Universe jobs, since with Standard Universe most system calls invoked by the job are actually sent over the network to be performed by the condor_shadow - thus the shadow could accumulate some pretty significant CPU usage. With vanilla universe and all other job universes, the shadows CPU usage should be very minimal / insignificant, thus HTCondor is not bothering to track it.

Hope the above helps
Todd