Some hints from my experiences doing scraping the output of the command line tools.
Note that, if you do this, you should avoid querying the schedd (condor_q) at all costs for any general stats unless absolutely necessary.
If you can handle limited delays then query the collector for those details via condor_status -submitter instead.
*never* use condor_q -global. The rest of your users will hate you for it.
Querying the schedd for a specific <ProcessId>.<JobId> is also much faster than querying for a ProcessId (in that once it finds a match it stops scanning it’s state)
You will need to deal with condor_q occasionally failing to respond as well.
I strongly suggest that when you do this you start using the -format options from the beginning, it will make you far more tolerant to changes in the command line tools behaviour. This will likely mean you need to be able to parse unix epoch based second timestamps (trivial on unix, not hard on windows with a function)
Gloucester Research Limited believes the information provided herein is reliable. While every care has been taken to ensure accuracy, the information is furnished to the recipients with no warranty as to the completeness and accuracy of its contents and on condition that any errors or omissions shall not be made the basis for any claim, demand or cause for action.
The information in this email is intended only for the named recipient. If you are not the intended recipient please notify us immediately and do not copy, distribute or take action based on this e-mail.
All messages sent to and from this email address will be logged by Gloucester Research Ltd and are subject to archival storage, monitoring, review and disclosure.
Gloucester Research Limited, 5th Floor, Whittington House, 19-30 Alfred Place, London WC1E 7EA.
Gloucester Research Limited is a company registered in England and Wales with company number 04267560.