[HTCondor-users] Storing HTCondor user log files centrally

[Matthew Forshaw <m.j.forshaw@xxxxxxxxxxxxxxx>]

Good afternoon all,

Hopefully a fairly simple configuration question for you all.

My aim is to collect detailed logs for all jobs going through our pool to carry out some analyses as part of my PhD. We currently collect fairly coarse-grained workload traces but are now interested in gaining a deeper insight into what is happening on a per-job level.

I was wondering if it is possible to specify a central location where all HTCondor user logs are stored, in addition to the location specified in the users' submit files.

1. I would like this to happen automatically without users having to change their submit scripts.

2. I would like logs to be stored even if a log file location has not been specified inside the job description.

3. Ideally I would like these logs to be in ClassAd XML format but I should be able to parse the output regardless of format.

If anyone has attempted something similar before, or could advise me on the best way to go about this, I would really appreciate it. Thanks in advance for any help you can offer!

All the best,

Matt