I think Todd touched on this in a previous thread:
Finally, re ND's Condor Log Analyzer, you may be interested in the HTCondor config
knob "EVENT_LOG" (see the Manual)- this specifies a file that receives job user log events, but across all users and user's jobs on a given schedd. I added a link to Condor Log Analyzer here:
I haven't tried this so I'm not sure if it records the usernames (my guess
is it maybe just the jobIDs).
Here, I cheat and wrap condor_submit so log files are always created in
a central location (even if the users
don't ask for them). This has the advantage that the logs are already broken
and per-submission date but of course it is possible to get around this
if users are particularly devious.
Dr Ian C. Smith,
Advanced Research Computing,
University of Liverpool UK.
On Behalf Of Matthew Forshaw
Sent: 11 December 2012 10:03
Subject: [HTCondor-users] Storing HTCondor user log files centrally
Hopefully a fairly simple configuration question
for you all.
My aim is to collect detailed logs for all jobs
going through our pool to carry out some analyses as part of my PhD. We currently collect fairly coarse-grained workload traces but are now interested in gaining a deeper insight into what is happening on a per-job level.
I was wondering if it is possible to specify a
central location where all HTCondor user logs are stored, in addition to the location specified in the users' submit files.
1. I would like this to happen automatically without
users having to change their submit scripts.
2. I would like logs to be stored even if a log
file location has not been specified inside the job description.
3. Ideally I would like these logs to be in ClassAd
XML format but I should be able to parse the output regardless of format.
If anyone has attempted something similar before,
or could advise me on the best way to go about this, I would really appreciate it. Thanks in advance for any help you can offer!