[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Splitting condor_history by user/task?
- Date: Mon, 28 Jan 2013 08:30:39 -0500
- From: Peter MacKinnon <pmackinn@xxxxxxxxxx>
- Subject: Re: [HTCondor-users] Splitting condor_history by user/task?
On 01/28/2013 05:04 AM, Max Fischer wrote:
our job-manager tool's Condor wrapper is showing increasing delays
when extracting information from condor_history (and to a lesser
extend, condor_q). This is especially the case with user pools that do
not split the history file regularly and have lots of user jobs,
preventing our wrapper from partitioning its search. The main problem
seems to be that regardless of constraints, condor_history will go
through the entire history file merely suppressing lines that do not
match the constraint.
What I'm looking for is a way to tell Condor to split its history/q
file (or make a copy of the relevant parts) either for each user or
preferably for a dynamic key of the job/task. Something akin to job
Log in combination with EVENT_LOG_JOB_AD_INFORMATION_ATTR, but the
important thing is that we need to be able to define it solely through
the job so that we do not require the pool to be configured
specifically for us.
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
You can also unsubscribe by visiting
The archives can be found at:
Not sure it's exactly what you're looking for but I've developed a
contrib that (among other things) stores job history ads into mongodb.
There's an existing tool to query the db much like condor_history based
on various parameters, including a user name (which is indexed).
Unfortunately there is no full ClassAd constraint expression parsing
capability so your queries are really what you can do from a mongodb
client such as the provided tool (written in pymongo).
(scroll to bottom for history tool details)
Let me know if you have further questions if any of that sounds
Cloud BU/MRG Grid
Red Hat Inc.