[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Splitting condor_history by user/task?


our job-manager tool's Condor wrapper is showing increasing delays when extracting information from condor_history (and to a lesser extend, condor_q). This is especially the case with user pools that do not split the history file regularly and have lots of user jobs, preventing our wrapper from partitioning its search. The main problem seems to be that regardless of constraints, condor_history will go through the entire history file merely suppressing lines that do not match the constraint.

What I'm looking for is a way to tell Condor to split its history/q file (or make a copy of the relevant parts) either for each user or preferably for a dynamic key of the job/task. Something akin to job Log in combination with EVENT_LOG_JOB_AD_INFORMATION_ATTR, but the important thing is that we need to be able to define it solely through the job so that we do not require the pool to be configured specifically for us.