[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] python api about history

Hi Iain,
    Thanks for your advice.
    I can catch your meaning.
    I implemented that function like yours.
    And the space of history files in our environments is 132G totally.
    When I  execute the script with the lines
        requirements = "EnteredCurrentStatus <= %s && EnteredCurrentStatus > %s" % (current_time,pre_time) 
        job_history = self.schedd.history(requirements, projection, 0),
    the script will take too long times.  I think self.schedd.history() will look over all 132G history files.
    How can I optimize it?

Jiang Xiaowei

NAME:Jiang Xiaowei
TEL:010 8823 6024
DEPARTMENT:Computing Center of IHEP
From: Iain Steers
Date: 2016-11-15 17:16
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] python api about history
history() can take a Requirement constraint as one of its arguments.
In order to do this efficiently i would suggest something like the following.
start_dt = datetime.datetime.now() - datetime.timedelta(minutes=30)
start_stamp = calendar.timegm(d.timetuple())
schedd = htcondor.Schedd()
jobs = schedd.history('EnteredCurrentStatus >= {0}'.format(start_stamp))
This should return an iterator of all the jobs which were added to the history in the last half hour.
Cheers, Iain
From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of jiangxw@xxxxxxxxxxxxxxx [jiangxw@xxxxxxxxxxxxxxx]
Sent: 15 November 2016 10:08
To: htcondor-users
Subject: [HTCondor-users] python api about history
Hi all,
    I used python api to get job history information then insert job information into database.
       The function is htcondor.Schedd.history().
    But when the amount of the history files in "/var/lib/condor/spool/" became large,
    htcondor.Schedd.history() will be executed slowly.
    I want to gain the job history information every half hour. So I just need to look over the history file generated last half hour.
    But the htcondor.Schedd.history() may look over all history files in "/var/lib/condor/spool" every time.
    I can not find  the parameter from htcondor.Schedd.history() like "condor_history -file".
    Are there some ways to solve it?
    Thanks for help.
Best regards,
Jiang XiaoWei
NAME:Jiang Xiaowei
TEL:010 8823 6024
DEPARTMENT:Computing Center of IHEP
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
The archives can be found at: