[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] python api about history



Thank you very mach.
Hope history() can provide a parameter setting history file like "-file" in condor_history command.


NAME:Jiang Xiaowei
MAIL:jiangxw@xxxxxxxxxxxxxxx
TEL:010 8823 6024
DEPARTMENT:Computing Center of IHEP
 
From: Iain Steers
Date: 2016-11-15 17:56
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] python api about history
Ah I was thinking of the since and stop constraints.
 
https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5642,56
 
Looks like they haven't been added to the python bindings yet.
________________________________________
From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of jiangxw@xxxxxxxxxxxxxxx [jiangxw@xxxxxxxxxxxxxxx]
Sent: 15 November 2016 10:53
To: htcondor-users
Subject: Re: [HTCondor-users] python api about history
 
Hi Iain,
    Thanks for your advice.
    I can catch your meaning.
    I implemented that function like yours.
    And the space of history files in our environments is 132G totally.
    When I  execute the script with the lines
        requirements = "EnteredCurrentStatus <= %s && EnteredCurrentStatus > %s" % (current_time,pre_time)
        job_history = self.schedd.history(requirements, projection, 0),
    the script will take too long times.  I think self.schedd.history() will look over all 132G history files.
    How can I optimize it?
    Thanks.
 
Cheers,
Jiang Xiaowei
________________________________
NAME:Jiang Xiaowei
MAIL:jiangxw@xxxxxxxxxxxxxxx
TEL:010 8823 6024
DEPARTMENT:Computing Center of IHEP
 
From: Iain Steers<mailto:iain.steers@xxxxxxx>
Date: 2016-11-15 17:16
To: HTCondor-Users Mail List<mailto:htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] python api about history
Hi,
 
history() can take a Requirement constraint as one of its arguments.
 
In order to do this efficiently i would suggest something like the following.
 
start_dt = datetime.datetime.now() - datetime.timedelta(minutes=30)
start_stamp = calendar.timegm(d.timetuple())
schedd = htcondor.Schedd()
jobs = schedd.history('EnteredCurrentStatus >= {0}'.format(start_stamp))
 
This should return an iterator of all the jobs which were added to the history in the last half hour.
 
Cheers, Iain
 
________________________________________
From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of jiangxw@xxxxxxxxxxxxxxx [jiangxw@xxxxxxxxxxxxxxx]
Sent: 15 November 2016 10:08
To: htcondor-users
Subject: [HTCondor-users] python api about history
 
Hi all,
    I used python api to get job history information then insert job information into database.
       The function is htcondor.Schedd.history().
    But when the amount of the history files in "/var/lib/condor/spool/" became large,
    htcondor.Schedd.history() will be executed slowly.
    I want to gain the job history information every half hour. So I just need to look over the history file generated last half hour.
    But the htcondor.Schedd.history() may look over all history files in "/var/lib/condor/spool" every time.
    I can not find  the parameter from htcondor.Schedd.history() like "condor_history -file".
    Are there some ways to solve it?
    Thanks for help.
 
Best regards,
Jiang XiaoWei
________________________________
NAME:Jiang Xiaowei
MAIL:jiangxw@xxxxxxxxxxxxxxx
TEL:010 8823 6024
DEPARTMENT:Computing Center of IHEP
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
 
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
 
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/