[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Python bindings and per job history files



But naively I would expect an iterator of 1 still be valid.
Anyway, I will see what I can do. Thanks for the hints.

RegardingÂLastRemoteHost, is there no way to inject it into the job ad? I've seen examples of things likeÂMemoryUsage:
STARTD_JOB_EXPRS = $(START_JOB_EXPRS) MemoryUsage

This should work for HEPSPEC06 as well or do I misunderstand the procedure?

Cheers,
Luke

On 7 March 2016 at 18:24, Iain Bradford Steers <iain.steers@xxxxxxx> wrote:
Also in reference to your original issue.

classad.parseAds expects a stream of classads e.g. (âcondor_history -l -match 4â) whereas youâre only giving it one file.

IIRC the function you need is parseOne(f).

Cheers, Iain

On Mar 7, 2016, at 19:21, Iain Steers <iain.steers@xxxxxxx> wrote:

Hi Luke,

Ah the htcondor-ce history files.

Unless something has changed in this latest release the advise is to parse the batch schedd history files instead.

IIRC thereâs some missing fields that are added in job routing and by the batch schedd that donât yet make it back to the CE schedd.

LastRemoteHost being one of the missing fields makes it difficult to apply per machine scaling factors to the jobs.

Cheers, Iain

On Mar 7, 2016, at 19:12, L Kreczko <L.Kreczko@xxxxxxxxxxxxx> wrote:

Dear experts,

I am currently trying to parse our per job history files. The example I got together so far is
import classad;
f = open("/var/lib/gratia/condorce_data/history.10.0");
ads = classad.parseAds(f);
for c in ads:
ÂÂÂ print c
where the printout looks similar to that of a classAd (indeed the type is a class ad).

However,
print c["JobStatus"]
fails with
Traceback (most recent call last):
 File "test_history.py", line 7, in <module>
ÂÂÂ print c["JobStatus"]
StopIteration

From the examples I've seen, this should be possible.

Fun fact: if I add
print dir(c )
to the for-loop, it works.
Equivalently, if I change the iterator to a list
for c in ads: -> for c in list(ads):
it works as well.
Is this a bug in the way the iterator is constructed or am I missing something?


The history file can be seen here:
https://gist.github.com/kreczko/9056874bbb54a1b4a21b#file-gistfile1-txt
I am running htcondor 8.4.3 and python 2.6.6.

Cheers,
Luke

--
*********************************************************
 Dr Lukasz Kreczko     ÂÂ
 Research Associate
 Department of Physics
 Particle Physics Group

 University of Bristol

 HH Wills Physics Lab
 University of Bristol
 Tyndall Avenue
 Bristol
 BS8 1TL


 +44 (0)117 928 8724Â
ÂÂ
 A top 5 UK university with leading employers (2015)
 A top 5 UK university for research (2014 REF)
 A world top 40 university (QS Ranking 2015)
*********************************************************
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



--
*********************************************************
 Dr Lukasz Kreczko     ÂÂ
 Research Associate
 Department of Physics
 Particle Physics Group

 University of Bristol

 HH Wills Physics Lab
 University of Bristol
 Tyndall Avenue
 Bristol
 BS8 1TL


 +44 (0)117 928 8724Â
ÂÂ
 A top 5 UK university with leading employers (2015)
 A top 5 UK university for research (2014 REF)
 A world top 40 university (QS Ranking 2015)
*********************************************************