Re: [Condor-users] registering a callback function to detect a job submission

["Kyungyong Lee" <klee@xxxxxxxxxxxx>]

Thank you for the information Alain,

I think using EventLog seems to provide much better way to detect job
submission than SchedD log file by lessening the number of unnecessary file
changes. I will post further questions if I get some while working on the
EventLog. Thanks.

Best,
Kyungyong Lee


-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Alain Roy
Sent: Monday, October 04, 2010 11:19 AM
To: Condor-Users Mail List
Subject: Re: [Condor-users] registering a callback function to detect a job
submission

On Sep 27, 2010, at 9:05 AM, Kyungyong Lee wrote:
> Hi All,
> 
> I have a question about a method to detect a condor job submission from
the third party application. The working scenario in my mind is:
> 
> A third party application registers a callback function to the
condor_schedd or condor_collector.
> 
> Whenever a job is submitted, I want to make the callback function to be
called.
> 
> 
> For this purpose, I used "SchedLog" file changes. At the third party
application, I registered a callback function for the "SchedLog" file
change. By detecting "SchedLog" file changes, I could monitor a job
submission, but the file seems to have too many unnecessary changes which
are not directly related to job submission. I am wondering if there is a
better way to detect a job submission as a callback type function. Thanks.

Hi Kyungyong,

As you guessed, there is no such callback provided by Condor. Using the
SchedLog file is iffy--it's not meant to be machine-parseable, and it may
change across versions of Condor. 

There is a poorly documented feature called the event log that might help
you. On a given schedd (not the collector) you can create a file that looks
just like the user job log file (the one you specify with "log = ..." in the
submit file), but it will log all events for all jobs for all users on that
schedd. You need to enable this in the schedd's configuration file:

http://www.cs.wisc.edu/condor/manual/v7.4/3_3Configuration.html#param:EventL
og

Hopefully this helps you out. I'll note that it's still not trivial to
implement what you want, if you care about catching every single event
reliably. The event log will rotate when it gets to big (you can control how
it's rotated with configuration). If lots of jobs are submitted, will you be
able to keep up? What if your program crashes and needs to pick up where it
left off? 

-alain

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/