Re: [Condor-users] registering a callback function to detect a job submission

[Alain Roy <roy@xxxxxxxxxxx>]

On Sep 27, 2010, at 9:05 AM, Kyungyong Lee wrote:
> Hi All,
> 
> I have a question about a method to detect a condor job submission from the third party application. The working scenario in my mind is:
> 
> A third party application registers a callback function to the condor_schedd or condor_collector.
> 
> Whenever a job is submitted, I want to make the callback function to be called.
> 
> 
> For this purpose, I used “SchedLog” file changes. At the third party application, I registered a callback function for the “SchedLog” file change. By detecting “SchedLog” file changes, I could monitor a job submission, but the file seems to have too many unnecessary changes which are not directly related to job submission. I am wondering if there is a better way to detect a job submission as a callback type function. Thanks.

Hi Kyungyong,

As you guessed, there is no such callback provided by Condor. Using the SchedLog file is iffy--it's not meant to be machine-parseable, and it may change across versions of Condor. 

There is a poorly documented feature called the event log that might help you. On a given schedd (not the collector) you can create a file that looks just like the user job log file (the one you specify with "log = ..." in the submit file), but it will log all events for all jobs for all users on that schedd. You need to enable this in the schedd's configuration file:

http://www.cs.wisc.edu/condor/manual/v7.4/3_3Configuration.html#param:EventLog

Hopefully this helps you out. I'll note that it's still not trivial to implement what you want, if you care about catching every single event reliably. The event log will rotate when it gets to big (you can control how it's rotated with configuration). If lots of jobs are submitted, will you be able to keep up? What if your program crashes and needs to pick up where it left off? 

-alain