[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Delay in job log writing
- Date: Thu, 15 Aug 2013 12:52:08 -0500
- From: Todd Tannenbaum <tannenba@xxxxxxxxxxx>
- Subject: Re: [HTCondor-users] Delay in job log writing
On 8/15/2013 9:15 AM, Smith, Ian wrote:
No answer to thisunfortunately but just to confirm that I have also seem this (with
> and it is a bit frustrating if you are polling the log files for
events rather than
> using condor_q (not a good idea as it can overload the scheduler).
> My guess is that Condor only updates the logfiles periodically and
this isn't down
> to an fsync() problem. If that's the case, is there any way to
configure Condor to
> update the logfiles more often ?
There is an answer - I sent the answer to the list on Monday and the
original poster confirmed things are working as I explained.
Post from Monday is here:
Relevant info from the post is:
The condor_q tool reports a job as "running" as soon as a condor_shadow
process is spawned for the job. However, the execute event does not show
up in the event log until the moment before the job is spawned on the
remote execute machine. Thus the time it takes the shadow to activate
the claim on the execute machine (aka start a condor_starter) and
potentially transfer any input files will show up as running time with
condor_q, but an execute event will not appear in the log until these
Hope the above helps,
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of ???
Sent: 12 August 2013 15:17
To: HTCondor-Users Mail List
Subject: [HTCondor-users] Delay in job log writing
Hi, I find that there is delay in job log file writing. In my test, condor_q showed one job were running, but there were only a submit event of it in the log file. After a while, more events occurs and they would be wrote together. I think this is because of the buffer in write/fwrite function.
I hope that my log reader program can know events as soon as possible, so is there any configuration can do this? I think condor_shadow should fsync after writing one event into log file.
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
You can also unsubscribe by visiting
The archives can be found at:
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing Department of Computer Sciences
HTCondor Technical Lead 1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132 Madison, WI 53706-1685