Re: [HTCondor-users] Delay in job log writing

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 8/15/2013 9:15 AM, Smith, Ian wrote:
> No answer to thisunfortunately but just to confirm that I have also seem this (with 
version 7.8.6)
> and it is a bit frustrating if you are polling the log files for 
events rather than
> using condor_q (not a good idea as it can overload the scheduler).
>
> My guess is that Condor only updates the logfiles periodically and 
this isn't down
> to an fsync() problem.  If that's the case, is there any way to 
configure Condor to
> update the logfiles more often ?


There is an answer - I sent the answer to the list on Monday and the 
original poster confirmed things are working as I explained.

Post from Monday is here:
https://www-auth.cs.wisc.edu/lists/htcondor-users/2013-August/msg00042.shtml

Relevant info from the post is:

The condor_q tool reports a job as "running" as soon as a condor_shadow 
process is spawned for the job. However, the execute event does not show 
up in the event log until the moment before the job is spawned on the 
remote execute machine. Thus the time it takes the shadow to activate 
the claim on the execute machine (aka start a condor_starter) and 
potentially transfer any input files will show up as running time with 
condor_q, but an execute event will not appear in the log until these 
operations complete.

Hope the above helps,
Todd


> cheers,
>
> -ian.
>
>
>
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of ???
> Sent: 12 August 2013 15:17
> To: HTCondor-Users Mail List
> Subject: [HTCondor-users] Delay in job log writing
>
> Hi, I find that there is delay in job log file writing. In my test, condor_q showed one job were running, but there were only a submit event of it in the log file. After a while, more events occurs and they would be wrote together. I think this is because of the buffer in write/fwrite function.
> I hope that my log reader program can know events as soon as possible, so is there  any configuration can do this? I think condor_shadow should fsync after writing one event into log file.
> Thanks!
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685