[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] job queue log file rotate

On 4/13/20 4:55 PM, Cheng-Jui Yu wrote:

We have a process that reads job-queue-log-file. Currently It must skip lines at the beginning of the new log file after a log-rotate to avoid duplicate transaction lines carried over from the previous file. The process skips all the lines until it sees a 105. We are looking for a better way to do this. Is there a way to tell schedd not to carry overÂtransaction lines after a rotate? Or is there a way to definitively know what lines or how many lines to skip. Thanks.


There's now way to tell the schedd not write write these lines before the 105 -- that's all the state of the schedd that it needs in case it restarts and needs to recover. I don't know what you need this data for, but the condor_job_router tails and ingests the job_queue.log, perhaps you could use the job router? Or, the global event log can write some job events to a separate file that is rotated, but not compressed like the job_queue.log -- would this be a useful source of data?