Re: [HTCondor-users] Configuration of logging

[Jaime Frey <jfrey@xxxxxxxxxxx>]

> On Apr 18, 2018, at 4:32 AM, Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx> wrote:
> 
> Dear HTCondor experts,
> 
> I just realized HTCondor takes care of logging and rotating the logs by itself, without leveraging existing tools such as logrotate. 
> This caused an issue because we had a schedd logging (due to a problematic job) almost continuously over night, and this morning, the 2 GB log partition was full. 
> Of course, I could reduce the log sizes and number of rotations using HTCondor parameters - but then I would (in the worst case) not be able to log more than a day in a few GB of storage. 
> 
> Is there any way to trigger automatic compression of logfiles after rotation, similar as to what logrotate can do out of the box? 
> For logfiles, this tends to save orders of magnitude of space. 
> 
> Also, is there a way to specify how long logs are kept (instead of going by size or number of rotations)? 
> Especially with the new GDPR it's common practice to (and I think everybody in Europe is even forced to...) specify how long log files are kept in units of "time" instead of "size". 


HTCondor doesnât have a way to automatically compress old log files. I suppose you could set up a cron job to do so.

You can specify a time instead of a file size for when the daemon logs should be rotated, like so:

# Save a dayâs worth of logs
MAX_SCHEDD_LOG = 4Hr
MAX_NUM_SCHEDD_LOG = 6

See MAX_DEFAULT_LOG in the manual for details.

Thanks and regards,
Jaime Frey
UW-Madison HTCondor Project