Re: [Condor-users] Cleaning up condor files

[matthew hope <matthew.hope@xxxxxxxxx>]

On Wed, 1 Dec 2004 14:25:00 -0000, Colin Gillespie
<c.gillespie@xxxxxxxxxxxxxxx> wrote:
> Dear All,
> 
> How do people clean up all the condor log files in a directory? A straight rm * 
> isn't possible since some of the jobs may be still running. What's the best way 
> to scan the files and determine which ones can be deleted.

A good point I've been thinking about.

note that:

If you NEVER reuse a log file for multiple runs then you can: (i.e if
it fails for some reason and you resubmit then the file name is
changed, tricky to achieve without some annoyance like injecting
timestamps)

grep for "XX:XX:XX Job terminated." and "XX:XX:XX Job was aborted by the user."
(substitue your fave regex for digits for X)
and delete only those files.

this however is not much use if you resubmit.

You may have do do what I am planning on doing which is look in the
file for the job id (actually I can look in the file name since I
control the naming but you may not have that luxury), then use
condor_q to see if it is still in the queue (annoying since this can
hold up the schedd so avoid doing it to often or too rapidly).

If it is in the queue leave it (note that quueu number resets provide
a source of false positives but no false negatives), if not felete.

Alternate 2
Use condor wait, when the job completes wait for x amount of time then
rm the files :)
(or variant thereof)
Note that it is finished with and at what time. every so often run a
job which checks this time against the file modified time - if
significantly different leave it, if not delete)

If you come up with a better way tell the group :)