Re: [HTCondor-users] Condor Jobs Logging for recurring jobs

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 3/14/2016 3:57 AM, Vamshikrishna Akundi wrote:
> Hi,
>
> I have to submit a job which runs every 1 hour.
>
> As of now, job is configured in such a way that the output of the job is
> written to a file which is mentioned in OUT.
>
> But, the logs in the file are overwritten whenever job runs.
>
> So I am looking for an alternative solution which covers either of
> following points:
>
> A.*Output file to append to the Main OUT file. (OR)*
>
> B.*Way to create dynamic output files, On basis of time or number.*

Hi Vamshi,

One way you can achieve the above is via the $$([ classad-expression ]) 
macro in your submit file. See the "Macros" section of the condor_submit 
man page at
  http://htcondor.org/manual/v8.4/condor_submit.html
for details.

So, for example, if you have a job run multiple times here is how to 
include a number that will increment with each run by using the 
"NumJobStarts" attribute in the job classad so that you will get 
foo.1.out, foo.2.out, etc.

  executable = foo.exe
  # Run every 15 minutes
  cron_minute = 0,15,30,45
  on_exit_remove = false
  # Have stdout file include numjobstarts value
  skip_filechecks = true
  output = foo.$$([ NumJobStarts ]).out
  queue

Similarly, you could use classad functions like formattime() to put the 
time into the file name:

  executable = foo.exe
  # Run every 15 minutes
  cron_minute = 0,15,30,45
  on_exit_remove = false
  # Have stdout file include numjobstarts value
  skip_filechecks = true
  output = foo.$$([formattime(time(),\"%H_%M\")]).out
  queue

Docs for the classad functions appear here: http://is.gd/JjH2h2

Hope the above helps
Todd

--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257