[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] maxidle for a dag with one node?

On Tue, 27 Sep 2011, Rob de Graaf wrote:

I was using the job log files to find completed jobs and do some post processing on their output, stitching them together to save inodes. For a quick fix I used dagman.out instead, but either way would work. Since I now have a DAG with many single-process jobs anyway, I could simply insert scheduler universe jobs every N nodes, problem solved.

One note on this -- for your postprocessing, I'd suggest using either a node status file (http://www.cs.wisc.edu/condor/manual/v7.7/2_10DAGMan_Applications.html#SECTION0031010000000000000000) or jobstate.log file (http://www.cs.wisc.edu/condor/manual/v7.7/2_10DAGMan_Applications.html#SECTION0031011000000000000000) instead of the dagman.out file. Both the node status file and the jobstate.log file have much simpler formats than the dagman.out file, and the formats are less likely to change with in any future versions of DAGMan. They are specifically designed to be easily machine-parseable, while the dagman.out file is not.

Note that neither the node status file nor the jobstate.log file is created by default -- you have to specify them in your DAG for them to get created.

Kent Wenger
Condor Team