[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Condor DAG spinning

On Wed, 28 Oct 2009, Hoover Sam wrote:

You can also increase your ulimit on your server, or limit the number
of DAGMAN jobs. I do both on my system. I increased the ulimit and set

DAGMAN_MAX_JOBS_IDLE = # of cores in condor cluster
DAGMAN_MAX_JOBS_SUBMITTED = 2 x # of cores in condor cluster

This keeps the queue to a reasonable size and limits the number of
open file descriptors.

Good point about ulimit -- I should have mentioned that.

One note, though -- on DAGMan versions prior to 7.3.2, setting DAGMAN_MAX_JOBS_IDLE and DAGMAN_MAX_JOBS_SUBMITTED won't have any effect on the number of file descriptors a given condor_dagman process consumes -- the log files all get opened at the beginning of the run, and closed at the end, no matter how many, or few, node jobs are in the queue at any given time.

But you may want to use DAGMAN_MAX_JOBS_IDLE and DAGMAN_MAX_JOBS_SUBMITTED for reasons other than the file descriptor limits.

Kent Wenger
Condor Team