Re: [Condor-users] DAGMan slow submission

["R. Kent Wenger" <wenger@xxxxxxxxxxx>]

On Tue, 9 Feb 2010, Mag Gam wrote:

> I have a process which has 2k parent processes. The reason why I am
> using DAGMan is to throttle the maximum number of jobs.
>
> I would like to have no more than 200 processes running therefore I am using:
>
> condor_submit_dag -maxjobs 100 dagfile
>
>
> I noticed that after every 1 minute it adds more and more jobs. Is it
> possible for DagMan to submit all of my jobs at 1 time?

Hmm -- are you saying that you have less than 100 jobs running?  If so,
something weird is going on.  The normal submit cycle for DAGMan is 5 
seconds, and the normal maximum number of submits per cycle is 5 -- so
if you are running at the default settings, it should take you around 100 
seconds to get 100 jobs into the system (assuming none of them have 
finished in the mean time).

You can tweak the submit cycles with the following configuration values:

  DAGMAN_USER_LOG_SCAN_INTERVAL
  DAGMAN_MAX_SUBMITS_PER_INTERVAL

(Assuming you are using a fairly recent version of DAGMan.)

Oh, yeah -- you don't have DAGMAN_SUBMIT_DELAY set, do you?

If none of the above things seem to be the answer, send me your dagman.out 
file and I'll see what I can figure out.

Kent Wenger
Condor Team