Re: [Condor-users] faster condor_submits with dagman

[Carsten Aulbert <carsten.aulbert@xxxxxxxxxx>]

Hi Peter

On Wednesday 30 June 2010 19:13:16 Peter Doherty wrote:
> SCHEDD_INTERVAL    = 30
> DAGMAN_MAX_JOBS_IDLE = 1000
> DAGMAN_SUBMIT_DELAY = 0
> DAGMAN_MAX_SUBMITS_PER_INTERVAL = 1000
> DAGMAN_USER_LOG_SCAN_INTERVAL = 1
> SCHEDD_INTERVAL_TIMESLICE = 0.10
> SUBMIT_SKIP_FILECHECKS = True
> HISTORY =
> NEGOTIATOR_INTERVAL = 30
> NEGOTIATOR_MAX_TIME_PER_SUBMITTER=20
> NEGOTIATOR_MAX_TIME_PER_PIESPIN=20

In case it's not the schedd or dagman but the Negotiator which is the 
bottleneck:

Have you looked into the CLAIM_WORKLIFE variable?

This has helped us a lot for very short jobs, currently it is set to 1 hour, 
i.e. 3600s thus and a negotiated slot won't be renegotiated then for one hour 
if the user has idle jobs in the queue.

Can you try if this helps? 

Cheers

Carsten

PS: this setting needs to be set on the worker node IIRC.