Re: [Condor-users] staggering the start of jobs

[Matthew Farrellee <matt@xxxxxxxxxx>]

On 01/26/2010 02:33 PM, Joe Boyd wrote:
> We submit all our jobs using dagmans.  We have some jobs which will
> overwhelm our database and sometimes our data handling systems.  We'd
> like to be able to stagger the start of those jobs so they don't hammer
> the database or data handling but let other jobs that don't exhibit this
> behavior start as quickly as possible.  We know which jobs are the
> problem beforehand of course.
> 
> I know we can use DAGMAN_MAX_JOBS_IDLE to set a max on how many real
> slots can start up per negotiation cycle per dagman job.  I'd like to
> also be able to limit the number of these jobs that start up
> simultaneously on a batch system wide scale.  So, one user might submit
> multiple dagmans each with these type of jobs that hammer the database. 
> Is there any mechanisms built into condor to limit the start rate of
> these jobs across multiple dagman jobs and even multiple users?  Is
> there a way to have the schedd throttle how fast it starts jobs that
> have some attribute set in the classads???
> 
> Thanks for any ideas,
> 
> joe

You can control how many run at any given time.

http://www.cs.wisc.edu/condor/manual/v7.4/3_13Setting_Up.html#37691

Best,


matt