[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] staggering the start of jobs

We submit all our jobs using dagmans. We have some jobs which will overwhelm our database and sometimes our data handling systems. We'd like to be able to stagger the start of those jobs so they don't hammer the database or data handling but let other jobs that don't exhibit this behavior start as quickly as possible. We know which jobs are the problem beforehand of course.

I know we can use DAGMAN_MAX_JOBS_IDLE to set a max on how many real slots can start up per negotiation cycle per dagman job. I'd like to also be able to limit the number of these jobs that start up simultaneously on a batch system wide scale. So, one user might submit multiple dagmans each with these type of jobs that hammer the database. Is there any mechanisms built into condor to limit the start rate of these jobs across multiple dagman jobs and even multiple users? Is there a way to have the schedd throttle how fast it starts jobs that have some attribute set in the classads???

Thanks for any ideas,