[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Multiple jobs in Idle state

On 11/11/2016 07:15 AM, Ashish Thool wrote:



My condor version is 8.2.10.

A single machine with 2 cores. There are 4 slots.


I’ve added 100 jobs to condor. Each job will run once 15 mins (cron_minute = 0,15,30,45 in job specification file). Each job will run for 10 minutes approximately and rescheduled.


Problem is only top 8 jobs (list from condor_q command) are picked up by condor for execution. Remaining jobs are in Idle state forever.

Currently, you are asking Condor to run 100 jobs * 4 times / hour * 10 minutes = 4,000 wallclock minutes per hour on a machine with 240 wallclock minutes per hour.  Something has to give.  You could

1) Tell condor the machine has 20 (virtual) cores, by setting NUM_CPUS = 20, and allow multiple jobs to run at the same time, if they are not cpu, disk or other resource-bound.
2)  Not try to run every job every fifteen minutes, but a-priori pick which jobs run in which hours of the day
3)  I think you might be able to set the job priority to be the _expression_ - EnteredCurrentStatus, which would sort the jobs based on when they last ran, but you'd still miss a lot of windows.

You also want to set cron_window to some higher value.