Re: [HTCondor-users] Multiple jobs in Idle state

[Greg Thain <gthain@xxxxxxxxxxx>]

On 11/11/2016 07:15 AM, Ashish Thool wrote:

Hi,

 

My condor version is 8.2.10.

A single machine with 2 cores. There are 4 slots.

 

I’ve added 100 jobs to condor. Each job will run once 15 mins (cron_minute = 0,15,30,45 in job specification file). Each job will run for 10 minutes approximately and rescheduled.

 

Problem is only top 8 jobs (list from condor_q command) are picked up by condor for execution. Remaining jobs are in Idle state forever.

Currently, you are asking Condor to run 100 jobs * 4 times / hour * 10 minutes = 4,000 wallclock minutes per hour on a machine with 240 wallclock minutes per hour.  Something has to give.  You could

1) Tell condor the machine has 20 (virtual) cores, by setting NUM_CPUS = 20, and allow multiple jobs to run at the same time, if they are not cpu, disk or other resource-bound.
2)  Not try to run every job every fifteen minutes, but a-priori pick which jobs run in which hours of the day
3)  I think you might be able to set the job priority to be the _expression_ - EnteredCurrentStatus, which would sort the jobs based on when they last ran, but you'd still miss a lot of windows.


You also want to set cron_window to some higher value.

-greg