[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Limit Number of Parallel Jobs



Sassy Natan <sassyn@xxxxxxxxx> wrote:
> This is off course assuming that User A and User B have the same priority.
> It is understood if a User B have a higher priority, once a job assign
> by User A will be finished ( around 5 hours), User B job will be
> started.
> 
> But what I would like to do is to prevent from user B to wait until
> one of the User A job will finished.

Now, you mentioned a desire to avoid preemption.  Why?  It seems
like "A has 80 jobs running, then 40 are preempted to give to B"
ends up working the same as "A is only allowed to run 40 jobs, so
40 slots are available to B."  Sure, some time is wasted running
jobs for A that don't complete (but will be automatically
restarted), but if B never shows up, A gets to use all of the
slots and finishes faster (potentially giving B all of the slots
later).

Assuming I've convinced you that preemption is a good idea:

A user's priority is a combination of their priority factor (the
number you set with condor_userprio, it defaults to 1) and the
real priority. The real priority is a number that roughly
correlates with the number of slots a user is using; you can
control the speed at which it approaches the number of slots of
PRIORITY_HALFLIFE.  By default PRIORITY_HALFLIFE is 86400 seconds
(1 day).

Condor will preempt a running to job to give it to a user with
higher priority.  PREEMPTION_REQUIREMENTS is used to determine
which running jobs are eligible for preemption.  The default will
only preempt jobs that have run for at least 1 hour, and when the
running job's user's priority is at least 20% worse than that of
the higher priority user's job.

If you want want more aggressive preemption, you might make
PREEMPTION_REQUIREMENTS more willing to preempt, and you might
make PRIORITY_HALFLIFE smaller.  Maybe something like:

# Preempt any job for a 20% higher priority user.
UWCS_PREEMPTION_REQUIREMENTS = \
    (RemoteUserPrio > TARGET.SubmitterUserPrio * 1.2) || (MY.NiceUser == True)
# 30 minutes
PRIORITY_HALF_LIFE= 1800

-- 
Alan De Smet                              Condor Project Research
adesmet@xxxxxxxxxxx                http://www.cs.wisc.edu/condor/