Re: [Condor-users] Condor priority model

[matthew hope <matthew.hope@xxxxxxxxx>]

On Wed, 15 Sep 2004 09:22:30 +0200, Steffen Grunewald
<steffen.grunewald@xxxxxxxxxx> wrote:
>
> On Wed, Sep 15, 2004 at 07:36:50AM +0100, matthew hope wrote:
> > The scheduling logic is a little opaque.
> 
> nicely said :-/

in fairness it's an NP(complete?) task to do perfectly 

> > If your job latency is important remember that condor is geared up for
> > high throughput not low latency...
> 
> In plain English: one may have to wait some time to get machine access,
> but once one has it (CLAIMED the VMs), the whole cluster will be pushed
> through, right? (Reminds me of that classical queue situation where the
> old lady asks to pass by since she only needs one little thing, and then
> remembers 1000s of others she needs too...)
> 
> Which means that high priority users (low prio factors) may have to wait
> for ages if there's a low prio user with 30000 jobs who just took the
> chance when the whole pool was idle. Or to setup preempting (which can
> be a pain with hundreds of VMs running long - more than a week or so -
> jobs). Right?
> Or the low prio user has to go and condor_hold her still idle jobs.

Without preemption basically yes.

an alternate is to run your own batch every so often that frees up the
recently started jobs with condor_vacate if it spots 'better' jobs
sitting idle for too long.

lot of hassle to maintain though so I'd think of other things first.
 
> Would be nice if there was an "-idleonly" option to condor_hold. Perhaps
> a suggestion for the next release? (In some cases it would also make
> sense to have a count limit to condor_release so only say 500 of 30000
> jobs would be released at one time...) (Of course it can be done using
> some long one-liner shell script, but most of our Condor users are
> users, not geeks.)

that's actually reasonably easy with 

condor_hold -constraint "JobStatus==1"

a condor release is also possible

condor_release "ClusterId==X && ProcId>=A && ProcId<B"

obviously with a shell script you could determine X, A and B
dynamically but that will do the job if your users are semi with it

think older versions may not have the -constraint options though.

Take a look at the retirement options and consider enabling preemption...