[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [Condor-users] about priorities



assumption - jobs are submitted in time order with sufficient gap between them to account for time differences between submit machines...

place 
RANK=-TARGET.QDate
on all execute machines.

assumption - the jobs are all submitted from one cluster..
place 
RANK=-TARGET.ProcId
on all execute machines.
If you wish to have multiple sets of these ordered jobs at once make sure you either include the cluster id as so (works for jobs up to 100000000 processes big!)
RANK= (-TARGET.ClusterId * 100000000) - TARGET.ProcId


assumption you can control the submission process such that every job is given a unique monotonic increasing/decreasing number (easy if all done from one cluster but still possible otherwise)
use for example
+MY_ORDER_NUMBER=[x]
in the submit file and ensure this ticks on every time

increasing
RANK=-TARGET.MY_ORDER_NUMBER

decreasing
RANK=TARGET.MY_ORDER_NUMBER

if you have to play nicely with other machines this rank expression must be modified to correctly handle jobs which don't have this attribute (using =!= and/or =?=)

Igf you haven't got control of the execute machines rank you'll have some issues...

also you will have to deal with the possibility that:

jobs 1-10 start
job 5 is vacated before finishing for some reasona at roughly the same time job 11 starts.
job 11 then completes before the negotiation cycle can swap 6 for it. job 11 has now run before 6 and nothing can change that.

This may not be a problem but this was not clear from your request.

all sorts of options available really.

> -----Original Message-----
> From: condor-users-bounces@xxxxxxxxxxx
> [mailto:condor-users-bounces@xxxxxxxxxxx]On Behalf Of
> Robert.Nordlund@xxxxxxxxxxxxxxxx
> Sent: 05 August 2004 14:51
> To: Condor-Users Mail List
> Subject: Re: [Condor-users] about priorities
> 
> 
> 
> 
> 
> 
> Hello all.
> 
> I am running a dedicated pool of Windows 2000 servers with one central
> submit machine.  I submit 20 jobs with multiple processes and 
> want to make
> sure that they are run in succession, i.e. Job1 is more important than
> Job2, etc.  If the first 10 jobs are concurrently running how 
> can I make
> sure that if Job1 fails due to a hardware failure that it 
> will preempt the
> the claim of the lowest priority job?  Is this something 
> fairly easy to
> accomplish with priorities/preemption?
> 
> Thank you,
> 
> Bob Nordlund
> 
> 
> 
> |---------+-------------------------------->
> |         |           Erik Paulson         |
> |         |           <epaulson@xxxxxxxxxxx|
> |         |           >                    |
> |         |           Sent by:             |
> |         |           condor-users-bounces@|
> |         |           cs.wisc.edu          |
> |         |                                |
> |         |                                |
> |         |           08/04/2004 10:43 PM  |
> |         |           Please respond to    |
> |         |           Condor-Users Mail    |
> |         |           List                 |
> |         |                                |
> |---------+-------------------------------->
>   
> >-------------------------------------------------------------
> -------------------------------------------------|
>   |                                                           
>                                                    |
>   |       To:       Condor-Users Mail List 
> <condor-users@xxxxxxxxxxx>                                    
>         |
>   |       cc:                                                 
>                                                    |
>   |       Subject:  Re: [Condor-users] about priorities       
>                                                    |
>   
> >-------------------------------------------------------------
> -------------------------------------------------|
> 
> 
> 
> 
> On Wed, Aug 04, 2004 at 06:16:45PM -0500, David A. Kotz wrote:
> > I think what you want to do is to set the following in your
> > condor_config:
> >
> > SUSPEND = FALSE
> > PREEMPT = FALSE
> > KILL = FALSE
> >
> 
> That will help prevent the local machine from evicting the 
> job (you'll also
> probably want have START = TRUE, or some such)
> 
> Those expresssions have nothing to do with preemption for priority
> purposes, though. Condor can "preempt" a job for two reasons - one,
> because the local conditions on the machine force us to get 
> rid of a job,
> or two, because there is a user with a higher priority who 
> wants to run a
> job. That preemption is controlled by PREEMPTION_REQUIREMENTS. It is a
> pool-wide setting that is evaluated only when Condor has a higher
> priority user who wants to run a job. It is tested against 
> each machine
> in the pool to see if the job currently running on that machine
> can be preempted.
> 
> To prevent priority preemption, set:
> 
> PREEMPTION_REQUIREMENTS = FALSE
> 
> in the config file of your central manager.
> 
> -Erik
> 
> each machine of the pool.
> > - dave
> >
> >
> > On Wed, 2004-08-04 at 03:44 -0700, Fernando Rannou wrote:
> > > Hello,
> > > I'm a little confused on how Condor handles priorities.
> > > On Section 2.7.2 of the Condor manual it reads:
> > > ---------
> > > "The priority of each individual user changes according 
> to the number
> > > of resources the individual is using.
> > > ...
> > >
> > > "For instance, if a low priority user is utilizing all available
> machines
> > > and suddenly a higher priority user submits jobs, Condor
> > > will immediately checkpoint and vacate jobs belonging to
> > > the lower priority user."
> > >
> > > ---------
> > > Which priority do they refer to? Effective, Real, Factor??
> > >
> > > We need that once a job is running it never, ever
> > > gets preempted, suspended, killed, or whatever
> > > no matter how long it's been running and
> > > no matter how low its priority is.
> > >
> > > How can I accomplished that?
> > > Thanks
> > >
> > > Fernando
> > >
> > >
> > > ----------------------------------------------------------
> > > IMPORTANT WARNING:  This email (and any attachments) is 
> only intended
> for the use of the person or entity to which it is addressed, and may
> contain information that is privileged and confidential.  You, the
> recipient, are obligated to maintain it in a safe, secure and 
> confidential
> manner.  Unauthorized redisclosure or failure to maintain 
> confidentiality
> may subject you to federal and state penalties. If you are not the
> recipient, please immediately notify us by return email, and 
> delete this
> message from your computer.
> > > ----------------------------------------------------------
> > > _______________________________________________
> > > Condor-users mailing list
> > > Condor-users@xxxxxxxxxxx
> > > http://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > --
> > David A. Kotz <dkotz@xxxxxxxxxxxxx>
> >
> > _______________________________________________
> > Condor-users mailing list
> > Condor-users@xxxxxxxxxxx
> > http://lists.cs.wisc.edu/mailman/listinfo/condor-users
> _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx
> http://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> 
> 
> 
> 
> **************************************************************
> ***********
> PRIVILEGED AND CONFIDENTIAL: This communication, including 
> attachments, is for the exclusive use of addressee and may 
> contain proprietary, confidential and/or privileged 
> information.  If you are not the intended recipient, any use, 
> copying, disclosure, dissemination or distribution is 
> strictly prohibited.  If you are not the intended recipient, 
> please notify the sender immediately by return e-mail, delete 
> this communication and destroy all copies.
> **************************************************************
> ***********
> 
> _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx
> http://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 


*****************************************************************
Gloucester Research Limited believes the information 
provided herein is reliable. While every care has been 
taken to ensure accuracy, the information is furnished 
to the recipients with no warranty as to the completeness 
and accuracy of its contents and on condition that any 
errors or omissions shall not be made the basis for any 
claim, demand or cause for action.
*****************************************************************