Re: [HTCondor-users] prefere one type of jobs on the negotiator

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi Todd,

there are actually 4 matching slots that are free and available but to me it looks like my jupyter job is not even considered by the matchmaker (?)

We do run an individual scheduler for these jobs and I hoped that might spedd up things too as there are very few jobs on this scheduler. 

I attached a negotiator-log, BIRD_jupyter.lite is the group in question ... 

Maybe you can tell me if the sort expression does work out and what I can do to make things more speedy please ? 

Best
Christoph

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "Todd Tannenbaum" <tannenba@xxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Montag, 12. August 2019 14:28:50
Betreff: Re: [HTCondor-users] prefere one type of jobs on the negotiator

> On Aug 12, 2019, at 6:47 AM, Beyer, Christoph <christoph.beyer@xxxxxxx> wrote:
> 
> 
> Hi,
> 
> I am trying to set up jupyter notebooks on htcondor, so far it is running but on a busy pool the spawned condor jobs from the jupyterhub take too long to start up. 
> 
> I tried to convince the negotiator to consider these jupyter jobs using: 
> 
> GROUP_SORT_EXPR = ifThenElse(DESYAcctGroup=?="BIRD_jupyter",3.2e+38,ifThenElse(GroupQuota>0,GroupResourcesInUse/GroupQuota,3.3e+38))
> 
> From my understanding the negotioator should consider the "BIRD_jupyter" jobs first but that does not seem to be the case or maybe it is but it's still not sufficient to get my jobs started in the time limit. 

Even if the negotiator is considering your Jupyter jobs first, that will only help if there are some resources eligible for matching. Are there? You can look in the negotiator log for the start of a preemption cycle and confirm that Jupyter group jobs are being considered first, and also see why no matches are being made.  If your pool is so busy that there is nothing left for Jupyter, consider a scheduling policy with either preemption and/or reservation. Or perhaps reduce the size of the resource request that your Jupyter sessions are requesting (e.g. your pool may be able to accommodate a lot of extra 1core/512MB slots but zero 1core/3GB slots....). 

Regards
Todd



> 
> Are there any ways to start a job fast ? 
> 
> I know about condor_now and it is a cool thing but it would raise some more questions if I would use it, hence I would like to know what else I can do to speed things up ??? 
> 
> Best
> Christoph
> 
> -- 
> Christoph Beyer
> DESY Hamburg
> IT-Department
> 
> Notkestr. 85
> Building 02b, Room 009
> 22607 Hamburg
> 
> phone:+49-(0)40-8998-2317
> mail: christoph.beyer@xxxxxxx
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: negotiator.log.gz
Description: GNU Zip compressed data