Re: [HTCondor-users] prefere one type of jobs on the negotiator

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

Hey Todd,

By "reservation" are you talking about something along the lines of having various slot types, or something else?

Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Todd Tannenbaum
Sent: Monday, August 12, 2019 8:29 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [External] Re: [HTCondor-users] prefere one type of jobs on the negotiator

> On Aug 12, 2019, at 6:47 AM, Beyer, Christoph <christoph.beyer@xxxxxxx> wrote:
> 
> 
> Hi,
> 
> I am trying to set up jupyter notebooks on htcondor, so far it is running but on a busy pool the spawned condor jobs from the jupyterhub take too long to start up. 
> 
> I tried to convince the negotiator to consider these jupyter jobs using: 
> 
> GROUP_SORT_EXPR = ifThenElse(DESYAcctGroup=?="BIRD_jupyter",3.2e+38,ifThenElse(GroupQuota>0,GroupResourcesInUse/GroupQuota,3.3e+38))
> 
> From my understanding the negotioator should consider the "BIRD_jupyter" jobs first but that does not seem to be the case or maybe it is but it's still not sufficient to get my jobs started in the time limit. 

Even if the negotiator is considering your Jupyter jobs first, that will only help if there are some resources eligible for matching. Are there? You can look in the negotiator log for the start of a preemption cycle and confirm that Jupyter group jobs are being considered first, and also see why no matches are being made.  If your pool is so busy that there is nothing left for Jupyter, consider a scheduling policy with either preemption and/or reservation. Or perhaps reduce the size of the resource request that your Jupyter sessions are requesting (e.g. your pool may be able to accommodate a lot of extra 1core/512MB slots but zero 1core/3GB slots....). 

Regards
Todd