Re: [HTCondor-users] Negotiation with partitionable slots

[Mathieu Bahin <mathieu.bahin@xxxxxxxxxxxxxxx>]

I can read in the manual that "This differs from scheduler matchmaking
in that multiple jobs can match with the partitionable slot during a
single negotiation cycle." (cf
http://research.cs.wisc.edu/htcondor//manual/v8.2.7/3_5Policy_Configuration.html#SECTION004510900000000000000).

But I don't really know how. Is it the regular behaviour with
partitionable slots? Because that's not what I note here in my tests...
I'm not sure to perfectly understand this section in the manual. Is
there something to do with the "CONSUMPTION_POLICY" to solve my issue?
Currently its value is False for us.

Cheers,
Mathieu

On 27/04/17 19:00, htcondor-users-request@xxxxxxxxxxx wrote:
> Send HTCondor-users mailing list submissions to
> 	htcondor-users@xxxxxxxxxxx
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> or, via email, send a message with subject or body 'help' to
> 	htcondor-users-request@xxxxxxxxxxx
>
> You can reach the person managing the list at
> 	htcondor-users-owner@xxxxxxxxxxx
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of HTCondor-users digest..."
>
>
> Today's Topics:
>
>    1. Negotiation with partitionable slots (Mathieu Bahin)
>    2. Re: Negotiation with partitionable slots (Steffen Grunewald)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 27 Apr 2017 09:23:16 +0200
> From: Mathieu Bahin <mathieu.bahin@xxxxxxxxxxxxxxx>
> To: htcondor-users@xxxxxxxxxxx
> Subject: [HTCondor-users] Negotiation with partitionable slots
> Message-ID: <59019C64.1000805@xxxxxxxxxxxxxxx>
> Content-Type: text/plain; charset=utf-8
>
> Hi,
>
> Our cluster, configured with partitionable slots and having very
> heterogeneous nodes, is being more loaded for a few months and we are
> facing a new issue: it's complicated for a user to run a job requesting
> big CPUs and/or memory.
>
> Ideally, we would have liked to have the smallest machines first filled
> up to ~80% (not to overload them when there is still space anywhere
> else) in order to leave free a few of the biggest machines when possible.
>
> We designed the NEGOCIATOR_JOB_POST_RANK value to something like that:
> (RemoteOwner =!= UNDEFINED) * ((floor(TotalCpus / 10) * -10000000) +
> KFlops - 1.0e10 * (Offline =?= True))
> Our nodes are then divided into 4 classes (our biggest node has 32
> CPUs), the one having less than 10 CPUs are now prioritary and within
> classes, the most powerful nodes comes first (with Kflops).
>
> Though, if we run a few dozens of jobs, from what I noted, the priority
> order is satisfying but only one job is allocated to the nodes of the
> more priority classes (I guess because with the partitionable slots,
> nodes only advertise one big free slot for one negotiation cycle) and
> the next ones are allocated, one by one also, to the less priority nodes.
> How can we do for several jobs really filling the highest priority nodes
> before considering the others?
>
> Cheers,
> Mathieu
>

-- 
---------------------------------------------------------------------------------------
| Mathieu Bahin
| IE CNRS
|
| Institut de Biologie de l'Ecole Normale Supérieure (IBENS)
| Biocomp team
| 46 rue d'Ulm
| 75230 PARIS CEDEX 05
| 01.44.32.23.56
---------------------------------------------------------------------------------------