[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Filling pool breadth-first with partitionable slots (Beyer, Christoph)



Hi Christoph,


Thank you for your email.

While the solution you suggested works for static slots, it doesnât work for p-slots. SlotID is always 1 for p-slots since p-slot names are slot1_1, slot1_2 etc instead of slot1, slot2.

Unless I am missing something?

Kind Regards
Jason




>
>Message: 3
>Date: Mon, 31 Oct 2016 07:24:00 +0100 (CET)
>From: "Beyer, Christoph" <christoph.beyer@xxxxxxx>
>To: htcondor-users <htcondor-users@xxxxxxxxxxx>
>Subject: Re: [HTCondor-users] Filling pool breadth-first with
>partitionable slots
>Message-ID: <101471472.19853023.1477895040409.JavaMail.zimbra@xxxxxxx>
>Content-Type: text/plain; charset="us-ascii"
>
>Hi Jason,
>
>you can use an expression like:
>
>NEGOTIATOR_POST_JOB_RANK = (RemoteOwner =?= UNDEFINED) * SlotID
>
>on your negotiator to resort the slots after the match, basically it means 'if RemoteOwner =?= UNDEFINED' equals to 1 if the slot has no RemoteOwner (in other words slot is free).
>
>For the next part, in the slot ranking the higher is the better so the higher the slot ID the higher the ranking which means if you run 8 core machines for ex. slot8 on all machines gets a job, then slot 7 on all machines and so on ...
>
>You can put some more logic in the sort statement of course ther might be examples in the manual or on the list for that ...
>
>best regards
>~christoph
>
>
>--
>/* Christoph Beyer | Office: Building 2b / 23 *
>* DESY | Phone: 040-8998-2317 *
>* - IT - | Fax: 040-8994-2317 *
>\* 22603 Hamburg | http://www.desy.de */
>
>
>Von: "Jason Liu" <jason.liu@xxxxxxxxxxxxx>
>An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
>Gesendet: Montag, 31. Oktober 2016 05:27:07
>Betreff: [HTCondor-users] Filling pool breadth-first with partitionable slots
>
>Hi,
>
>We have a need to fill our HTCondor pool breadth-first to improve IO efficiency.
>
>We are using p-slots in our pool, since our jobs come in all sorts of shapes and sizes.
>
>I am a bit lost as to how to configure a pool to fill breadth-first. By default it seems HTCondor fills depth-first.
>
>After some research, I have found this article:
>
>https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToFillPoolBreadthFirst
>
>which shows a way to fill the pool breadth-first. However the solution presented has one major drawback for p-slots, which is that each machine can only match one new job per negotiation cycle.
>
>We are looking for a way to fill our pool breadth first, without the limitation of only being able to match one job per negotiation cycle per machine. Does HTCondor have a way to overcome this?
>
>We are using HTCondor 8.4.7. In addition, we are also using concurrency limits and consumption policy is enabled on all machines in the pool.
>
>Any help would be greatly appreciated.
>
>Thanks.
>
>Kind Regards,
>Jason
PRIVACY AND CONFIDENTIALITY NOTICE
The information contained in this message is intended for the named recipients only. It may contain confidential information and if you are not the intended recipient, you must not copy, distribute or take any action in reliance on it. If you have received this message in error please destroy it and reply to the sender immediately or contact us at the above telephone number.
VIRUS DISCLAIMER
While we take every precaution against presence of computer viruses on our system, we accept no responsibility for loss or damage arising from the transmission of viruses to e-mail recipients.