[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Filling pool breadth-first with partitionable slots



Hi Jason,

you can use an _expression_ like:

NEGOTIATOR_POST_JOB_RANK = (RemoteOwner =?= UNDEFINED) * SlotID

on your negotiator to resort the slots after the match, basically it means 'if RemoteOwner =?= UNDEFINED' equals to 1 if the slot has no RemoteOwner (in other words slot is free).

For the next part, in the slot ranking the higher is the better so the higher the slot ID the higher the ranking which means if you run 8 core machines for ex. slot8 on all machines gets a job, then slot 7 on all machines and so on ...

You can put some more logic in the sort statement of course ther might be examples in the manual or on the list for that ...

best regards
        ~christoph


--
/*   Christoph Beyer     |   Office: Building 2b / 23     *
 *   DESY                |    Phone: 040-8998-2317        *
 *   - IT -              |      Fax: 040-8994-2317        *
\*   22603 Hamburg       |     http://www.desy.de         */


Von: "Jason Liu" <jason.liu@xxxxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Montag, 31. Oktober 2016 05:27:07
Betreff: [HTCondor-users] Filling pool breadth-first with partitionable slots

Hi,

We have a need to fill our HTCondor pool breadth-first to improve IO efficiency.

We are using p-slots in our pool, since our jobs come in all sorts of shapes and sizes.

I am a bit lost as to how to configure a pool to fill breadth-first. By default it seems HTCondor fills depth-first. 

After some research, I have found this article:

https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToFillPoolBreadthFirst

which shows a way to fill the pool breadth-first. However the solution presented has one major drawback for p-slots, which is that each machine can only match one new job per negotiation cycle. 

We are looking for a way to fill our pool breadth first, without the limitation of only being able to match one job per negotiation cycle per machine. Does HTCondor have a way to overcome this?

We are using HTCondor 8.4.7. In addition, we are also using concurrency limits and consumption policy is enabled on all machines in the pool. 

Any help would be greatly appreciated.

Thanks.

Kind Regards,
Jason


PRIVACY AND CONFIDENTIALITY NOTICE
The information contained in this message is intended for the named recipients only. It may contain confidential information and if you are not the intended recipient, you must not copy, distribute or take any action in reliance on it. If you have received this message in error please destroy it and reply to the sender immediately or contact us at the above telephone number.
VIRUS DISCLAIMER
While we take every precaution against presence of computer viruses on our system, we accept no responsibility for loss or damage arising from the transmission of viruses to e-mail recipients.


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/