Re: [HTCondor-users] Best Practice method to limit number of slots on a machine

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi Lyle,

to be on the safe side, you can try the int() function [1]

Cheers,
  Thomas

[1]
https://htcondor.readthedocs.io/en/latest/man-pages/classads.html?highlight=int()#term-integer-INT-expr

On 09/09/2022 09.03, Lyle Pakula wrote:
> Hi Christoph,
>
> Yes the issue is the number of CPU's varies across the cluster so that, 
> using a hard cpu limit, would tend to saturate the weaker machines while 
> leaving the powerful machines underutilised.
>
> Indeed ROUND would solve that problemÂ- i'll give it a try and see what 
> condor does!
>
> Thanks Lyle
>
>
>
> On Fri, Sep 9, 2022 at 3:47 PM Beyer, Christoph <christoph.beyer@xxxxxxx 
> <mailto:christoph.beyer@xxxxxxx>> wrote:
>
>     Hi Lyle,
>
>     you can limit the number of CPUs that are used to create slots
>     (that's if you use partitionable slots) using:
>
>     NUM_CPUS -> lie about the number of detected cpus
>     MAX_NUM_CPUS -> limit the max number of cpus
>
>     The numbers should be integer - hence I am not sure if something
>     like '0,75 *Â $(DETECTED_CPUS)' will work - you have to try it I
>     guess ;)
>
>     If you have static slots it is very easy, just create less slots ...
>
>     Best
>     christoph
>
>
>     -- 
>     Christoph Beyer
>     DESY Hamburg
>     IT-Department
>
>     Notkestr. 85
>     Building 02b, Room 009
>     22607 Hamburg
>
>     phone:+49-(0)40-8998-2317
>     mail: christoph.beyer@xxxxxxx <mailto:christoph.beyer@xxxxxxx>
>
>     ------------------------------------------------------------------------
>     *Von: *"Lyle Pakula" <Lyle@xxxxxxxxxxxxxxxx
>     <mailto:Lyle@xxxxxxxxxxxxxxxx>>
>     *An: *"HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx
>     <mailto:htcondor-users@xxxxxxxxxxx>>
>     *Gesendet: *Freitag, 9. September 2022 00:43:01
>     *Betreff: *[HTCondor-users] Best Practice method to limit number of
>     slots on aÂÂÂÂÂÂÂÂmachine
>
>     Hi Everyone,
>     I'm wondering what people's best practice is to easily limit the
>     number of jobs running on a machine as not to saturate that machine.
>
>     Our jobs are heavy I/O rather than compute. So when a job is
>     submitted we try to spread it across the cluster using the below rank
>
>     # Define Load Balancing on the AE Pool
>     NEGOTIATOR_PRE_JOB_RANK=( \$(NEGOTIATOR_PRE_JOB_RANK) ) * SlotID
>
>     But big jobs will still saturate. So what's the best way to limit
>     the number of slots use on a machine, noting our machines vary from
>     4-8 cores. Ideally we would limit the number of slots ot sayÂ75% of
>     what the max is. Below is a bit heavy handed.
>
>     # DON'T COUNT HYPERTHREADED CPUS AS THIS LEADS TO SATURATION
>     COUNT_HYPERTHREAD_CPUS=FALSE
>
>     Thanks, Lyle
>
>     -- 
>     AE CAPITAL
>     15 William St, Level 19, Melbourne VIC, Australia
>
>     p +61 3 9020 7801
>     m +61 (0)434 872 054
>     w http://www.aecapital.com.au <http://www.aecapital.com.au>
>
>
>     AE Capital Pty Limited (ACN 153 242 865) is regulated by the
>     Australian Securities & Investments Commission and is a Corporate
>     Authorised Representative of JFM Pty Limited (ACN 125 150 656),
>     holder of an Australian Financial Services Licence (AFSL 314585). 
>     AE Capital Pty Limited is a member of the National Futures
>     Association (ID 0498660).
>
>     _______________________________________________
>     HTCondor-users mailing list
>     To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>     <mailto:htcondor-users-request@xxxxxxxxxxx> with a
>     subject: Unsubscribe
>     You can also unsubscribe by visiting
>     https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>     <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>
>
>     The archives can be found at:
>     https://lists.cs.wisc.edu/archive/htcondor-users/
>     <https://lists.cs.wisc.edu/archive/htcondor-users/>
>     _______________________________________________
>     HTCondor-users mailing list
>     To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>     <mailto:htcondor-users-request@xxxxxxxxxxx> with a
>     subject: Unsubscribe
>     You can also unsubscribe by visiting
>     https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>     <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>
>
>     The archives can be found at:
>     https://lists.cs.wisc.edu/archive/htcondor-users/
>     <https://lists.cs.wisc.edu/archive/htcondor-users/>
>
>
>
> --
> AE CAPITAL
> 15 William St, Level 19, Melbourne VIC, Australia
>
> p +61 3 9020 7801
> m +61 (0)434 872 054
> w http://www.aecapital.com.au <http://www.aecapital.com.au>
>
>
> AE Capital Pty Limited (ACN 153 242 865) is regulated by the Australian 
> Securities & Investments Commission and is a Corporate Authorised 
> Representative of JFM Pty Limited (ACN 125 150 656), holder of an 
> Australian Financial Services Licence (AFSL 314585). AE Capital Pty 
> Limited is a member of the National Futures Association (ID 0498660).
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature