[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Filling pool breadth-first with partitionable slots (Beyer, Christoph)

I like depth-first owing as it is an anti-fragmentation tool, but breadth-first is, in my experience, better for input file transfers in the real-world because my storage can easily swamps the 1GbE network interface on an N-core machine run N jobs.

Tom Downes
Senior Scientist and Data CenterÂManager
Center for Gravitation, Cosmology and Astrophysics
University of Wisconsin-Milwaukee

On Mon, Oct 31, 2016 at 11:09 AM, Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx> wrote:
I whipped this up a couple of years ago:

DynamicSlotID = ifThenElse(DynamicSlot, regexps("slot\d+_(\d+).*", Name, "\1", "i"), UNDEFINED)

However I don't think this will help much with ranking.

There's also a NumDynamicSlots attribute which may be what you're looking for - you'd rank the partitionable slots with the lowest number of dynamic slots highest. I'm not sure which version that attribute appeared in, though.

Personally, I prefer depth-first fill because the Linux buffer cache coupled with a 12-hour automount timeout helps limit the amount of NFS traffic generated by jobs starting up, as most of our jobs pull inputs from NFS filesystems rather than an input transfer.

    -Michael Pelletier.

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Jason Liu
Sent: Monday, October 31, 2016 3:39 AM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] Filling pool breadth-first with partitionable slots (Beyer, Christoph)

Hi Christoph,

Thank you for your email.

While the solution you suggested works for static slots, it doesnât work for p-slots. SlotID is always 1 for p-slots since p-slot names are slot1_1, slot1_2 etc instead of slot1, slot2.

Unless I am missing something?

Kind Regards

HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at: