[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Help Excluding node from preemption



Ok, so after bashing my head against this a little more with more debugging, I think I figured it out. Will put it here for others to find as well as to see if anyone sees a problem with this approach:

 

JobHost = (splitSlotName(Machine))[1]

PREEMPTION_REQUIREMENTS = debug($(StateTimer) > (2 * $(DAY)) && ($(JobHost) != "specialNode.illinois.edu"))

 

Thanks,

 

-------------------------------------

Gianni Pezzarossi

Computational System Analyst

Research Services

Engineering IT Shared Services

University of Illinois @ Urbana-Champaign

 

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Pezzarossi, Gianni
Sent: Friday, August 19, 2022 2:51 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Help Excluding node from preemption

 

Hey all,

 

So, I am struggling to figure out what classad I need to look at for this, or if it is even possible.

So this is in htcondor 8.8, and I have a preemption configuration like this (copied from one of the UW-Madison presentations iirc)

 

StateTimer = (time() - EnteredCurrentState)

DAY = 86400

PREEMPTION_REQUIREMENTS = $(StateTimer) > (2 * $(DAY)) && RemoteUserPrio > SubmittorPrio * 1.2

 

And the research group wants to set aside a single node for some special long-running jobs that shouldn’t be interrupted. So the first thing I did for the node was change the START config to only allow jobs from the two users that will be running these jobs to start on the node. However, I also wanted to avoid any preemption from happening between these two users. The quick solution I came up with was:

 

PREEMPTION_REQUIREMENTS = $(StateTimer) > (2 * $(DAY)) && RemoteUserPrio > SubmittorPrio * 1.2 && ((RemoteUser != "specialUser1") || (RemoteUser != "specialUser2"))

 

Which seems to work, except that means their jobs can’t be preempted cluster-wide. So I was thinking if there was a way to instead “disable” preemption on just the one special node, something like:

 

PREEMPTION_REQUIREMENTS = $(StateTimer) > (2 * $(DAY)) && RemoteUserPrio > SubmittorPrio * 1.2 && (RemoteHost != “specialNode.illinois.edu”)

 

 

I know RemoteHost has the format of “slot1_1@xxxxxxxxxxxxxxxxxxxxxxxxx” so I tried using the splitSlotName() function to strip out the hostname:

 

StateTimer = (time() - EnteredCurrentState)

DAY = 86400

JobHost = splitSlotName(RemoteHost)[1]

PREEMPTION_REQUIREMENTS = $(StateTimer) > (2 * $(DAY)) && RemoteUserPrio > SubmittorPrio * 1.2 && (JobHost != “specialNode.illinois.edu”)

 

but it would always evaluate to undefined. I’m assuming the Negotiator doesn’t see the RemoteHost classad? Or I’ve just screwed up some syntax somewhere.

 

If anyone has some insight as to how wrong I am, it’d be appreciated.

 

Thanks,

 

-------------------------------------

Gianni Pezzarossi

Computational System Analyst

Research Services

Engineering IT Shared Services

University of Illinois @ Urbana-Champaign