Re: [HTCondor-users] scheduled downtime configuration

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi Ben,

always good to hear from you :) 

Thanks for the insight, that's what I had in mund more or less, I do skip the file/startdcron part for now and just made it remote controllable through condor_config_val.

It's not very sophisticated but I put it here for whoever might be looking for something similar anyway :) 

On the workernode: 

InStageDrain = False
ShutdownTime = 0
Drain = ((InStageDrain =?= True && (time() + MaxJobRetirementTime < ShutdownTime)) || InStageDrain =?= False)
STARTD_ATTRS = InStageDrain, ShutdownTime, StartJobs, $(STARTD_ATTRS)
STARTD.SETTABLE_ATTRS_ADMINISTRATOR = StartJobs, InStageDrain, ShutdownTime
START = (NODE_IS_HEALTHY =?= True) && (StartJobs =?= True) && $(Drain)

Remote control: 

zitpcx35701%  date -d "May 30 14:59:48 CEST 2019" +%s
1559221188

condor_config_val -name <workernode> -startd -set "ShutdownTime = 1559221188"
condor_config_val -name <workernode> -startd -set "InStageDrain = True"
condor_reconfig <workernode> -daemon startd                      

Cheers
Chris

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "Ben Jones" <ben.dylan.jones@xxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Mittwoch, 15. Mai 2019 19:19:29
Betreff: Re: [HTCondor-users] scheduled downtime configuration

Hi Christoph,

Here's what we do...

We have the following clause in our START:

((InStagedDrain =?= True && (time() + MaxRuntime < ShutdownTime)) || InStagedDrain =?= False)

We have a startd cron that runs this:


====
#!/bin/bash

FILE=/etc/shutdowntime

if [ -f $FILE ];
then
    UNIX_SHUTDOWN=`awk '{print $0}' $FILE`
    echo "InStagedDrain = True"
    echo "ShutdownTime = $UNIX_SHUTDOWN"
else
    echo "InStagedDrain = False"
    echo "ShutdownTime = 0"
fi

====

Then we have tooling which populates /etc/shutdowntime, which is just a timestamp of when we want the host to drain by. 

cheers,
Ben
________________________________________
From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of Beyer, Christoph [christoph.beyer@xxxxxxx]
Sent: 15 May 2019 18:51
To: htcondor-users
Subject: [HTCondor-users] scheduled downtime configuration

Hi condor-team et al,

I am looking for some input for a common task I think. We do have scheduled downtimes and I want to drain the workernodes to a point in time more or less 60 min before the official downtime begins.

My initial idea is to come up with a classadd that defines the rest of the uptime of the machine or the point in time when the machine will go down and then alter the start definition to only accept jobs that have a requested runtime smaller than the uptime of the workernode.

I think out of the back of my head that at least partly I have seen something similar in a request string once but can't find it anywhere, hence as I don't want to reinvent the wheel my question is wether there is a potentially usable host classadd already or a condition/requirement string that can be used to get the described behaviour ???

Best
Christoph


--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/