[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Machine activity for partitionable slots



Thank you all very much for the prompt responses!

Condor's built-in power management looks interesting, but if I'm not mistaken this is about hibernation rather than fully powering down/up a machine, correct? What I'm working on currently uses an IPMI/redfish interface to obtain and change the power state of the machine. This allows us to have some more fine-tailored decision making processes to determine what machines will be turned on when, and what for. Having access to information about when a dynamic slot was last created or destroyed would therefore be absolutely fantastic, and most certainly a feature I'm patiently awaiting.

In the interim, the most optimal solution indeed seems to be to get the tool to remember when it last saw dynamic slots for a machine, as the maximum idle time before shutdown should be flexible (based on other data we have collected, and possibly different per machine, or based on how many machines are currently on and idle). Ideally I'd want the tool to be as hands-off as possible and base its decision to change a machine's power state entirely on machine ClassAd data.

Again, thank you very much for the responses, it is really appreciated.

- Niels


On Wed, Jun 30, 2021 at 8:58 PM Todd L Miller <tlmiller@xxxxxxxxxxx> wrote:
    Assuming the machines can turn themselves off, you could set

STARTD_NOCLAIM_SHUTDOWN = 3600

to have the startd shut itself off after an hour idle. You can then add

MASTER.DAEMON_SHUTDOWN_FAST = ( STARTD_StartTime == 0 ) && ((CurrentTime - DaemonStartTime) > 60)

to have the master shutdown if the startd shuts down, and

DEFAULT_MASTER_SHUTDOWN_SCRIPT = /path/to/shutdown-h-now.sh

to have the master shutting turn off the whole machine. (Or run a script
to alert your monitoring system, etc.)

- ToddM
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/