Re: [HTCondor-users] Backfill on an OpenStack system

[Steven C Timm <timm@xxxxxxxx>]

There are 2 ways it can be done.  One is to install the optional EC2 openstack emulator

and use the aws features of htcondor to launch virtual machines.  

The other way is the so called "VAC" system in which there is a daemon running on each cloud

node to self-launch a VM that was developed by GridPP in the UK.. basically the idea that the VM's 

launch out of the "vacuum" and join a htcondor pool.  The latter can run on any pool, doesn't necessarily

need openstack.

I am fairly new to running openstack myself so am not sure if they have the equivalent of VM's that

can be pre-empted. but if you have a startd you could use htcondor to condor_off the startd if the

VM is needed back and have the VM then programed to exit.

HTCondor at one point was going to add a feature to talk directly to the OpenStack "Nova" API but

I don't think that it is functional yet,.

Steve Timm

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of West, Matthew <M.T.West@xxxxxxxxxxxx>
Sent: Saturday, September 4, 2021 1:20 PM
To: htcondor-users@xxxxxxxxxxx <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Backfill on an OpenStack system

 

Hi All,

Here at Exeter, IT is setting up an OpenStack system to support researchers who want DRAM heavy bespoke workstation-like environments. Because I don't expect the system to be full up with active users 24/7, I am wondering what the optimal way to setup an HTCondor pool on it to run jobs as backfill. Would this be similar to how you would do it for any other spare resources: have a VM start up on a node and announce itself to the collector daemon as an available worker if idle conditions of the machine are met?

It reminds me of the method to expand one's resources into corporate cloud servers but I am not sure what tools are useful in this case.

Cheers,

Matt