Re: [HTCondor-users] metascheduler anybody ?

["Fischer, Max (SCC)" <max.fischer@xxxxxxx>]

Hi Christoph,

we have used our opportunistic resource meta-scheduler (think cloud bursting but more generic) COBalD/TARDIS for something like this. The initial design case is to acquire resources from other resource providers and integrate them into our own HTCondor pool â think HTC on top of HPC. But we are now also using the tooling to âacquireâ our own resources and âintegrateâ them into other local schedulers for our users â think Spark/YARN, Dask, but also Condor, SLURM if needed.

Weâve been slacking off with publications lately, but thereâs a nice slide set with pictures here for a glimpse of the main ideas:

https://indico.physik.uni-muenchen.de/event/117/contributions/535/attachments/273/638/interactive_distributed_computing_hep_multi_managed_cluster.pdf

Cheers,

Max

On 10. Jun 2022, at 10:14, Beyer, Christoph <christoph.beyer@xxxxxxx> wrote:

Hi,

every now and then we think about installing a kind of more universal meta scheduler in order to give useres a more generic, formal entry point to the different computing pools we provide (HTC/HPC, condor/slurm, local pool/remote pool etc.)

Is anybody out there planning to uses such a setup or maybe has already a protoype or a running implementation of something alike ?

Best
christoph

--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: smime.p7s
Description: S/MIME cryptographic signature