[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Have submit nodes inside a Kubernetes cluster



Hi,

 

I know it is possible to run condor entirely inside Kubernetes.

But what if I have an existing pool outside of the Kubernetes cluster, with the central manager and the executor nodes, could I spawn one or several pods in the Kubernetes cluster and have them act as submit nodes ?

 

As a quick test I tried the following:

  • Have a pool running on different machines (one CM and two nodes)
  • On another machine, not a member of the pool but part of the same network, I start a Docker in Docker container
  • Inside this dind container, I start a CentOS container , on which I run “curl -fsSL https://get.htcondor.org | GET_HTCONDOR_PASSWORD="some_secret_password" /bin/bash -s -- --no-dry-run --submit my.cm.adress”

After doing this, from inside the CentOS container I can use condor_status fine, but when I submit jobs they stay IDLE forever and the negotiator log says something like:

 

Failed to connect to condor@xxxxxxxxxxxxxxxxxx (<172.18.0.1:9618?addrs=172.18.0.1-9618&alias=condor.test.submit&noUDP&sock=schedd_412_7d83>)

Error: Ignoring submitter for this cycle

 

So it looks like the CM has trouble connecting back to the submit container.

I tried the same setup but outside of the dind container (spawning the CentOS container directly on the docker host machine) and there it works well.

 

So, seeing that, I am starting to have my doubts about how having Kubernetes pods connecting to the pool and submitting jobs.

Was this ever done before ? Did it work ? Could it work ?

 

Thanks,

Gaëtan

 


Gaetan Geffroy
Junior Software Engineer, Space

Terma GmbH
Europaarkaden II, Bratustraße 7, 64293 Darmstadt, Germany
T +49 6151 86005 43 (direct)  •  T +49 6151 86005-0
Terma GmbH - Sitz Darmstadt  •  Handelsregister Nr.: HRB 7411, Darmstadt
Geschäftsführer: Poul Vigh / Steen Vejby Sørensen
www.terma.com • 
LinkedinTwitterInstagramYoutube


Attention:
This e-mail (and attachment(s), if any) - intended for the addressee(s) only - may contain confidential, copyright, or legally privileged information or material, and no one else is authorized to read, print, store, copy, forward, or otherwise use or disclose any part of its contents or attachment(s) in any form. If you have received this e-mail in error, please notify me by telephone or return e-mail, and delete this e-mail and attachment(s). Thank you.