[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] HTCondor on GCP images unavailable



Hi Daniel:

I took over managementÂof the Google Cloud - HTCondor relationshipÂearlier this year. The Cloud project (kbatch-public) that hosts the images was temporarily disabled. Some security notificationsÂwent out just as I was getting ownershipÂof the project. I've resolved theÂunderlying problems and it should not happen again. You should have direct access to the images and I was able to deploy a simple cluster using the Marketplace solution .

Earlier in August, I also resolved a Marketplace problem concerning deploying machines without GPUs. I suspect you may have been the one to report the problem to the team at UW?

Bigger picture:
  • The team at U. Wisconsin wants to take over the Marketplace solution but there are some legal challenges (U. of Wisconsin is officially part of the gov't)
  • The images themselves have become out-of-date (HTCondor v8.8.1)
As a side project, I've been working on modernizing the solution to support 9.x security (IDTOKENs, etc) using Terraform, Packer and Ansible. I would like to release these as open source projects for the HTCondor community but I need to get approval and work out other details. My hope is that the Packer code would maintain up-to-date images for theÂMarketplace solution and also serve users like you.

Since you are a customer, I mayÂ(may!) be able to share these interim solutions withÂyou directly under a non-OSS license. Please contact me at tpdownes@xxxxxxxxxx if you are interested. I would also appreciate an e-mail describing how you are using HTCondor on GCP. Knowing that our customers use HTCondor helps justify our partnership with the U. Wisconsin.

Anyone else making use of HTCondor on Google Cloud should reach out, too!

Tom

On Mon, Sep 20, 2021 at 1:01 PM Tom Downes <tpdownes@xxxxxxxxx> wrote:
Hi Daniel-

I'm a Google Cloud consultant who follows this list reasonably carefully. For the moment, I just want to acknowledge the e-mail and let you know that I'll follow up soon.

If you're interested, my colleague Ross Thomson and I will be presenting that the HTCondor Workshop tomorrow on some Google Cloud and "general" cloud development in HTCondor (first listed presentation):

https://indico.cern.ch/event/1059494/sessions/412060/#20210921

Tom


On Mon, Sep 20, 2021 at 4:21 AM Daniel Krebs <d.krebs@xxxxxxxxxx> wrote:
Hi,

we've been using HTCondor on GCP [1] for some time now and first of all
I want to say thank for you such a great project!

It looks like a recent update has broken the deployment scripts though.
When deploying a new instance of HTCondor-on-GCP, I get the following erros:

The resource 'projects/kbatch-public/global/images/condor-submit-v2-0-0'
is not ready

The resource
'projects/kbatch-public/global/images/condor-master-centos7-v8-8-1' is
not ready

Moreover, our existing instance broke because the same issue happens
with the "old" compute tier images (condor-compute-v2-0-1).

Are the GCP Compute images open source and available somewhere? I'd be
interested to maintain them inside our GCP project so our services would
not be affected by outside changes.


Cheers,
Daniel Krebs

[1]
https://console.cloud.google.com/marketplace/product/kbatch-public/htcondor-on-gcp
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/