Re: [HTCondor-users] Jenkins -> Condor

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

I wrote a plugin to launch workers via HTCondor submissions a while back, but haven't put it into production as yet, or even really tested it particularly hard.

One trick I had to write was a requirements expression that the worker job has to start running on an exec node within the slave startup timeout (CurrentTRime - Qdate > ~JenkinsTimeoutSeconds~) which Jenkins enforces, and a periodic remove expression to drop moot workers from the job queue.

Also, bear in mind that the workers need to be able to reach the master which submitted them, so if your exec node or Jenkins server has firewalls in place, the worker would need to traverse them.

I seem to recall that I also set up parameters to allow Jenkins to specify disk, memory, and cpu requirements which translated into the submit.

Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company
________________________________________
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Gergely Debreczeni via HTCondor-users
Sent: Tuesday, June 11, 2019 4:58 PM
To: htcondor-users@xxxxxxxxxxx
Cc: Gergely Debreczeni <gergely.debreczeni@xxxxxxxxxxxx>
Subject: [External] [HTCondor-users] Jenkins -> Condor

Hello,

We have a Jenkins system for various build purposes containing significant amount of slave nodes. These nodes are not utilized 100%, so I want these nodes to be available for other purposes as well. I thought that the simplest solution would be to 

1. integrate Jenkins slave nodes into our already existing Condor pool as execute hosts
2. and transform Jenkins build jobs to Condor jobs and Jenkins server to a Condor submit machine and submit build jobs through Condor

Does anybody have experience in such setup ? Any advice, best practice here ?

Thanks a lot in advance,
Gergely