Re: [HTCondor-users] Jenkins -> Condor

[Gergely Debreczeni <gergely.debreczeni@xxxxxxxxxxxx>]

Thank you very much !

In the meantime I'm wondering on an alternative solution, which is the following:

I install Condor execute node on Jenkins slave without letting Jenkins know about it. Jenkins jobs will seen by Condor as "Owner" activity, and if I configure so, Condor will evict/suspend any job running on the Jenkins build node when a new build is submitted as such Owner activity increased. (In our case build jobs has absolute priority...)

What do you think ? Is that a solution worth to start with ?

Cheers,
Gergely

---
Gergely Debreczeni, Ph.D.
Chief Scientist, Aimotive
T: +36 1 770 7204 | M: +36 30 324 7551 | W: aimotive.com
A: Aimotive Kft, Szepvolgyi ut 22, Budapest, 1025 Hungary



-----Original Message-----
From: Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>
Sent: Wednesday, June 12, 2019 6:55 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Gergely Debreczeni <gergely.debreczeni@xxxxxxxxxxxx>
Subject: RE: Jenkins -> Condor

I wrote a plugin to launch workers via HTCondor submissions a while back, but haven't put it into production as yet, or even really tested it particularly hard.

One trick I had to write was a requirements expression that the worker job has to start running on an exec node within the slave startup timeout (CurrentTRime - Qdate > ~JenkinsTimeoutSeconds~) which Jenkins enforces, and a periodic remove expression to drop moot workers from the job queue.

Also, bear in mind that the workers need to be able to reach the master which submitted them, so if your exec node or Jenkins server has firewalls in place, the worker would need to traverse them.

I seem to recall that I also set up parameters to allow Jenkins to specify disk, memory, and cpu requirements which translated into the submit.

Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company
________________________________________
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Gergely Debreczeni via HTCondor-users
Sent: Tuesday, June 11, 2019 4:58 PM
To: htcondor-users@xxxxxxxxxxx
Cc: Gergely Debreczeni <gergely.debreczeni@xxxxxxxxxxxx>
Subject: [External] [HTCondor-users] Jenkins -> Condor

Hello,

We have a Jenkins system for various build purposes containing significant amount of slave nodes. These nodes are not utilized 100%, so I want these nodes to be available for other purposes as well. I thought that the simplest solution would be to

1. integrate Jenkins slave nodes into our already existing Condor pool as execute hosts 2. and transform Jenkins build jobs to Condor jobs and Jenkins server to a Condor submit machine and submit build jobs through Condor

Does anybody have experience in such setup ? Any advice, best practice here ?

Thanks a lot in advance,
Gergely
________________________________
This e-mail and any files transmitted with it contain confidential and may contain privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized use, copying, disclosure or distribution of the material in this e-mail is strictly forbidden.