Re: [HTCondor-users] condor_now with jupyter notebooks

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi Todd,

thanks for getting back to me with this :) 

While I thought a bit more about this it does not seem to be a managable way to get fast jupyter slots running as the krb-token integration would not work as expected. 

How can I configure a second startd for dedicated jobs and would that avoid the negotiation cycle ? 

At the moment it seems to be the main issue that the collector is not getting aware of the jupyter job fast enough. I did try a separate scheduler, exclusively for jupyter jobs but then these get ignored completely and take more like 10 minutes to show up in the negotiator log if at all. 

Is there a way to prioritize a scheduler on the collector side, maybe by running several collector, one of those exclusively for one scheduler ? 

I have got around 70 static slots now that only run jupyter jobs and jupyter jobs do have a corresponding classadd attribute. Is it still necessary to tweak the ranking of the slots and jobs as these jobs are only able to run on these slots anyway ?

Quotas and prios for these jobs are high as a kite ! 

We see a bigger demand for jupyter environments at DESY and I would love to drag these people on the condor pool but at the moment I am a bit of a lame duck :( 

I will update the scheduler which unfortunately takes some time in order to use condor_now as a fallback. 

If the first try does not succeed I will start a sleeping job for the user and ask him to try again in a couple of minutes. hopefully in the meantime the sleep job is running and gets replaced with the notebook job at the next login attempt. Not a perfect solution but still something I might be able to sell to the users ! 

Still if there are any means to speed up the process of collecting and negotiating please give me a hint !!!!!!

Best
Christoph
 

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "Todd L Miller" <tlmiller@xxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Dienstag, 10. September 2019 00:14:31
Betreff: Re: [HTCondor-users] condor_now with jupyter notebooks

> - take one of the idle jobs and alter it's ownership to match the user

 	This is not something HTCondor allows.  (I assume you mean "sleep" 
here, rather than "idle", given the way condor_now works.)

> Is there anybody doing something similar maybe or could give me a hint 
> if and how this could be done ?

 	The condor_now manpage claims that the queue super-user can run a 
now-job on a victim-job's resources even if those jobs have different 
owners.  I don't know of anyone who's done this in production, so you'll 
want to test this with a schedd, startd, and jobs that you don't care 
about, but you don't actually need to change the job's ownership.

 	On the other hand, it may well be easier to configure a startd or 
two to only run notebook jobs than to reserve slots with jobs, even if 
that's not quite as nice a design.  (Or to configure 12 startds with a 
reserved slot, whatever.)

- ToddM
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/