Re: [HTCondor-users] Limit Maximum Running Jobs per User

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi Steven,

just add materialize_max_idle to your submit file

materialize_max_idle = 10

queue 10000

In your submit file means, submit 10.000 jobs and always keep 10 jobs idle in the sched queue (when the first job vanishes and the number of idle queued  jobs becomes 9 the next job from your submission will get 'materialized' in the queue).

I think the scheduler must have the option

SCHEDD_ALLOW_LATE_MATERIALIZATION

set tot true for this to work but I am not sure if that is still necessary ...

Best

christoph

--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

---

Von: "Steven Clark" <s.clark@xxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Donnerstag, 23. September 2021 14:09:45
Betreff: [HTCondor-users] Limit Maximum Running Jobs per User

Hello HTCondor Users,

I am wondering if there is a way to control the maximum number of running jobs as a user. Ideally, I would like to be able to submit large numbers of jobs but only allow a certain number of jobs to run at a given time, the rest are held idle until other jobs finish. I understand that DAG can be used to set parent jobs, but this may be overkill for me as the jobs are independent. I simply want to ensure that there are always a certain number of nodes open to other users so that I can submit more jobs than there are total nodes but never be using every node as a single user. I know the priority system will let other users' submissions run as jobs finish, but I've had the problem of submitting jobs that take a long time to finish and thus lock other users out for many hours until they finish running.

I have searched the forum and found a little info about the MAX_JOBS_PER_OWNER tool, but it is not clear to me how to use this and looks like it may only be used at the level of administrator? Is there a way to do this as a user?

Thanks in advance for any help, please let me know if I can clarify in any way.

-Steven Clark

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/