Re: [HTCondor-users] Limit Maximum Running Jobs per User

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

arg, 

I can not properly read my own documentation, the feature you are looking for is closely related though (maybe): 

max_materialize = <n>

Will limit the number of running jobs to the given <n> 


Best
christoph

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "Christoph Beyer" <christoph.beyer@xxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Donnerstag, 23. September 2021 14:23:55
Betreff: Re: [HTCondor-users] Limit Maximum Running Jobs per User

Hi Steven, 

just add materialize_max_idle to your submit file 



materialize_max_idle = 10 

queue 10000 





In your submit file means, submit 10.000 jobs and always keep 10 jobs idle in the sched queue (when the first job vanishes and the number of idle queued jobs becomes 9 the next job from your submission will get 'materialized' in the queue). 




I think the scheduler must have the option 





SCHEDD_ALLOW_LATE_MATERIALIZATION 





set tot true for this to work but I am not sure if that is still necessary ... 





Best 


christoph 



-- 
Christoph Beyer 
DESY Hamburg 
IT-Department 

Notkestr. 85 
Building 02b, Room 009 
22607 Hamburg 

phone:+49-(0)40-8998-2317 
mail: christoph.beyer@xxxxxxx 


Von: "Steven Clark" <s.clark@xxxxxxxxxxx> 
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx> 
Gesendet: Donnerstag, 23. September 2021 14:09:45 
Betreff: [HTCondor-users] Limit Maximum Running Jobs per User 

Hello HTCondor Users, 

I am wondering if there is a way to control the maximum number of running jobs as a user. Ideally, I would like to be able to submit large numbers of jobs but only allow a certain number of jobs to run at a given time, the rest are held idle until other jobs finish. I understand that DAG can be used to set parent jobs, but this may be overkill for me as the jobs are independent. I simply want to ensure that there are always a certain number of nodes open to other users so that I can submit more jobs than there are total nodes but never be using every node as a single user. I know the priority system will let other users' submissions run as jobs finish, but I've had the problem of submitting jobs that take a long time to finish and thus lock other users out for many hours until they finish running. 

I have searched the forum and found a little info about the MAX_JOBS_PER_OWNER tool, but it is not clear to me how to use this and looks like it may only be used at the level of administrator? Is there a way to do this as a user? 

Thanks in advance for any help, please let me know if I can clarify in any way. 

-Steven Clark 

_______________________________________________ 
HTCondor-users mailing list 
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a 
subject: Unsubscribe 
You can also unsubscribe by visiting 
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users 

The archives can be found at: 
https://lists.cs.wisc.edu/archive/htcondor-users/ 

_______________________________________________ 
HTCondor-users mailing list 
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a 
subject: Unsubscribe 
You can also unsubscribe by visiting 
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users 

The archives can be found at: 
https://lists.cs.wisc.edu/archive/htcondor-users/