Mailing List ArchivesPublic Access |
![[Computer Systems Lab]](https://www-auth.cs.wisc.edu/pics/csl_logo.gif){kind=logo}
|
Mailing List ArchivesPublic Access |
|
|
max_materialize seems to be the closest feature to what I am looking for. Testing it now, but it seems that this feature will allow the user to set the maximum running jobs from a single submit file while holding any more jobs on idle. This will definitely
help me in the time being and I will make use of it, thanks for the suggestion.
This command however is not exactly what I am looking for as it requires a single submit file to submit multiple jobs. I would really like to be able to set some sort of global max_materialize for a user regardless of the submit file from which the jobs are
sent. For example, if a user submits 1000 jobs, I would like to be able to set a maximum of 50 running jobs and all others are held on idle. Ideally, this behavior would be the same whether one uses a single submit file to submit 1000 jobs or uses 1000 submit
files each submitting one single job
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Beyer, Christoph <christoph.beyer@xxxxxxx>
Sent: 23 September 2021 08:28 To: htcondor-users <htcondor-users@xxxxxxxxxxx> Subject: Re: [HTCondor-users] Limit Maximum Running Jobs per User arg,
I can not properly read my own documentation, the feature you are looking for is closely related though (maybe): max_materialize = <n> Will limit the number of running jobs to the given <n> Best christoph -- Christoph Beyer DESY Hamburg IT-Department Notkestr. 85 Building 02b, Room 009 22607 Hamburg phone:+49-(0)40-8998-2317 mail: christoph.beyer@xxxxxxx ----- Ursprüngliche Mail ----- Von: "Christoph Beyer" <christoph.beyer@xxxxxxx> An: "htcondor-users" <htcondor-users@xxxxxxxxxxx> Gesendet: Donnerstag, 23. September 2021 14:23:55 Betreff: Re: [HTCondor-users] Limit Maximum Running Jobs per User Hi Steven, just add materialize_max_idle to your submit file materialize_max_idle = 10 queue 10000 In your submit file means, submit 10.000 jobs and always keep 10 jobs idle in the sched queue (when the first job vanishes and the number of idle queued jobs becomes 9 the next job from your submission will get 'materialized' in the queue). I think the scheduler must have the option SCHEDD_ALLOW_LATE_MATERIALIZATION set tot true for this to work but I am not sure if that is still necessary ... Best christoph -- Christoph Beyer DESY Hamburg IT-Department Notkestr. 85 Building 02b, Room 009 22607 Hamburg phone:+49-(0)40-8998-2317 mail: christoph.beyer@xxxxxxx Von: "Steven Clark" <s.clark@xxxxxxxxxxx> An: "htcondor-users" <htcondor-users@xxxxxxxxxxx> Gesendet: Donnerstag, 23. September 2021 14:09:45 Betreff: [HTCondor-users] Limit Maximum Running Jobs per User Hello HTCondor Users, I am wondering if there is a way to control the maximum number of running jobs as a user. Ideally, I would like to be able to submit large numbers of jobs but only allow a certain number of jobs to run at a given time, the rest are held idle until other jobs finish. I understand that DAG can be used to set parent jobs, but this may be overkill for me as the jobs are independent. I simply want to ensure that there are always a certain number of nodes open to other users so that I can submit more jobs than there are total nodes but never be using every node as a single user. I know the priority system will let other users' submissions run as jobs finish, but I've had the problem of submitting jobs that take a long time to finish and thus lock other users out for many hours until they finish running. I have searched the forum and found a little info about the MAX_JOBS_PER_OWNER tool, but it is not clear to me how to use this and looks like it may only be used at the level of administrator? Is there a way to do this as a user? Thanks in advance for any help, please let me know if I can clarify in any way. -Steven Clark _______________________________________________ HTCondor-users mailing list To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://nam02.safelinks.protection.outlook.com/?url=""> The archives can be found at: https://nam02.safelinks.protection.outlook.com/?url=""> _______________________________________________ HTCondor-users mailing list To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://nam02.safelinks.protection.outlook.com/?url=""> The archives can be found at: https://nam02.safelinks.protection.outlook.com/?url=""> _______________________________________________ HTCondor-users mailing list To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://nam02.safelinks.protection.outlook.com/?url=""> The archives can be found at: https://nam02.safelinks.protection.outlook.com/?url=""> |