[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] MAX_JOBS_PER_OWNER



Are you sure, MAX_JOBS_PER_OWNER is set to 8? 

Try running

 

     condor_config_val -v -dump MAX_JOBS

 

Also, where should be a message in the SchedLog whever the MAX_JOBS_PER_OWNER limit Is exceeded.

What does that message say?

 

-tj

 

 

From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Alessandro Italiano
Sent: Monday, October 3, 2016 1:39 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] MAX_JOBS_PER_OWNER

 

Hi

 

thanks for the answer

 

thatâs the point, all the jobs are idle in the queue and even if the user has reached his job limit in the queue

he can still continue to submit jobs

 

in the following example 

 

- MAX_JOB_PER_OWNER = 8

- the cluster 216 has been accepted when there are 8 idle jobs

- the cluster 220 has been accepted when there are 12 idle jobs

 

 

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 214.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 215.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 216.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 220.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 221.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ 

[italiano@ui01 ~]$ 

[italiano@ui01 ~]$ 

[italiano@ui01 ~]$ 

[italiano@ui01 ~]$ condor_q

 

 

-- Schedd: ui01.recas.ba.infn.it : <90.147.168.55:9618?...

 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD               

 214.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 214.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 214.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 214.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 

20 jobs; 0 completed, 0 removed, 20 idle, 0 running, 0 held, 0 suspended

[italiano@ui01 ~]$ 

 

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 224.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 225.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 226.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 232.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)....

4 job(s) submitted to cluster 233.

[italiano@ui01 ~]$ condor_submit  sleep

Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ condor_submit  sleep

\Submitting job(s)

ERROR: Failed to create proc

Number of submitted jobs would exceed MAX_JOBS_PER_OWNER

[italiano@ui01 ~]$ 

[italiano@ui01 ~]$ 

[italiano@ui01 ~]$ condor_q

 

 

-- Schedd: ui01.recas.ba.infn.it : <90.147.168.55:9618?...

 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD               

 214.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 214.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 214.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 214.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 215.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 216.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 220.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 221.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 224.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 224.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 224.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 224.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 225.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 225.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 225.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 225.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 226.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 226.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 226.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 226.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 232.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 232.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 232.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 232.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 233.0   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 233.1   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 233.2   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 233.3   italiano       10/3  20:29   0+00:00:00 I  0   0.0  sleep 600

 

40 jobs; 0 completed, 0 removed, 40 idle, 0 running, 0 held, 0 suspended

[italiano@ui01 ~]$ 



 

On 03 Oct 2016, at 16:37, Todd Tannenbaum <tannenba@xxxxxxxxxxx> wrote:

 

On 10/3/2016 6:02 AM, Alessandro Italiano wrote:

Hi

I need to limit the number of jobs a user can submit to the schedd.

I tried to use MAX_JOBS_PER_SUBMISSION, and it works as expected.
Submission with a queue command >= MAX_JOBS_PER_SUBMISSION fails.

[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s).......
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_SUBMISSION

How MAX_JOBS_PER_OWNER should work ?

In the following example I tried to submit a job with a "queue 4" command.
as you can see cluster 116 has not been submitted while cluster 119 has
been correctly submitted.

is it the expected behaviour ?



Yes, I would expected cluster 119 to successfully submit assuming that four or more earlier submitted jobs completed or otherwise were removed from the queue.  How long do your sleep jobs run?  Is it possible that between submission of cluster 113 and cluster 119 that four of your jobs completed?

regards,
Todd






[root@ui01 ~]# condor_status -version
$CondorVersion: 8.4.9 Sep 29 2016 BuildID: 382747 $
$CondorPlatform: x86_64_RedHat6 $


[root@ui01 ~]# condor_config_val -dump| grep MAX_JOBS_PER
MAX_JOBS_PER_OWNER = 8
MAX_JOBS_PER_SUBMISSION = 8
[root@ui01 ~]#
[root@ui01 ~]#

[italiano@ui01 ~]$ grep queue sleep
queue 4
[italiano@ui01 ~]$

[italiano@ui01 ~]$
[italiano@ui01 ~]$
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 113.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 114.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 115.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 119.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 120.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)
ERROR: Failed to create proc
Number of submitted jobs would exceed MAX_JOBS_PER_OWNER
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 127.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 128.
[italiano@ui01 ~]$ condor_submit  sleep
Submitting job(s)....
4 job(s) submitted to cluster 129.
[italiano@ui01 ~]$


thanks in advance

Ale




_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



-- 
Todd Tannenbaum <
tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to 
htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/