Re: [HTCondor-users] HTCondor concurrency limit

[Marc Volovic <marcv@xxxxxxxxxx>]

J

 

Thanks. I am acquainted with his page. That's the problem – I set up everything according to what I seem to read there, set my own personal concurrency limit to 1 and – lo and behold – can run a gazillion jobs.

 

M

 

 

From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Miha Ahronovitz
Sent: Sunday, August 10, 2014 7:46 PM
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] HTCondor concurrency limit

 

Ok... Maybe I do not understand the concurrency.

 

You are not the only one :-(  Most of the people on this thread do not understand it, but they are afraid to admit it

 

A blogger decided to publish his own discoveries Concurrency Limits: Protecting shared resources

 

See if it inspires you.

 

M

 

--- --- --- --- --- --- --- --- --- --- --- --- ---

Miha Ahronovitz

Principal | Ahrono Associates

Web: http://www.ahrono.com/

Blog: http://my-inner-voice.blogspot.com/

c: 408 422 2757

e: miha.ahronovitz@xxxxxxxxxx

tw: @myinnervoice

--- --- --- --- --- --- --- --- --- --- --- --- ---

 

 

On Sun, Aug 10, 2014 at 4:55 AM, Marc Volovic <marcv@xxxxxxxxxx> wrote:

Hello, sorry for the latish response - we were on a weekend :-)...

Concurrency limits are indeed listed in negotiator log and in jobs.

As a test, I set my own concurrency to 1 and started initiating condor_run 'sleep 10000' jobs.

Slowly, bit by bit, there came more and more of them running. Not what I wants :(....

Ok... Maybe I do not understand the concurrency.

Assume I am the only user of the whole cluster. I want no more than 2 of my jobs running at any given time, no matter how many free slots there are. Should I set 'marcv_limit = 2' and set SCHED_ATTRS = Owner, etc, or should I do something more violent?

Many thanks in advance,

marc

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Marc Volovic
Sent: Friday, August 08, 2014 7:29 AM
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] HTCondor concurrency limit

I did a condor_reconfig -full -all and am pretty sure limits are in the ads. Will check logs.

The annoying bit is that a test cluster works, the production one does not.


Sent from my iPad

> On 7 Aug 2014, at 22:36, "Ben Cotton" <ben.cotton@xxxxxxxxxxxxxxxxxx> wrote:
>
> Marc,
>
> I was able to enforce limits with a configuration as you described. I
> assume you did a condor_reconfig on both your central manager and your
> scheduler? Do the concurrency limits appear in the job ad? Does
> "Concurrency Limit" appear in your negotiator log?
>
>
> Thanks,
> BC
>
> --
> Ben Cotton
> main: 888.292.5320
>
> Cycle Computing
> Leader in Utility HPC Software
>
> http://www.cyclecomputing.com
> twitter: @cyclecomputing
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/