Re: [Condor-users] Max. amount of Jobs in queue

["Matt Hope" <Matt.Hope@xxxxxxxxxxxxxxx>]

I can only speak for the windows series - I would guess they aren't up
to the linux ones in that regard (certainly not with XP's silly 10 TCP
connection limit, now removed in Vista SP2) 

Hard numbers definitely depend on the load per job, the clustering (both
explicit and automatic) and how often you call condor_q (the latter is
the killer really)
This only really makes a difference if your schedd has many more
(hundreds) slots to run its jobs, otherwise its basically a limitation
of memory and disk.

Good vague rule of thumb - if you want to head into tens of thousands of
jobs and/or hundreds of execute slots you may want to consider whether
you should attempt to dristribute this load beyond a single queue...

Matt

-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Ian Chesal
Sent: 28 May 2009 16:37
To: Condor-Users Mail List
Subject: Re: [Condor-users] Max. amount of Jobs in queue

> On the 6.8 series the limit is more like 15K (and slightly lower on XP

> verses a server based install).

We've routinely pushed our Linux 6.8.x series schedds to 40,000 jobs
with about a 100:1 job:cluster ratio. With 500 startd's pulling and
running jobs from one schedd. The trick is auto-cluster configuration
and controlling the jobs coming in. You really want to keep a tight
reign on how jobs look when they enter the system. Lots of consistency
is important for good auto-clustering.

And you don't want anyone running condor_q calls against a schedd that's
working that hard.

> Hopefully you're on a seven series so the 150K sounds reasonable.

Or you can, as Matt pointed out, skip 6.x and save yourself all the
trouble. I think we only a handful of startd's you don't have a lot to
worry about.

- Ian

Confidentiality Notice.
This message may contain information that is confidential or otherwise
protected from disclosure. If you are not the intended recipient, you
are hereby notified that any use, disclosure, dissemination,
distribution,  or copying  of this message, or any attachments, is
strictly prohibited.  If you have received this message in error, please
advise the sender by reply e-mail, and delete the message and any
attachments.  Thank you.

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at: 
https://lists.cs.wisc.edu/archive/condor-users/

----
Gloucester Research Limited believes the information provided herein is reliable. While every care has been taken to ensure accuracy, the information is furnished to the recipients with no warranty as to the completeness and accuracy of its contents and on condition that any errors or omissions shall not be made the basis for any claim, demand or cause for action.
The information in this email is intended only for the named recipient.  If you are not the intended recipient please notify us immediately and do not copy, distribute or take action based on this e-mail.
All messages sent to and from this email address will be logged by Gloucester Research Ltd and are subject to archival storage, monitoring, review and disclosure.
Gloucester Research Limited, 5th Floor, Whittington House, 19-30 Alfred Place, London WC1E 7EA.
Gloucester Research Limited is a company registered in England and Wales with company number 04267560.
----