[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Wnidows schedd job limits (Was: RE: Hooks in the Scheduler Universe?)


We have a small pool so we cannot concurrently run more than 110 jobs. I am sure you have already done this, but I thought I would ask anyhow. Did you increase the default heap size in the registry on the submit machines. With the default setting, one can only run around 200 jobs at any one time. I would be interested in hearing from others as well because our goal is to continue increasing our pool size over the next year and our pool is mostly Windows, some VMs and a couple servers, which I will be adding in the next month or so. I have not attempted any of the considerations you mentioned below, so I don't have any suggestions here--sorry.


From: Matt Hope <Matt.Hope@xxxxxxxxxxxxxxx>
To: Condor-Users Mail List <condor-users@xxxxxxxxxxx>
Date: 10/07/2010 01:18 AM
Subject: [Condor-users] Wnidows schedd job limits (Was: RE: Hooks in the Scheduler Universe?)
Sent by: condor-users-bounces@xxxxxxxxxxx

I am currently thinking over how to work around the limitations of the windows schedd/shadow structure.  
Using windows server 2003 64bit and tweaking the registry for a few things we can only stably run 200 jobs per submit node, which is a real pain. Thankfully running it in a VM appears to be acceptable so we’ve pretty much headed towards per USER submit VM’s
Some of the condor based solutions I was considering:
·         Job hooks
o   In the hive mind opinion should I not consider even testing using job hooks (for replacement of schedd/negotiator) on windows right now?
·         Multiple per user daemons per box
o   I doubt this would actually improve things
o   Also not clear if anyone uses this heavily on windows
·         Remote submit to linux based schedd’s                
o   Remote submission is ultimately a bit of a hack, and forces the client side to do a lot more state checking
Any one here on windows done anything to get that MAX_JOBS_RUNNING higher than 200 on windows?
I’d take a short term win in pushing those numbers higher…


Gloucester Research Limited believes the information provided herein is reliable. While every care has been taken to ensure accuracy, the information is furnished to the recipients with no warranty as to the completeness and accuracy of its contents and on condition that any errors or omissions shall not be made the basis for any claim, demand or cause for action.

The information in this email is intended only for the named recipient.  If you are not the intended recipient please notify us immediately and do not copy, distribute or take action based on this e-mail.

All messages sent to and from this email address will be logged by Gloucester Research Ltd and are subject to archival storage, monitoring, review and disclosure.

Gloucester Research Limited, 5th Floor, Whittington House, 19-30 Alfred Place, London WC1E 7EA.

Gloucester Research Limited is a company registered in England and Wales with company number 04267560.

Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at: