[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] maxidle for a dag with one node?

On 09/22/2011 08:45 PM, Alan De Smet wrote:
If you're using "queue 10000" or lots of calls to "queue" in a
single submit file, they different processes will be submitted
all at once.

What's your problem with submitting large numbers of jobs at
once?  If it's a matter of not being able to have lots of jobs
running at the same time, there may be other solutions.

When I tried to submit 300,000 jobs at once, the schedd ran out of memory and became unusably slow. It's a somewhat older server with limited RAM. I may need to increase the number of jobs to several million in a later stage. Would a recent schedd with new hardware handle that many queued jobs?