[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] Scheduling N of M jobs



I am working with a set of jobs that are I/O bound, and need to submit several hundred of them to our cluster. Once 10 or 20 are running, the storage system they are using becomes saturated, so running more than that just wastes CPUs in the cluster. Is there a way, as a normal user, to tell the scheduler that, of this group of 400 jobs, I want 10 to be running at any given time?

This seems to be a fairly common problem, but all I have run into as a solution are scripts that periodically check the current queue state and either submit or release jobs if the running number has dropped below a threshold, which seems like an ugly hack.
-Nathan