[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Dagman/submit is slow


It should be possible for us to speed this up, given certain important constraints.

If all Condor jobs in your DAG do not share a single userlog file, then DAGMan must rely on the userlog event timestamps to correctly sequence events when reading the logs to recover after a crash or reboot. Because the Condor userlog timestamps have one-second resolution, DAGMan submits at most one job per second to ensure that the timestamps can be sorted properly.

However, if you use only one userlog DAGMan could rely solely on the order of events in the log, and submit as fast as condor_submit can be called for each job (which may be somewhat faster than once per second).

I will try to add this as an automatic optimization in the single-userlog case, and let you know when it's available.



On Jun 13, 2005, at 4:19 AM, Cargnelli, Matthieu wrote:


I'm trying to use condor to distrbute jobs within a cluster. I'm using a
dagman to describe the 1200 tasks I wish to execute in parallel. It
takes about 10 minutes to have a hundred of these tasks running. I have
set the number of jobs the dagman can submit per interval to its maximum
: 1000, and the interval to 0. But it submits only one or two tasks each
second. It's sufficient to feed the procs I use for now, but I plan on
using hundreds of processors soon, so I'd like to know if there's a way
to speed things up.
The datafile I'm dealing with is approximately 2MB, and the network is
one GB. From my experience in transmitting files over that network it
mustn't take more than 0.2 s to be passed between the master and the nodes.
The nodes are configured to accept any work, never to preempt their jobs
and are dedicated, do they are never interrupted by keypress or mouse.
The universe I use is vanilla.

Thanks in advance for your help.

Matthieu Cargnelli

-- Peter Couvares University of Wisconsin-Madison Condor Project Research Department of Computer Sciences pfc@xxxxxxxxxxx 1210 W. Dayton St. Rm #4241 (608) 265-8936 Madison, WI 53706-1685