Re: [Condor-users] slow scheduling of dagman jobs

[Patty Bragger <braggepj@xxxxxxxxx>]

100 separate calls takes about 11 seconds

$ time for i in $(seq 0 99); do condor_submit grab1.sub>/dev/null; done   
real    0m11.257s
user    0m6.597s
sys     0m2.237s

multiple tests resulted in pretty similar time results, 
real    0m11.235s
real    0m10.812s
real    0m11.000s
real    0m11.103s

So an average of about 9 jobs/sec, which is faster (but only a little) than submitting through dag.  What kind of rates are you guys getting?  Maybe this is this normal?

thanks,
Patty

On Thu, Sep 8, 2011 at 1:13 PM, Dan Bradley <dan@xxxxxxxxxxxx> wrote:

On 9/8/11 11:35 AM, Patty Bragger wrote:

But if that is the case, that this is just the overhead of submitting individual jobs, why don't I see the same overhead when submitting 100 separate jobs through a non-dag submit?  What is the difference between submitting a dag file that has 100 entries and submitting a non-dag file that has 100 job submissions (not through queue 100, but through 100 separate job definitions in the file.)  I would expect those to have pretty much the same overhead.

There are some optimizations that take place when submissions are combined into a single submit file, even if many separate queue statements are used.  It would be good to test 100 separate invocations of condor_submit in your environment.

--Dan

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxedu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/