[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] slow scheduling of dagman jobs
- Date: Thu, 8 Sep 2011 14:40:30 -0400
- From: Patty Bragger <braggepj@xxxxxxxxx>
- Subject: Re: [Condor-users] slow scheduling of dagman jobs
100 separate calls takes about 11 seconds
$ time for i in $(seq 0 99); do condor_submit grab1.sub>/dev/null; done
multiple tests resulted in pretty similar time results,
So an average of about 9 jobs/sec, which is faster (but only a little) than submitting through dag. What kind of rates are you guys getting? Maybe this is this normal?
On Thu, Sep 8, 2011 at 1:13 PM, Dan Bradley <dan@xxxxxxxxxxxx>
There are some optimizations that take place when submissions are combined into a single submit file, even if many separate queue statements are used. It would be good to test 100 separate invocations of condor_submit in your environment.
On 9/8/11 11:35 AM, Patty Bragger wrote:
But if that is the case, that this is just the overhead of submitting individual jobs, why don't I see the same overhead when submitting 100 separate jobs through a non-dag submit? What is the difference between submitting a dag file that has 100 entries and submitting a non-dag file that has 100 job submissions (not through queue 100, but through 100 separate job definitions in the file.) I would expect those to have pretty much the same overhead.