[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Condor development release 6.9.5 now available



On Nov 29, 2007 8:45 PM, Dan Bradley <dan@xxxxxxxxxxxx> wrote:
>
>
> Matt Hope wrote:
> >> MAX_CONCURRENT_UPLOADS
> >>
> > Worth knowing about. If I had (say) a dedicated windows multi cpu
> > submit machine with fast (Gigabit) network to a few hundred execute
> > nodes which could all activate at once can I push the default higher
> > than 10 and if so what number would you suggest :)
> >
>
> There should be no problem in Condor handling larger values for
> MAX_CONCURRENT_UPLOADS or _DOWNLOADS.  However, in many cases, I doubt
> you will get higher throughput than when you use a lower level of
> concurrency.  Depends on how many disk heads the files are sitting on.
> It would be nice if Condor kept track of the underlying resource
> utilization, but we thought adding these two transfer queues would at
> least be a good improvement over the previous solution, which was to
> blindly limit the rate at which jobs start up.

Oh in my case disk heads won't come into it, the file have been
entirely cached by both the OS and if not the raid controller (it's
small). All the heavy lifting is done from the execute nodes via a
clustered file system. I have some users with rather more sizeable
return data though...

> I hope it helps!

much better thanks :)