[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] dagman: Possible to run a "PRE" script on same node as the main program will run?

Hi Tim

On Friday 23 July 2010 15:24:09 Timothy St. Clair wrote:
> You may want to check out (concurrency limits):
> http://www.cs.wisc.edu/condor/manual/v7.5/3_13Setting_Up.html#36823

I've briefly looked at that before and would have wished there was a solution 
which the user herself could set.

However, I don't think this will help me much, as the user's job will copy 
data to the local node which should take a couple of minutes and then should 
run for 1-2 hours on this data. With concurrency limits I will only limit the 
total number of jobs as I still cannot tell Condor that only the copying job 
needs to be limited, not the CPU intensive follow-up job, right?

Thus with concurrency limits I would maybe prevent an inrush at the start of a 
job cluster, but when most nodes are busy with CPU, no more network I/O 
intensive jobs would be able to run even if some CPU cores were idle as the 
limit has been reached.

Do you agree?