On Thu, Sep 2, 2010 at 6:23 PM, Diane Trout
<diane@xxxxxxxxxxx> wrote:
I was wondering if there was anything like the CpuBusy macro that could I could use to throttle launching new processes if the disks were too busy?
There isn't anything in Condor to do this out-of-the-box but you could run Startd cron jobs on your machines that periodically run iostat or vmstat and return a "disk activity" number that gets stored and set as a ClassAd attribute on the machine's ad.
If you called the attribute CurrentDiskActivity and higher meant more disk I/O load, you could steer your jobs away from heavily loaded machines with:
rank = 1/CurrentDiskActivity
In your submit tickets. Or stop them from going to heavily loaded machines altogether with:
requirements = CurrentDiskActivity < <some threshold value>
Other possibilities:
* Is there an easy way in a condor_submit script to limit the number of simultaneous jobs?
There's no throttling on condor_submit but there are two ways to achieve this:
* Or can I query the load average of a different machine in my START _expression_?
If you use the Startd Cron approach I mentioned above you could make the ad you publish part of your machine's START _expression_. But tread carefully here. Switching START from True -> False for currently running jobs can have the unintended side effect of causing them to be vacated as Condor thinks the machine has been taken over and should be in the Owner state. Better would be to reference the attribute in your job's Requirements _expression_.
Or am I missing some better way of dealing with multi-gigabyte file format conversions?
Nope. You've got your bases covered. There are a few ways to approach it, which approach works best is up to you.
- Ian
![[Computer Systems Lab]](https://www-auth.cs.wisc.edu/pics/csl_logo.gif){kind=logo}