[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] Looking for something like CpuBusy for Disk


My lab has been running a condor cluster with around 40 cores available. One of the problems we've encountered is that some workloads when distributed over the whole cluster easily overwhelm our nfs servers. 

I was wondering if there was anything like the CpuBusy macro that could I could use to throttle launching new processes if the disks were too busy?

Other possibilities:
 * Is there an easy way in a condor_submit script to limit the number of simultaneous jobs?
 * Or can I query the load average of a different machine in my START expression?

Or am I missing some better way of dealing with multi-gigabyte file format conversions?


Attachment: signature.asc
Description: Digital signature