Re: [Condor-users] Looking for something like CpuBusy for Disk

[Lans Carstensen <Lans.Carstensen@xxxxxxxxxxxxxx>]

Greetings!  Diane Trout wrote:
> On Thu, Sep 02, 2010 at 08:27:11PM -0400, Ian Chesal wrote:
> > Hi Diane,
> >
> > On Thu, Sep 2, 2010 at 6:23 PM, Diane Trout <diane@xxxxxxxxxxx>
> > wrote:
> >
> > > I was wondering if there was anything like the CpuBusy macro that
> > > could I could use to throttle launching new processes if the
> > > disks were too busy?
> > There isn't anything in Condor to do this out-of-the-box but you
> > could run Startd cron jobs on your machines that periodically run
> > iostat or vmstat and return a "disk activity" number that gets
> > stored and set as a ClassAd attribute on the machine's ad.
>
> Can you define an expression that queries a different machines
> attributes?
>
> For example:
>
> The main file server publishes an attribute
> PercentUtilization.Partition1
>
> Can another machine query the attribute for instance
> FileServer.PercentUtilization.Partion1? Or would I need to use an
> external script to push the attribute into each machines local
> ClassAd?
>
> Thank you again, Diane

No, there are two approaches for managing information like that based on 
my experience:

1.  Report it strictly from the client perspective.  For recent Linux + 
NFS, that involves analyzing /proc/self/mountstats.  You can measure NFS 
service latency that way for a particular mountpoint - but I'd be 
interested in your means (or others on this list) of analyzing which 
mountpoints apply to which job.

2.  Publishing information from your servers periodically through 
generic ad's and then writing startd's crons to read them and re-use the 
information.

-- Lans Carstensen