Re: [Condor-users] tuning a file server

[David McBride <dwm@xxxxxxxxxxxx>]

Steffen Grunewald wrote:

Hi,

> some weeks ago, I had to move our file systems (/home and some data) to 
> another server, equipped with 3ware 9000 cards, and using 8 400GB disks
> in RAID5 with hot spare (2.3TB net capacity). To manage the disk space,
> I had to switch to Linux kernel 2.6 - and there are indications that this
> move caused some problems.

3ware tend to make excellent Linux-compatible RAID solutions, so it is 
unlikely to be a RAID-level problem, as your basic read benchmark 
suggested.

I'd suggest also running the 'bonnie++' benchmark on your filesystem to 
get a better idea of real-world performance (file creates/erasures, 
small loads, etc.)

> The file server has to serve 360 VMs in 180 dual-CPU machines, and if 
> there are lots of small I/O operations, the iowait percentage goes up to
> more than 80%! I have never seen that before (the previous server was
> almost identical, kernel 2.4, 1.4TB each).

Some important missing information:

* How (if at all) are the fileserver contents exported? NFS?
* What filesystem(s) are you using?

> Now I suspect that the i/o strategies introduced with 2.6 kernels are
> badly configured... are there any suggestions?

The 2.6 IO schedulers tend to _better_ than the 2.4 one.  The default is 
to use the 'anticipatory' scheduler which tends to be excellent for most 
needs.

Cheers,
David
--
David McBride <dwm@xxxxxxxxxxxx>
Department of Computing, Imperial College, London