Playing around with NFS+XFS

[Federico Sevilla III <jijo@leathercollection.ph>]

Hi everyone,

By popular demand of some fellow members in the Philippine Linux Users'
Group (PLUG), I decided to show up at work at midnight when everyone else
was away, to mess around with the server's hard drives. The objective:
stress-test NFS+XFS and show the world it's more than ready for primetime.
;>

I'm using XFS (of course) with Linux kernel 2.4.9, with NFSv3 on both the
client and server. I have four IBM hard drives (yes, the dreaded IBM GXPs)
hosted by a 3ware Escalade 6400 in hardware RAID5.

I ran two tests.

The first test was to see if I could transfer a 4GB file. I created a
concatenated tarball that has a size 4891125760 bytes, which when I divide
by 1024 thrice is about 4.56GB. I didn't exactly "transfer" it, though.
On the client side I mounted the NFS exported volume, then did a "cat
foo.tar > /dev/null".

I think it did well to test the server's hard drives, NFS for reading, and
the wire speed. I initially did a plain-old-copy, but the client's hard
drive was limiting things and I wanted to push everything I could push on
the server's side. Comments on my methodology as far as this is concerned
are welcome.

The second test was aimed at over-all stressing, using bonnie++. I ran
bonnie++ with the following parameters:

# bonnie++ -d /opt/test/bonnie++ -s 1G -n 5:1M:1M:256 -u root

The mount options for NFS are "rsize=8192,wsize=8192". I think I can push
things further by bumping this up to 32768 since I'm using NFSv3, but
maybe I should sleep first. ;>

Anyway here are the results:

Version  1.01d      ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
Gusi-NFS-XFS     1G  4200  82  5937   7  1988   4  4037  76 10512   9 120.3   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP /sec %CP
Gusi-NFS-XFS  5:1:0    80   2   252   5    56   1   118   4   341   7 41   0

A more readable copy is in <http://jijo.leathercollection.ph/bonnie.html>.

I'm a definite newbie when it comes to this benchmarking business. I
prefer to read everybody else's statistics. Maybe someone has some other
decent stress tests to recommend?

 --> Jijo

--
Federico Sevilla III  :: jijo@leathercollection.ph
Network Administrator :: The Leather Collection, Inc.
GnuPG Key: <http://jijo.leathercollection.ph/jijo.gpg>