I went thru the same hardware config gymnastics (confounded by bad disks, a
bad controller, bad hotswap cages, and some SW config issues - see:
http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001067.html).
The 3ware info is partially reported here:
http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001134.html
and I believe I also sent a similar report to this list, but google doesn't
show it.
Here's the results of my bonnie tests on a similar system:
8x250GB WD SD series disks on a 3ware 9500S-8, on an IWILL 2x Opteron mobo,
4GB RAM, with XFS.
The XFS parameters are with a 64K stripe to match the RAID card; other params
more or less vanilla.
Below are some bonnie timing results with differing filesystems (1 run with
ext3, 3 with XFS with differnt file sizes - unwrap in an editor to compare in
columns:
XFS sand,
7000M,52315,99,104402,22,32242,8,32018,60,127998,20,435.5,0,80,1724,15,+++++,
+++, 3248, 16,1718,15,+++++,+++,750, 4
ext3 sand, 7000M,40682,91, 47732,23,25432,9,38027,72,179352,27,311.1,0,80,
416,99,+++++,+++,53250,100, 423,99,+++++,+++,560,58
XFS sand,
7000M,50040,98,106324,24,33046,8,31269,59,112240,17,416.0,0,80,1930,17,+++++,
+++, 3753, 17,1913,17,+++++,+++,449, 2
XFS sand,
15000M,51065,99,101659,24,26884,7,35344,69,141223,23,263.0,0,80,1666,14,
+++++,+++, 4565, 22,1700,16,+++++,+++,793, 4
From my reading (see URL above for resourcelist), XFS is quite bad for tiny
files - we use it for very large files (>GB size); using XFS for this would
generally be a bad thing. We do NOT get tremendous performance out of it;
but the performance is much better than with ext3 and the CPU usage is lower,
sometimes dramtically so. Real life experience with some benchmarks confirms
this - we get approximately ~ the same real life thruput as we do on a IBM
SP2 8way module with a direct attach disk.
We are now considering adding a local PVFS2 system to a small cluster for very
fast IO under MPI
On Monday 25 July 2005 18:11, Gaspar Bakos wrote:
> Dear all,
>
> The purpose of this email is twofold:
> - to share the results of the many tests I performed with
> a 3ware RAID card + RAID-5 + XFS, pushing for better file I/O,
> - and to initiate some brainstorming on what parameters can be tuned for
> getting a good performance out of this hardware under 2.6.* kernels.
>
> I started all these tests because the performance was quite poor, meaning
> that the write speed was slow, the read speed was barely acceptable, and
> the system load went very high (10.0) during bonnie++ tests.
> My questions are marked below with "Q".
<clipped>
|