Consistent throughput challenge -- fragmentation?

[Brian Cain]

All,

I have been observing some odd behavior regarding write throughput to an
XFS partition (the baseline kernel version is 2.6.32.27).  I see
consistently high write throughput (close to the performance of the raw
block device) to the filesystem immediately after a mkfs, but after a few
test cycles, there is sporadic poor performance.

The test mechanism is like so:

[mkfs.xfs <blockdev>] (no flags/options, xfsprogs ver 3.1.1-0.1.36)
...
1. remove a previous test cycle's directory
2. create a new directory
3. open/write/close a small file (4kb) in this directory
4. open/read/close this same small file (by the local NFS server)
5. open[O_DIRECT]/write/write/write/.../close a large file (anywhere from
~100MB to 200GB)

Step #5 contains the high-throughput metrics which becomes an order of
magnitude worse several test cycles after a mkfs.  Omitting steps 1-3 does
not show the poor performance behavior.

Can anyone provide any suggestions as to an explanation for the behavior or
a way to mitigate it?  Running xfs_fsr didn't seem to improve the results.

I'm happy to share benchmarks, specific results data, or describe the
hardware being used for the measurements if it's helpful.

-- 
-Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20130225/58f21dbb/attachment.html>