XFS slows down on used partions with bonnie++

[Paul Schutte <paul@it.up.ac.za>]

Hi,

I have being playing around with bonnie++ from
http://www.coker.com.au/bonnie++/

I found an interesting thing.

When I run bonnie++ on a newly created XFS filesystem I get the
following results:

mkfs.xfs -f -l size=8192b /dev/sda7

meta-data=/dev/sda7              isize=256    agcount=19, agsize=262144
blks
data     =                       bsize=4096   blocks=4843589, imaxpct=25

         =                       sunit=0      swidth=0 blks, unwritten=0

naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=8192
realtime =none                   extsz=65536  blocks=0, rtextents=0

mount -o logbufs=8 /dev/sda7 /mnt

cd /mnt

/mnt#time bonnie++ -u root -s0 -b -n 10:100000:1000:1000

Version 1.02b       ------Sequential Create------ --------Random
Create--------
kendy2.up.ac.za     -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
10:100000:1000/1000   206   8   154   3  1463  20   192   7    49   1
1081  18
kendy2.up.ac.za,,,,,,,,,,,,,,10:100000:1000/1000,206,8,154,3,1463,20,192,7,49,1,

1081,18
0.300u 15.540s 6:35.00 4.0%     0+0k 0+0io 215pf+0w

/mnt#time bonnie++ -u root -s0 -b -n 10:100000:1000:1000

Version 1.02b       ------Sequential Create------ --------Random
Create--------
kendy2.up.ac.za     -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
10:100000:1000/1000   196   7    83   1  1215  23   191   8    49   1
1023  20
kendy2.up.ac.za,,,,,,,,,,,,,,10:100000:1000/1000,196,7,83,1,1215,23,191,8,49,1,1

023,20
0.370u 16.520s 7:31.92 3.7%     0+0k 0+0io 219pf+0w

I created the file system.
Run bonnie++ with the parameters as above.
I run bonnie++ immediately for a second time.
If you look at the sequential read field, you will see that it is nearly
half the amount of the first run.
According to this test XFS seems to lose sequential read speed as the
filesystem gets used.
You can umount and even reboot the machine and run bonnie++ again and
still get the slowdown phenomenon,
provided you mount the same filesystem again without mkfs'ing it.

I repeated this test on several other machines with the same result.
I also did it with other filesystems.

Here is the result with ext2:

Version 1.02b       ------Sequential Create------ --------Random
Create--------
kendy2.up.ac.za     -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
10:100000:1000/1000   142   2   142   2   585   3   150   3    46   0
430   2
kendy2.up.ac.za,,,,,,,,,,,,,,10:100000:1000/1000,142,2,142,2,585,3,150,3,46,0,43

0,2
0.240u 8.950s 7:56.63 1.9%      0+0k 0+0io 218pf+0w
/mnt#time bonnie++ -u root -s0 -b -n 10:100000:1000:1000
Version 1.02b       ------Sequential Create------ --------Random
Create--------
kendy2.up.ac.za     -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
10:100000:1000/1000   154   3   143   3   540   2   152   2    47   0
449   2
kendy2.up.ac.za,,,,,,,,,,,,,,10:100000:1000/1000,154,3,143,3,540,2,152,2,47,0,44

9,2
0.300u 9.080s 7:42.20 2.0%      0+0k 0+0io 220pf+0w

No slow down.
I can include results for reiserfs and JFS, but will add size to this
message without adding additional info
regarding this issue.


The machine is a Dell PE2550
1G RAM (I used mem=256M kernel param, otherwise everything runs from
cache)
4x18G 15k RPM seagate cheethas in RAID 10
2x1.133GHz P4 CPUs

Regards

Paul Schutte