Hi,<br><br>I am using XFS + 2.6.29 on my MySQL servers, they perform great. <br><br>I am testing XFS on SSD these days, due to the fact that FITRIM support of XFS was <br>shipped with Linux kernel 2.6.38 or newer, I tested XFS + 2.6.38 and XFS + 2.6.39, but<br>
it surprises me that the performance of XFS with these two versions of kernel drops so<br>much.<br><br>Here are the results of my tests with fio, all these two tests were taken on the same hardware<div>with same testing environment (except for different kernel version).<br>
<br>====== XFS + 2.6.29 ======<br><br># mount | grep /mnt/xfs<br>/dev/sdc1 on /mnt/xfs type xfs (rw,noatime,nodiratime,nobarrier,logbufs=8)<br># fio --filename=/mnt/xfs/test --direct=1 --rw=randrw --bs=16k --size=50G --numjobs=16 --runtime=120 --group_reporting --name=test --rwmixread=90 --thread --ioengine=psync<br>
test: (g=0): rw=randrw, bs=16K-16K/16K-16K, ioengine=psync, iodepth=1<br>...<br>test: (g=0): rw=randrw, bs=16K-16K/16K-16K, ioengine=psync, iodepth=1<br>fio 1.58<br>Starting 16 threads<br>test: Laying out IO file(s) (1 file(s) / 51200MB)<br>
Jobs: 16 (f=16): [mmmmmmmmmmmmmmmm] [100.0% done] [181.5M/21118K /s] [11.4K/1289 iops] [eta 00m:00s]<br>test: (groupid=0, jobs=16): err= 0: pid=8446<br>read : io=21312MB, bw=181862KB/s, iops=11366 , runt=120001msec<br>clat (usec): min=80 , max=146337 , avg=1369.72, stdev=1026.26<br>
lat (usec): min=81 , max=146338 , avg=1370.87, stdev=1026.27<br>bw (KB/s) : min= 6998, max=13600, per=6.26%, avg=11376.13, stdev=499.42<br>write: io=2369.4MB, bw=20218KB/s, iops=1263 , runt=120001msec<br>clat (usec): min=67 , max=145760 , avg=268.28, stdev=894.06<br>
lat (usec): min=67 , max=145761 , avg=269.46, stdev=894.09<br>bw (KB/s) : min= 509, max= 2166, per=6.26%, avg=1265.42, stdev=213.82<br>cpu : usr=11.09%, sys=44.83%, ctx=26015341, majf=0, minf=8396<br>IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%<br>
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%<br>complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%<br>issued r/w/d: total=1363980/151635/0, short=0/0/0<br>lat (usec): 100=0.11%, 250=5.85%, 500=3.79%, 750=0.32%, 1000=5.51%<br>
lat (msec): 2=80.06%, 4=1.26%, 10=3.07%, 20=0.01%, 50=0.01%<br>lat (msec): 100=0.01%, 250=0.01%<br><br>Run status group 0 (all jobs):<br>READ: io=21312MB, aggrb=181862KB/s, minb=186227KB/s, maxb=186227KB/s, mint=120001msec, maxt=120001msec<br>
WRITE: io=2369.4MB, aggrb=20217KB/s, minb=20703KB/s, maxb=20703KB/s, mint=120001msec, maxt=120001msec<br><br>Disk stats (read/write):<br>sdc: ios=1361926/151423, merge=0/0, ticks=1793432/27812, in_queue=1820240, util=99.99%<br>
<br><br><br><div><br></div><div>====== XFS + 2.6.39 ======</div><div><br># mount | grep /mnt/xfs<br>/dev/sdc1 on /mnt/xfs type xfs (rw,noatime,nodiratime,nobarrier,logbufs=8)<br># fio --filename=/mnt/xfs/test --direct=1 --rw=randrw --bs=16k --size=50G --numjobs=16 --runtime=120 --group_reporting --name=test --rwmixread=90 --thread --ioengine=psync<br>
test: (g=0): rw=randrw, bs=16K-16K/16K-16K, ioengine=psync, iodepth=1<br>...<br>test: (g=0): rw=randrw, bs=16K-16K/16K-16K, ioengine=psync, iodepth=1<br>fio 1.58<br>Starting 16 threads<br>test: Laying out IO file(s) (1 file(s) / 51200MB)<br>
Jobs: 16 (f=16): [mmmmmmmmmmmmmmmm] [100.0% done] [58416K/6680K /s] [3565 /407 iops] [eta 00m:00s]<br>test: (groupid=0, jobs=16): err= 0: pid=26902<br> read : io=6507.1MB, bw=55533KB/s, iops=3470 , runt=120004msec<br> clat (usec): min=155 , max=356038 , avg=4562.52, stdev=4748.18<br>
lat (usec): min=156 , max=356038 , avg=4562.69, stdev=4748.19<br> bw (KB/s) : min= 1309, max= 4864, per=6.26%, avg=3479.03, stdev=441.47<br> write: io=741760KB, bw=6181.2KB/s, iops=386 , runt=120004msec<br> clat (usec): min=71 , max=348236 , avg=390.11, stdev=3106.30<br>
lat (usec): min=71 , max=348236 , avg=390.31, stdev=3106.30<br> bw (KB/s) : min= 28, max= 921, per=6.29%, avg=389.02, stdev=114.68<br> cpu : usr=3.43%, sys=11.12%, ctx=21598477, majf=0, minf=7762<br>
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%<br>
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%<br> complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%<br> issued r/w/d: total=416508/46360/0, short=0/0/0<br>
lat (usec): 100=2.65%, 250=0.98%, 500=6.58%, 750=31.88%, 1000=0.27%<br> lat (msec): 2=0.08%, 4=0.23%, 10=55.04%, 20=1.76%, 50=0.49%<br> lat (msec): 100=0.02%, 250=0.01%, 500=0.01%<br><br>Run status group 0 (all jobs):<br>
READ: io=6507.1MB, aggrb=55532KB/s, minb=56865KB/s, maxb=56865KB/s, mint=120004msec, maxt=120004msec<br> WRITE: io=741760KB, aggrb=6181KB/s, minb=6329KB/s, maxb=6329KB/s, mint=120004msec, maxt=120004msec<br><br>Disk stats (read/write):<br>
sdc: ios=416285/46351, merge=0/1, ticks=108136/8768, in_queue=116368, util=93.60%<br><br><br></div><div>as the tests result shows, the IOPS of XFS + 2.6.29 is about 12600, but it drops to about 3900</div><div>with XFS + 2.6.39.</div>
<div><br></div><div>I tried different XFS format options and different mount options, but it did not help.</div><div><br></div><div>Any thought?</div><div><br>--<br>Xupeng Yun<br><a href="http://about.me/xupeng">http://about.me/xupeng</a></div>
</div>