No subject
Thu Oct 6 05:08:24 CDT 2011
performance of 'ext4' on this workload, and that =ABExt4 can be up
20-50x times than XFS when data is also being written as well
(e.g. untarring kernel tarballs).=BB only when both data and
metadata are written to RAM by 'ext4'.
One can spend a lot of time changing parameters, as in using
'delaylog' or 'nobarrier' etc.
I have tried with my favourite rather "tighter" flusher
parameters, some comparisons that I find interesting:
----------------------------------------------------------------
# egrep ' (/tmp|/tmp/(ext4|xfs))' /proc/mounts; sysctl vm | egrep '=5F=
(bytes|centisecs)' | sort
none /tmp tmpfs rw 0 0
/dev/sdd3 /tmp/ext4 ext4 rw,barrier=3D1,data=3Dordered 0 0
/dev/sdd8 /tmp/xfs xfs rw,nouuid,attr2,inode64,logbsize=3D256k,sunit=3D=
8,swidth=3D8,noquota 0 0
vm.dirty=5Fbackground=5Fbytes =3D 900000000
vm.dirty=5Fbytes =3D 100000
vm.dirty=5Fexpire=5Fcentisecs =3D 200
vm.dirty=5Fwriteback=5Fcentisecs =3D 100
# (cd /tmp/ext4; rm -rf linux-2.6.32; sync; time tar -x -f /tmp/linux-=
2.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
real 0m6.776s
user 0m0.107s
sys 0m1.260s
Dirty: 1776 kB
Writeback: 0 kB
real 0m0.231s
user 0m0.000s
sys 0m0.197s
----------------------------------------------------------------
# (cd /tmp/xfs; rm -rf linux-2.6.32; sync; time tar -x -f /tmp/linux-2=
.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
real 2m25.805s
user 0m0.135s
sys 0m1.812s
Dirty: 2372 kB
Writeback: 84 kB
real 0m1.683s
user 0m0.000s
sys 0m0.196s
----------------------------------------------------------------
That's a bit of a surprise, because time to completion on both
when the flusher parameters allowed writing entirely to memory
for both with 'eatmydata tar' were the same. It looks like that
when flushing 'xfs' still does a fair bit of implicit metadata
commits, as switching off barriers shows:
----------------------------------------------------------------
# mount -o remount,barrier=3D0 /dev/sdd8 /tmp/ext4
# (cd /tmp/ext4; rm -rf linux-2.6.32; sync; time tar -x -f /tmp/linux-=
2.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
real 0m7.388s
user 0m0.127s
sys 0m1.235s
Dirty: 508 kB
Writeback: 0 kB
real 0m0.243s
user 0m0.000s
sys 0m0.199s
----------------------------------------------------------------
# mount -o remount,nobarrier /dev/sdd3 /tmp/xfs
# (cd /tmp/xfs; rm -rf linux-2.6.32; sync; time tar -x -f /tmp/linux-2=
.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
real 0m31.047s
user 0m0.124s
sys 0m1.880s
Dirty: 2324 kB
Writeback: 24 kB
real 0m0.269s
user 0m0.000s
sys 0m0.195s
----------------------------------------------------------------
While it seems likely 'ext4' runs headlong without commits on
either metadata or data ('ext4' and 'ext3' in effect have a
rather loose 'delaylog'). XFS however seems to be a bit at a
disadvantage though as with 'nobarrier' and 'eatmydata tar' the
time to completion should be the same. The partition for XFS is
on inner tracks, but that does not make that much of a
difference.
Also compare with 'ext4' using 'eatmydata tar' with no barriers
and using 'star' with no barrier and also 'data=3Dwriteback':
----------------------------------------------------------------
base# umount /tmp/ext4; mount -t ext4 -o defaults,barrier=3D0,data=3D=
writeback /dev/sdd3 /tmp/ext4
base# (cd /tmp/ext4; rm -rf linux-2.6.32; sync; time tar -x -f /tmp/=
linux-2.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
real 0m6.158s
user 0m0.123s
sys 0m1.233s
Dirty: 1704 kB
Writeback: 0 kB
real 0m0.247s
user 0m0.001s
sys 0m0.194s
----------------------------------------------------------------
base# (cd /tmp/ext4; rm -rf linux-2.6.32; sync; time star -x -f /tmp=
/linux-2.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
star: 37343 blocks + 0 bytes (total of 382392320 bytes =3D 373430.00k).=
real 0m32.101s
user 0m0.196s
sys 0m1.718s
Dirty: 24 kB
Writeback: 48 kB
real 0m0.217s
user 0m0.000s
sys 0m0.193s
----------------------------------------------------------------
Finally here is on XFS, with 'delaylog', on a system with a
3.x kernel and a rather fast (especially on small random writes)
SSD drive (and my usual tighter flusher parameters):
----------------------------------------------------------------
# uname -a
Linux.ty.sabi.co.UK 3.0.0-15-generic #26~lucid1-Ubuntu SMP Wed Jan 25 1=
5:37:10 UTC 2012 x86=5F64 GNU/Linux
# egrep ' (/tmp|/tmp/(ext4|xfs))' /proc/mounts; sysctl -a 2>/dev/null =
| egrep '=5F(bytes|centisecs)' | sort
none /tmp tmpfs rw,relatime,size=3D1024000k 0 0
/dev/sda6 /tmp/xfs xfs rw,noatime,nodiratime,attr2,delaylog,discard,ino=
de64,logbsize=3D256k,sunit=3D16,swidth=3D8192,noquota 0 0
/dev/sda3 /tmp/ext4 ext4 rw,nodiratime,relatime,errors=3Dremount-ro,use=
r=5Fxattr,acl,barrier=3D1,data=3Dordered,discard 0 0
fs.xfs.age=5Fbuffer=5Fcentisecs =3D 1500
fs.xfs.filestream=5Fcentisecs =3D 3000
fs.xfs.xfsbufd=5Fcentisecs =3D 100
fs.xfs.xfssyncd=5Fcentisecs =3D 3000
vm.dirty=5Fbackground=5Fbytes =3D 900000000
vm.dirty=5Fbytes =3D 100000000
vm.dirty=5Fexpire=5Fcentisecs =3D 200
vm.dirty=5Fwriteback=5Fcentisecs =3D 100
----------------------------------------------------------------
# (cd /tmp/xfs; rm -rf linux-2.6.32; sync; time tar -x -f /tmp/linux-2=
.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
real 0m5.148s
user 0m0.300s
sys 0m2.876s
Dirty: 50052 kB
Writeback: 0 kB
WritebackTmp: 0 kB
real 0m0.784s
user 0m0.000s
sys 0m0.100s
----------------------------------------------------------------
# (cd /tmp/xfs; rm -rf linux-2.6.32; sync; time star -x -f /tmp/linux-=
2.6.32.tar; egrep 'Dirty|Writeback' /proc/meminfo; time sync)
star: 37343 blocks + 0 bytes (total of 382392320 bytes =3D 373430.00k).=
real 6m21.946s
user 0m0.808s
sys 0m11.321s
Dirty: 0 kB
Writeback: 0 kB
WritebackTmp: 0 kB
real 0m0.097s
user 0m0.000s
sys 0m0.044s
----------------------------------------------------------------
The effect of 'delaylog' is pretty obvious there.
The numbers above with their wide variation depending on changes
in the level of safety requested amply demonstrate that it takes
the skills of a propagandist or a buffoon to boast about the
"performance" of 'delaylog' and comparisons with 'ext4' without
prominently mentioning the big safety tradeoffs involved.
More information about the xfs
mailing list