xfs
[Top] [All Lists]

Deleting files with extended attributes is dead slow

To: linux-xfs@xxxxxxxxxxx
Subject: Deleting files with extended attributes is dead slow
From: Bernd Schubert <bernd.schubert@xxxxxxxxxxxxxxxxxx>
Date: Fri, 12 Aug 2011 20:19:53 +0200
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110626 Iceowl/1.0b2 Icedove/3.1.11
Hi there,

with FhGFS we may use extended attributes to store file meta data and with ext3/ext4 that also works very well and the rate to create files and to write those EAs (create() + fsetxattr() is about 2.5 to 3 times faster than with a create() + write(). Size of those EA data is about 256 bytes depending on the number of storage stripes. However, with XFS using extended attributes is *extremely* slow. Here are some numbers with ext4 and xfs using a patched [1] bonnie++

schubert@fsdevel3 bonnie++-1.96>./bonnie++ -d /mnt/fstestXFS/ -s0 -n 
1:256:256:1 -r 0 -X
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Create------ --------Random Create--------
fsdevel3            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
          1:256:256   330   6 +++++ +++    47   1   469   9 +++++ +++    31   1
Latency               962ms      36us     140ms     866ms      28us     311ms
1.96,1.96,fsdevel3,1,1314378941,,,,,,,,,,,,,,1,256,256,,,330,6,+++++,+++,47,1,469,9,+++++,+++,31,1,,,,,,,962ms,36us,140ms,866ms,28us,311ms



schubert@fsdevel3 bonnie++-1.96>./bonnie++ -d /mnt/fstestEXT4/ -s0 -n 
100:256:256:10 -r 0 -X
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Create------ --------Random Create--------
fsdevel3            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
     100:256:256/10 21348  52 +++++ +++ 32878  61 25286  60 +++++ +++ 25873  53
Latency               746us    1926us    2553us     653us     118us   32250us
1.96,1.96,fsdevel3,1,1314379136,,,,,,,,,,,,,,100,256,256,,10,21348,52,+++++,+++,32878,61,25286,60,+++++,+++,25873,53,,,,,,,746us,1926us,2553us,653us,118us,32250us


NOTE: For ext4 I had to increase the number of files by factor 100 to get any sane number (it would only print '+++++' otherwise). Running the benchmark with the same numbers on xfs deleted in so slow delete numbers, that it probably still would be busy to delete files by tomorrow.


The xfs file system was formated with these parameters:
mkfs.xfs -f -i size=512 -i maxpct=90  -l lazy-count=1 -n size=64k 
/dev/mapper/vg0fsdev3-XFStest


ext4 was formated to have an inode size of 512B and to have a joural size of 400MB.

Both file systems are mounted with "nobarrier" (real FhGFS production systems usually have raid controllers with battery backups).


Any ideas why xfs is so slow with extended attributes? Without extended attributes, so skipping the "-X" option in our patched bonnie also results in slower Create numbers (about 4500 on xfs vs. 7500 on ext4) compared to ext4, but that is 'only' factor 1.67 and not 50 or more as with EAs.


Thanks,
Bernd


[1] Updated bonnie to support extended attributes:
https://bitbucket.org/aakef/bonnie/

<Prev in Thread] Current Thread [Next in Thread>