Removing files is not XFS's strong point, there is a known way to implement
a fix for this, but it involves major surgery, and it has never been attempted.
Basically you are looking at the most disk bound operation in XFS, the
remove operation can contain synchronous disk I/O - this is why your
cpu is not 100% busy during it.
I will dig out the description of what needs doing (which involves
intimate knowledge of xfs workings), in the meantime, you can make
things better with two configuration changes.
1. use a larger than default log size
mkfs -t xfs -f -l size=32768b /dev/xxx
this cannot be applied retro actively to existing filesystems.
2. use the following mount option
mount -o logbufs=4
or
mount -o logbufs=8
The should both help the situation, but will not totally fix it.
The reason ext2 is so much slower is that you keep modifying the same
buffers in memory without writing them to disk. With XFS you keep writing
new transactions into the log - and in the process of reconsuming log space
have to flush metadata that goes with it. The mkfs option makes the log
bigger, the mount option gives you more in memory buffers so you can have
more log writes going in parallel.
I have also found that multiple parallel removes of different trees will
go faster than a single one of the whole tree.
Steve
>
> Howdy.. I just converted my laptop's root partition to XFS; while
> doing some hacking I ended up doing a quick-and-dirty benchmark
> between XFS and ext2.
>
> The freedb database (www.freedb.org) is a 59MB .tar.bz2 that expands
> out into 1.1GB worth of little files, all 1-2K in size, mostly towards
> the 1K size. (Why they use the filesystem as the database and not use
> libdb or something is beyond me. Anyway.) So, I timed the untar and
> rm -rf times of this tree.
>
> The two machines involved are an IBM X20 laptop, P3-600, 392MB ram,
> 20GB IBM Travelstar 4200 RPM drive, UDMA mode3, xfs filesystem. On my
> desktop, which is an Athlon 700, VIA VT82Cxxx Apollo chipset, 512MB
> ram, tar file on an UATA/100 drive on the primary on-board interface,
> destination drive on a promise UATA/100 IDE controller with an IBM
> Deskstar 7200 RPM 45GB drive on it, with an ext2 filesystem:
>
> On my laptop (xfs),
>
> tar xjf ../freedb-complete-20010219.tar.bz2 138.88s user
> 207.17s system 27% cpu 21:17.84 total
> rm -rf db 1.45s user 130.99s system 9% cpu 24:24.66 total
>
> On my desktop (ext2),
>
> tar xjf ~vladimir/freedb-complete-20010219.tar.bz2 153.76s user
> 2772.26s system 98% cpu 49:28.47 total
> rm -rf db 0.16s user 73.39s system 94% cpu 1:18.24 total
>
> Now, I converted the filesystem on my desktop to XFS. Here's the same
> numbers again, this time for xfs.
>
> tar xjf ~vladimir/freedb-complete-20010219.tar.bz2 156.77s user
> 199.28s system 61% cpu 9:41.88 total
> rm -rf db 1.73s user 131.59s system 19% cpu 11:07.25 total
>
>
> So:
>
> System tar time cpu rm time cpu
> P3-600/4200rpm, xfs 21:17.84 27% 24:24.66 9%
> K7-700/7200rpm, xfs 9:41.88 61% 11:07.25 19%
> K7-700/7200rpm, ext2 49:28.47 98% 1:18.24 94%
>
> So, while xfs obviously kicks ext2's butt, I'm curious as to the cpu
> usage numbers of both the untar and the remove, and also of xfs's
> significantly slower remove times (even though its cpu usage was
> also significantly lower).. is there anything about xfs that
> causes removes to be significantly slower? (I just realized that
> I didn't time a 'sync' after the relevant removes though, so
> a good chunk of this could have even been buffered, maybe...)
>
> I realize that this was an ad-hoc benchmark, but the numbers seemed
> interesting nonetheless. (If anyone's interested in testing the same,
> the database is available at ftp://ftp.freedb.org/pub/freedb/ ).
>
> - Vlad
|