xfs
[Top] [All Lists]

is xfs good if I have millions of files and thousands of hardlinks?

To: xfs@xxxxxxxxxxx
Subject: is xfs good if I have millions of files and thousands of hardlinks?
From: Tomasz Chmielewski <mangoo@xxxxxxxx>
Date: Tue, 19 Feb 2008 14:53:57 +0100
Sender: xfs-bounce@xxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.8) Gecko/20061110 Mandriva/1.5.0.8-1mdv2007.1 (2007.1) Thunderbird/1.5.0.8 Mnenhy/0.7.4.666
I have a ext3 filesystem with almost 200 million files (1.2 TB fs, ~65% full); most of the files are hardlinked multiple times, some of them are hardlinked thousands of times.

I described my problem yesterday on linux-fsdev list:

http://marc.info/?t=120333985100003


In general, because new files and hardlinks are being added all the time and the old ones are being removed, this leads to a very, very poor performance.


When I want to remove a lot of directories/files (which will be hardlinks, mostly), I see disk write speed is down to
50 kB/s - 200 kB/s (fifty - two hundred kilobytes/s) - this is the "bandwidth" used during the deletion.



Also, the filesystem is very fragmented ("dd if=/dev/zero of=some_file bs=64k" writes only about 1 MB/s).



Will xfs handle a large number of files, including lots of hardlinks, any better than ext3?




--
Tomasz Chmielewski
http://wpkg.org


<Prev in Thread] Current Thread [Next in Thread>