<html><head></head><body>I figured someone would fill in the gaps in my experience-assumptions. Excellent info, especially on those fs comparisons.<br>
-- <br>
Sent from my Android phone with K-9 Mail. Please excuse my brevity.<br><br><div class="gmail_quote">Dave Chinner <david@fromorbit.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre style="white-space: pre-wrap; word-wrap:break-word; font-family: sans-serif">On Fri, Oct 07, 2011 at 06:58:53AM -0700, Bryan J Smith wrote:<br />> [ Not really adding any technical meat, but just some past<br />> experience with XFS, plus Ext3 experience ]<br />> <br />> I remember running into this a long time ago when I was first<br />> playing with XFS for /tmp and /var (I was still a Linux/XFS noob<br />> at the time, not that I'm an expert today). I ran into the same<br />> case where both free block and inodes were still available<br />> (although similarly well utilized), and the median file size was<br />> around 1KiB. It was also in the case of many small files being<br />> written out in a short period.<br />> <br />> In my case, I didn't use the XFS debugger to get into the<br />> allocation of the extents (would have if I wasn't such a noob,<br />> good, discrete command to know, thanx!).<br />> <br />> Exte
nts are
outstanding for data and similar directories, ordering<br />> and placing large and small files to mitigate fragmentation. But<br />> in this case, and correct me if I'm wrong, it's really just a<br />> wasteful use for the extents approach, as the files typically fit<br />> in a single data block or two. <br /><br />And single blocks still an extent, so there's nothing "wasted" by<br />having a single block extent.<br /><br />....<br /><br />> I've used Ext3 with around 8 million files with a median size well<br />> under 4KiB (under 32GiB total). It works "well enough." I'm<br />> curious how Ext4 would do though. I think Ric Wheeler's team (at<br />> Red Hat) has done some benchmarks on 7+ figure file counts on Ext3<br />> and Ext4.<br /><br />And against XFS, too. In case you didn't realise, you're talking to<br />the person who ran a large number of those tests. ;)<br /><br />The results were ext4 is good for create/delete workloads up
to
2-4<br />threads and about 100k files per directory on a decent disk<br />subsystem (4000 iops). It's much better than ext3, and for those<br />workloads about 2x as fast as XFS at 1-2 threads. This pattern held<br />true as long as the disk subsystem could handle the number of iops<br />that ext4 threw at it. XFS performance came at a much, much lower<br />iops cost (think order of magnitude), so shoul dbe more consistent<br />on a wider range of storage hardware than ext4.<br /><br />However, XFS was about 3x faster on cold cache lookups than ext4, so<br />if you're workload is dominated by lookups, XFS is definitely the<br />faster filesystem to use even if creates/unlinks on ext4 are<br />faster..<br /><br />As soon as you have more parallelism than 2-4 threads or large<br />directories, XFS create/unlink speed surpasses ext4 by a large<br />amount - the best I got out of ext4 was ~80k creates a second, while<br />XFS topped 130k creates/s at 8 threads. And the lookup spe
ed<br
/>differential increase in XFS's favour at larger thread counts as<br />well.<br /><br />So it really depends on your workload as to which filesystem will<br />handle your small files best. Mail spools tend to have lots of<br />parallelism, which is why XFS works pretty well, even though it is<br />a small file workload.<br /><br />Cheers,<br /><br />Dave.<br />-- <br />Dave Chinner<br />david@fromorbit.com<br /></pre></blockquote></div></body></html>