XFS use within multi-threaded apps

[Angelo McComis]

Stewart, and others:

On Tue, Oct 19, 2010 at 12:24 AM, Stewart Smith <stewart at flamingspork.com>wrote:

> On Mon, 18 Oct 2010 09:42:04 -0400, Angelo McComis <angelo at mccomis.com>
> wrote:
> > I have a use case where I'd like to forward the use of XFS. This is for
> > large (multi-GB, say anywhere from 5GB to 300GB) individual files, such
> as
> > what you'd see under a database's data file / tablespace.
>
> The general advice from not only those of us who hack on database
> systems for a living (and hobby), but those that also run it in
> production on more systems than you'll ever be able to count is this for
> database system performance tuning (i.e. after making your SQL not
> completely nuts)
>
> Step 1) Use XFS.
>
> Nothing, and I do mean nothing comes close to reliability and consistent
> performance.
>
> We've seen various benchmarks where X was faster.... most of the
> time. Then suddenly your filesystem takes a mutex for 15 seconds and
> you're database performance goes down the crapper.
>
>
I have been running iozone benchmarks, both head to head ext3 versus XFS,
single LUN, default mkfs options, etc.  And the short answer is XFS wins
hands down on writes and random writes.  Ext3 wins a couple of the other
tests, but not by nearly the margin that XFS wins on the other ones.  That
was the KB/sec tests. The IOPS test was even more telling, and showed XFS
winning by orders of magnitudes on a few tests, and being close or a tie on
the ones that it didn't win.  I took two SAN luns and ran local FS versus
SAN-presented (this is FC, attached to IBM DS8k storage), and ran the tests
there.  When done with the head to head tests, I concatenated the LUNS to
make a RAID 0 / simple 2-stripe set, and ran the tests some more.

I can't say that the numbers make a lot of sense, since it's a 4gbit FC
connection


>  > My database vendor (who, coincidentally markets their own filesystems
> and
> > operating systems) says that there are certain problems under XFS with
> > specific mention of corruption issues, if a single root or the metadata
> > become corrupted, the entire filesystem is gone, and it has performance
> > issues on a multi-threaded workload, caused by the single root filesystem
> > for metadata becoming a bottleneck.
>
> XFS has anything but performance problems on multithreaded
> workloads. It is *the* best of the Linux filesystems
> (actually... possibly any file system anywhere) for multithreaded
> IO. You can either benchmark it or go and read the source - check out
> the direct IO codepaths and what locks get taken (or rather, what locks
> aren't taken).
>
> Generally speaking, most DBMSs don't do much filesystem metadata
> operations, the most common being extending the data file. So what you
> really care about is multithreaded direct IO performance, scalability
> and reliability.
>
> > This feedback from the vendor is surely taken with a grain of salt as
> they
> > have marketing motivations of their own product to consider.
>
> If the vendor is who I suspect, and the filesystem being pushed is
> starting with two letters down the alphabet than XFS... I
> wouldn't. While a great file system for a number of applications, it is
> nowhere near ready for big database IO loads - to the extent that last I
> heard it still wasn't being recommended for the various DBs I care about
> (at least by the DB support guys).
>

Well -  I mentioned it above. Their current recommendation for Linux is to
stick with ext3... and for big file/big IO operations, switch to ext4. And
we had a meeting wherein we had a discussion that went like "well, ext3 has
problems whenever the kernel journal thread wakes up to flush under heavy
I/O, and ext4
is not available to us..."  and Dave Chinner on an earlier post regarding
the maturity level of ext4 and its present status.

I have been able to schedule a meeting with the folks at my vendor, on the
database software side. Aside from the questions I have, points to make,
etc., I'm curious if there's anything else, based on anyone here's input, I
should be asking them? This is a pretty grand opportunity to sit down and
grill them.

Thanks to everyone who participates on this list - you are all a great
resource and a perfect example of what the open source community is all
about.

Regards,
Angelo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20101020/19b9586f/attachment.htm>