[Top] [All Lists]

Re: XFS: Abysmal write performance because of excessive seeking (allocat

To: Stefan Ring <stefanrin@xxxxxxxxx>
Subject: Re: XFS: Abysmal write performance because of excessive seeking (allocation groups to blame?)
From: Christoph Hellwig <hch@xxxxxxxxxxxxx>
Date: Thu, 5 Apr 2012 17:37:40 -0400
Cc: xfs@xxxxxxxxxxx
In-reply-to: <CAAxjCEwBMbd0x7WQmFELM8JyFu6Kv_b+KDe3XFqJE6shfSAfyQ@xxxxxxxxxxxxxx>
References: <CAAxjCEwBMbd0x7WQmFELM8JyFu6Kv_b+KDe3XFqJE6shfSAfyQ@xxxxxxxxxxxxxx>
User-agent: Mutt/1.5.21 (2010-09-15)
Hi Stefan,

thanks for the detailed report.

The seekwatcher makes it very clear that XFS is spreading I/O over the
4 allocation groups, while ext4 isn't.   There's a couple of reasons why
XFS is doing that, including to max out multiple devices in a
multi-device setup, and not totally killing read speed.

Can you try a few mount options for me both all together and if you have
some time also individually.

 -o inode64

        This allows inodes to be close to data even for >1TB
        filesystems.  It's something we hope to make the default soon.

 -o filestreams

        This keeps data written in a single directory group together.
        Not sure your directories are large enough to really benefit
        from it, but it's worth a try.

 -o allocsize=4k

        This disables the agressive file preallocation we do in XFS,
        which sounds like it's not useful for your workload.

> I ran the tests with a current RHEL 6.2 kernel and also with a 3.3rc2
> kernel. Both of them exhibited the same behavior. The disk hardware
> used was a SmartArray p400 controller with 6x 10k rpm 300GB SAS disks
> in RAID 6. The server has plenty of RAM (64 GB).

For metadata intensive workloads like yours you would be much better
using a non-striping raid, e.g. concatentation and mirroring instead of
raid 5 or raid 6.  I know this has a cost in terms of "wasted" space,
but for IOPs bound workload the difference is dramatic.

P.s. please ignore Peter - he's made himself a name as not only beeing
technically incompetent but also extremly abrasive.  He is in no way
associated with the XFS development team.

<Prev in Thread] Current Thread [Next in Thread>