XFS: Abysmal write performance because of excessive seeking (allocation groups to blame?)

[Stefan Ring]

> It seems the 1078 is simply not that quick with anything but pure
> striping.  Hardware RAID10 write performance appears only about 50%
> faster than RAID6.  The RAID6 speed is roughly 1/3rd of the RAID0 speed.
>  So exporting the individual drives as I previously mentioned and using
> mdraid6 should yield at least  a 3x improvement, assuming your CPUs
> aren't already loaded down.

Whatever the problem with the controller may be, it behaves quite
nicely usually. It seems clear though, that, regardless of the storage
technology, it cannot be a good idea to schedule tiny blocks in the
order that XFS schedules them in my case.

This:
AG0 *   *   *
AG1  *   *   *
AG2   *   *   *
AG3    *   *   *

cannot be better than this:

AG0 ***
AG1    ***
AG2       ***
AG3          ***

Yes, in theory, a good cache controller should be able to sort this
out. But at least this particular controller is not able to do so and
could use a little help. Also, a single consumer-grade drive is
certainly not helped by this write ordering.