XFS: Abysmal write performance because of excessive seeking (allocation groups to blame?)

[Stefan Ring]

> What was the location of the KVM images you were copying?  Is it
> possible the source device simply slowed down?  Or network congestion if
> this was an NFS copy?

Piped via ssh from another host. No, everything was completely idle otherwise.

>> So I threw XFS back in, restarted the restore, and it went very
>> smoothly while still providing acceptable interactivity.
>
> It's nice to know XFS "saved the day" but I'm not so sure XFS deserves
> the credit here.  The EXT4 driver itself/alone shouldn't cause the lack
> of responsiveness behavior you saw.  I'm guessing something went wrong
> on the source side of these file copies, given your report of dropping
> to 30-40MB/s on the writeout.

Maybe it shouldn’t, but something sure did. And the circumstances seem
to point at ext4. Since the situation persisted for minutes after I
had stopped the transfer, it cannot possibly have been related to the
source.

I have a feeling that with appropriate vm.dirty_ratio tuning (and
probably related settings), I could have remedied this. But that’s
just one more thing I’d have to tinker with just to get to get
acceptable behavior out of this machine. I don’t mind if I don’t get
top-notch performance out of the box, but this is simply too much. I
don’t want to be expected to hand-tune every damn thing.

>> XFS is not a panacea (obviously), and it may be a bit slower in many
>> cases, and doesn’t seem to cope well with fragmented free space (which
>> is what this entire thread is really about),
>
> Did you retest fragmented freespace writes with the linear concat or
> RAID10?  If not you're drawing incorrect conclusions due to not having
> all the facts.

Yes, I did this. It performed very well. Only slightly slower than on
a completely empty file system.