xfs
[Top] [All Lists]

Re: kernels 3.4 slower due to allocation workqueue

To: Yann Dupont <Yann.Dupont@xxxxxxxxxxxxxx>
Subject: Re: kernels 3.4 slower due to allocation workqueue
From: Mark Tinguely <tinguely@xxxxxxx>
Date: Mon, 15 Apr 2013 08:45:16 -0500
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <516BCACE.1040900@xxxxxxxxxxxxxx>
References: <516BCACE.1040900@xxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120122 Thunderbird/9.0
On 04/15/13 04:39, Yann Dupont wrote:
Hello,
last week we received new machines (DELL R720xd) for an extension of our
ceph cluster.
(64 Gb ram, 2x Xeon E5-2650, PERC H710P (really LSI MEGARAID), and 12x3
TB disks + 2SSD (not used as cachecade))

I was doing test on the raid card with kernel 3.4.38 to try to find what
I can get of this beast with RAID5, when I noticed an unusual slow
values on compilebench. The difference is very visible on the initial
create tests (can detail more if needed).

I finally observed that ONLY 3.4 kernels exhibit that behaviour ;
3.3.xxx and before are OK, 3.5.xxx and later are back to good values.

I bisected the problem to this commit

c999a223c2f0d31c64ef7379814cea1378b2b800 is the first bad commit
commit c999a223c2f0d31c64ef7379814cea1378b2b800
Author: Dave Chinner <dchinner@xxxxxxxxxx>
Date: Thu Mar 22 05:15:07 2012 +0000

xfs: introduce an allocation workqueue

I understand this regression is not a bug, and probably just a corner
case of the new code, that was certainly corrected after during 3.5
development (didn't tried to bisect this one, maybe dave know what is
the corrective patch ?)

The problem is that 3.4 is the last long-term kernel for the moment, and
it's unfortunate it shows this regression.

Maybe a backport of the fix (if this backport is possible AND not very
intrusive) could be a good idea ?

Cheers,


Here are the allocation worker changes.

The biggest performance commit should be aa292847, which limits the callers to the worker.

commit 3b876c8f2a361ceeed3fed894980c69066f903a0
Author: Jeff Liu <jeff.liu@xxxxxxxxxx>
Date:   Thu Jun 7 15:44:32 2012 +0800

    xfs: fix debug_object WARN at xfs_alloc_vextent()

commit aa292847b9fc6e187547110de833a7d3131bbddf
Author: Dave Chinner <dchinner@xxxxxxxxxx>
Date:   Thu Jul 12 07:40:43 2012 +1000

    xfs: don't defer metadata allocation to the workqueue

 commit 2455881c0b52f87be539c4c7deab1afff4d8a560
Author: Dave Chinner <dchinner@xxxxxxxxxx>
Date:   Fri Oct 5 11:06:58 2012 +1000

    xfs: introduce XFS_BMAPI_STACK_SWITCH

commit e04426b9202bccd4cfcbc70b2fa2aeca1c86d8f5
Author: Dave Chinner <dchinner@xxxxxxxxxx>
Date:   Fri Oct 5 11:06:59 2012 +1000

    xfs: move allocation stack switch up to xfs_bmapi_allocate

commit 9e96fe6df44425b69ed89f6ac20352cec1f127d7
Author: Brian Foster <bfoster@xxxxxxxxxx>
Date:   Thu Jan 17 13:11:29 2013 -0500

    xfs: pull up stack_switch check into xfs_bmapi_write


The last 3 patches address an AGF buffer hang with the allocation worker.

--Mark.

<Prev in Thread] Current Thread [Next in Thread>