Re: [BUG] Slab corruption during XFS writeback under memory pressure

[Calvin Owens <calvinowens@xxxxxx>]

On 07/18/2016 07:05 PM, Calvin Owens wrote:
> On 07/17/2016 11:02 PM, Dave Chinner wrote:
> > On Sun, Jul 17, 2016 at 10:00:03AM +1000, Dave Chinner wrote:
> > > On Fri, Jul 15, 2016 at 05:18:02PM -0700, Calvin Owens wrote:
> > > > Hello all,
> > > >
> > > > I've found a nasty source of slab corruption. Based on seeing similar symptoms
> > > > on boxes at Facebook, I suspect it's been around since at least 3.10.
> > > >
> > > > It only reproduces under memory pressure so far as I can tell: the issue seems
> > > > to be that XFS reclaims pages from buffers that are still in use by
> > > > scsi/block. I'm not sure which side the bug lies on, but I've only observed it
> > > > with XFS.
> > [....]
> > > But this indicates that the page is under writeback at this point,
> > > so that tends to indicate that the above freeing was incorrect.
> > >
> > > Hmmm - it's clear we've got direct reclaim involved here, and the
> > > suspicion of a dirty page that has had it's bufferheads cleared.
> > > Are there any other warnings in the log from XFS prior to kasan
> > > throwing the error?
> > Can you try the patch below?
> Thanks for getting this out so quickly :)
>
> So far so good: I booted Linus' tree as of this morning and reproduced the ASAN
> splat. After applying your patch I haven't triggered it.
>
> I'm a bit wary since it was hard to trigger reliably in the first place... so I
> lined up a few dozen boxes to run the test case overnight. I'll confirm in the
> morning (-0700) they look good.
All right, my testcase ran 2099 times overnight without triggering anything.

For the overnight tests, I booted the boxes with "mem=" to artificially limit 
RAM,
which makes my repro *much* more reliable (I feel silly for not thinking of that
in the first place). With that setup, I hit the ASAN splat 21 times in 98 runs 
on
vanilla 4.7-rc7. So I'm sold.

Tested-by: Calvin Owens <calvinowens@xxxxxx>

Again, really appreciate the quick response :)

Thanks,
Calvin

> Thanks,
> Calvin
>
> > -Dave.