xfs
[Top] [All Lists]

Re: XFS crashing system with general protection fault

To: Bruno Prémont <bonbons@xxxxxxxxxxxxxxxxx>
Subject: Re: XFS crashing system with general protection fault
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Fri, 6 Feb 2015 09:15:16 +1100
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <20150205151007.7c954c01@xxxxxxxxxxxxxxxx>
References: <20141224111403.54d7226b@xxxxxxxxxxxx> <20141228115127.GN24183@dastard> <20141229084452.615e1900@xxxxxxxxxxxxxxxx> <20150113081742.6c3a5823@xxxxxxxxxxxxxxxx> <20150205151007.7c954c01@xxxxxxxxxxxxxxxx>
User-agent: Mutt/1.5.21 (2010-09-15)
On Thu, Feb 05, 2015 at 03:10:07PM +0100, Bruno Prémont wrote:
> Hi Dave,
> 
> New crash, new trace, this time on 3.18.2.
> It looks like this time a NULL dereference happened prior to touched memory 
> poison being detected.
> 
> Once again it's during normal system operation (no mount/umount activity)

Can you rebuild the kernel with CONFIG_XFS_WARN=y and see if that
throws any interesting messages into logs?

However:

> [1900390.261491] 
> =============================================================================
> [1900390.272989] BUG task_struct (Tainted: G      D W     ): Poison 
> overwritten
> [1900390.283021] 
> -----------------------------------------------------------------------------
> [1900390.283021] 
> [1900390.297056] INFO: 0xffff880213d651b3-0xffff880213d651b3. First byte 0x6d 
> instead of 0x6b
> [1900390.309044] INFO: Slab 0xffffea00084f5800 objects=16 used=16 fp=0x       
>    (null) flags=0x8000000000004080
> [1900390.323087] INFO: Object 0xffff880213d64ba0 @offset=19360 
> fp=0xffff880213d61e40
> [1900390.323087] 
> [1900390.336988] Bytes b4 ffff880213d64b90: 60 2d d6 13 02 88 ff ff 5a 5a 5a 
> 5a 5a 5a 5a 5a  `-......ZZZZZZZZ
> [1900390.350988] Object ffff880213d64ba0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
> 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
> [1900390.364943] Object ffff880213d64bb0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
> 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
....
> [1900391.674636] Object ffff880213d651b0: 6b 6b 6b 6d 6b 6b 6b 6b 6b 6b 6b 6b 
> 6b 6b 6b 6b  kkkmkkkkkkkkkkkk
                                                     ^^

There's a single bit that has been flipped in the task_struct slab.
So more than just XFS is seeing memory corruption - this is in core
kernel structure slab caches. I'm not sure, either, how XFS could
cause corruption in this slab.

So, I'd be checking all the previous memory corruptions to see if
they are single bit errors, and if there is any pattern to the
addresses at which they occur. The above bit flip makes me think
"hardware issue" and everything else stems from that...

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>