xfs
[Top] [All Lists]

Re: [PATCH 1/3] xfs: don't shutdown log recovery on validation errors

To: Ben Myers <bpm@xxxxxxx>
Subject: Re: [PATCH 1/3] xfs: don't shutdown log recovery on validation errors
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Thu, 13 Jun 2013 12:08:27 +1000
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <20130613010441.GX20932@xxxxxxx>
References: <1371003548-4026-1-git-send-email-david@xxxxxxxxxxxxx> <1371003548-4026-2-git-send-email-david@xxxxxxxxxxxxx> <20130613010441.GX20932@xxxxxxx>
User-agent: Mutt/1.5.21 (2010-09-15)
On Wed, Jun 12, 2013 at 08:04:41PM -0500, Ben Myers wrote:
> Hey Dave,
> 
> On Wed, Jun 12, 2013 at 12:19:06PM +1000, Dave Chinner wrote:
> > From: Dave Chinner <dchinner@xxxxxxxxxx>
> > 
> > Unfortunately, we cannot guarantee that items logged multiple times
> > and replayed by log recovery do not take objects back in time. When
> > theya re taken back in time, the go into an intermediate state which
> > is corrupt, and hence verification that occurs on this intermediate
> > state causes log recovery to abort with a corruption shutdown.
> > 
> > Instead of causing a shutdown and unmountable filesystem, don't
> > verify post-recovery items before they are written to disk. This is
> > less than optimal, but there is no way to detect this issue for
> > non-CRC filesystems If log recovery successfully completes, this
> > will be undone and the object will be consistent by subsequent
> > transactions that are replayed, so in most cases we don't need to
> > take drastic action.
> > 
> > For CRC enabled filesystems, leave the verifiers in place - we need
> > to call them to recalculate the CRCs on the objects anyway. This
> > recovery problem canbe solved for such filesystems - we have a LSN
> > stamped in all metadata at writeback time that we can to determine
> > whether the item should be replayed or not. This is a separate piece
> > of work, so is not addressed by this patch.
> 
> Is there a test case for this one?  How are you reproducing this?

The test case was Dave Jones running sysrq-b on a hung test machine.
The machine would occasionally end up with a corrupt home directory.

http://oss.sgi.com/pipermail/xfs/2013-May/026759.html

Analysis from a metdadump provided by Dave:

http://oss.sgi.com/pipermail/xfs/2013-June/026965.html

And Cai also appeared to be hitting this after a crash on 3.10-rc4,
as it's giving exactly the same "verifier failed during log recovery"
stack trace:

http://oss.sgi.com/pipermail/xfs/2013-June/026889.html

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>