xfs
[Top] [All Lists]

Re: xfs umount with i/o error hang/memory corruption

To: Bob Mastors <bob.mastors@xxxxxxxxxxxxx>
Subject: Re: xfs umount with i/o error hang/memory corruption
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Sat, 5 Apr 2014 08:57:12 +1100
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <CALjwKZCsBbZ27DE4m=-jN7yWPCqn=QxNgtZZpUcdFvfCPvH5tw@xxxxxxxxxxxxxx>
References: <CALjwKZAJ-R8dS13Rsj3+K3hM9p0z08qvi4ZVTYbDWKT1Biu=-Q@xxxxxxxxxxxxxx> <20140404212016.GY17603@dastard> <CALjwKZCsBbZ27DE4m=-jN7yWPCqn=QxNgtZZpUcdFvfCPvH5tw@xxxxxxxxxxxxxx>
User-agent: Mutt/1.5.21 (2010-09-15)
On Fri, Apr 04, 2014 at 03:40:38PM -0600, Bob Mastors wrote:
> On Fri, Apr 4, 2014 at 3:20 PM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
> 
> > On Fri, Apr 04, 2014 at 12:15:23PM -0600, Bob Mastors wrote:
> > > Greetings,
> > >
> > > I am new to xfs and am running into a problem
> > > and would appreciate any guidance on how to proceed.
> > >
> > > After an i/o error from the block device that xfs is using,
> > > an umount results in a message like:
> > > [  370.636473] XFS (sdx): Log I/O Error Detected.  Shutting down
> > filesystem
> > > [  370.644073] XFS (h           ïïïh"h          ïïïH#h          ïïïbsg):
> > > Please umount the filesystem and rectify the problem(s)
> > > Note the garbage on the previous line which suggests memory corruption.
> > > About half the time I get the garbled log message. About half the time
> > > umount hangs.
> >
> > I got an email about this last night with a different trigger - thin
> > provisioning failing log IO in the unmount path. I know what the
> > problem is, I just don't have a fix for it yet.
> >
> > To confirm it's the same problem, can you post the entirity of the
> > dmesg where the error occurs?
> >
> Here is the console output from putting a BUG() call  in
> xfs_do_force_shutdown
> after the "Please umount ..." message.
> Let me know if want something else.
> 
> I don't know how much help I can be on this, but I can run more tests
> or try out fixes.

I'll let you know when I have a fix for you to try - probably early
next week as it's Saturday morning right now...

> [  129.548644] XFS (sdb): metadata I/O error: block 0x19022 ("xlog_iodone")
> error 19 numblks 64
> [  129.618739] XFS (sdb): xfs_do_force_shutdown(0x2) called from line 1170
> of file fs/xfs/xfs_log.c.  Return address = 0xffffffffa04c8001
> [  129.699768] XFS (sdb): Log I/O Error Detected.  Shutting down filesystem
> [  129.699924] XFS (sdb): Unable to update superblock counters. Freespace
> may not be correct on next mount.
> [  129.699927] XFS (sdb): xfs_log_force: error 5 returned.
> [  129.716031] XFS (På/): Please umount the filesystem and rectify the

Yup, that is identical to the bug report I got from the dm-thinp
folks. Thanks for the quick followup!

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>