help xfs filesystem crash

[Joe Landman]

On 01/05/2013 08:47 AM, 符永涛 wrote:
> Dear xfs experts,
> 
> We're running glusterfs over top of xfs and recently we have
> encountered xfs filesystem crash two times on two different servers.
> Can you kindly help to give me some insight of how to debug this kind
> of failure?
> Bellow is xfs filesystem crash messages:
> server1:
> Dec 28 16:55:01 localhost kernel: XFS (dm-0): xfs_iunlink_remove:
> xfs_inotobp() returned error 22.
> Dec 28 16:55:01 localhost kernel: XFS (dm-0): xfs_inactive: xfs_ifree
> returned error 22
> Dec 28 16:55:01 localhost kernel: XFS (dm-0):
> xfs_do_force_shutdown(0x1) called from line 1184 of file
> fs/xfs/xfs_vnodeops.c. Return address = 0xffffffffa029f20a
> Dec 28 16:55:01 localhost kernel: XFS (dm-0): I/O Error Detected.
> Shutting down filesystem
> Dec 28 16:55:01 localhost kernel: XFS (dm-0): Please umount the
> filesystem and rectify the problem(s)
> Dec 28 16:55:20 localhost kernel: XFS (dm-0): xfs_log_force: error 5 returned.

xfs sitting atop dm here.  This looks like a dm issue, possibly the dm
LV going away, or going offline.  I'd start with tracing at the device
level upwards to the LV, see if this is still alive and well behaved