xfs_repair v3.1.6 - Segmentation fault AND XFS internal error xfs_btree_check_sblock

[Richard Ems]

Hi all !

We have a XFS that started giving errors some days ago.
This is on an openSUSE 11.4 64 bit system.
The XFS is 12 TB big, 9.8 TB are used. Hardware RAID 6 on an Areca 1680 controller.

Mounting the XFS with ro,norecovery works almost always.

But xfs_repair crashes with a Segmentation fault. I tried both v3.1.4 from 
openSUSE 11.4 and xfs_repair v3.1.6 downloaded from the git repo.

Now after a reboot - the ___production___system completely freezed while running 
the last xfs_repair v3.1.6 !!! - the XFS got mounted rw, but just trying to 
touch a file generated the following error:


Oct 17 16:33:02 c3m kernel: [  794.628715] Filesystem "sdb1": XFS internal error xfs_btree_check_sblock at line 120 of file /usr/src/packages/BUILD/kernel-default-2.6.37.6/linux-2.6.37/fs/xfs/xfs_btree.c.  Caller 0xffffffffa0376cbe
Oct 17 16:33:02 c3m kernel: [  794.628718] 
Oct 17 16:33:02 c3m kernel: [  794.628722] Pid: 9066, comm: touch Not tainted 2.6.37.6-0.7-default #1
Oct 17 16:33:02 c3m kernel: [  794.628724] Call Trace:
Oct 17 16:33:02 c3m kernel: [  794.628737]  [<ffffffff81005819>] dump_trace+0x69/0x2e0
Oct 17 16:33:02 c3m kernel: [  794.628744]  [<ffffffff814ba5c3>] dump_stack+0x69/0x6f
Oct 17 16:33:02 c3m kernel: [  794.628776]  [<ffffffffa0376666>] xfs_btree_check_sblock+0x86/0x120 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.628864]  [<ffffffffa0376cbe>] xfs_btree_read_buf_block.clone.0+0x9e/0xc0 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.628947]  [<ffffffffa0378a3e>] xfs_btree_increment+0x1ee/0x290 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.629036]  [<ffffffffa038e522>] xfs_dialloc+0x5e2/0x900 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.629148]  [<ffffffffa0390bd5>] xfs_ialloc+0x75/0x6d0 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.629259]  [<ffffffffa03aaa25>] xfs_dir_ialloc+0x95/0x340 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.629409]  [<ffffffffa03adbb6>] xfs_create+0x406/0x6c0 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.629560]  [<ffffffffa03b9fdf>] xfs_vn_mknod+0xaf/0x1d0 [xfs]
Oct 17 16:33:02 c3m kernel: [  794.629717]  [<ffffffff811552c3>] vfs_create+0x113/0x190
Oct 17 16:33:02 c3m kernel: [  794.629724]  [<ffffffff81155c52>] do_last+0x572/0x600
Oct 17 16:33:02 c3m kernel: [  794.629730]  [<ffffffff81155e88>] do_filp_open+0x1a8/0x610
Oct 17 16:33:02 c3m kernel: [  794.629736]  [<ffffffff81147496>] do_sys_open+0x66/0x110
Oct 17 16:33:02 c3m kernel: [  794.629743]  [<ffffffff81002e4b>] system_call_fastpath+0x16/0x1b
Oct 17 16:33:02 c3m kernel: [  794.629754]  [<00007fbb0de57ce0>] 0x7fbb0de57ce0


The last lines before the " xfs_repair -n -P /dev/sdb1 " Segmentation fault where:

would clear forw/back pointers in block 0 for attributes in inode 4319273
bad attribute leaf magic # 0x250 for dir ino 4319273
problem with attribute contents in inode 4319273
would clear attr fork
bad nblocks 2 for inode 4319273, would reset to 1
bad anextents 1 for inode 4319273, would reset to 0
-bash: line 5:  6488 Segmentation fault      /opt/xfsprogs-3.1.6/sbin/xfs_repair -n -P /dev/sdb1

The complete " xfs_repair -n -P /dev/sdb1 " output file is 1.2 MB gzipped. If anyone wants 
to have a look at it please ask and I will send it as a private mail.


What can I do to recover/repair this XFS ? We have important data in there !

Many thanks,
Richard


-- 
Richard Ems       mail: Richard.Ems at Cape-Horn-Eng.com

Cape Horn Engineering S.L.
C/ Dr. J.J. Dómine 1, 5º piso
46011 Valencia
Tel : +34 96 3242923 / Fax 924
http://www.cape-horn-eng.com