xfs_repair Err 990

[Frank Hellmann <frank@xxxxxxxxxxxxx>]

Hi!

Filesystem is 950GB XFS partition on an external Fibre-Channel Array. 
xfs_progs is version 2.6.25. System is running gentoo linux with a 
vanilla 2.6.8.1 kernel. The system crashed due to a power-failure.

I ran into the infamous err 990 during xfs_repair:

...
Phase 5 - rebuild AG headers and trees...
        - reset superblock...
Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - ensuring existence of lost+found directory
        - traversing filesystem starting at / ...
rebuilding directory inode 128
corrupt dinode 2537480171, extent total = 1, nblocks = 0.  Unmount and 
run xfs_repair.
fatal error -- couldn't map inode 2537480171, err = 990

and stops there. The filesystem is mounting fine after the half finished 
run of xfs_repair, but accesses going to that inode (or others not 
repaired) make it barf (see attached dmesg.txt).

Anything I can do to fix this via xfs_db? Like hiding that part from the 
users until I have a chance to do the xfs_dump+mkfs.xfs+xfs_restore routine?

        Thanks. Cheers,
                        Frank...
--
--------------------------------------------------------------------------
Frank Hellmann          Optical Art GmbH           Waterloohain 7a
DI Supervisor           http://www.opticalart.de   22769 Hamburg
frank@xxxxxxxxxxxxx     Tel: ++49 40 5111051       Fax: ++49 40 43169199

 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c028df12>] linvfs_readdir+0x202/0x23b
 [<c0166e29>] vfs_lstat+0x1c/0x58
 [<c01791bf>] update_atime+0x92/0xd5
 [<c01674dd>] sys_lstat64+0x1b/0x39
 [<c0170df4>] vfs_readdir+0xb4/0xb6
 [<c01710d7>] filldir64+0x0/0xeb
 [<c0171263>] sys_getdents64+0xa1/0xab
 [<c01710d7>] filldir64+0x0/0xeb
 [<c0105117>] syscall_call+0x7/0xb
Filesystem "sdc1": corrupt dinode 763502840, extent total = 1, nblocks = 0.  
Unmount and run xfs_repair.
0x0: 49 4e 81 b6 01 02 00 01 00 00 03 e9 00 00 00 64
Filesystem "sdc1": XFS internal error xfs_iformat(1) at line 475 of file 
fs/xfs/xfs_inode.c.  Caller 0xc0264b3a
 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c0166e29>] vfs_lstat+0x1c/0x58
 [<c01674dd>] sys_lstat64+0x1b/0x39
 [<c0105117>] syscall_call+0x7/0xb
Filesystem "sdc1": corrupt dinode 3629094670, extent total = 1, nblocks = 0.  
Unmount and run xfs_repair.
0x0: 49 4e 81 b6 01 02 00 01 00 00 03 e9 00 00 00 64
Filesystem "sdc1": XFS internal error xfs_iformat(1) at line 475 of file 
fs/xfs/xfs_inode.c.  Caller 0xc0264b3a
 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c014d89f>] do_anonymous_page+0x130/0x1b7
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c0166e29>] vfs_lstat+0x1c/0x58
 [<c01674dd>] sys_lstat64+0x1b/0x39
 [<c0116cd7>] do_page_fault+0x0/0x56f
 [<c0105ba1>] error_code+0x2d/0x38
 [<c0105117>] syscall_call+0x7/0xb
Filesystem "sdc1": corrupt dinode 2537480171, extent total = 1, nblocks = 0.  
Unmount and run xfs_repair.
0x0: 49 4e 41 ed 01 02 00 02 00 00 03 e9 00 00 00 64
Filesystem "sdc1": XFS internal error xfs_iformat(1) at line 475 of file 
fs/xfs/xfs_inode.c.  Caller 0xc0264b3a
 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c028df12>] linvfs_readdir+0x202/0x23b
 [<c0166e29>] vfs_lstat+0x1c/0x58
 [<c01674dd>] sys_lstat64+0x1b/0x39
 [<c0170df4>] vfs_readdir+0xb4/0xb6
 [<c0127101>] update_process_times+0x45/0x51
 [<c01141f3>] smp_apic_timer_interrupt+0xe6/0x14e
 [<c0105117>] syscall_call+0x7/0xb
Filesystem "sdc1": corrupt dinode 3611258870, extent total = 1, nblocks = 0.  
Unmount and run xfs_repair.
0x0: 49 4e 81 ed 01 02 00 01 00 00 00 00 00 00 00 00
Filesystem "sdc1": XFS internal error xfs_iformat(1) at line 475 of file 
fs/xfs/xfs_inode.c.  Caller 0xc0264b3a
 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c0166e29>] vfs_lstat+0x1c/0x58
 [<c01674dd>] sys_lstat64+0x1b/0x39
 [<c0105117>] syscall_call+0x7/0xb
Filesystem "sdc1": corrupt dinode 2403297492, extent total = 1, nblocks = 0.  
Unmount and run xfs_repair.
0x0: 49 4e 41 ed 01 02 00 02 00 00 03 e9 00 00 00 64
Filesystem "sdc1": XFS internal error xfs_iformat(1) at line 475 of file 
fs/xfs/xfs_inode.c.  Caller 0xc0264b3a
 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c029fb14>] vsprintf+0x27/0x2b
 [<c02a0a2a>] copy_to_user+0x52/0x62
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c0166dd1>] vfs_stat+0x1f/0x5b
 [<c01674a4>] sys_stat64+0x1b/0x39
 [<c0106fb5>] math_state_restore+0x28/0x42
 [<c0105117>] syscall_call+0x7/0xb
Filesystem "sdc1": corrupt dinode 2403297492, extent total = 1, nblocks = 0.  
Unmount and run xfs_repair.
0x0: 49 4e 41 ed 01 02 00 02 00 00 03 e9 00 00 00 64
Filesystem "sdc1": XFS internal error xfs_iformat(1) at line 475 of file 
fs/xfs/xfs_inode.c.  Caller 0xc0264b3a
 [<c0263702>] xfs_iformat+0x2b5/0x5c2
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0264b3a>] xfs_iread+0x1cf/0x218
 [<c0261c7f>] xfs_iget_core+0x101/0x6ce
 [<c0178bf6>] iget_locked+0xbc/0xfd
 [<c02623a3>] xfs_iget+0x157/0x189
 [<c027e552>] xfs_dir_lookup_int+0xb4/0x12b
 [<c0283cad>] xfs_lookup+0x50/0x88
 [<c0290317>] linvfs_lookup+0x67/0x9f
 [<c016b2c1>] real_lookup+0xe1/0x104
 [<c016b582>] do_lookup+0x96/0xa1
 [<c016bd9e>] link_path_walk+0x811/0xfdf
 [<c024d6dd>] xfs_dir2_put_dirent64_direct+0x0/0x96
 [<c016c869>] path_lookup+0xad/0x1d2
 [<c016cb44>] __user_walk+0x49/0x7b
 [<c0166dd1>] vfs_stat+0x1f/0x5b
 [<c01674a4>] sys_stat64+0x1b/0x39
 [<c0105117>] syscall_call+0x7/0xb
XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1583 of file 
fs/xfs/xfs_alloc.c.  Caller 0xc0229136
 [<c0227e6f>] xfs_free_ag_extent+0x454/0x78a
 [<c0229136>] xfs_free_extent+0x100/0x125
 [<c0229136>] xfs_free_extent+0x100/0x125
 [<c0289280>] kmem_zone_alloc+0x50/0x96
 [<c027dae7>] xfs_trans_get_efd+0x38/0x46
 [<c023c2c6>] xfs_bmap_finish+0x13e/0x1d2
 [<c02654f3>] xfs_itruncate_finish+0x22a/0x452
 [<c0283c09>] xfs_inactive+0x504/0x558
 [<c0293ed9>] vn_rele+0xfd/0x119
 [<c02927fe>] linvfs_clear_inode+0x18/0x30
 [<c0177d6f>] clear_inode+0x9b/0xa7
 [<c0178e56>] generic_delete_inode+0x10a/0x13c
 [<c0179070>] iput+0x62/0x7c
 [<c016e309>] sys_unlink+0x13a/0x172
 [<c0171263>] sys_getdents64+0xa1/0xab
 [<c01710d7>] filldir64+0x0/0xeb
 [<c0105117>] syscall_call+0x7/0xb
xfs_force_shutdown(sdc1,0x8) called from line 4049 of file fs/xfs/xfs_bmap.c.  
Return address = 0xc0293420
Filesystem "sdc1": Corruption of in-memory data detected.  Shutting down 
filesystem: sdc1
Please umount the filesystem, and rectify the problem(s)