Last week one of my fileserver has rebooted due to power failure, and found
out there were corrupted files with the following error messages:
Jan 13 08:01:17 fs5-11 kernel: Filesystem "sd(8,7)": corrupt dinode
101980910, extent total = 522866323, nblocks = 3608. Unmo
unt and run xfs_repair.
Jan 13 08:01:17 fs5-11 kernel: 0x0: 49 4e 81 80 01 02 00 01 00 00 03 e9 00
00 03 e9
Jan 13 08:01:17 fs5-11 kernel: Filesystem "sd(8,7)": XFS internal error
xfs_iformat(1) at line 472 of file xfs_inode.c. Calle
r 0xe099535b
Jan 13 08:01:17 fs5-11 kernel: dca71c68 e099423f e09db2b4 00000001 dde26800
e09db292 000001d8 e099535b
Jan 13 08:01:17 fs5-11 kernel: e099535b 1f2a3320 1f2a4e93 00000000
00000000 00000000 00000000 dde26800
Jan 13 08:01:17 fs5-11 kernel: c6e03298 e099535b c6e03298 c386ae00
c386ae00 c6e033c4 00000001 00000000
Jan 13 08:01:17 fs5-11 kernel: Call Trace: [<e099423f>] xfs_iformat [xfs]
0x20f (0xdca71c6c))
Jan 13 08:01:17 fs5-11 kernel: [<e09db2b4>] .LC14 [xfs] 0x3d3 (0xdca71c70))
Jan 13 08:01:17 fs5-11 kernel: [<e09db292>] .LC14 [xfs] 0x3b1 (0xdca71c7c))
Jan 13 08:01:17 fs5-11 kernel: [<e099535b>] xfs_iread [xfs] 0xdb
(0xdca71c84))
Jan 13 08:01:17 fs5-11 kernel: [<e099535b>] xfs_iread [xfs] 0xdb
(0xdca71c88))
Jan 13 08:01:17 fs5-11 kernel: [<e099535b>] xfs_iread [xfs] 0xdb
(0xdca71cac))
Jan 13 08:01:17 fs5-11 kernel: [<e0992fb6>] xfs_iget_core [xfs] 0x1b6
(0xdca71ce8))
Jan 13 08:01:17 fs5-11 kernel: [<e099335d>] xfs_iget [xfs] 0x7d
(0xdca71d38))
Jan 13 08:01:17 fs5-11 kernel: [<e09a9f81>] xfs_dir_lookup_int [xfs] 0x61
(0xdca71d7c))
Jan 13 08:01:17 fs5-11 kernel: [<e09ae7be>] xfs_lookup [xfs] 0x3e
(0xdca71db8))
Jan 13 08:01:17 fs5-11 kernel: [<e09bb52f>] linvfs_lookup [xfs] 0x3f
(0xdca71dec))
Jan 13 08:01:17 fs5-11 kernel: [<c0156d70>] cached_lookup [kernel] 0x10
(0xdca71e10))
Jan 13 08:01:17 fs5-11 kernel: [<c0157e71>] lookup_hash [kernel] 0x91
(0xdca71e24))
Jan 13 08:01:17 fs5-11 kernel: [<c0157f09>] lookup_one_len [kernel] 0x59
(0xdca71e40))
Jan 13 08:01:17 fs5-11 kernel: [<e0a8a660>] nfsd_lookup [nfsd] 0x340
(0xdca71e64))
Jan 13 08:01:17 fs5-11 kernel: [<c011a5ef>] __wake_up [kernel] 0x4f
(0xdca71ed0))
Jan 13 08:01:17 fs5-11 kernel: [<e0a531a4>] svc_sock_enqueue [sunrpc] 0x184
(0xdca71ef8))
Jan 13 08:01:17 fs5-11 kernel: [<e0a53a43>] svc_udp_recvfrom [sunrpc] 0x2d3
(0xdca71f10))
Jan 13 08:01:17 fs5-11 kernel: [<e0a90418>] nfsd3_proc_lookup [nfsd] 0xd8
(0xdca71f38))
Jan 13 08:01:17 fs5-11 kernel: [<e0a99a8c>] nfsd_procedures3 [nfsd] 0x6c
(0xdca71f58))
Jan 13 08:01:17 fs5-11 kernel: [<e0a87567>] nfsd_dispatch [nfsd] 0xb7
(0xdca71f64))
Jan 13 08:01:17 fs5-11 kernel: [<e0a993d8>] nfsd_version3 [nfsd] 0x0
(0xdca71f7c))
Jan 13 08:01:17 fs5-11 kernel: [<e0a52e18>] svc_process_Rsmp_01d929dc
[sunrpc] 0x368 (0xdca71f80))
Jan 13 08:01:17 fs5-11 kernel: [<e0a99a8c>] nfsd_procedures3 [nfsd] 0x6c
(0xdca71f9c))
Jan 13 08:01:17 fs5-11 kernel: [<e0a993d8>] nfsd_version3 [nfsd] 0x0
(0xdca71fa0))
Jan 13 08:01:17 fs5-11 kernel: [<e0a993f8>] nfsd_program [nfsd] 0x0
(0xdca71fa4))
Jan 13 08:01:17 fs5-11 kernel: [<e0a8736a>] nfsd [nfsd] 0x1ca (0xdca71fc0))
Jan 13 08:01:17 fs5-11 kernel: [<e0a871a0>] nfsd [nfsd] 0x0 (0xdca71fe0))
Jan 13 08:01:17 fs5-11 kernel: [<c0107275>] kernel_thread_helper [kernel]
0x5 (0xdca71ff0))
Jan 13 08:01:17 fs5-11 kernel:
Jan 13 08:01:17 fs5-11 kernel: nfsd: non-standard errno: -990
Tried to run xfs_repair but failed with fatal error:
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
zero_log: head block 59683 tail block 59683
- scan filesystem freespace and inode maps...
- found root inode chunk
Phase 3 - for each AG...
- scan and clear agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
corrupt block 0 in directory inode 158139
will junk block
no . entry for directory 158139
no .. entry for directory 158139
corrupt block 3 in directory inode 1231523
will junk block
.
.
.
Phase 5 - rebuild AG headers and trees...
- reset superblock...
Phase 6 - check inode connectivity...
- resetting contents of realtime bitmap and summary inodes
- ensuring existence of lost+found directory
- traversing filesystem starting at / ...
corrupt block 8 in directory inode 838926683: junking block
rebuilding directory inode 838926683
corrupt block 5 in directory inode 170001259: junking block
bad hash table for directory inode 170001259 (no data entry): rebuilding
disconnected dir inode 2081119363, moving to lost+found
disconnected dir inode 2116550700, moving to lost+found
.
.
.
Phase 7 - verify and correct link counts...
resetting inode 158139 nlinks from 3 to 2
resetting inode 16967337 nlinks from 3 to 2
resetting inode 335545743 nlinks from 2806 to 2805
fatal error -- couldn't map inode 2133733748, err = 22
I found some other list posting with similar problem, did any fix being
release regarding this issue?
xfs_info as follow:
|