xfs_metadump
Johannes Weydt
Johannes.Weydt at EMEA.NEC.COM
Fri Jun 12 07:49:24 CDT 2015
Hello SGI,
We experienced a I/O error on one of our XFS filesystems, xfs_repair -n advised to
"Please capture the filesystem metadata with xfs_metadump and
report it to xfs at oss.sgi.com<mailto:xfs at oss.sgi.com>."
before it ended with a Segmentation fault.
The metadata is almost 3GB big, how do you recommend to transfer the file?
Do you have an open FTP for us?
Also attached are some console outputs of first investigation attempts.
Log of xfs_repair -n (1st run)
root at daw1:~ # xfs_repair -n /dev/mapper/bindata_roma2-bindata_roma2
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- scan filesystem freespace and inode maps...
block (93,180061223-180061223) multiply claimed by bno space tree, state - 1
block (93,180062247-180062247) multiply claimed by bno space tree, state - 1
block (93,180067744-180067744) multiply claimed by bno space tree, state - 1
block (93,180068775-180068775) multiply claimed by bno space tree, state - 1
block (93,180072871-180072871) multiply claimed by bno space tree, state - 1
block (93,180074983-180074983) multiply claimed by bno space tree, state - 1
block (93,180075936-180075936) multiply claimed by bno space tree, state - 1
block (93,180092864-180092864) multiply claimed by bno space tree, state - 1
block (93,180146905-180146905) multiply claimed by bno space tree, state - 1
block (93,180256625-180256625) multiply claimed by bno space tree, state - 1
block (93,131740544-131740544) multiply claimed by cnt space tree, state - 2
block (93,131740722-131740722) multiply claimed by cnt space tree, state - 2
block (93,156891590-156891590) multiply claimed by cnt space tree, state - 2
agf_freeblks 20604555, counted 20607360 in ag 93
sb_icount 858240, counted 856960
sb_ifree 17344, counted 17717
sb_fdblocks 5921605452, counted 5920660936
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 7
- agno = 8
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- agno = 15
- agno = 16
- agno = 17
- agno = 18
- agno = 19
- agno = 20
- agno = 21
- agno = 22
- agno = 23
imap claims a free inode 99087474454 is in use, would correct imap and clear inode
- agno = 24
- agno = 25
- agno = 26
- agno = 27
- agno = 28
- agno = 29
- agno = 30
- agno = 31
- agno = 32
- agno = 33
- agno = 34
- agno = 35
- agno = 36
- agno = 37
- agno = 38
- agno = 39
- agno = 40
- agno = 41
- agno = 42
- agno = 43
- agno = 44
- agno = 45
- agno = 46
- agno = 47
- agno = 48
- agno = 49
- agno = 50
- agno = 51
- agno = 52
- agno = 53
- agno = 54
- agno = 55
- agno = 56
- agno = 57
- agno = 58
- agno = 59
- agno = 60
- agno = 61
- agno = 62
- agno = 63
- agno = 64
- agno = 65
- agno = 66
- agno = 67
- agno = 68
- agno = 69
- agno = 70
- agno = 71
- agno = 72
- agno = 73
- agno = 74
- agno = 75
- agno = 76
- agno = 77
- agno = 78
- agno = 79
- agno = 80
- agno = 81
- agno = 82
- agno = 83
- agno = 84
- agno = 85
- agno = 86
- agno = 87
- agno = 88
- agno = 89
- agno = 90
- agno = 91
- agno = 92
bad nblocks 6601 for inode 395205468183, would reset to 6602
bad nextents 5825 for inode 395205468183, would reset to 5826
bad back pointer in block 8389120 for directory inode 395205468183
- agno = 93
- agno = 94
- agno = 95
- agno = 96
- agno = 97
- agno = 98
- agno = 99
- agno = 100
- agno = 101
- agno = 102
- agno = 103
- agno = 104
- agno = 105
- agno = 106
- agno = 107
- agno = 108
- agno = 109
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 7
- agno = 21
- agno = 36
- agno = 39
- agno = 11
- agno = 48
- agno = 42
- agno = 9
- agno = 45
- agno = 31
- agno = 52
- agno = 65
- agno = 54
- agno = 74
- agno = 43
- agno = 51
- agno = 38
- agno = 63
- agno = 64
- agno = 67
- agno = 50
- agno = 69
- agno = 71
- agno = 46
- agno = 86
- agno = 97
- agno = 57
- agno = 59
- agno = 14
- agno = 60
- agno = 44
- agno = 70
- agno = 10
- agno = 49
- agno = 72
- agno = 12
- agno = 25
- agno = 27
- agno = 13
- agno = 76
- agno = 75
- agno = 77
- agno = 79
- agno = 29
- agno = 83
- agno = 84
- agno = 93
- agno = 87
- agno = 92
- agno = 88
- agno = 90
- agno = 91
- agno = 89
- agno = 56
- agno = 47
- agno = 96
- agno = 95
- agno = 99
- agno = 58
- agno = 104
- agno = 105
- agno = 101
- agno = 107
- agno = 61
- agno = 15
- agno = 62
- agno = 108
- agno = 66
- agno = 68
- agno = 33
- agno = 24
- agno = 26
- agno = 73
- agno = 80
- agno = 100
- agno = 30
- agno = 32
- agno = 17
- agno = 20
- agno = 40
- agno = 23
- agno = 22
- agno = 94
- agno = 28
- agno = 8
- agno = 103
- agno = 16
- agno = 81
- agno = 102
- agno = 85
- agno = 41
- agno = 98
- agno = 82
- agno = 35
- agno = 78
- agno = 106
- agno = 109
- agno = 34
- agno = 53
- agno = 37
- agno = 19
- agno = 55
- agno = 18
bad nblocks 6601 for inode 395205468183, would reset to 6602
bad nextents 5825 for inode 395205468183, would reset to 5826
entry "C3E_MAIN_FC_ROUT_25374.14582" at block 698 offset 1056 in directory inode 395205468183 references free inode 395272675342
would clear inode number in entry at offset 1056...
entry "C3E_MAIN_FC_ROUT_25374.14583" at block 698 offset 1096 in directory inode 395205468183 references free inode 395272675343
would clear inode number in entry at offset 1096...
entry "C3E_MAIN_FC_ROUT_25374.14584" at block 698 offset 1136 in directory inode 395205468183 references free inode 395272675344
would clear inode number in entry at offset 1136...
entry "C3E_MAIN_FC_ROUT_25374.14585" at block 698 offset 1176 in directory inode 395205468183 references free inode 395272675345
would clear inode number in entry at offset 1176...
entry "C3E_MAIN_FC_ROUT_25374.14586" at block 698 offset 1216 in directory inode 395205468183 references free inode 395272675346
would clear inode number in entry at offset 1216...
entry "C3E_MAIN_FC_ROUT_25374.14587" at block 698 offset 1256 in directory inode 395205468183 references free inode 395272675347
would clear inode number in entry at offset 1256...
entry "C3E_MAIN_FC_ROUT_25374.14588" at block 698 offset 1296 in directory inode 395205468183 references free inode 395272675348
would clear inode number in entry at offset 1296...
entry "C3E_MAIN_FC_ROUT_25374.14589" at block 698 offset 1336 in directory inode 395205468183 references free inode 395272675349
would clear inode number in entry at offset 1336...
entry "C3E_MAIN_FC_ROUT_25374.14590" at block 698 offset 1376 in directory inode 395205468183 references free inode 395272675350
would clear inode number in entry at offset 1376...
entry "C3E_MAIN_FC_ROUT_25374.14591" at block 698 offset 1416 in directory inode 395205468183 references free inode 395272675351
would clear inode number in entry at offset 1416...
entry "C3E_MAIN_FC_ROUT_25374.14592" at block 698 offset 1456 in directory inode 395205468183 references free inode 395272675352
would clear inode number in entry at offset 1456...
entry "C3E_MAIN_FC_ROUT_25374.14593" at block 698 offset 1496 in directory inode 395205468183 references free inode 395272675353
would clear inode number in entry at offset 1496...
entry "C3E_MAIN_FC_ROUT_25374.14594" at block 698 offset 1536 in directory inode 395205468183 references free inode 395272675354
would clear inode number in entry at offset 1536...
entry "C3E_MAIN_FC_ROUT_25374.14595" at block 698 offset 1576 in directory inode 395205468183 references free inode 395272675355
would clear inode number in entry at offset 1576...
entry "C3E_MAIN_FC_ROUT_25374.14596" at block 698 offset 1616 in directory inode 395205468183 references free inode 395272675356
would clear inode number in entry at offset 1616...
entry "C3E_MAIN_FC_ROUT_25374.14597" at block 698 offset 1656 in directory inode 395205468183 references free inode 395272675357
would clear inode number in entry at offset 1656...
entry "C3E_MAIN_FC_ROUT_25374.14598" at block 698 offset 1696 in directory inode 395205468183 references free inode 395272675358
would clear inode number in entry at offset 1696...
entry "C3E_MAIN_FC_ROUT_25374.14599" at block 698 offset 1736 in directory inode 395205468183 references free inode 395272675359
would clear inode number in entry at offset 1736...
entry "C3E_MAIN_FC_ROUT_25374.14600" at block 698 offset 1776 in directory inode 395205468183 references free inode 395272675360
would clear inode number in entry at offset 1776...
entry "C3E_MAIN_FC_ROUT_25374.14601" at block 698 offset 1816 in directory inode 395205468183 references free inode 395272675361
would clear inode number in entry at offset 1816...
entry "C3E_MAIN_FC_ROUT_25374.14602" at block 698 offset 1856 in directory inode 395205468183 references free inode 395272675362
would clear inode number in entry at offset 1856...
entry "C3E_MAIN_FC_ROUT_25374.14603" at block 698 offset 1896 in directory inode 395205468183 references free inode 395272675363
would clear inode number in entry at offset 1896...
entry "C3E_MAIN_FC_ROUT_25374.14604" at block 698 offset 1936 in directory inode 395205468183 references free inode 395272675364
would clear inode number in entry at offset 1936...
entry "C3E_MAIN_FC_ROUT_25374.14605" at block 698 offset 1976 in directory inode 395205468183 references free inode 395272675365
would clear inode number in entry at offset 1976...
entry "C3E_MAIN_FC_ROUT_25374.14606" at block 698 offset 2016 in directory inode 395205468183 references free inode 395272675366
would clear inode number in entry at offset 2016...
entry "C3E_MAIN_FC_ROUT_25374.14607" at block 698 offset 2056 in directory inode 395205468183 references free inode 395272675367
would clear inode number in entry at offset 2056...
entry "C3E_MAIN_FC_ROUT_25374.14608" at block 698 offset 2096 in directory inode 395205468183 references free inode 395272675368
would clear inode number in entry at offset 2096...
entry "C3E_MAIN_FC_ROUT_25374.14609" at block 698 offset 2136 in directory inode 395205468183 references free inode 395272675369
would clear inode number in entry at offset 2136...
entry "C3E_MAIN_FC_ROUT_25374.14610" at block 698 offset 2176 in directory inode 395205468183 references free inode 395272675370
would clear inode number in entry at offset 2176...
entry "C3E_MAIN_FC_ROUT_25374.14612" at block 698 offset 2216 in directory inode 395205468183 references free inode 395272675371
would clear inode number in entry at offset 2216...
entry "C3E_MAIN_FC_ROUT_25374.14611" at block 698 offset 2256 in directory inode 395205468183 references free inode 395272675372
would clear inode number in entry at offset 2256...
entry "C3E_MAIN_FC_ROUT_25374.14613" at block 698 offset 2296 in directory inode 395205468183 references free inode 395272675373
would clear inode number in entry at offset 2296...
entry "C3E_MAIN_FC_ROUT_25374.14614" at block 698 offset 2336 in directory inode 395205468183 references free inode 395272675374
would clear inode number in entry at offset 2336...
entry "C3E_MAIN_FC_ROUT_25374.14615" at block 698 offset 2376 in directory inode 395205468183 references free inode 395272675375
would clear inode number in entry at offset 2376...
entry "C3E_MAIN_FC_ROUT_25374.14616" at block 698 offset 2416 in directory inode 395205468183 references free inode 395272675376
would clear inode number in entry at offset 2416...
entry "C3E_MAIN_FC_ROUT_25374.14617" at block 698 offset 2456 in directory inode 395205468183 references free inode 395272675377
would clear inode number in entry at offset 2456...
entry "C3E_MAIN_FC_ROUT_26054.8978" at block 2124 offset 416 in directory inode 98814672640 references free inode 99087474454
would clear inode number in entry at offset 416...
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
entry "C3E_MAIN_FC_ROUT_26054.8978" in directory inode 98814672640 points to free inode 99087474454, would junk entry
corrupt dinode 395205468183, (btree extents). This is a bug.
Please capture the filesystem metadata with xfs_metadump and
report it to xfs at oss.sgi.com.
corrupt dinode 395205468183, (btree extents). This is a bug.
Please capture the filesystem metadata with xfs_metadump and
report it to xfs at oss.sgi.com.
corrupt dinode 395205468183, (btree extents). This is a bug.
Please capture the filesystem metadata with xfs_metadump and
report it to xfs at oss.sgi.com.
Segmentation fault (core dumped)
root at daw1:~ #
Log of xfs_repair -n (2nd run)
root at daw1:~ # xfs_repair -n /dev/mapper/bindata_roma2-bindata_roma2
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- scan filesystem freespace and inode maps...
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 7
- agno = 8
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- agno = 15
- agno = 16
- agno = 17
- agno = 18
- agno = 19
- agno = 20
- agno = 21
- agno = 22
- agno = 23
- agno = 24
- agno = 25
- agno = 26
- agno = 27
- agno = 28
- agno = 29
- agno = 30
- agno = 31
- agno = 32
- agno = 33
- agno = 34
- agno = 35
- agno = 36
- agno = 37
- agno = 38
- agno = 39
- agno = 40
- agno = 41
- agno = 42
- agno = 43
- agno = 44
- agno = 45
- agno = 46
- agno = 47
- agno = 48
- agno = 49
- agno = 50
- agno = 51
- agno = 52
- agno = 53
- agno = 54
- agno = 55
- agno = 56
- agno = 57
- agno = 58
- agno = 59
- agno = 60
- agno = 61
- agno = 62
- agno = 63
- agno = 64
- agno = 65
- agno = 66
- agno = 67
- agno = 68
- agno = 69
- agno = 70
- agno = 71
- agno = 72
- agno = 73
- agno = 74
- agno = 75
- agno = 76
- agno = 77
- agno = 78
- agno = 79
- agno = 80
- agno = 81
- agno = 82
- agno = 83
- agno = 84
- agno = 85
- agno = 86
- agno = 87
- agno = 88
- agno = 89
- agno = 90
- agno = 91
- agno = 92
entry count 592 too large in block 8389120 for directory inode 395205468183
- agno = 93
- agno = 94
- agno = 95
- agno = 96
- agno = 97
- agno = 98
- agno = 99
- agno = 100
- agno = 101
- agno = 102
- agno = 103
- agno = 104
- agno = 105
- agno = 106
- agno = 107
- agno = 108
- agno = 109
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 30
- agno = 67
- agno = 72
- agno = 77
- agno = 84
- agno = 7
- agno = 75
- agno = 40
- agno = 17
- agno = 29
- agno = 78
- agno = 28
- agno = 55
- agno = 81
- agno = 57
- agno = 82
- agno = 80
- agno = 58
- agno = 83
- agno = 85
- agno = 16
- agno = 88
- agno = 100
- agno = 103
- agno = 102
- agno = 107
- agno = 108
- agno = 62
- agno = 10
- agno = 32
- agno = 34
- agno = 26
- agno = 71
- agno = 73
- agno = 50
- agno = 59
- agno = 38
- agno = 86
- agno = 8
- agno = 36
- agno = 60
- agno = 61
- agno = 45
- agno = 94
- agno = 49
- agno = 18
- agno = 39
- agno = 9
- agno = 22
- agno = 97
- agno = 98
- agno = 11
- agno = 90
- agno = 24
- agno = 99
- agno = 19
- agno = 12
- agno = 63
- agno = 64
- agno = 15
- agno = 65
- agno = 66
- agno = 68
- agno = 69
- agno = 51
- agno = 70
- agno = 74
- agno = 54
- agno = 87
- agno = 89
- agno = 42
- agno = 91
- agno = 43
- agno = 92
- agno = 93
- agno = 25
- agno = 20
- agno = 52
- agno = 13
- agno = 95
- agno = 53
- agno = 41
- agno = 96
- agno = 21
- agno = 101
- agno = 46
- agno = 56
- agno = 104
- agno = 31
- agno = 109
- agno = 76
- agno = 37
- agno = 44
- agno = 23
- agno = 105
- agno = 33
- agno = 106
- agno = 35
- agno = 14
- agno = 47
- agno = 27
- agno = 79
- agno = 48
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
No modify flag set, skipping filesystem flush and exiting.
Between the two xfsrepairs we mounted, checked for the files and unmounted the filesystem again.
The check resulted in a lot of
ls: cannot access 5/C3E_MAIN_FC_ROUT_25374.14618: Structure needs cleaning
xfsprogs is installed in version:
root at daw1:/localtmp # rpm -qa |grep xfsprogs
xfsprogs-3.1.1-10.el6.x86_64
Can you advise something?
Best regards / Mit freundlichen Grüßen
Johannes Weydt
System Analyst, HPCE Division
Telefon: 069-8062-2376
Mobile: 01522-2851526
Fax: 069-8062-3267
NEC Deutschland GmbH, Prinzenallee 11, D-40549 Duesseldorf, Germany
Geschäftsführer: Michael Stückmann Handelsregister Düsseldorf HRB 579413
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20150612/4e966b1c/attachment-0001.html>
More information about the xfs
mailing list