xfs
[Top] [All Lists]

xfs_metadump

To: "xfs@xxxxxxxxxxx" <xfs@xxxxxxxxxxx>
Subject: xfs_metadump
From: Johannes Weydt <Johannes.Weydt@xxxxxxxxxxxx>
Date: Fri, 12 Jun 2015 12:49:24 +0000
Accept-language: en-US
Cc: Frühauf Andreas <Andreas.Fruehauf@xxxxxx>, "Reiter Manuel" <manuel.reiter@xxxxxx>, "'dwdos@xxxxxxxxxxxx'" <dwdos@xxxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
Thread-index: AdClCjN4ZipKDoF6QiW9/DRltdfHuw==
Thread-topic: xfs_metadump

Hello SGI,

 

We experienced a I/O error on one of our XFS filesystems, xfs_repair –n advised to

“Please capture the filesystem metadata with xfs_metadump and

report it to xfs@xxxxxxxxxxx.”

before it ended with a Segmentation fault.

 

The metadata is almost 3GB big, how do you recommend to transfer the file?

Do you have an open FTP for us?

 

Also attached are some console outputs of first investigation attempts.

 

Log of xfs_repair –n (1st run)

root@daw1:~ # xfs_repair -n /dev/mapper/bindata_roma2-bindata_roma2

Phase 1 - find and verify superblock...

Phase 2 - using internal log

        - scan filesystem freespace and inode maps...

block (93,180061223-180061223) multiply claimed by bno space tree, state - 1

block (93,180062247-180062247) multiply claimed by bno space tree, state - 1

block (93,180067744-180067744) multiply claimed by bno space tree, state - 1

block (93,180068775-180068775) multiply claimed by bno space tree, state - 1

block (93,180072871-180072871) multiply claimed by bno space tree, state - 1

block (93,180074983-180074983) multiply claimed by bno space tree, state - 1

block (93,180075936-180075936) multiply claimed by bno space tree, state - 1

block (93,180092864-180092864) multiply claimed by bno space tree, state - 1

block (93,180146905-180146905) multiply claimed by bno space tree, state - 1

block (93,180256625-180256625) multiply claimed by bno space tree, state - 1

block (93,131740544-131740544) multiply claimed by cnt space tree, state - 2

block (93,131740722-131740722) multiply claimed by cnt space tree, state - 2

block (93,156891590-156891590) multiply claimed by cnt space tree, state - 2

agf_freeblks 20604555, counted 20607360 in ag 93

sb_icount 858240, counted 856960

sb_ifree 17344, counted 17717

sb_fdblocks 5921605452, counted 5920660936

        - found root inode chunk

Phase 3 - for each AG...

        - scan (but don't clear) agi unlinked lists...

        - process known inodes and perform inode discovery...

        - agno = 0

        - agno = 1

        - agno = 2

        - agno = 3

        - agno = 4

        - agno = 5

        - agno = 6

        - agno = 7

        - agno = 8

        - agno = 9

        - agno = 10

        - agno = 11

        - agno = 12

        - agno = 13

        - agno = 14

        - agno = 15

        - agno = 16

        - agno = 17

        - agno = 18

        - agno = 19

        - agno = 20

        - agno = 21

        - agno = 22

        - agno = 23

imap claims a free inode 99087474454 is in use, would correct imap and clear inode

        - agno = 24

        - agno = 25

        - agno = 26

        - agno = 27

        - agno = 28

        - agno = 29

        - agno = 30

        - agno = 31

        - agno = 32

        - agno = 33

        - agno = 34

        - agno = 35

        - agno = 36

        - agno = 37

        - agno = 38

        - agno = 39

        - agno = 40

        - agno = 41

        - agno = 42

        - agno = 43

        - agno = 44

        - agno = 45

        - agno = 46

        - agno = 47

        - agno = 48

        - agno = 49

        - agno = 50

        - agno = 51

        - agno = 52

        - agno = 53

        - agno = 54

        - agno = 55

        - agno = 56

        - agno = 57

        - agno = 58

        - agno = 59

        - agno = 60

        - agno = 61

        - agno = 62

        - agno = 63

        - agno = 64

        - agno = 65

        - agno = 66

        - agno = 67

        - agno = 68

        - agno = 69

        - agno = 70

        - agno = 71

        - agno = 72

        - agno = 73

        - agno = 74

        - agno = 75

        - agno = 76

        - agno = 77

        - agno = 78

        - agno = 79

        - agno = 80

        - agno = 81

        - agno = 82

        - agno = 83

        - agno = 84

        - agno = 85

        - agno = 86

        - agno = 87

        - agno = 88

        - agno = 89

        - agno = 90

        - agno = 91

        - agno = 92

bad nblocks 6601 for inode 395205468183, would reset to 6602

bad nextents 5825 for inode 395205468183, would reset to 5826

bad back pointer in block 8389120 for directory inode 395205468183

        - agno = 93

        - agno = 94

        - agno = 95

        - agno = 96

        - agno = 97

        - agno = 98

        - agno = 99

        - agno = 100

        - agno = 101

        - agno = 102

        - agno = 103

        - agno = 104

        - agno = 105

        - agno = 106

        - agno = 107

        - agno = 108

        - agno = 109

        - process newly discovered inodes...

Phase 4 - check for duplicate blocks...

        - setting up duplicate extent list...

        - check for inodes claiming duplicate blocks...

        - agno = 0

        - agno = 1

        - agno = 2

        - agno = 3

        - agno = 4

        - agno = 5

        - agno = 6

        - agno = 7

        - agno = 21

        - agno = 36

        - agno = 39

        - agno = 11

        - agno = 48

        - agno = 42

        - agno = 9

        - agno = 45

        - agno = 31

        - agno = 52

        - agno = 65

        - agno = 54

        - agno = 74

        - agno = 43

        - agno = 51

        - agno = 38

        - agno = 63

        - agno = 64

        - agno = 67

        - agno = 50

        - agno = 69

        - agno = 71

        - agno = 46

        - agno = 86

        - agno = 97

        - agno = 57

        - agno = 59

        - agno = 14

        - agno = 60

        - agno = 44

        - agno = 70

        - agno = 10

        - agno = 49

        - agno = 72

        - agno = 12

        - agno = 25

        - agno = 27

        - agno = 13

        - agno = 76

        - agno = 75

        - agno = 77

        - agno = 79

        - agno = 29

        - agno = 83

        - agno = 84

        - agno = 93

        - agno = 87

        - agno = 92

        - agno = 88

        - agno = 90

        - agno = 91

        - agno = 89

        - agno = 56

        - agno = 47

        - agno = 96

        - agno = 95

        - agno = 99

        - agno = 58

        - agno = 104

        - agno = 105

        - agno = 101

        - agno = 107

        - agno = 61

        - agno = 15

        - agno = 62

        - agno = 108

        - agno = 66

        - agno = 68

        - agno = 33

        - agno = 24

        - agno = 26

        - agno = 73

        - agno = 80

        - agno = 100

        - agno = 30

        - agno = 32

        - agno = 17

        - agno = 20

        - agno = 40

        - agno = 23

        - agno = 22

        - agno = 94

        - agno = 28

        - agno = 8

        - agno = 103

        - agno = 16

        - agno = 81

        - agno = 102

        - agno = 85

        - agno = 41

        - agno = 98

        - agno = 82

        - agno = 35

        - agno = 78

        - agno = 106

        - agno = 109

        - agno = 34

        - agno = 53

        - agno = 37

        - agno = 19

        - agno = 55

        - agno = 18

bad nblocks 6601 for inode 395205468183, would reset to 6602

bad nextents 5825 for inode 395205468183, would reset to 5826

entry "C3E_MAIN_FC_ROUT_25374.14582" at block 698 offset 1056 in directory inode 395205468183 references free inode 395272675342

        would clear inode number in entry at offset 1056...

entry "C3E_MAIN_FC_ROUT_25374.14583" at block 698 offset 1096 in directory inode 395205468183 references free inode 395272675343

        would clear inode number in entry at offset 1096...

entry "C3E_MAIN_FC_ROUT_25374.14584" at block 698 offset 1136 in directory inode 395205468183 references free inode 395272675344

        would clear inode number in entry at offset 1136...

entry "C3E_MAIN_FC_ROUT_25374.14585" at block 698 offset 1176 in directory inode 395205468183 references free inode 395272675345

        would clear inode number in entry at offset 1176...

entry "C3E_MAIN_FC_ROUT_25374.14586" at block 698 offset 1216 in directory inode 395205468183 references free inode 395272675346

        would clear inode number in entry at offset 1216...

entry "C3E_MAIN_FC_ROUT_25374.14587" at block 698 offset 1256 in directory inode 395205468183 references free inode 395272675347

        would clear inode number in entry at offset 1256...

entry "C3E_MAIN_FC_ROUT_25374.14588" at block 698 offset 1296 in directory inode 395205468183 references free inode 395272675348

        would clear inode number in entry at offset 1296...

entry "C3E_MAIN_FC_ROUT_25374.14589" at block 698 offset 1336 in directory inode 395205468183 references free inode 395272675349

        would clear inode number in entry at offset 1336...

entry "C3E_MAIN_FC_ROUT_25374.14590" at block 698 offset 1376 in directory inode 395205468183 references free inode 395272675350

        would clear inode number in entry at offset 1376...

entry "C3E_MAIN_FC_ROUT_25374.14591" at block 698 offset 1416 in directory inode 395205468183 references free inode 395272675351

        would clear inode number in entry at offset 1416...

entry "C3E_MAIN_FC_ROUT_25374.14592" at block 698 offset 1456 in directory inode 395205468183 references free inode 395272675352

        would clear inode number in entry at offset 1456...

entry "C3E_MAIN_FC_ROUT_25374.14593" at block 698 offset 1496 in directory inode 395205468183 references free inode 395272675353

        would clear inode number in entry at offset 1496...

entry "C3E_MAIN_FC_ROUT_25374.14594" at block 698 offset 1536 in directory inode 395205468183 references free inode 395272675354

        would clear inode number in entry at offset 1536...

entry "C3E_MAIN_FC_ROUT_25374.14595" at block 698 offset 1576 in directory inode 395205468183 references free inode 395272675355

        would clear inode number in entry at offset 1576...

entry "C3E_MAIN_FC_ROUT_25374.14596" at block 698 offset 1616 in directory inode 395205468183 references free inode 395272675356

        would clear inode number in entry at offset 1616...

entry "C3E_MAIN_FC_ROUT_25374.14597" at block 698 offset 1656 in directory inode 395205468183 references free inode 395272675357

        would clear inode number in entry at offset 1656...

entry "C3E_MAIN_FC_ROUT_25374.14598" at block 698 offset 1696 in directory inode 395205468183 references free inode 395272675358

        would clear inode number in entry at offset 1696...

entry "C3E_MAIN_FC_ROUT_25374.14599" at block 698 offset 1736 in directory inode 395205468183 references free inode 395272675359

        would clear inode number in entry at offset 1736...

entry "C3E_MAIN_FC_ROUT_25374.14600" at block 698 offset 1776 in directory inode 395205468183 references free inode 395272675360

        would clear inode number in entry at offset 1776...

entry "C3E_MAIN_FC_ROUT_25374.14601" at block 698 offset 1816 in directory inode 395205468183 references free inode 395272675361

        would clear inode number in entry at offset 1816...

entry "C3E_MAIN_FC_ROUT_25374.14602" at block 698 offset 1856 in directory inode 395205468183 references free inode 395272675362

        would clear inode number in entry at offset 1856...

entry "C3E_MAIN_FC_ROUT_25374.14603" at block 698 offset 1896 in directory inode 395205468183 references free inode 395272675363

        would clear inode number in entry at offset 1896...

entry "C3E_MAIN_FC_ROUT_25374.14604" at block 698 offset 1936 in directory inode 395205468183 references free inode 395272675364

        would clear inode number in entry at offset 1936...

entry "C3E_MAIN_FC_ROUT_25374.14605" at block 698 offset 1976 in directory inode 395205468183 references free inode 395272675365

        would clear inode number in entry at offset 1976...

entry "C3E_MAIN_FC_ROUT_25374.14606" at block 698 offset 2016 in directory inode 395205468183 references free inode 395272675366

        would clear inode number in entry at offset 2016...

entry "C3E_MAIN_FC_ROUT_25374.14607" at block 698 offset 2056 in directory inode 395205468183 references free inode 395272675367

        would clear inode number in entry at offset 2056...

entry "C3E_MAIN_FC_ROUT_25374.14608" at block 698 offset 2096 in directory inode 395205468183 references free inode 395272675368

        would clear inode number in entry at offset 2096...

entry "C3E_MAIN_FC_ROUT_25374.14609" at block 698 offset 2136 in directory inode 395205468183 references free inode 395272675369

        would clear inode number in entry at offset 2136...

entry "C3E_MAIN_FC_ROUT_25374.14610" at block 698 offset 2176 in directory inode 395205468183 references free inode 395272675370

        would clear inode number in entry at offset 2176...

entry "C3E_MAIN_FC_ROUT_25374.14612" at block 698 offset 2216 in directory inode 395205468183 references free inode 395272675371

        would clear inode number in entry at offset 2216...

entry "C3E_MAIN_FC_ROUT_25374.14611" at block 698 offset 2256 in directory inode 395205468183 references free inode 395272675372

        would clear inode number in entry at offset 2256...

entry "C3E_MAIN_FC_ROUT_25374.14613" at block 698 offset 2296 in directory inode 395205468183 references free inode 395272675373

        would clear inode number in entry at offset 2296...

entry "C3E_MAIN_FC_ROUT_25374.14614" at block 698 offset 2336 in directory inode 395205468183 references free inode 395272675374

        would clear inode number in entry at offset 2336...

entry "C3E_MAIN_FC_ROUT_25374.14615" at block 698 offset 2376 in directory inode 395205468183 references free inode 395272675375

        would clear inode number in entry at offset 2376...

entry "C3E_MAIN_FC_ROUT_25374.14616" at block 698 offset 2416 in directory inode 395205468183 references free inode 395272675376

        would clear inode number in entry at offset 2416...

entry "C3E_MAIN_FC_ROUT_25374.14617" at block 698 offset 2456 in directory inode 395205468183 references free inode 395272675377

        would clear inode number in entry at offset 2456...

entry "C3E_MAIN_FC_ROUT_26054.8978" at block 2124 offset 416 in directory inode 98814672640 references free inode 99087474454

        would clear inode number in entry at offset 416...

No modify flag set, skipping phase 5

Phase 6 - check inode connectivity...

        - traversing filesystem ...

entry "C3E_MAIN_FC_ROUT_26054.8978" in directory inode 98814672640 points to free inode 99087474454, would junk entry

corrupt dinode 395205468183, (btree extents).  This is a bug.

Please capture the filesystem metadata with xfs_metadump and

report it to xfs@xxxxxxxxxxxx

corrupt dinode 395205468183, (btree extents).  This is a bug.

Please capture the filesystem metadata with xfs_metadump and

report it to xfs@xxxxxxxxxxxx

corrupt dinode 395205468183, (btree extents).  This is a bug.

Please capture the filesystem metadata with xfs_metadump and

report it to xfs@xxxxxxxxxxxx

Segmentation fault (core dumped)

root@daw1:~ #

 

Log of xfs_repair –n (2nd run)

root@daw1:~ # xfs_repair -n /dev/mapper/bindata_roma2-bindata_roma2

Phase 1 - find and verify superblock...

Phase 2 - using internal log

        - scan filesystem freespace and inode maps...

        - found root inode chunk

Phase 3 - for each AG...

        - scan (but don't clear) agi unlinked lists...

        - process known inodes and perform inode discovery...

        - agno = 0

        - agno = 1

        - agno = 2

        - agno = 3

        - agno = 4

        - agno = 5

        - agno = 6

        - agno = 7

        - agno = 8

        - agno = 9

        - agno = 10

        - agno = 11

        - agno = 12

        - agno = 13

        - agno = 14

        - agno = 15

        - agno = 16

        - agno = 17

        - agno = 18

        - agno = 19

        - agno = 20

        - agno = 21

        - agno = 22

        - agno = 23

        - agno = 24

        - agno = 25

        - agno = 26

        - agno = 27

        - agno = 28

        - agno = 29

        - agno = 30

        - agno = 31

        - agno = 32

        - agno = 33

        - agno = 34

        - agno = 35

        - agno = 36

        - agno = 37

        - agno = 38

        - agno = 39

        - agno = 40

        - agno = 41

        - agno = 42

        - agno = 43

        - agno = 44

        - agno = 45

        - agno = 46

        - agno = 47

        - agno = 48

        - agno = 49

        - agno = 50

        - agno = 51

        - agno = 52

        - agno = 53

        - agno = 54

        - agno = 55

        - agno = 56

        - agno = 57

        - agno = 58

        - agno = 59

        - agno = 60

        - agno = 61

        - agno = 62

        - agno = 63

        - agno = 64

        - agno = 65

        - agno = 66

        - agno = 67

        - agno = 68

        - agno = 69

        - agno = 70

        - agno = 71

        - agno = 72

        - agno = 73

        - agno = 74

        - agno = 75

        - agno = 76

        - agno = 77

        - agno = 78

        - agno = 79

        - agno = 80

        - agno = 81

        - agno = 82

        - agno = 83

        - agno = 84

        - agno = 85

        - agno = 86

        - agno = 87

        - agno = 88

        - agno = 89

        - agno = 90

        - agno = 91

        - agno = 92

entry count 592 too large in block 8389120 for directory inode 395205468183

        - agno = 93

        - agno = 94

        - agno = 95

        - agno = 96

        - agno = 97

        - agno = 98

        - agno = 99

        - agno = 100

        - agno = 101

        - agno = 102

        - agno = 103

        - agno = 104

        - agno = 105

        - agno = 106

        - agno = 107

        - agno = 108

        - agno = 109

        - process newly discovered inodes...

Phase 4 - check for duplicate blocks...

        - setting up duplicate extent list...

        - check for inodes claiming duplicate blocks...

        - agno = 0

        - agno = 1

        - agno = 2

        - agno = 3

        - agno = 4

        - agno = 5

        - agno = 6

        - agno = 30

        - agno = 67

        - agno = 72

        - agno = 77

        - agno = 84

        - agno = 7

        - agno = 75

        - agno = 40

        - agno = 17

        - agno = 29

        - agno = 78

        - agno = 28

        - agno = 55

        - agno = 81

        - agno = 57

        - agno = 82

        - agno = 80

        - agno = 58

        - agno = 83

        - agno = 85

        - agno = 16

        - agno = 88

        - agno = 100

        - agno = 103

        - agno = 102

        - agno = 107

        - agno = 108

        - agno = 62

        - agno = 10

        - agno = 32

        - agno = 34

        - agno = 26

        - agno = 71

        - agno = 73

        - agno = 50

        - agno = 59

        - agno = 38

        - agno = 86

        - agno = 8

        - agno = 36

        - agno = 60

        - agno = 61

        - agno = 45

        - agno = 94

        - agno = 49

        - agno = 18

        - agno = 39

        - agno = 9

        - agno = 22

        - agno = 97

        - agno = 98

        - agno = 11

        - agno = 90

        - agno = 24

        - agno = 99

        - agno = 19

        - agno = 12

        - agno = 63

        - agno = 64

        - agno = 15

        - agno = 65

        - agno = 66

        - agno = 68

        - agno = 69

        - agno = 51

        - agno = 70

        - agno = 74

        - agno = 54

        - agno = 87

        - agno = 89

        - agno = 42

        - agno = 91

        - agno = 43

        - agno = 92

        - agno = 93

        - agno = 25

        - agno = 20

        - agno = 52

        - agno = 13

        - agno = 95

        - agno = 53

        - agno = 41

        - agno = 96

        - agno = 21

        - agno = 101

        - agno = 46

        - agno = 56

        - agno = 104

        - agno = 31

        - agno = 109

        - agno = 76

        - agno = 37

        - agno = 44

        - agno = 23

        - agno = 105

        - agno = 33

        - agno = 106

        - agno = 35

        - agno = 14

        - agno = 47

        - agno = 27

        - agno = 79

        - agno = 48

No modify flag set, skipping phase 5

Phase 6 - check inode connectivity...

        - traversing filesystem ...

        - traversal finished ...

        - moving disconnected inodes to lost+found ...

Phase 7 - verify link counts...

No modify flag set, skipping filesystem flush and exiting.

 

Between the two xfsrepairs we mounted, checked for the files and unmounted the filesystem again.

The check resulted in a lot of

ls: cannot access 5/C3E_MAIN_FC_ROUT_25374.14618: Structure needs cleaning

 

xfsprogs is installed in version:

root@daw1:/localtmp # rpm -qa |grep xfsprogs

xfsprogs-3.1.1-10.el6.x86_64

 

Can you advise something?

 

Best regards / Mit freundlichen Grüßen

 

Johannes Weydt

System Analyst, HPCE Division

Telefon: 069-8062-2376

Mobile:  01522-2851526

Fax:         069-8062-3267

 

 

NEC Deutschland GmbH, Prinzenallee 11,  D-40549 Duesseldorf, Germany

Geschäftsführer: Michael Stückmann     Handelsregister Düsseldorf HRB 579413

 

 

<Prev in Thread] Current Thread [Next in Thread>