xfs
[Top] [All Lists]

Re: XFS on 2.4 Tib raid

To: Dave Chinner <dgc@xxxxxxx>, linux-xfs@xxxxxxxxxxx, naude@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: XFS on 2.4 Tib raid
From: Leon Vismer <lvismer@xxxxxxxxxxxxxxxx>
Date: Wed, 11 May 2005 13:57:12 +0200
In-reply-to: <20050511084450.A91712@melbourne.sgi.com>
References: <200505092210.25667.lvismer@picturenet.co.za> <1115712231.42806ae75b4d1@picturenet.co.za> <20050511084450.A91712@melbourne.sgi.com>
Reply-to: lvismer@xxxxxxxxxxxxxxxx
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: KMail/1.8
Hi Dave

I did the following:

# parted /dev/sdc
parted> mklabel gpt
parted> mkpart primary 0 2384080
parted> quit

I saw the following in tail /var/log/messages

May 11 13:37:40 mail kernel: program parted is using a deprecated SCSI ioctl, 
please convert it to SG_IO
May 11 13:37:50 mail kernel: sym1:4:0:phase change 2-7 16@01b75f60 resid=10.

# xfs_check /dev/sdc1 (gives the following)

bad sb magic # 0xb0b25aab in ag 30
bad sb version # 0x9d8b in ag 30
bad agf magic # 0x9be1c556 in ag 30
bad agf version # 0xf2c55d8a in ag 30
bad agi magic # 0xe0467f8a in ag 30
bad agi version # 0xb4f92355 in ag 30
can't seek in filesystem at bb 426861406330510323
can't read agfl block for ag 2915391330
can't seek in filesystem at bb 426861438191846816
can't read btree block 2915391330/3982667062
can't seek in filesystem at bb 426861412315354400
can't read btree block 2915391330/748105510
can't seek in filesystem at bb 426861435013558208
can't read btree block 2915391330/3585380986
agf_freeblks 1761148316, counted 0 in ag 30
agf_longest 2463815067, counted 0 in ag 30
agi_count 2563941498, counted 0 in ag 30
agi_freecount 3663028107, counted 0 in ag 30
agi unlinked bucket 0 is 462714605 in ag 30 (inode=16568841965)
agi unlinked bucket 1 is 1724631010 in ag 30 (inode=16757016546)
agi unlinked bucket 2 is 4215237414 in ag 30 (inode=17100139302)
agi unlinked bucket 3 is 3231072623 in ag 30 (inode=16115974511)
agi unlinked bucket 4 is 2859018972 in ag 30 (inode=16817662684)
agi unlinked bucket 5 is 2760201574 in ag 30 (inode=16718845286)
agi unlinked bucket 6 is 3855234389 in ag 30 (inode=16740136277)
agi unlinked bucket 7 is 3405867853 in ag 30 (inode=16290769741)
agi unlinked bucket 8 is 1827066604 in ag 30 (inode=16859452140)
agi unlinked bucket 9 is 3401839999 in ag 30 (inode=16286741887)
agi unlinked bucket 10 is 1390827773 in ag 30 (inode=16423213309)
agi unlinked bucket 11 is 1180067325 in ag 30 (inode=16212452861)
agi unlinked bucket 12 is 3699362302 in ag 30 (inode=16584264190)
agi unlinked bucket 13 is 3058210065 in ag 30 (inode=17016853777)
agi unlinked bucket 14 is 1077317540 in ag 30 (inode=16109703076)
agi unlinked bucket 15 is 4205651413 in ag 30 (inode=17090553301)
agi unlinked bucket 16 is 2894855762 in ag 30 (inode=16853499474)
agi unlinked bucket 17 is 3366759320 in ag 30 (inode=16251661208)
agi unlinked bucket 18 is 3837338121 in ag 30 (inode=16722240009)
agi unlinked bucket 19 is 246785257 in ag 30 (inode=16352912617)
agi unlinked bucket 20 is 3164927206 in ag 30 (inode=17123570918)
agi unlinked bucket 21 is 2584733074 in ag 30 (inode=16543376786)
agi unlinked bucket 22 is 1073273742 in ag 30 (inode=17179401102)
agi unlinked bucket 23 is 1078564132 in ag 30 (inode=16110949668)
agi unlinked bucket 24 is 3071981037 in ag 30 (inode=17030624749)
agi unlinked bucket 25 is 3155280692 in ag 30 (inode=17113924404)
agi unlinked bucket 26 is 2661588059 in ag 30 (inode=16620231771)
agi unlinked bucket 27 is 4261928854 in ag 30 (inode=17146830742)
agi unlinked bucket 28 is 1478809154 in ag 30 (inode=16511194690)
agi unlinked bucket 29 is 1530263199 in ag 30 (inode=16562648735)
agi unlinked bucket 30 is 1965008675 in ag 30 (inode=16997394211)
agi unlinked bucket 31 is 2129756125 in ag 30 (inode=17162141661)
agi unlinked bucket 32 is 1790905946 in ag 30 (inode=16823291482)
agi unlinked bucket 33 is 612504315 in ag 30 (inode=16718631675)
agi unlinked bucket 34 is 2367087999 in ag 30 (inode=16325731711)
agi unlinked bucket 35 is 2036780234 in ag 30 (inode=17069165770)
agi unlinked bucket 36 is 3748229838 in ag 30 (inode=16633131726)
agi unlinked bucket 37 is 656196899 in ag 30 (inode=16762324259)
agi unlinked bucket 38 is 2597802675 in ag 30 (inode=16556446387)
agi unlinked bucket 39 is 1727913098 in ag 30 (inode=16760298634)
agi unlinked bucket 40 is 2900347851 in ag 30 (inode=16858991563)
agi unlinked bucket 41 is 140805993 in ag 30 (inode=16246933353)
agi unlinked bucket 42 is 2746809821 in ag 30 (inode=16705453533)
agi unlinked bucket 43 is 3270611966 in ag 30 (inode=16155513854)
agi unlinked bucket 44 is 120717675 in ag 30 (inode=16226845035)
agi unlinked bucket 45 is 1402607980 in ag 30 (inode=16434993516)
agi unlinked bucket 46 is 4177169118 in ag 30 (inode=17062071006)
agi unlinked bucket 47 is 2608780005 in ag 30 (inode=16567423717)
agi unlinked bucket 48 is 2696235053 in ag 30 (inode=16654878765)
agi unlinked bucket 49 is 1494642567 in ag 30 (inode=16527028103)
agi unlinked bucket 50 is 3971373067 in ag 30 (inode=16856274955)
agi unlinked bucket 51 is 2234035046 in ag 30 (inode=16192678758)
agi unlinked bucket 52 is 1210913975 in ag 30 (inode=16243299511)
agi unlinked bucket 53 is 2388247726 in ag 30 (inode=16346891438)
agi unlinked bucket 54 is 1707649758 in ag 30 (inode=16740035294)
agi unlinked bucket 55 is 950720200 in ag 30 (inode=17056847560)
agi unlinked bucket 56 is 2822954051 in ag 30 (inode=16781597763)
agi unlinked bucket 57 is 3945987840 in ag 30 (inode=16830889728)
agi unlinked bucket 58 is 2475018245 in ag 30 (inode=16433661957)
agi unlinked bucket 59 is 66143967 in ag 30 (inode=16172271327)
agi unlinked bucket 60 is 374620976 in ag 30 (inode=16480748336)
agi unlinked bucket 61 is 1415867961 in ag 30 (inode=16448253497)
agi unlinked bucket 62 is 1494057121 in ag 30 (inode=16526442657)
agi unlinked bucket 63 is 1636941515 in ag 30 (inode=16669327051)
bad sb magic # 0x9cb156fe in ag 31
bad sb version # 0x5c55 in ag 31
bad agf magic # 0x362aef53 in ag 31
bad agf version # 0x1573498a in ag 31
bad agi magic # 0xf27db7fb in ag 31
bad agi version # 0xb199b154 in ag 31
can't seek in filesystem at bb 425981987078782683
can't read agfl block for ag 2909385045
can't seek in filesystem at bb 425981993282677816
can't read btree block 2909385045/775486892
can't seek in filesystem at bb 425981998602996096
can't read btree block 2909385045/1440526677
can't seek in filesystem at bb 425981999647932048
can't read btree block 2909385045/1571143671
agf_freeblks 2326620443, counted 0 in ag 31
agf_longest 360132570, counted 0 in ag 31
agi_count 2334420026, counted 0 in ag 31
agi_freecount 3060805593, counted 0 in ag 31
agi unlinked bucket 0 is 83776965 in ag 31 (inode=16726775237)
agi unlinked bucket 1 is 1836045795 in ag 31 (inode=16868431331)
agi unlinked bucket 2 is 4267464407 in ag 31 (inode=17152366295)
agi unlinked bucket 3 is 2103323737 in ag 31 (inode=17135709273)
agi unlinked bucket 4 is 1525630295 in ag 31 (inode=17094886743)
agi unlinked bucket 5 is 364345899 in ag 31 (inode=17007344171)
agi unlinked bucket 6 is 2333571627 in ag 31 (inode=16829086251)
agi unlinked bucket 7 is 2326657722 in ag 31 (inode=16822172346)
agi unlinked bucket 8 is 3849629434 in ag 31 (inode=16734531322)
agi unlinked bucket 9 is 3135458031 in ag 31 (inode=17094101743)
agi unlinked bucket 10 is 2865027426 in ag 31 (inode=16823671138)
agi unlinked bucket 11 is 2941998428 in ag 31 (inode=16900642140)
agi unlinked bucket 12 is 1432868311 in ag 31 (inode=17002124759)
agi unlinked bucket 13 is 358479221 in ag 31 (inode=17001477493)
agi unlinked bucket 14 is 3311269231 in ag 31 (inode=16733042031)
agi unlinked bucket 15 is 4071799381 in ag 31 (inode=16956701269)
agi unlinked bucket 16 is 3209953024 in ag 31 (inode=17168596736)
agi unlinked bucket 17 is 723146157 in ag 31 (inode=16829273517)
agi unlinked bucket 18 is 1727987890 in ag 31 (inode=16760373426)
agi unlinked bucket 19 is 3637689322 in ag 31 (inode=17059462122)
agi unlinked bucket 20 is 4076798006 in ag 31 (inode=16961699894)
agi unlinked bucket 21 is 883654319 in ag 31 (inode=16989781679)
agi unlinked bucket 22 is 775203674 in ag 31 (inode=16881331034)
agi unlinked bucket 23 is 427779866 in ag 31 (inode=17070778138)
agi unlinked bucket 24 is 1438602860 in ag 31 (inode=17007859308)
agi unlinked bucket 25 is 1773590197 in ag 31 (inode=16805975733)
agi unlinked bucket 26 is 2762837293 in ag 31 (inode=16721481005)
agi unlinked bucket 27 is 2935663775 in ag 31 (inode=16894307487)
agi unlinked bucket 28 is 3415029177 in ag 31 (inode=16836801977)
agi unlinked bucket 29 is 1636925287 in ag 31 (inode=16669310823)
agi unlinked bucket 30 is 441938335 in ag 31 (inode=17084936607)
agi unlinked bucket 31 is 4239157515 in ag 31 (inode=17124059403)
agi unlinked bucket 32 is 1499987712 in ag 31 (inode=17069244160)
agi unlinked bucket 33 is 723146186 in ag 31 (inode=16829273546)
agi unlinked bucket 34 is 3539927189 in ag 31 (inode=16961699989)
agi unlinked bucket 35 is 2519393938 in ag 31 (inode=17014908562)
agi unlinked bucket 36 is 1442524042 in ag 31 (inode=17011780490)
agi unlinked bucket 37 is 3151402431 in ag 31 (inode=17110046143)
agi unlinked bucket 38 is 1729452322 in ag 31 (inode=16761837858)
agi unlinked bucket 39 is 2894841416 in ag 31 (inode=16853485128)
agi unlinked bucket 40 is 4185894072 in ag 31 (inode=17070795960)
agi unlinked bucket 41 is 2857988512 in ag 31 (inode=16816632224)
agi unlinked bucket 42 is 3925303262 in ag 31 (inode=16810205150)
agi unlinked bucket 43 is 1538259740 in ag 31 (inode=17107516188)
agi unlinked bucket 44 is 1525231515 in ag 31 (inode=17094487963)
agi unlinked bucket 45 is 3384200701 in ag 31 (inode=16805973501)
agi unlinked bucket 46 is 3410567162 in ag 31 (inode=16832339962)
agi unlinked bucket 47 is 3098354139 in ag 31 (inode=17056997851)
agi unlinked bucket 48 is 3413808191 in ag 31 (inode=16835580991)
agi unlinked bucket 49 is 4013806751 in ag 31 (inode=16898708639)
agi unlinked bucket 50 is 3971107991 in ag 31 (inode=16856009879)
agi unlinked bucket 51 is 742286517 in ag 31 (inode=16848413877)
agi unlinked bucket 52 is 4214010495 in ag 31 (inode=17098912383)
agi unlinked bucket 53 is 2911549813 in ag 31 (inode=16870193525)
agi unlinked bucket 54 is 1835982843 in ag 31 (inode=16868368379)
agi unlinked bucket 55 is 3950794005 in ag 31 (inode=16835695893)
agi unlinked bucket 56 is 4264191327 in ag 31 (inode=17149093215)
agi unlinked bucket 57 is 2317233546 in ag 31 (inode=16812748170)
agi unlinked bucket 58 is 2720922833 in ag 31 (inode=16679566545)
agi unlinked bucket 59 is 1832384401 in ag 31 (inode=16864769937)
agi unlinked bucket 60 is 3802764410 in ag 31 (inode=16687666298)
agi unlinked bucket 61 is 2510520014 in ag 31 (inode=17006034638)
agi unlinked bucket 62 is 716535181 in ag 31 (inode=16822662541)
agi unlinked bucket 63 is 1066866403 in ag 31 (inode=17172993763)

After seeing all of this I ran:

# xfs_repair /dev/sdc1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
        - scan filesystem freespace and inode maps...
bad on-disk superblock 30 - bad magic number
primary/secondary superblock 30 conflict - AG superblock geometry info 
conflicts with filesystem geometry
bad magic # 0x0 for agf 30
bad version # 0 for agf 30
bad sequence # 0 for agf 30
bad length 0 for agf 30, should be 18302063
bad magic # 0x0 for agi 30
bad version # 0 for agi 30
bad sequence # 0 for agi 30
bad length # 0 for agi 30, should be 18302063
reset bad sb for ag 30
reset bad agf for ag 30
reset bad agi for ag 30
bad agbno 0 for btbno root, agno 30
bad agbno 0 for btbcnt root, agno 30
bad agbno 0 for inobt root, agno 30
bad on-disk superblock 31 - bad magic number
primary/secondary superblock 31 conflict - AG superblock geometry info 
conflicts with filesystem geometry
bad magic # 0x0 for agf 31
bad version # 0 for agf 31
bad sequence # 0 for agf 31
bad length 0 for agf 31, should be 18302063
bad magic # 0x0 for agi 31
bad version # 0 for agi 31
bad sequence # 0 for agi 31
bad length # 0 for agi 31, should be 18302063
reset bad sb for ag 31
reset bad agf for ag 31
reset bad agi for ag 31
bad agbno 0 for btbno root, agno 31
bad agbno 0 for btbcnt root, agno 31
bad agbno 0 for inobt root, agno 31
        - found root inode chunk
Phase 3 - for each AG...
        - scan and clear agi unlinked lists...
error following ag 30 unlinked list
error following ag 31 unlinked list
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - clear lost+found (if it exists) ...
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
Phase 5 - rebuild AG headers and trees...
        - reset superblock...
Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - ensuring existence of lost+found directory
        - traversing filesystem starting at / ...
        - traversal finished ...
        - traversing all unattached subtrees ...
        - traversals finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
done

The running xfs_check again gives the same as above

Many thanks
--
Leon

> > I include the full error from /var/log/messages. As one copies
> > files it stops at some point and the following error loops and the
> > machine needs to be rebooted.
> >
> > 0x0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > Filesystem "sdc": XFS internal error xfs_alloc_read_agf at line 2195 of
> > file fs/ xfs/xfs_alloc.c.  Caller 0xd0b638ca
> >  [<d0b63ced>] xfs_alloc_read_agf+0x12d/0x1f0 [xfs]
> >  [<d0b638ca>] xfs_alloc_fix_freelist+0x47a/0x490 [xfs]
> >  [<d0b6407d>] xfs_alloc_vextent+0x2cd/0x500 [xfs]
>
> So we're trying to allocate an extent and the AGF read from disk
> is full of zeros.
>
> Can you do the following:
>
>  - write a known pattern to the disk before making the
>    filesystem (0xa5 is a good one)
>  - make the filesystem
>  - run xfs_check on the device before mounting to validate
>    the filesystem was made properly
>  - mount the filesystem
>  - run your copy until it breaks
>  - run xfs_check on the filesystem (if you needed to reboot
>    to get here, you should mount and unmount the filesystem
>    to replay the log first)
>  - run xfs_repair on the filesystem if xfs_check finds
>    errors
>
> And post the output of any errors that are found?
>
> If there really is a AGF full of zeros (or 0xa5!) in the filesystem,
> xfs_check will find it.
>
> BTW, did you build the kernel with CONFIG_LBD=y (i.e. support block
> devices larger than 2TiB)?
>
> Cheers,
>
> Dave.


<Prev in Thread] Current Thread [Next in Thread>