xfs
[Top] [All Lists]

Re: XFS File system in trouble

To: Roger Willcocks <roger@xxxxxxxxxxxxxxxx>, Leslie Rhorer <lrhorer@xxxxxxxxxxxx>
Subject: Re: XFS File system in trouble
From: Eric Sandeen <sandeen@xxxxxxxxxxx>
Date: Sat, 15 Aug 2015 19:32:54 -0500
Cc: Brian Foster <bfoster@xxxxxxxxxx>, Kris Rusocki <kszysiu@xxxxxxxxxx>, "Rhorer, Leslie" <Leslie.Rhorer@xxxxxxxxxx>, "xfs@xxxxxxxxxxx" <xfs@xxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <55CF8C4F.1080906@xxxxxxxxxxx>
References: <20150720111747.GA53450@xxxxxxxxxxxxxxx> <55B73365.1050908@xxxxxxxxxxxx> <20150728123307.GC38784@xxxxxxxxxxxxxxx> <55B79BFD.6020509@xxxxxxxxxxxx> <20150728221150.GA26604@xxxxxxxxxxxxxxx> <55BE7C75.4060604@xxxxxxxxxxxx> <55C06F41.4030502@xxxxxxxxxxxx> <20150804224240.GU16638@dastard> <55C8006C.8070807@xxxxxxxxxxxx> <55CC375C.10902@xxxxxxxxxxxx> <20150814012635.GT3902@dastard> <55CE75CA.5070506@xxxxxxxxxxxx> <74DC7EBF-0029-4E5E-9D96-DF193E2BE83F@xxxxxxxxxxxxxxxx> <55CF8972.8020306@xxxxxxxxxxx> <55CF8C4F.1080906@xxxxxxxxxxx>
On 8/15/15 2:00 PM, Eric Sandeen wrote:
> On 8/15/15 1:48 PM, Eric Sandeen wrote:
>> On 8/15/15 7:28 AM, Roger Willcocks wrote:
>>> xfs_repair 3.2.1 runs cleanly.
>>>
>>> xfs_repair 3.1.1 complains about a load of stuff, including:
>>
>> I wouldn't expect v3.1.1 to work at all, because:
>>
>> # db/xfs_db -V
>> xfs_db version 4.2.0-rc1
>> # db/xfs_db /mnt/test2/leslie/md0.img -c version
>> versionnum [0xbdb4+0x8a] = 
>> V4,NLINK,DIRV2,ATTR,ALIGN,DALIGN,LOGV2,EXTFLG,SECTOR,MOREBITS,ATTR2,LAZYSBCOUNT,PROJID32BIT
>>
>> the filesystem has 32-bit project IDs, and:
>>
>> # git log --oneline | grep -i "projid32bit"
>> dd536e1 xfsprogs: Note projid32bit default change in mkfs.xfs manpage
>> 22bc10e xfsprogs: projid32bit handling
>>
>> # git describe --contains 22bc10e
>> v3.1.4~2
>>
>> that feature support didn't show up until v3.1.4.  Were you running a stock 
>> v3.1.1?
>>
>> Anyway, in my testing, up to v3.2.0, repair finds a lot of errors (and 
>> spends some
>> time looking for a proper superblock)
>>
>> v3.2.1 finds no errors.
>>
>> The errors stopped showing up as of:
>>
>> commit d085fb486f8b33304f2fdf704411313ffc8bcc0c
>> Author: Dave Chinner <dchinner@xxxxxxxxxx>
>> Date:   Tue Jul 8 10:36:39 2014 +1000
>>
>>     repair: fix quota inode handling in secondary superblocks
> 
> I take that back, a bit:  "errors vs. no errors" was for xfs_repair -n.
> 
> a "full" repair with v3.1.4 shows only:
> 
> # repair/xfs_repair /mnt/test2/leslie/md0.img 
> Phase 1 - find and verify superblock...
> Phase 2 - using internal log
>         - zero log...
>         - scan filesystem freespace and inode maps...
> non-null user quota inode field in superblock 5
> reset bad sb for ag 5
> non-null user quota inode field in superblock 10
> reset bad sb for ag 10
> non-null user quota inode field in superblock 18
> reset bad sb for ag 18
> non-null user quota inode field in superblock 23
> reset bad sb for ag 23
>         - found root inode chunk
> Phase 3 - for each AG...
>         - scan and clear agi unlinked lists...
>         - process known inodes and perform inode discovery...
>         - agno = 0
> ...
>         - agno = 29
> bad version number 0x3 on inode 124656869424
> bad version number 0x3 on inode 124656869424, resetting version number

And that does match the runtime error he was hitting:

It works for a while, because some of the directories already exist, but when 
it started to try to create directories, it started getting FS errors and I had 
to wind up rebooting.  After reboot, I get the following when I try to run the 
command from the directory again:

[  380.556635] XFS (md0): xfs_iread: validation failed for inode 124656869424 
failed

FWIW, this inode is unused; when he hit it runtime, we were attempting to 
allocate it.

I suppose the bad version could be chalked up to a bit-flip from the bad memory 
that was in the box.

-Eric

<Prev in Thread] Current Thread [Next in Thread>