xfs
[Top] [All Lists]

Re: need help how to debug xfs crash issue xfs_iunlink_remove: xfs_inoto

To: çææ <yongtaofu@xxxxxxxxx>
Subject: Re: need help how to debug xfs crash issue xfs_iunlink_remove: xfs_inotobp() returned error 22
From: Eric Sandeen <sandeen@xxxxxxxxxxx>
Date: Tue, 09 Apr 2013 12:10:31 -0500
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <CADFMGuL7968v6L-3=j3FY3YYjeA_XH1CyuLgnL88u-abxiwHvg@xxxxxxxxxxxxxx>
References: <CADFMGuJm5bPPwbbUtYwrCVDL23KExJTw_-VRX2UEEdZjo+i5oA@xxxxxxxxxxxxxx> <51642E5E.3040403@xxxxxxxxxxx> <CADFMGuL7968v6L-3=j3FY3YYjeA_XH1CyuLgnL88u-abxiwHvg@xxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5
On 4/9/13 10:18 AM, çææ wrote:
> The servers are back to service now and It's hard to run xfs_repair. It 
> always happen bellow is the xfs_repair log when it happens on another server 
> several days ago.

...

> çäæ
> repairçlog
> 
> sh-4.1$ sudo xfs_repair /dev/glustervg/glusterlv
> Phase 1 - find and verify superblockâ
> Phase 2 - using internal log
>         - zero logâ
>         - scan filesystem freespace and inode mapsâ
> agi unlinked bucket 0 is 4046848 in ag 0 (inode=4046848)
> agi unlinked bucket 5 is 2340485 in ag 0 (inode=2340485)
> agi unlinked bucket 6 is 2326854 in ag 0 (inode=2326854)
> agi unlinked bucket 8 is 1802120 in ag 0 (inode=1802120)
> agi unlinked bucket 14 is 495566 in ag 0 (inode=495566)
> agi unlinked bucket 16 is 5899536 in ag 0 (inode=5899536)
> agi unlinked bucket 19 is 4008211 in ag 0 (inode=4008211)
> agi unlinked bucket 21 is 4906965 in ag 0 (inode=4906965)
> agi unlinked bucket 23 is 2022231 in ag 0 (inode=2022231)
> agi unlinked bucket 24 is 1626200 in ag 0 (inode=1626200)
> agi unlinked bucket 25 is 938585 in ag 0 (inode=938585)
> agi unlinked bucket 30 is 4226526 in ag 0 (inode=4226526)
> agi unlinked bucket 34 is 4108962 in ag 0 (inode=4108962)
> agi unlinked bucket 37 is 1740389 in ag 0 (inode=1740389)
> agi unlinked bucket 39 is 247399 in ag 0 (inode=247399)
> agi unlinked bucket 40 is 6237864 in ag 0 (inode=6237864)
> agi unlinked bucket 43 is 3404331 in ag 0 (inode=3404331)
> agi unlinked bucket 45 is 2092717 in ag 0 (inode=2092717)
> agi unlinked bucket 48 is 4041008 in ag 0 (inode=4041008)
> agi unlinked bucket 50 is 1459762 in ag 0 (inode=1459762)
> agi unlinked bucket 56 is 852024 in ag 0 (inode=852024)

If this machine is still around in similar state, can you do a

# find /path/to/mount -inum $INODE_NUMBER

for the inode numbers above, and see what files they are?
That might give us a clue about what operations were happening
to them.  Dumping the gluster xattrs on those files
might also be interesting.  Just guesses here, but it'd be a
little more data.

(if this is an old repair, maybe doing the same for your most
recent incident would be best)

Thanks,
-Eric

>         - found root in ode chunk
> Phase 3 - for each AGâ
>         - scan and clear agi unlinked listsâ
>         - process known inodes and perform inode discoveryâ
>         - agno = 0
> 7f8220be6700: Badness in key lookup (length)
> bp=(bno 123696, len 16384 bytes) key=(bno 123696, len 8192 bytes)

(FWIW the above warnings look like an xfs_repair bug, not related)

-Eric

<Prev in Thread] Current Thread [Next in Thread>