xfs_iunlink_remove: xfs_inotobp() returned error 22 -- debugging

[Eric Sandeen]

On 4/20/13 4:38 AM, Brian Foster wrote:
> On 04/20/2013 06:10 AM, 符永涛 wrote:
>> Dear Eric,
>> I have applied your latest patch and collected the following log:
>>
>> /var/log/message
>> Apr 20 17:28:23 10 kernel: XFS (sdb): xfs_iunlink_remove: xfs_inotobp()
>> returned error 22 for inode 0x1b20b ag 0 agino 1b20b
>> Apr 20 17:28:23 10 kernel:
>> Apr 20 17:28:23 10 kernel: XFS (sdb): xfs_inactive: xfs_ifree returned
>> error 22
>> Apr 20 17:28:23 10 kernel: XFS (sdb): xfs_do_force_shutdown(0x1) called
>> from line 1184 of file fs/xfs/xfs_vnodeops.c.  Return address =
>> 0xffffffffa02d4d0a
>> Apr 20 17:28:23 10 kernel: XFS (sdb): I/O Error Detected. Shutting down
>> filesystem
>> Apr 20 17:28:23 10 kernel: XFS (sdb): Please umount the filesystem and
>> rectify the problem(s)
>> Apr 20 17:28:37 10 kernel: XFS (sdb): xfs_log_force: error 5 returned.
>> Apr 20 17:29:07 10 kernel: XFS (sdb): xfs_log_force: error 5 returned.
>> Apr 20 17:29:37 10 kernel: XFS (sdb): xfs_log_force: error 5 returned.
>> Apr 20 17:30:07 10 kernel: XFS (sdb): xfs_log_force: error 5 returned.
>>
>> debugfs trace:
>> https://docs.google.com/file/d/0B7n2C4T5tfNCTlZGUVpnZENrZ3M/edit?usp=sharing
>>
> 
> FWIW...
> 
> <...>-6908  [001]  8739.967623: xfs_iunlink: dev 8:16 ino 0x83a8b mode
> 0100000, flags 0x0
> <...>-6909  [001]  8739.970252: xfs_iunlink: dev 8:16 ino 0x83a8b mode
> 0100000, flags 0x0

Interesting, this time it was on the same CPU [001]

I'd hoped that we'd see this same inode in some of the new tracepoints
but we don't, I'm not sure why; perhaps it overflowed the trace buffer?

> 0x83a8b and 0x1b20b both hash to unlinked list bucket 11.

which is why the error triggered on 0x1b20b, since the list is now corrupt. 

-Eric