need help how to debug xfs crash issue xfs_iunlink_remove: xfs_inotobp() returned error 22

[符永涛]

Next time I'll dump it according to your suggestions, thank you.


2013/4/12 符永涛 <yongtaofu at gmail.com>

> Sorry I didn't dump the meta right after unmounting post-failure, I have
> dumped meta after a mount/umount, I'll share the meta to you soon.
>
>
> 2013/4/12 Eric Sandeen <sandeen at sandeen.net>
>
>> On 4/11/13 6:26 PM, Brian Foster wrote:
>> > On 04/11/2013 03:11 PM, 符永涛 wrote:
>> >> It happens tonight again on one of our servers, how to debug the root
>> >> cause? Thank you.
>> >>
>> >
>> > Hi,
>> >
>> > I've attached a system tap script (stap -v xfs.stp) that should
>> > hopefully print out a bit more data should the issue happen again. Do
>> > you have a small enough number of nodes (or predictable enough pattern)
>> > that you could run this on the nodes that tend to fail and collect the
>> > output?
>> >
>> > Also, could you collect an xfs_metadump of the filesystem in question
>> > and make it available for download and analysis somewhere? I believe the
>> > ideal approach is to mount/umount the filesystem first to replay the log
>> > before collecting a metadump, but somebody could correct me on that (to
>> > be safe, you could collect multiple dumps: pre-mount and post-mount).
>>
>> Dave suggested yesterday that this would be best: metadump right
>> after unmounting post-failure, then mount/umount & generate another
>> metadump.
>>
>> -Eric
>>
>> > Could you also describe your workload a little bit? Thanks.
>> >
>> > Brian
>> >
>> >> Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_iunlink_remove:
>> >> xfs_inotobp() returned error 22.
>> >> Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_inactive: xfs_ifree
>> returned
>> >> error 22
>> >> Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_do_force_shutdown(0x1)
>> >> called from line 1184 of file fs/xfs/xfs_vnodeops.c.  Return address =
>> >> 0xffffffffa02ee20a
>> >> Apr 12 02:32:10 cqdx kernel: XFS (sdb): I/O Error Detected. Shutting
>> >> down filesystem
>> >> Apr 12 02:32:10 cqdx kernel: XFS (sdb): Please umount the filesystem
>> and
>> >> rectify the problem(s)
>> >> Apr 12 02:32:19 cqdx kernel: XFS (sdb): xfs_log_force: error 5
>> returned.
>> >> Apr 12 02:32:49 cqdx kernel: XFS (sdb): xfs_log_force: error 5
>> returned.
>> >> Apr 12 02:33:19 cqdx kernel: XFS (sdb): xfs_log_force: error 5
>> returned.
>> >> Apr 12 02:33:49 cqdx kernel: XFS (sdb): xfs_log_force: error 5
>> returned.
>> >>
>>
>>
>
>
> --
> 符永涛
>



-- 
符永涛
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20130412/59c1e9dd/attachment-0001.html>