possible null pointer in xlog_iodone

[Vlad Bespalov]

CentOS 6.4 kernel 2.6.32-358.2.1.el6.x86_64
i`ll try to make a script that can be used out-of-house.

several requests may have failed (un)conveniently, so we have passed
through setting XLOG_IO_ERROR in xfs_log_force_unmount() and
eventually skipped xlog_wait altogether:
-----------------
<1>XFS (<device>): metadata I/O error: block 0xa03e1d ("xlog_iodone")
error 5 buf count 1024
<5>XFS (<device>): xfs_do_force_shutdown(0x2) called from line 1052 of
file fs/xfs/xfs_log.c.  Return address = 0xffffffffa0559e51
<1>XFS (<device>): Log I/O Error Detected.  Shutting down filesystem
<1>XFS (<device>): Please umount the filesystem and rectify the problem(s)
<4>XFS (<device>): xfs_log_force: error 5 returned.
<5>XFS (<device>): xfs_do_force_shutdown(0x1) called from line 1063 of
file fs/xfs/linux-2.6/xfs_buf.c.  Return address = 0xffffffffa0575
<4>XFS (<device>): xfs_log_force: error 5 returned.
<4>XFS (<device>): xfs_log_force: error 5 returned.
...
<1>XFS (mtab~): metadata I/O error: block 0xa03e1b ("xlog_iodone")
error 5 buf count 1024
---------------
Thanks.

On 15 June 2013 05:03, Dave Chinner <david at fromorbit.com> wrote:
> On Fri, Jun 14, 2013 at 09:15:04PM +0400, Vlad Bespalov wrote:
>> i`m running an xfs filesystem over device going offline/online and
>> sometimes offline may be done in parallel with unmounting
>>
>> at some point i got several crashes with null pointer panic in
>> xlog_iodone: xlog_t structure taken from input buffer is null
>>
>> i wonder if the following call path combined with disk online/offline
>> handling could have led to this crash:
>>
>> --------------
>> xfs_unmountfs()
>>     xfs_log_unmount_write(mp)
>>         xlog_state_release_iclog(log)
>>             xlog_sync(log, iclog = log->l_iclog)
>>             (bp=iclog->ic_bp)
>>                 xlog_bdstrat(bp)
>>                 (iclog->ic_state != XLOG_STATE_ERROR ? )
>>                     xfs_buf_iorequest(bp)
>>                         xfs_buf_ioend (called with scheduling (*) )
>>                         (queues  : bp->b_iodone_work,
>>                          callback: xlog_iodone)
>
> Which is followed by:
>
>                 if (!(iclog->ic_state == XLOG_STATE_ACTIVE ||
>                       iclog->ic_state == XLOG_STATE_DIRTY)) {
>                         if (!XLOG_FORCED_SHUTDOWN(log)) {
>                                 xlog_wait(&iclog->ic_force_wait,
>                                                         &log->l_icloglock)
>
> Which is supposed to wait for the log IO to complete and hence
> xlog_iodone() is supposed to have been run by the time this code
> completes.
>
> What kernel are you tesing on? Do you have a script that reproduces
> it?
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david at fromorbit.com