On Mon, Mar 21, 2016 at 10:16:38AM +1100, Dave Chinner wrote:
> On Sun, Mar 20, 2016 at 09:43:01AM -0400, Sage Weil wrote:
> > Hi all,
> >
> > For the last few kernel releases we've been seeing this pretty regularly:
> >
>
> [snip]
>
> Yup, splice still has locking problems. Always has, and the
> write_iter rework a few releases ago just moved the warnings
> elsewhere.
>
> > #012-> #2 (&pipe->mutex/1){+.+.+.}:
>
> xfs_file_splice_read
> xfs_iolock(XFS_IOLOCK_SHARED)
> ....
> pipe_lock(pipe)
>
> XFS takes the XFS_IOLOCK_SHARED here to serialise splice read
> against truncate.
>
> > #012-> #1 (&(&ip->i_iolock)->mr_lock){++++++}:
>
> mutex_lock(inode->i_mutex)
> ....
> xfs_vn_setattr
> xfs_iolock(XFS_IOLOCK_EXCL)
>
> Truncate lock order for IO serialisation (no pipe lock).
>
> > #012-> #0 (&sb->s_type->i_mutex_key#19){+.+.+.}:
>
> iter_file_splice_write
> pipe_lock(pipe)
> xfs_file_buffered_aio_write
> mutex_lock(inode->i_mutex)
> xfs_iolock(XFS_IOLOCK_EXCL)
>
> XFS takes i_mutex (required for buffered IO path) and
> XFS_IOLOCK_EXCL (for atomic write semantics and truncate exclusion),
> but the splice write path has put this under the pipe lock....
>
> i.e. because the read/write path take the IOLOCK different ways,
> lockdep seems to think it's OK. Hence IO testing doesn't report
> issues, but the moment you throw in another operation that takes
> both the i_mutex and XFS_IOLOCK_EXCL, lockdep will through a lock
> order violation report.
>
> > We're not actually doing racing truncate and splice (or any racing
> > operations for that matter) to the same files, so we don't actually hit
> > this, but the lockdep warning is enough to make our test runs fail, and
> > we'd rather not whitelist this if it's something that should get fixed
> > anyway.
>
> We can't fix it in XFS - the high level splice infrastructure which
> that read IO serialisation is done entirely based on page cache page
> locking and so can hold the pipe lock across ->splice_read callouts.
^^^^^^^^^^^
vfs_iter_write()
Got my read and write callouts mixed up there. (i.e. splice_read is
assumed to have no filesystem locking above the pipe lock, so it is
assumed to be safe to take inode locks in vfs_iter_write() whilst
under the pipe lock).
Cheers,
Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx
|