If we get two unaligned direct IO's to the same filesystem block that is marked as a new allocation (i.e. buffer_new), then both IOs will zero the portion of the block they are not writing data to. A
Urgh. Yuck. Urgh. Yuck. Could we perhaps handle this by making an IO instantiate a page cache page for partial writes, and forcing that portion of the IO through the page cache? The second IO would h
It's better than silent data corruption. I don't want any direct IO for XFS to go through the page cache - unaligned or not. using the page cache for the unaligned blocks would also be much worse for
I can confirm that, it fixes corruption of my VM images when using AIO +DIO. (cache=none,aio=native). I haven't reviewed the patch closely, but 1) can we do this only for AIO+DIO combination ? For re
Not for filesystems that do their own locking. In most cases XFS does not take the i_mutiex during DIO writes, and when it does it drops it long before we call into the generic direct IO code that do
Does this problem also possible for DIO and non AIO case? (Ext4 case this only happy with AIO+DIO+unaligned). If not, could we simply force unaligned AIO+DIO to be synchronous? Still direct IO... I a
.... There is nothing specific to AIO about this bug. XFS (at least) allows concurrent DIO writes to the same inode regardless of whether they are dispatched via AIO or multiple separate threads and
Okay..yeah ext4 prevent direct IO write to the same inode from multiple threads, so this is not a issue for non-aio case. How does XFS serialize direct IO (aligned) to the same file offset(or overlap
This seems to address both two IOs are unaligned direct IO. If the first IO is aligned direct IO, then it is not tracked? I am also concerned about the aligned direct IO case... 1) first thread aio+d
That is clearly and unmistakably an application bug - it should not be issuing concurrent, overlapping IO to the same block(s) regardless of whether they are unaligned, aligned or a mixture of both.
It doesn't. The 1996 usenix paper describes it well: http://oss.sgi.com/projects/xfs/papers/xfs_usenix/index.html See section 6.2 "Performing File I/O", sepcifically the sections on "Using Direct I/O
while there are multiple applications running on the same filesystem, they could possible touching the same files concurrently. How could applications know there is other apps to change the same file
That's for the applications to figure out. A typical way of acheiving serialisation at application level is for all the applications doing dio to the same files is to use byte-range locking... Rememb