Thank you for the straight answers to my questions, and for not trying
to argue with me about my real life experiences. I honestly appreciate
All this information was very helpful. I've always been fuzzy about
the interaction between filesystem mount options, like commit=5, and
the vm tunables. So am I correct in thinking that
dirty_writeback_centisecs tells pdflush how often to wake up and look
around for data to flush. That the commit=5 mount parameter to extX
says "flush the journaled metadata to disk every 5 seconds, unless the
vm tells us to do it before that". And If I set commit=45, I'd still
get 30-35 second metadata flushes due to dirty_expire_centisecs=3000
and dirty_writeback_centisecs=500? And that that actual data will also
gets flusehd at least every 30-35 seconds for the same reasons? And
that those vm parameters basically apply to XFS in the same way?
(Except XFS has some setting to flush metadata more frequently, I'd
I follow LWN, even though I rarely post anymore. I'd read that article
before. But I reviewed it again on (your?) recommendation, yesterday.
I think Ric recommended it to me, as well.
Now about ext3 behavior. (Sorry, I know this is an XFS list, but I've
been dying to have this clarified. And this does kind of relate to
XFS.) Not only does data get flushed more frequently. But it gets
careful treatment. It gets written out immediately before the
metadata. But both are (almost?) always written at almost exactly the
same time. Obviously, this comes with a significant performance price
tag for the whole system, which understandably upsets a lot of people.
But more than just the more frequent data updates, doesn't that
contribute substantially to ext3's pony-magic? The reason that I'm
particularly interested in this is that I don't really care that much
whether I lose 5 seconds of data or 30 seconds of data. That doesn't
make much difference. What I do care about is losing a whole day's
worth of data. And I have definitely observed a difference here
between ext3 & ext4. (And I'm assuming XFS would act very much like
ext4 in this context.) What happens with ext4, but *never* in my
experience with ext3 (though I acknowledge that it's *possible* and
just a matter of relative likelihood) is that I end up with C/ISAM
(not database!) files that were being appended to that are not just
missing data at the end, but are actually corrupted. Sometimes in such
a way that the rebuild utility cannot repair them and everything is
lost. This is not common. But I've seen it happen a couple of times
since I've been using ext4. Is the careful ordering, and the fact that
metadata is always written at the same time (and just after the data)
in ext3 likely to be responsible for that difference?