Date: Mon, 29 Dec 2008 11:03:29 -0800
From: Chris Wedgwood <cw@f00f.org>
To: Martin Steigerwald <Martin@Lichtvoll.de>
Cc: xfs@oss.sgi.com
Subject: Re: massively truncated files with XFS with sudden power loss on
	2.6.27 and 2.6.28
Message-ID: <20081229190329.GA17289@puku.stupidest.org>
References: <200812291920.34123.Martin@Lichtvoll.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200812291920.34123.Martin@Lichtvoll.de>

On Mon, Dec 29, 2008 at 07:20:33PM +0100, Martin Steigerwald wrote:

> But I had a hard crash this time while shutting down the system
> regularily and the KDE addressbook, KDE settings, additional sidebar
> all was lost due to truncated files. This was without barriers but
> also without write cache.

I've seen this but not for a very long time.

It used be be (perhaps still is) that KDE updates configurations with
open O_TRUNC & rewrite.

This means there is a window when you can lose data.


I suggested that they should open temp, write, fsync then rename (some
time ago) and I recall seeing some changes in CVS the next day to do
that, but i think that was with ktmpfile or something only).

The other thing is XFS has a much smaller window now than it used to
on the open w/ truncate case, I think now writeout begins as soon as
the file is closed.


Older versions of firefox did this with bookmarks too, so you would
get cases there were you lost data.  Now it uses sqlite as a store
which is much more sane in it's write patterns.

> Do you have any idea on how to help to get down to the cause of this
> - without risking precious data?

ball-peen hammer?

> Did anyone else see this? Does anyone use XFS on laptops and had
> recent power losses or crashes?

I use XFS on laptops, have done for years and don't typically see
this.

> I expect to loose the changes for a dirtied file thats in the page
> cache.

Right.

> But I do not expect to loose the current (old) file on disk in that
> case, unless the crash happens when its actually written out at that
> time.

But you do, when it opens the old file and truncates it, that event is
logged and at which point the file is zero-length containing nothing.

The data hits the disk later on and the size is updated

If you lose power before then, you get zero length files.

> Actually I am considering to switch to ext3/4.

If it is what i explained above, you can still this this though it's
much harder.

Basically, developers shouldn't rewrite critical data in place.

(didn't Jim Gray say something like "Update in Place is a Poison
Apple"?)

> Is there interest in digging this?

Check how KDE writes out configuration files, strace might be easier
than figuring it out from the code.