On Wed, Dec 05, 2001 at 02:19:27PM -0500, Xianglong Yuan wrote:
> >-Andi Kleen [05 Dec 2001 19:26 +0100] wrote:
> >
> > > Thanks, Steve, it becomes much clear to me now as why it is. This
> > > problem is critical to us as we run structure simulation for up
> > > to several weeks and dump intermediate data every couple hours
> > > and append them to few large files. If for some reason the system
> > > crash during data output (though rarely), all the results from
> > > few weeks running will gone which is unacceptable for us.
> > > Probably I should looking for FS journaling both meta-data and
> > > file-data, although I don't really need file-data journaling if
> > > ever I could get my old content back.
> >
> > The scenario Steve described obviously does not apply to appending to big
> > files, because there is no rename() from a temporary file involved.
> > If it has been flushed they will be on disk. You can force flushing
> > by using a fsync() for example.
> >
> > -Andi
>
> But only be on disk after flushing, correct? What if crashing
Yes, but until that the old data is not gone because the file is not truncated
.
The problem with the editors is that the old data is usually lost in some
temporary file; that won't be the problem here.
> during data dumping before it can issue a fsync(), or even during
> fsync()? I think the best bet is to remove the old file only
Then you lose a few seconds at worst.
> after ensuring the new one is there, or fallback to old one, an
> asynchronous trasaction which is I believe what Steve said he is
> working on. Same technique as ordered mode in ext3?
The problem is really that the delete part of rename is synchronous,
which causes a log flush at an very inconvenient moment if you're using an
editor
and crashed shortly afterwards. For appending to existing files it is no problem
though again (first no delete and even if there was a flush it wouldn't truncate
your file)
-Andi
|