On Wed, 2001-12-05 at 13:19, Xianglong Yuan wrote:
> >-Andi Kleen [05 Dec 2001 19:26 +0100] wrote:
> >
> > > Thanks, Steve, it becomes much clear to me now as why it is. This
> > > problem is critical to us as we run structure simulation for up
> > > to several weeks and dump intermediate data every couple hours
> > > and append them to few large files. If for some reason the system
> > > crash during data output (though rarely), all the results from
> > > few weeks running will gone which is unacceptable for us.
> > > Probably I should looking for FS journaling both meta-data and
> > > file-data, although I don't really need file-data journaling if
> > > ever I could get my old content back.
> >
> > The scenario Steve described obviously does not apply to appending to big
> > files, because there is no rename() from a temporary file involved.
> > If it has been flushed they will be on disk. You can force flushing
> > by using a fsync() for example.
> >
> > -Andi
>
> But only be on disk after flushing, correct? What if crashing
> during data dumping before it can issue a fsync(), or even during
> fsync()? I think the best bet is to remove the old file only
> after ensuring the new one is there, or fallback to old one, an
> asynchronous trasaction which is I believe what Steve said he is
> working on. Same technique as ordered mode in ext3?
You can also use O_SYNC when you open the file, and with xfs the
file data and associated metadata will be on disk before the write
call returns. There is a mount option osyncisdsync which makes go
faster by not transactionally updating file modification times.
Performance will be disk bound, but if you are moving large amounts
of data you may well be disk bound anyway.
The ordered mode in ext3 is almost certainly a better performing
solution than this, it would be nice to get this into xfs, but
probably quite a while before it could be worked on.
Steve
>
> Xianglong
--
Steve Lord voice: +1-651-683-3511
Principal Engineer, Filesystem Software email: lord@xxxxxxx
|