pcp
[Top] [All Lists]

Re: [pcp] pmlogrewrite questions from the developer meeting

To: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Subject: Re: [pcp] pmlogrewrite questions from the developer meeting
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Sun, 13 Apr 2014 20:49:10 -0400 (EDT)
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <01c501cf56de$54b6d370$fe247a50$@internode.on.net>
References: <01c501cf56de$54b6d370$fe247a50$@internode.on.net>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: Ac9WkUroZbfqdTajSgqHF7LgHS1u4ntiUdPu
Thread-topic: pmlogrewrite questions from the developer meeting
Hi Ken,

----- Original Message -----
> [..]
> 1. temporary files are in the same directory as the input archive, so
> renaming does not imply any copying, just directory updates
> 
> 2. the input archive is never overwritten
> 
> 3. the input archive files is maintained via hard links (using a second set
> of temporary names) throughout and any error causes the old names to be
> reinstated
> 

There is a metadata vs data ordering assumption, I believe.  We need to fsync
the newly created data files before the rename otherwise we could end up with
empty files on-disk - or files-filled-with-zeroes depending on the filesystem
implementation (in this case: directory metadata IO & new file inode metadata
IO completes up to the open(O_CREAT), but no data writes/allocation happen).

> Feedback from others would be welcome, but I think the concerns raised at the
> meeting are not substantiated by the code.

I suspect I obfuscated the matter by incorrectly thinking that the temporary
file creation was happening in some other TMPDIR - I was totally wrong there,
as your audit found - apologies for the misunderstanding!

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>