> -----Original Message-----
> From: Frank Ch. Eigler [mailto:fche@xxxxxxxxxx]
> Sent: Friday, 25 April 2014 9:09 PM
> ...
> For added exercise of the metadata/fflush code, this might give a thorough
> workout:
>
> log mandatory on default {
> proc
> }
>
> (with some serious forking/etc. going on in the background).
I would expect this to produce _less_ failures in the old code ... because
the indom will change with every fetch, which causes the metadata file to be
written to, which triggers a fflush() of all the output files.
However, it does expose a remaining (as of Fri) issue with the new code ...
I had deliberately not converted the metadata and index writes to unbuffered
and one logical record per fwrite(), and of course this example exposed a
failure of the killer script (about 1 in 20 failures).
So I've committed changes to make all the archive I/O unbuffered and use one
fwrite() per logical record, which means there is no need for any flush
operations (which used fflush() alone).
What this means is I've moved the memmove() from stdio into the libpcp
routines (so no real change in work), and we do more write() calls for small
pmResults, pmDesc + name records and small pmInDoms, but fewer write() calls
for (the more common) larger pmResults and larger pmInDoms.
With these changes
(a) there is no real difference in CPU load from the numbers I reported
before
(b) the killer script passes solidly, even for the proc metrics case
|