Another one from the developer’s meeting.
The –u option to pmlogger is not really unbuffered. We’re still using stdio and stdio buffers and stdio default buffer flushing, but we’re adding fflush() calls in the correct order (meta data file, data file, index file) at the end of each logical record written to the archive (after each pmFetch).
This closes, but does not remove the time window in which the physical files are not consistent and aligned with the end of a logical archive record.
-u can be passed to all the pmloggers being managed by pmlogger_check and friends by adding –u to each line in the control file.
As I see it, we have 3 options:
1. leave as is, other than some clarification of the –u option in the usage verbage and the man page
2. make –u the default (and introduce some new option, like –b, to change back to the previous buffering policy in case someone is logging very short records and does not like the additional write() calls that –u would imply in this case. Note that the default pmlogger config for the machine I’m composing this mail on writes 8700 byte records every 6 0seconds with a stdio buffer size of 8K, so –u would make very little difference to the write() rate
3. Same as 2. plus really do unbuffered I/O and avoid the stdio copying and not expose odd half records between a normal stdio write on buffer full and the fflush() … we would not need the fflush() in this case.
Thoughts? Comments?