Thanks for the great feedback!
I'll address it all at once, so as not to spawn multiple email
threads.
On 11/11/2015 12:03 AM, Nathan Scott
wrote:
[...]
Tools which use (opts->flags & PM_OPTFLAG_MULTI) continue to work as before.
I wonder if this flag should be propagated more (as in, almost everywhere) - to
anticipate cases where it might not be desirable for certain tools to support
multiple-archives (pmdumplog? pmlogcheck? some PMAPI 3rd party? not sure).
I'll add this to my list of future considerations.
Single PMNS for the entire context
This is needed for those APIs which have a need to examine the entire PMNS of
the context. Examples include pmTraversePMNS(3) and pmLookup*(3).
I propose that this PMNS be built up as each individual archive is accessed.
The main reason is that consistency checking can then also be performed as
each archive is accessed. In the case of a consistency issue, it is then
possible (even probable) that useful data will have been provided to the
client before the problem is encountered. The label of each archive still
needs to be examined when the context is opened, in order to determine their
ordering in overall the time line, but it is not necessary to examine the
PMNS (.meta) of each until the metrics within them are to be examined.
In the case of an API call, like pmTraversePMNS(3), we can bite the bullet
and complete the PMNS of the entire archive set as needed.
There might be quite few API calls in that set - pmLookupName, pmLookupInDom
- but that may be OK as long as not every client always issues one of those
calls as the first thing it does. :)
It might not be a problem to read all the .meta files, they tend to be fairly
small (heh, except in certain unusual situations - hi Martins!).
My feeling on this is that it will not be difficult to delay reading
all of the .meta data until necessary and that I see no need to
force this upon tools which do not need to do so, even if they turn
out to be in the minority I will make sure to isolate the code so
that this decision can be easily reversed.
The design of data structures and policies for retention could potentially
depend on what kinds of usage scenarios we envision. We must also keep in
mind that, in the case of directories of archives, new archives could be
dynamically appearing via an active pmlogger or via some other means. They
could also be dynamically disappearing, however this is just as easy to
detect and should probably be treated as an error situation.
These things are guaranteed, and will be the normal situation (archives both
appearing and disappearing) - i.e. in the daily log rotation situations. So
I think these need to work seamlessly (i.e. its not an error when an archive
is removed - its expected, every night, in steady state operation).
But are disappearing archives expected to be handled within an open
libpcp context? Even the current single-archive contexts only keep
one volume open at a time. If we were to return to a previously used
volume which has disappeared, for any reason, including log
rotation, while the context is open, an error will occur when we
attempt to re-open the now-nonexistent volume. I don't think that a
set of archives within a multi-archive context can or should be
treated any differently.
There are only two ways that I can see to handle this.
- leave the file descriptors open, even after moving on to
another archive
- read all of the metric and retain them in memory
both of which run afoul of scaling concerns.
With respect to archives appearing in the middle of the time line, I
tend to agree with Frank that attempting to handle them would cause
all manners of inconsistency with respect to data which has
potentially already been reported for points in the time line which
follow the insertion of the new archive. I am leaning toward having
libpcp automatically handle new archives which appear at the end of
the overall timeline.
I feel that we should cater to the possibility of new archives being created
within directories but only at the end of the time line for each directory
(if any). pmlogger(1) would create new archives in this way. I believe that
handing the creation of new archives at random points in the time line
whenever an arbitrary archive boundary is crossed, would be a waste of time.
I'm not following what makes it more difficult to handle this case than just
appearing at the end of the timeline? It might mean using a tree structure,
perhaps, rather than a simple list/array ... or is there more to it?
Thinking out loud - all archives will have a fixed start time, which *cannot*
ever change - so perhaps that could be the key used to index the (tree?) data
structure. Since the end point can change (and there's no simple way to tell
if an archive is actively being written), we can't rely on that at all - but
start time may be enough to quickly find candidate archives.
Tough case to keep in mind: think of two active pmloggers, both recording for
the same host, in the same directory, one logging once a second, the other
once an hour. At PMAPI-client startup time, the two might appear to not be
overlapping... (and the initial consistency checks might all pass), but that
would change later when the long sampling interval elapses once more. (ow!)
IOW, its not easy (or even possible?) to tell if any archive is "finished"
being logged to, so I'd focus away from that as a key/accessor for any data
structures you're using.
Other than for the case you give above of two actively-being-written
archives within the same directory, the difficulty comes not in
inserting new archives into their proper place in the overall
timeline but with with difficulty with new metrics being introduced
in the middle of an area of the timeline which has already been
traversed.
The best example I can come with would be having archives A and C
initially and having traversed from archive A into archive C while
scaling the results of some counter. The final sample from A and the
initial sample from C will be interpolated to produce a rate that is
constant for queries within the gap between A and C. Archive B now
appears in that gap and the same timespan is traversed again. This
time the detailed rates are given for the timespan of B. Is this a
problem? Maybe, maybe not.
Frank, can you come up with a better example?
Dave
|
|