Hi, Ken -
> [...] The pmlogrewrite issue is real and needs to be elevated from
> the TODO to the DONE list.
OK.
> Also, the long-term semantic issues associated with mark records,
> counter wraps, system reboots, and PMDA restarts are all made worse
> in the one big log model.
In what way are these semantic issues different with logs that merge a
few times into a day-long one, versus logs that merge a few times a
week into (say) a fortnight-long one? We already must handle wraps,
reboots, restarts whenever they occur.
> I think we need to look at the use cases for PCP archives.
>
> 1. What the hell happened today? - the classical approach works fine
> 2. What the hell happened on 17 Jan 2014 (and was it the same as today)? -
Could it be that because of the tradition of day-granularity merging,
we've been trained to think about a day as the natural and sole data
granularity? What about "last two hours" being asked at 1AM? Or
"the current week"?
> 3. When did the load average start going over 100? Or when in the
> last month did the load average go over 100? - here we want to
> ITERATE over a set of archives (not process a CONCATENATED archive)
> [...]
Why not?
> 4. Show me trends over time [...]
> There are 2 key parts to addressing this need (a) concatenating the data
> into a single set, and (b) temporal data reduction [...]
Doing this via alternate pmmgr configurations is in the TODO.
> Unfortunately the current incarnation of pmlogreduce is neither robust nor
> semantically correct all the time.
We'll need to fix those problems.
- FChE
|