[pcp] RFC - pmlogrewrite

[Ken McDonell]

The suggestion to split the proc metrics out of the linux pmda has
raised the question (again) of controlled rewriting of PCP archives to
handle changes in metadata over time.

pmlogextract cannot merge archives where the metadata is different for
the same metric across the input archives, e.g. type changes from 32-bit
to 64-bit with some PMDA upgrade, units change, or the name of a metric
changes or the PMID of a metric changes (which is the problem with
splitting the proc metrics out of the linux PMDA).

The proposed solution is to create pmlogrewrite and then allow
pmlogextract to preprocess all input archives with pmlogwrite before
merging.

Of course pmlogrewrite could be used stand alone as well.

The changes to be made by pmlogrewrite are defined in a control file
that would be something like the attached strawman syntax proposal.

I'd be interested in comments/feedback.  For example, I'm not convinced
of the usefulness of the "new" instance specification.

Cheers, Ken.


-------------- next part --------------
Tool to Rewrite PCP Archive Logs

label {
    hostname -> "abc"
    tz -> "timezone"
    time -> +[hh:[mm:]]ss[.usec] | -[hh:[mm:]]ss[.usec]
		: usec is 6 decimal digits
		: adjust the label record(s) and the timestamps in all
		  pmInDom metadata and pmResults
}

indom domain.serial {
    pmindom -> domain.serial
    name "abc" -> "def" | cull
    inst 123 -> 456 | cull
    new inst 123 "foo"
    		: add new instances for an existing domain, or define
		: instances for a new instance domain
}

metric [name|domain.cluster.ordinal] {
    pmid -> domain.cluster.ordinal
    type -> XXX (for some PM_TYPE_XXX)
    		: PM_TYPE_NOSUPPORT => drop all values in pmResults
		: PM_TYPE_UNKOWN and PM_TYPE_EVENT are not allowed
    indom -> NULL | domain.serial
    sem -> XXX (for some PM_SEM_XXX)
    units -> dimSpace, dimTime, dimCount, scaleSpace, scaleTime, scaleCount
}