[pcp] [resend] Derived Metrics - RFC

Ken McDonell kenj at internode.on.net
Sat Nov 7 00:25:53 CST 2009

On Fri, 2009-11-06 at 10:20 +1100, Nathan Scott wrote:
> We haven't used the existing infrastructure for this kind of thing
> yet (and really should at some point) - the ld(1) --version-script
> does allow multiple API versions to be specified in a single .so &
> we haven't explored using that at all so far.
> > For this reason I'm hesitant to change the libpcp API/ABI.
> > 
> > But I'm willing to be persuaded on this point.
> An example of the above --version-script file syntax:
> http://git.savannah.gnu.org/cgit/acl.git/tree/exports (that's as
> used by libacl on Linux - pretty straightforward syntax and I think
> does some of the things you're after here... maybe, hopefully?).

Fair point ... I'll do some research.

> > There is certainly no architectural obstacle to putting the
> > functionality into libpcp rather than as wrappers to the libpcp
> > routines as I was proposing.
> I'd be much happier with that, esp. having the functionality made
> available transparently & in the same way for all platforms.

OK, consider this decided ... the wrapper option is dead.

> > 
> > My goal here was to NOT require any application changes ... hence the
> > streams model analogy.  The examples you give suggest that the
> > applications might be modified to understand about derived metrics.
> > 
> > On reflection I think we may be able to do both.
> > 
> Should focus on doing it in the ideal way, and if applications need to
> change to make use of the new functionality, then so be it.

I am not sure there is an "ideal" way.  There are pros and cons, but
more on this topic below.

> > As an early initialization (this bit is a little tricky, but I think
> > pmLoadNameSpace or pmNewContext have to be called before anything we
> > care about could be called), I'll check for $PCP_DERIVED_CONFIG being
> > defined in the environment and if it use, register (see below) the
> > derived metrics from that configuration file.  As a prop for
> > platforms
> > that don't have environment variables the routine
> > 	pmLoadDerivedConfig(char *path)
> > may be used to explicitly register derived metrics from a config
> > file ... so this could be used with command line glue to load one (or
> > indeed more) configuration files as and when an application chooses.
> Its not clear that PCP_DERIVED_METRIC buys us anything?  Seems a very
> painful-to-use interface when compared to just specifying a formula for
> the new metric "inline" as one would specify any other metric name.

There are problems with the inline method ....

1. I don't want to have to remember the formula for derived.tricky every
time I want to use it ... putting the declaration in a file and having
derived.tricky just appear in the PMNS would be preferable.  This is
even more compelling when you consider common derived metrics that one
might want to make available for all PCP apps, perhaps across multiple
client machines.
2. While inline does look to work for pmval, it is probably the easiest
case.  I don't think this will work at all (or at best requires a whole
lot of additional work that is not needed in the derived metrics just
appear in the PMNS) for pmie nor pmlogger nor pmdumptext nor pmlc.
3. I'm not preventing inline syntax when that makes sense, but since the
majority of apps are unlikely to want to use this, I'd prefer to push
the effort back to the app that wants to use the inline syntax.

> > To register a derived metric, the routine
> > 	pmRegisterDerived(char *name, char *expr)
> > is used.  The name needs to come from the expression definer, because
> > this (a) acts as the handle for subsequent PMAPI calls, and
> > pmLookupName() in particular, and (b) defines where the derived
> > metric should be spliced into the PMNS.
> > 
> > By exposing this routine, ...
> > 1. I am committing to the "change libpcp" path, so the earlier
> > comments
> >    about ELF and $LD_PRELOAD are not longer relevant
> > 2. pmval could be modified to parse
> > 	pmval "cputime = simple.time.user + simple.time.sys" ...
> >    (the delta() is not needed here, but would be needed in the
> >    delta(v1) / delta(v2) class of expressions) and then pmval could
> > call
> >    pmRegisterDerived("cputime", "simple.time.user + simple.time.sys")
> > 3. pmchart can go crazy with the definition of derived metrics via
> > the
> >    GUI (and possibly saving them for later reloading or auto
> > reloading)
> Thats good, but still seems a little clumsy - not clear that routine
> is needed - could pmLookupName just do it all, if it detects a name
> assignment, along the lines:
> char *names = { "new.metric = sample.one + sample.two", "sample.one" };
> if (pmLookupName(2, &names, &pmids) < 0) ...

But many tools use pmGetChildren and pmTraversePMNS to expose the PMNS,
and there is no similar hook that could be used in these case to insert
the derived metric definition automatically.

> (obviously, thats more hard-coded than most tools would use... but any
> metric "name" containing an assignment is invalid as a traditional name
> and would be easy to detect in pmLookupName).
> > Is that closer to meeting your needs?
> Yep, definately getting there.

There are actually enough problems already in getting this right, so I'm
going to try an initial implementation with pmRegisterDerived() and
pmLoadDerivedConfig() in the API.  I'll skip the $PCP_DERIVED_CONFIG
stuff for the moment, and just use -d derived_config_file at the command
line for a few apps to get pmLoadDerivedConfig() called see how it all

The "problems" encountered so far are ...

+ cannot use flex/lex or yacc/bison for parsing expressions due to ugly
symbol space pollution in libpcp that will likely kill any application
that uses flex/lex or yacc/bison (so pmlogger, pmie, pmlc, dbpmda,
pmlogextract, hotproc PMDA) ... sigh
+ need one set of data structures _per_ context, and some syntax and
semantic checks must be deferred from parse time to the time
pmNewContext has been called successfully
+ tricky semantics around some aggregate functions
+ do we need constants in the expressions?  if so, they may need to be

More information about the pcp mailing list