[pcp] pcp updates - derived metrics

Ken McDonell kenj at internode.on.net
Thu Dec 31 01:21:43 CST 2009


Here's a little something Santa left under the tree.

I believe this is complete, and passes QA (-x remote) with the
accompanying batch of pcpqa changes that are on their way.

But there is a lot of code here ... it is most of pmie re-implemented in
libpcp (!) or three times bigger than interp.c ... so I'd be astounded
if there are not things I've missed.

Please yell if you find a problem, especially a regression in existing
stuff which really should not have happened.

If you want to get a quick look at what's been done, check the
pmRegisterDerived man page.

Changes committed to git://oss.sgi.com/kenj/pcp.git dev

 man/man1/pcpintro.1            |   17 
 man/man3/GNUmakefile           |    5 
 man/man3/pmderivederrstr.3     |   38 
 man/man3/pmloadderivedconfig.3 |   70 
 man/man3/pmlookupname.3        |    2 
 man/man3/pmparsehostspec.3     |    2 
 man/man3/pmregisterderived.3   |  429 +++++
 src/include/GNUmakefile        |    2 
 src/include/impl.h             |   18 
 src/include/pmapi.h            |    7 
 src/libpcp/src/GNUmakefile     |    9 
 src/libpcp/src/context.c       |   14 
 src/libpcp/src/derive.c        | 3121 ++++++++++++++++++++++++++++++-----------
 src/libpcp/src/derive.h        |  102 +
 src/libpcp/src/derive_fetch.c  | 1388 +++++++++++++++++-
 src/libpcp/src/desc.c          |   21 
 src/libpcp/src/fetch.c         |   39 
 src/libpcp/src/fetchlocal.c    |    3 
 src/libpcp/src/logutil.c       |   25 
 src/libpcp/src/p_pmns.c        |   10 
 src/libpcp/src/pmns.c          |  322 +++-
 src/libpcp/src/util.c          |   34 
 src/libpcp_gui/src/record.c    |    7 
 src/pmcd/pmdaproc.sh           |   70 
 src/pmdas/mmv/GNUmakefile      |    4 
 src/pmdas/mmv/help             |   28 
 src/pmdas/mmv/mmv.c            |   93 -
 src/pmdas/sample/help          |   24 
 src/pmdas/sample/pmns          |   34 
 src/pmdas/sample/src/sample.c  |  145 +
 src/pmdbg/pmdbg.c              |    6 
 src/pminfo/pminfo.c            |   13 
 32 files changed, 5038 insertions(+), 1064 deletions(-)

commit 0bf5e5b70de6e46b58c15e6c999575ae7ecaed25
Author: Ken McDonell <kenj at internode.on.net>
Date:   Thu Dec 31 17:32:17 2009 +1100

    Derived metrics - fix last TODO
    
    Problem with matching instances for binary operators when the number
    of instances (and hence the instance identifiers) do not match between
    the left and right operands.

commit 730bf349c78e121e9f9781829e1253eed35a42d4
Author: Ken McDonell <kenj at internode.on.net>
Date:   Thu Dec 31 15:07:32 2009 +1100

    Derived Metrics - last stage
    
    This checkin removes all but 1 of the TODOs, completes the metadata
    handling (especially dimension and scale), fixes a problem with the
    value scaling (need both multiplier and divisor in the general case),
    adds man pages, etc.

commit 5876649b19eda9af943719ee14739a3254023ebe
Author: Ken McDonell <kenj at internode.on.net>
Date:   Wed Dec 30 12:00:44 2009 +1100

    sample pmda - more metrics
    
    Added a bunch of additional metrics to support all of the semantic
    and aggregate function QA needed for derived metrics.

commit fd7425a6fd00ab212c88329a583fbd20995a7871
Author: Ken McDonell <kenj at internode.on.net>
Date:   Wed Dec 30 11:59:22 2009 +1100

    Derived metrics - Stage VI
    
    All of the aggregate functions - avg(), count(), delta(), max(), min()
    and sum() are now implemented.
    
    Started the man pages for the derived metrics routines in libpcp.

commit f3d88a4dbfe521a377ea5f08fbbe70614df2e89d
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Dec 21 21:56:23 2009 +1100

    Derived metrics - Stage V
    
    Complete some of the TODOs in the earlier work, plus added initial
    support for delta(metric).  These changes have been tested with QA
    passing -x remote.

commit 0ce5b4558c1805a1fcb6068b54cc5922d49fb03c
Author: Ken McDonell <kenj at internode.on.net>
Date:   Wed Dec 16 22:12:14 2009 +1100

    Derived metrics - Stage IV
    
    With these changes, basic fetching is working and the PMNS services
    have been fixed to the point where pmchart is able to plot derived
    metrics.

commit d57b16a43e091bb41d0d16b748630dc3d6eb834e
Author: Ken McDonell <kenj at internode.on.net>
Date:   Thu Dec 10 17:37:11 2009 +1100

    Derived Metrics - interim commit
    
    With these changes, the basic fetch and value propagation and
    pmResult rewriting is done.
    Still no arithmetic in the evaluation phase.

commit 5f66071c2253807490f542bb463921c75f76e4be
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Dec 7 07:33:44 2009 +1100

    Derived metrics - stage II
    
    Parser is done.
    Basic hooks in place around context create and destroy.
    PMNS services done.
    pmLookupDesc working although some more work is needed on the mapping
    and checking of pmUnits.

commit 74bd314e6cabaaaa7d7d76ccd56b898de1bf1a8d
Merge: 248bff8 b6f50cf
Author: Ken McDonell <kenj at internode.on.net>
Date:   Tue Dec 1 17:25:04 2009 +1100

    Merge branch 'dev' into derive

commit b6f50cf52934be8f47c3e4e67d8e51c882d54718
Author: Ken McDonell <kenj at internode.on.net>
Date:   Tue Dec 1 17:20:11 2009 +1100

    mmv pmda - add mmv.debug
    
    Add the metric, store support so it can be set on the fly, help text.
    Also added some initial diagnostics to help with debugging.

commit 55b57946782ac3ade01c30cbbd8864d1fce32b7f
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Nov 30 07:06:42 2009 +1100

    mmv help text file removed

commit 764e3002c0cb1fd93bd4f1f2f3b2c21116b1b223
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Nov 30 07:03:54 2009 +1100

    mmv.reload help text changes
    
    Because mmv.reload is below the root of a dynamic PMNS subtree at
    mmv, the name mmv.reload cannot be resolved by newhelp before the
    PMDA is installed.
    
    So the PMDA needs to supply the help text for _all_ metrics of the
    form mmv.* ... and the "help" file is not needed.

commit 51fca57b05721897bcef40fb4fdddc8602c2f66c
Author: Ken McDonell <kenj at internode.on.net>
Date:   Sun Nov 29 06:59:18 2009 +1100

    record.c replace mktemp() with mkstemp() when available
    
    Removes a compilation warning in the build, and use of mktemp() that
    has been removed elsewhere in pcp.

commit 248bff81782ac24af97b2a8e01f936991e92e1bc
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Nov 16 16:45:05 2009 +1100

    Derived Metrics - preliminary checkin
    
    Nothing is really working yet ... add some new functions for libpcp,
    add -c dmfile for pminfo to force loading of a set of derived metric
    definitions and some pmns hooks.

commit d0aa0a18a283e8188965a0f62a37ebe3eb7ffa36
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Nov 16 16:44:35 2009 +1100

    Fix troff formatting error.

commit c46e28a597bf2d85a67a70707bcda0d062edb8b8
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Nov 16 16:41:14 2009 +1100

    Revert "pmdaproc.sh Fix pminfo usage"
    
    This reverts commit d439aa938f9e9edd20b4bb8761251839008ea378.
    
    Back out my changes to redo after discussion with Nathan.

commit d439aa938f9e9edd20b4bb8761251839008ea378
Author: Ken McDonell <kenj at internode.on.net>
Date:   Mon Nov 16 06:18:00 2009 +1100

    pmdaproc.sh Fix pminfo usage
    
    pminfo is used in several places within the script ...
    	pminfo
    	pminfo -h localhost
    	pminfo -n $NAMESPACE
    	pminfo -h localhost -n $NAMESPACE
    
    Remove all the -h localhost stuff as this is simply not needed.
    
    Remove the -n $NAMESPACE stuff as this is wrong (need to be interrogating
    pmcd's view of the PMNS) and this does not work correctly in the presence
    of metrics that use the dynamic PMNS features.
    
    Also cleaned up some of the "Usage" reporting.

commit 4e31303ac9dbc7517bfb1c52f6e6c4e817fa4dc1
Author: Ken McDonell <kenj at internode.on.net>
Date:   Tue Nov 10 14:25:52 2009 +1100

    Add DBG_TRACE_DERIVE
    
    Tracing infrastructure for derived metrics.
    Note use with DBG_TRACE_APPL2 for very verbose output, at least in
    the intial stages of developing derived metrics support.




More information about the pcp mailing list