pcp
[Top] [All Lists]

pmie support for string-typed metrics

To: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>, Lukas Berk <lberk@xxxxxxxxxx>, Mark Goodwin <mgoodwin@xxxxxxxxxx>, Dave Brolley <brolley@xxxxxxxxxx>
Subject: pmie support for string-typed metrics
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Wed, 13 Apr 2016 00:46:46 -0400 (EDT)
Cc: PCP <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <220069218.39805602.1460521052029.JavaMail.zimbra@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: HSFF6hsZsCdljDhsHMSymJYhlNuLYQ==
Thread-topic: pmie support for string-typed metrics
Hi Ken,

We've come across a few cases in the last little while that led
Lukas to suggest yesterday that having pmie firing one-time-logging
requests at pmlogger could be a useful general facility.

But it hit a snag - namely, pmie only supporting numeric metrics.
So we're wondering, how involved would it be to extend pmie for some
(possibly just limited?) support for interpreting string metrics?
These are the cases I know of:

- Mark wanted to start logging disk.dev.scheduler all the time to
  record when supported customers change scheduler at his request
  (as in, exact time).  But it very rarely changes under normal
  conditions, so logging the same value all the time in-between
  is fairly wasteful.

- Dave hit problems at daylight saving cross-over, in terms of having
  the correct timezone reported for multi-archive contexts.
  The first part of solving that (which is an old issue, but now much
  more visible, for the "multi-archive auto-spanning many days" case)
  could involve more frequently logging of pmcd.timezone to detect the
  change.
  The second part might be to extend pmPrintstamp and friends to be
  able to use a more dynamic notion of timezones, somehow, instead of
  set once per context - to correctly report around the switch (exact
  details there are still being pondered.)

- Folk like Aconex have seen cases in the past where they want to know
  exactly if/when a java upgrade occurred (perhaps looking back across
  several months of data) - which is represented as a string too.


They're all somewhat similar cases - all use PM_TYPE_STRING metrics -
and so could theoretically be tackled with pmie rules like:

timezone = ( pmcd.timezone @0 != pmcd.timezone @1 )
-> print "Timezone changed to %v";

versions = ( mmv.MyApp.java.jvm_version @0 != mmv.MyApp.java.jvm_version @1 )
-> print "Java upgraded to %v";

iosched = ( some_inst ( disk.dev.scheduler @0 != disk.dev.scheduler @1 ) )
-> print "I/O scheduler changed to %v for block device %i\n";


And if say, instead of "print", we did something like:

metric = "disk.dev.scheduler";
iosched = ( some_inst ( $metric @0 != $metric @1 ) )
-> shell "echo log mandatory on once $metric | pmlc -P"

... we could potentially have a neat solution there.  Esp. considering
the pmie sampling frequency could be relatively high (so we'd log once
quite close to the transition points, instead at a low frequency to
offset the needless log growth).  Thoughts?

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>