pcp
[Top] [All Lists]

Re: [pcp] pcpstats

To: "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Subject: Re: [pcp] pcpstats
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Mon, 26 May 2014 19:52:07 -0400 (EDT)
Cc: Michele Baldessari <michele@xxxxxxxxxx>, pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <y0m7g58mmq5.fsf@xxxxxxxx>
References: <20140524215548.GA13540@xxxxxxxxxxxxxxx> <1951313532.14634422.1401085287050.JavaMail.zimbra@xxxxxxxxxx> <y0m7g58mmq5.fsf@xxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: WjJmzVGGcsRriQmIrPVGKf2utqRmNQ==
Thread-topic: pcpstats

----- Original Message -----
> 
> nathans wrote:
> > [...]
> >> https://github.com/mbaldessari/pcpstats
> > Neat idea!
> 
> Yes.  Before too long, imagine a web-interactive version of almost
> exactly that.
> 

Stop, stop - you're making Marko drool all over his keyboard.

> > [...]  - Values of counter semantics metrics (PM_SEM_COUNTER) are
> > commonly reported after rate conversion (requires successive
> > samples, and the value plotted is the value-delta divided by the
> > time-delta), so that MB/sec, or utilizations, are reported instead
> > of raw values. [...]
> 
> Given that this comes up for multiple applications in multiple
> languages, have you considered including this functionality within
> libpcp proper?

Yeah, anything is up for consideration.  Difficulty lies in how to
tackle this; the PMAPI is a bit low-level for this, needs a higher
level API.  Other difficulties include mapping C constructs to the
object-oriented things, and the mismatch there (helper functions in
the C API not enough for this sort of thing; new, relatively complex
data structures would be needed).  Then the fact that a pmResult will
contain both counter & non-counter metrics, and for every sample we
need a value for each, even when its initially "no value yet" for
counters awaiting a second sample, so ultimately it'll end up in
multiple passes over either the pmResult structure or some other data
structure.  And if its some other data structure, which it probably
would be, thats often ideally a language-specific high-level data
structure like a hash, accessed with language primitives, not via
a special helper API.

Its a non-trivial problem.  pcp.pmcc solves a large portion of it
already, and can be made object-oriented & pythonic in ways direct
C APIs cannot - then there's the ~50 C tools in pcp that have not
wanted such an API to date (its *too* high-level for many tools).
*shrug*, so its not clear libpcp is a good spot - maybe a new lib
above libpcp, but even then ... which tools is it targeting?  The
C++ tools use libpcp_qmc already for this purpose, so are not
likely to use it (unless it could be plugged into that, under the
covers - although that's all object-oriented C++ code as well and
uses Qt classes/data structures).  In the past, higher level APIs
haven't really taken off (like optfetch in libpcp ... used by only
a few tools, and that's been there forever).

But, as always, experiment and have a crack if you'd like, of course
- I'd recommend a library layering on the functionality above libpcp.
I also recommend picking some real, existing tools that would suit
to make use of it and converting them, rather than a "build it and
they will come" model, cos if they don't, it's a maintenance load
forever (case in point - libpcp_trace, which really needs a brain
transplant and then some actual users).

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>