I wrote:
> - The API is really wordy for basic tasks such as extracting and
> scaling some metrics. [...] We need a higher level API that
> combines the lookup/indom/fetch/extract/scale steps (doing internal
> caching/lookups as necessary), with the goal of making it easy to
> interface with an enclosing application. [...]
Time to unveil a first sketch at a possible partial solution. (Are
those enough qualifiers?) Let's say hello to "fetch groups": a way
for a PMAPI client to identify metrics it will want to load, what
scaling/conversion it would like, and the place to put the results.
The client will then fetch the group one or more times, and consume
the data in its preassigned locations.
Hello, fetch groups!
int pmCreateFetchGroup(); // create a fetchgroup for current pcp context
// specify what to fetch
int pmExtendFetchGroup_TYPE(int, const char*, const char *, const char *, TYPE
*);
int pmExtendFetchGroup_indom_TYPE(int, const char*, const char *, char***, TYPE
**);
// (where TYPE goes from uint32 ... double)
int pmExtendFetchGroup_timestamp(int, struct timeval*); // specify where to put
timestamp
int pmFetchGroup(int); // (re)fetch the group as many times as required
int pmDestroyFetchGroup(int); // destroy the fetchgroup
All the API magic is clearly in the pmExtendFetchGroup_* functions, so
let's take a look at one:
int pmExtendFetchGroup_int32(
int pmfg,
const char *metric,
const char *scale,
const char *instance,
__int32_t *value)
Example usage from a dramatically shrunk pmclient.c:
pmExtendFetchGroup_uint32(pmfg, "disk.all.total",
"count/second", NULL, &info.dkiops);
and
pmExtendFetchGroup_float(pmfg, "kernel.all.load",
NULL, "1 min", &info.load1);
The former asks libpcp to prepare for fetching the disk.all.total
metric, type-convert, rescale, and rate-convert the data to
counts/second, and deposit the result into the given unsigned int.
(Rate-conversion would come built-in, as would a reverse of the
pmUnitsStr function.)
The latter asks libpcp to prepare for fetching the kernel.all.load
metric, no scaling, just to pick one named instance from the indom,
and again deposit the result into the given float.
OK, OK, that's pretty clear. How about pulling whole instance domains?
char **cpu_user_name = NULL;
float *cpu_user = NULL;
pmExtendFetchGroup_indom_float(pmfg, "kernel.percpu.cpu.user",
"second/second", &cpu_user_name, &cpu_user);
This asks libpcp to prepare for fetching all the instances of the
named metric, do conversion to plain unit rates (from
millisecond-counter), and deposit the instance names & values in
parallel arrays. (libpcp would allocate both those arrays with
realloc.)
OK, with that one-time preparation out of the way, how would the main
loop look? Too simple:
if ((sts = pmFetchGroup(pmfg)) < 0) { /* tragedy */ }
printf ("%u %f\n", info.dkiops, =info.load1)
for (i=0; cpu_user_name[i] != NULL; i++)
printf ("%s:%f\n", cpu_user_name[i], cpu_user[i]);
... and I guess do a pmDestroyFetchGroup at the end, for
valgrind-correctness.
(Internal implementation would involve some data structures stored
within libpcp associated with that pmfg integer, including all the
pmDesc and pmID goo, prior pmResults for rate conversion, etc. The
pmExtendFetchGroup would do name/desc resolution & prep. pmFetchGroup
would do a pmFetch(), then a table-driven extraction of the pmResult
into the user-specified locations. I haven't started building it out
but nothing in there worries me.)
Some questions remain.
First of all, what do you think of the general flavour?
Second, do you endorse the type-safe nature of passing float*
etc. destinations for the metric values, tolerating N sibling
foo_TYPE() functions, instead of something icky like the pmAtomValue
unions or void pointers?
Third, I haven't fully sketched out what to do with
PM_TYPE_{STRING,AGGREGATE}; probably have the user pass a destination
buffer/length, or else have libpcp allocate, or (why not) both as
alternatives. Fine?
Fourth, I don't know what to do with events. They're fiendishly
complicated in the pmValue structures, and would have a whale of a
time encoding its recursive subdivision as one function call. Maybe
skip it? Maybe some nasty varargs monstrosity?
Fifth, it is entirely possible for values to come back absent from a
pmFetch, and thus a pmFetchGroup. Encoding the absence of data is
probably necessary, but I'm not sure whether better to do it with a
formal flag (another bool* argument at the *Extend* functions), or
maybe just use sentinel values (0, NaN) on the output. Or both?
Sixth, not sure how/whether to represent indom profiles. Maybe not
needed; just have the user invoke the non-_indom variant of the
*Extend* functions, enumerating the instances and places where to put
each's data. Or maybe have a variant of the _indom_ functions where
the instance names are already provided?
Seventh, several of the above represent optionalities that could lead
to a cartesian-product-style explosion of the number of individual
functions in the API. I don't mind that FWIW, but someone might.
Eigth, aw heck, why not say that *Extend* with pmfg index #0 means
"do it right now", i.e., the equivalent of
tmp = pmCreateFetchGroup()
pmExtendFetchGroup(tmp, .....)
pmFetchGroup(tmp)
pmDestroyFetchGroup(tmp)
What could be simpler for one-shot inquiries?
Ninth, this should apply OK to archives too (for consecutive fetches);
should we bother wrap pmSetMode, or just let the client call it
directly before pmFetchGroup()?
Feedback welcome!
- FChE
|