----- "Ken McDonell" <kenj@xxxxxxxxxxxxxxxx> wrote:
> OK, I have a proposal for your consideration and feedback.
>
> This is a bit long, apologies for that.
Looks great, just what I was hoping for.
> New pmimport proposal
>
> Assumptions
>
> - there is no need to directly support the old pmimport API, as the
> only
> plugins were for various versions of sar on irix and unicos
>
> - some pmimport API users may have little knowledge of PCP internal
> data models
>
> - there is a case for both C/C++ and Perl APIs
(or python, or... but, doesn't change anything here, and
we can use Perl as our all-platforms initial implementation
and others can take it further if/when needed)
> - it makes more sense for each customization piece to include a
> main loop fetching data and calling the pmimport API to produce
> a PCP archive (as opposed to the old way of the mainloop in
> pmimport and the customization providing callback implementations)
All valid assumptions I believe.
> int pmiGetHdl(char *name, char *instance)
> int pmiPutValueHdl(int handle, pmAtomValue *atom)
(I agree these should be *Handle)
> For each of the routines above there would be a Perl wrapper
> (pmiFooBar()
> is wrapped by foo_bar()) and some constant functions and helper
> routines,
> so a Perl pmimport program might look like:
>
> use PCP::IMPORT;
Can we call it PCP::LogImport please? IMPORT doesn't really
describe it that well IMO. Happy to keep the libpcp_import
name for the C API though.
> use vars qw( $import $myindom $inst $ctr $now $time );
> $import = PCP::IMPORT->new('');
That seems like maybe it should take more (optional?) args?
(log name, host name, timezone).
> $myindom = 1;
>
> $import->add_metric('myown.counter', PM_ID_NULL, PM_TYPE_U32,
> PM_INDOM_NULL, PM_SEM_COUNTER, units(0,0,1,0,0,PM_COUNT_ONE));
> $import->add_metric('myown.time', PM_ID_NULL, PM_TYPE_FLOAT,
> $myindom, PM_SEM_INSTANT, units(0,1,0,0,PM_TIME_SEC,0));
> ...
(hmm, whats the algorithm for auto-picking PMIDs going to be like?
esp, how will the domain# be chosen & guaranteed not to conflict?
I like the idea of auto selecting IDs, just need to make sure its
deterministic & non-conflicting with existing agents I guess).
> $import->add_instance($myindom, "sleep", PM_IN_NULL);
> $import->add_instance($myindom, "eat", PM_IN_NULL);
> $import->add_instance($myindom, "play", PM_IN_NULL);
> ...
>
> # main loop once per output record
> ...
> # loop per metric
> $import->put_value('myown.counter', PM_IN_NULL, $ctr);
> ...
> # loop over instances
> $import->put_value('myown.time', $inst, $time);
> ...
> $import->write($now);
>
>
Looks good.
> I've toyed with a pmiConfig() routine as a generic metadata parser
> ... I cannot see how to marry up the resultant metadata with
> the data stream.
I'm not sure I understand the problem ...
> Consider a 5 column spreadsheet and the config (in Nathan's syntax):
>
> instance name="some" id=0 indom=7
> instance name="other" id=1 indom=7
> metric name=db.cache.hits id=128.4.23 indom=7 type=uint32 sem=counter
> units=0,0,1,0,0,count
> metric name=db.transactions id=128.4.24 type=uint64 sem=counter
> units=0,0,1,0,0,count
> metric name=db.cache.misses id=128.4.25 indom=7 type=uint32
> sem=counter
> units=0,0,1,0,0,count
>
> The problem is we have defined 5 possible metric-instance pairs
> db.cache.hits[some]
> db.cache.hits[other]
> db.transactions
> db.cache.misses[some]
> db.cache.misses[other]
> but there is no way to
> (a) tell which metric-instance pair is associated with each column of
> the
> spreadsheet (and the problem only gets worse when not all
> possible
> metric-instance pairs are actually in the spreadsheet)
> (b) reach back inside the metadata in the library to know how to
> parse
> each cell of the spreadsheet and prepare the pmResult or the
> parameters for calls to pmiPutValue() or pmiPutValueHdl()
But the call to pmiConfig would effectively replace all the calls to
$import->add_metric and $import->add_instance, no? Those don't give
any hint as to which column in the spreadsheet they match up to either
... thats done via "put_value('metric name', ...)" or a handle, which
would still be the case with a pmiConfig call. I'm missing something
here I think...?
cheers.
--
Nathan
|