----- "Ken McDonell" <kenj@xxxxxxxxxxxxxxxx> wrote:
> > > If there was consensus on exactly how this meta data was to be
> > > encoded in the file, then it would be possible to write a generic
> > > "csv" plugin for pmimport.
>
> Leaving aside the implementation on the pmimport side for the moment
> (I'll return to that further down this mail), I think there is a case
> for a "standard" CSV (or spreadsheet, I don't really care) format
> that will work in a data-driven way with a "standard" PCP import plugin.
> ... it would be nice to agree that this is something that should be done.
I think we should do this. I wonder, however, whether encoding the
metadata in the CSV is ideal - perhaps we should add a -c <config>
option to pmimport which will construct all the metadata (metrictab,
indomtab, etc) for the plugin in a "standard" way ... that could
then be used with other (non-CSV) importers too.
Perhaps something like...
hostname db1.com.au
timezone EST-10
instance name="sales database" id=0 indom=7
instance name="finance database" id=1 indom=7
metric name=db.cache.hits id=128.4.23 indom=7 type=counter units=0,0,1,0,0,count
metric name=db.transactions id=128.4.24 type=counter units=0,0,1,0,0,count
metric name=db.bytes_sent id=128.3.0 type=counter units=1,0,0,mbyte,0,0
... etc, etc ...
[parsing this should be a libpcp_import C API routine, and should then
be callable from C frontend or Perl script to setup global metrictab/...]
I have been involved with evaluating some low-end SAN solutions recently,
and these all come with console software providing their own custom perf
monitoring software - all have an export-to-CSV option. I've put some
sample output at http://oss.sgi.com/~nathans/, to throw some additional
real life samples into the mix.
> >
> > The plan I had in mind for the pmimport tool and API was to make
> > a Perl wrapper around a plugin API so that script plugins could
> > be written. With that kind of approach we'd be giving punters
> > access to CPAN for their data import situations (databases, a CSV
> > parser that handles the corner cases, any date formats, and even
> > direct spreadsheet parsing) and it's alot simpler and quicker to
> > write scripts than C code.
>
>
> [warning gross Perl ignorance area ahead]
>
> I can sort of see how
>
> pmimport (C) <-->
> API wrapper (C) <-->
> API wrapper (Perl) <-->
> punter's plugin (Perl)
>
> might work, except that the punter's plugn has to be a package or
> somesuch (the main program is still pmimport).
>
> Alternatively I could see an implementation architecture like
>
> punter's main prog (Perl) with callback routines <-->
> call pmimport_setup() (may not be needed, as primeImportFile() sort
> of
> gets done in the punter's main prog ... perhaps some API to deal with
> setting the host and timezone
> calls to pmimport_register to register the callbacks
> call pmimport_main() ... which is really the existing pmimport main
> program hidden behinds a Perl wrapper
> <--> call backs into the punter's Perl for getPmResult(),
> getPmDesc()
> and getIndom()
>
> Is this even close to what you had in mind?
Yes. We'd pretty much push everything possible into libpcp_import,
and then have a small wrapper main() in C code which then becomes a
pmimport binary (like the existing one), and then have APIs to glue
the scripting code to the underlying C code. I think people would
expect that their Perl code would have a ->run() method or similar
at the end, which pretty much calls the same thing pmimport's main
method does.
>
> Is the front-end driver you're describing here similar to the first
> architecture I explained above (with the main program in Perl rather
> than C)?
>
Both should be feasible really. Probably wouldn't have the C variant
(binary) calling Perl modules, nor the Perl vesion calling C modules.
I'd start out with writing a skeleton script which does IO & parsing,
and then work backwards from there (script programmer would expect to
be doing those two things himself, I think, and not much more - rest
should be C API calls shared with pmimport.c I'd hope).
cheers.
--
Nathan
|