----- "Ken McDonell" <kenj@xxxxxxxxxxxxxxxx> wrote:
> Petr,
>
> I'm promoting this discussion to the PCP list, as there may be others
> with an interest in the topic.
*nod*, yep, very interested.
> I've attached the two man pages for the parts of pmimport (which have
> been released by SGI, but are not in the PCP tree on oss.sgi.com yet,
> pending a decision on what to do with pmimport).
We should put them in the tree, just not installed (like the code).
> If there was consensus on exactly how this meta data was to be
> encoded in the file, then it would be possible to write a generic
> "csv" plugin for pmimport.
The plan I had in mind for the pmimport tool and API was to make
a Perl wrapper around a plugin API so that script plugins could
be written. With that kind of approach we'd be giving punters
access to CPAN for their data import situations (databases, a CSV
parser that handles the corner cases, any date formats, and even
direct spreadsheet parsing) and it's alot simpler and quicker to
write scripts than C code.
Then an example Perl script or two for importing specific formats
(using http://search.cpan.org/~hmbrand/DBD-CSV-0.29/lib/DBD/CSV.pm
for CSV, perhaps, to give a DBD example importer - and/or using
http://search.cpan.org/~hmbrand/Spreadsheet-Read-0.40/Read.pm for
a direct spreadsheet->pcp archive export example).
The PCP::PMDA module would be a good starting point - it wraps up
the libpcp_pmda API, in much the same way as we'd want to capture
the pmimport plugin API. Some pmimport code refactoring would be
needed to make it more amenable to use from scripts though (perhaps
the pmimport binary should become a libpcp_import with a Perl
front-end driver script, which can use either shared lib or script
plugins, for example) ... its a fair bit of work, but I think the
end result would be alot better than just a CSV parser.
Also, if libpcp_import refactoring done specifically for scripts,
it should be relatively straightfwd to do Python bindings, etc.
> > ...
> > I have attached the file which is actually OpenOffice calc
> > export to csv
> > of such a real-world data which comes from industrial device.
> > Most of the variables there are temperatures and pressures.
This attachment got lost in the forwarding (guess its not needed
really, you gave the gist of it) - be nice to not have to export
to CSV and then export again to PCP (possibly with hand-tweaking
in-between), but rather go directly with the Spreadsheet::Read &
a new PCP::LogImport module!
cheers.
--
Nathan
|