Petr,
I'm promoting this discussion to the PCP list, as there may be others
with an interest in the topic.
I've attached the two man pages for the parts of pmimport (which have
been released by SGI, but are not in the PCP tree on oss.sgi.com yet,
pending a decision on what to do with pmimport).
You are correct, in addition to the metric names (probably in the first
row of the data) you'd need some way of describing the metadata which
would include
- PMID (although this could safely be assigned by pmimport if we reserve
a domain number for the purpose)
- data type (32, U32, 64, U64, FLOAT, DOUBLE or STRING)
- instance domain? and if yes, then instance name for this column
- data semantics (COUNTER, INSTANT or DISCRETE)
- data units - dimension and scale for space, time and count ...
although this could be extracted by parsing a string representation such
as "none", "nanoseconds", "bytes", "Mbytes / msec", etc
If there was consensus on exactly how this meta data was to be encoded
in the file, then it would be possible to write a generic "csv" plugin
for pmimport.
On Mon, 2010-06-21 at 11:59 +0200, Petr Matoušů wrote:
> Hello Ken,
> regarding making build available for oss.
>
> On Mon, 2010-06-21 at 07:31 +1000, Ken McDonell wrote:
> > commit 8b800c265ba80fd4e88d0301879d126ed50634f3
> > Author: Nathan Scott <nathans@xxxxxxxxxx>
> > Date: Mon Mar 16 10:06:04 2009 +1100
> >
> > pmimport - a tool/API for creating PCP logs from external data.
> >
> > This is a straight port of the code SGI recently made available.
> > It is now platform-independent though (not clear it even worked
> > on Linux before?). I've added it to the tree, but decided not
> > to add it to the installed set of tools/APIs/docs yet, until we
> > have a real-world use in oss-land (in proprietary-land, we had
> > sar plugins for various flavours of IRIX and Unicos).
> >
> >
>
> I can release some sample data, but it has nothing to do with computer
> world. If that is enough for making it available for oss,
> see the attachment.
> Basically, the data consist of separated columns where the first column
> is timestamp of some format (this varies), typically the excel format,
> but can be any. The next columns are dependent variables, so the header
> could be like this:
> #time | var1 | var2 | var3 | var4 | .... varN
> #
> Usually the file contains some header describing columns and can contain
> more headers if necessary for pmimport.
>
> The number of lines can be big, for example year history with 2min
> interval.
> The data source file format is csv like. Separator may vary.
> N is never fixed, depends on data source.
>
> Typically such a data source is often subject of statistical analysis
> (for example sw Statistica by StatSoft, or R in oss). No such a sw I
> know is giving the possibility to replay the data or handle it and
> visualize by the way as PCP does, therefore I am searching how to do it.
>
> I have attached the file which is actually OpenOffice calc export to csv
> of such a real-world data which comes from industrial device.
> Most of the variables there are temperatures and pressures.
>
> To allow general usage the pcpimport could be able to read some metadata
> info for every variable ... like the units, etc? So maybe the first x
> rows should contain metadata, which will be pmimport using? Just asking.
>
> It would be nice if there is something like general "csv2pcp" plugin
> which could pipe csv data into pcp, and if any metadata for the import
> are needed, they could exist as named rows at the beginning of file
> holding coma separated parameters for each variable (column).
>
>
> Best regards,
> Petr
>
pmimport.1
Description: Text Data
pmimport.3
Description: Text Data
|