pcp
[Top] [All Lists]

Re: [pcp] pcp2influxdb - a clone of pcp2graphite

To: Nathan Scott <nathans@xxxxxxxxxx>
Subject: Re: [pcp] pcp2influxdb - a clone of pcp2graphite
From: Alec Ten Harmsel <alec@xxxxxxxxxxxxxxxxxx>
Date: Wed, 6 Jul 2016 13:57:07 -0400
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=1Kud2rFo/L5kHOhZoPm9NEQi8lc=; b=oewcV Hs4vy5b3Q1+C//REAK8Z61O+wuFShz2g7A5RKOFerSSSo+vR+cMlXSiQe+AWWjG5 FL2/mb2WWDagiQOciCr2RF4Npvznh0eGqd127xkWCFPRJKYCTlYy8tqQmk0kv9+5 FVZ9jO7bKm5HTH+ArVKwCK6tZnkuIxy/VkFlj8=
In-reply-to: <343267617.3556068.1467601341893.JavaMail.zimbra@xxxxxxxxxx>
References: <20160630184840.GB880@xxxxxxxxxxxxxxxxxxxxxxxxxxx> <1219763127.3321732.1467327849186.JavaMail.zimbra@xxxxxxxxxx> <b7aedda5-507c-f001-d8c9-940d760cf6fb@xxxxxxxxxxxxxxxxxx> <343267617.3556068.1467601341893.JavaMail.zimbra@xxxxxxxxxx>
User-agent: Mutt/1.5.24 (2015-08-30)
Hi Nathan,

On Sun, Jul 03, 2016 at 11:02:21PM -0400, Nathan Scott wrote:
> Hi Alec,
> 
> ----- Original Message -----
> > [...]
> > >> We've been testing it for a couple of days and have not had any issues
> > >> with the patch.
> > > The one thing the patch lacks is the influxdb equivalent of regression
> > > test pcp/qa/667 which would take pcp2influxdb through its paces.  Could
> > > you add something like that little shell script, for influxdb?
> > >
> > 
> > I will certainly do this, but it'll take a few days. We are pretty busy
> > getting ready for some maintenance that will happen in a couple of weeks.
> > 
> 
> OK, thanks and no problem.  Let me know if any assistance is needed there -
> there's some high level notes about the test scripts in qa/README too.
>

I have added a test - qa/9001 - in the new version of the patch,
attached to this mail. I did not want to pick a number in the low 1000's
just in case it would be taken by now; that can be changed to whatever.
Sorry if that causes any trouble. Updates to qa/group and other metadata
are not included in the patch so that it can be easily re-numbered.

The test that I added is not as full featured as qa/667. `socat' quits
after receiving EOF, which makes it not useful for testing
pcp2influxdb. pcp2influxdb aggregates metrics for a single request as
much as possible, but two separate calls to Relay.send() will generate
two separate connections. Without using another program or writing a
little python HTTP server to use for testing, I wasn't sure how to add
that to the test script.

I did some refactoring, with the only change being that
multiple-instance metrics are now recorded in a better way. A metric
like `disk.partitions.read' would have created a handful of
"measurements" in InfluxDB, each with a single "field" with a key of
"value":

* disk_partitions_read_sda1
* disk_partitions_read_sda2
* disk_partitions_read_sda3

Now, just a single `disk_partitions_read' "measurement" is created, with
multiple "fields":

* _sda1
* _sda2
* _sda3

In InfluxDB query language, "measurements" are similar to SQL tables and
"fields" are similar to SQL columns.

If there's anything else - docs, bugs, etc. - I will of course be happy
to take a look.

Thanks,

Alec

Attachment: pcp2influxdb-2.patch
Description: Text Data

<Prev in Thread] Current Thread [Next in Thread>