systemtap/pcp integration pmda 0.01

[David Smith <dsmith@xxxxxxxxxx>]

Here's version 0.01 of my systemtap/pcp integration work that uses
systemtap (https://sourceware.org/systemtap/) to export JSON data and a
pcp (http://www.performancecopilot.org/) python pmda that reads and
processes the JSON data.

At this point, the systemtap side of things is *way* behind the pcp
side. The attached systemtap script has hand-crafted JSON schema and
data output functions. The script itself is based on some work that Will
Cohen has been doing to measure network latency. Note that the script
does output live data.

The pcp side of things in the attached python pmda is much farther along
than the systemtap side of th ings. It parses the JSON schema and data
that the systemtap script creates, and creates single-valued numeric or
string metrics for pcp.

Here's what the schema looks like that the systemtap script creates:

====
{
  "type": "object",
  "title": "root",
  "additionalProperties": false,
  "properties": {
    "generation": {
      "type": "integer",
      "additionalProperties": false
    },
    "data": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "interface": {
          "type": "string",
          "description": "network xmit device",
          "minLength": 0,
          "additionalProperties": false
        },
        "xmit_count": {
          "type": "integer",
          "description": "number of packets for xmit device",
          "minimum": 0,
          "default": 0,
          "additionalProperties": false
        },
        "xmit_latency": {
          "type": "integer",
          "description": "sum of latency for xmit device",
          "minimum": 0,
          "default": 0,
          "additionalProperties": false
        }
      },
      "required": [
        "interface",
        "xmit_count",
        "xmit_latency"
      ]
    }
  }
}
====

Here's what the data produced by the systemtap script looks like:

====
{
  "generation": 1,
  "data": {
    "interface": "eth0",
    "xmit_count": 652046,
    "xmit_latency": 90
  }
}
====

Once the systemtap script is running and the pcp pmda is installed,
here's what the data produced in pcp looks like:

====
# pminfo -df stap_json

stap_json.root.xmit_latency
    Data Type: 64-bit int  InDom: PM_INDOM_NULL 0xffffffff
    Semantics: counter  Units: none
    value 2040

stap_json.root.xmit_count
    Data Type: 64-bit int  InDom: PM_INDOM_NULL 0xffffffff
    Semantics: counter  Units: none
    value 8856603

stap_json.root.interface
    Data Type: string  InDom: PM_INDOM_NULL 0xffffffff
    Semantics: instant  Units: none
    value "eth0"
====

So why is this version 0.01? Because there is a long way to go.

On the systemtap side of things, we don't really want to make a user
hand-craft his own JSON schema and data strings. So, a lot of work is
needed there.

On the pcp side of things, errors aren't handled well (the script will
just abort in several places), it only supports single-valued metrics,
it only supports a single systemtap script, etc. So, a lot of work is
needed there also.

But, I thought I'd go ahead and show this work in hopes of getting
feedback before I go too far down a wrong path.

(Note that I didn't attach some of the files needed by the pcp pmda to
actually get it installed, but I've checked them into the pcp.dsmith git
repository.)

-- 
David Smith
dsmith@xxxxxxxxxx
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

pmdastap_json.python
Description: Text document

net_xmit_json3.stp
Description: Text document