[Top] [All Lists]

Re: [pcp] suitability of PCP for event tracing

To: Nathan Scott <nathans@xxxxxxxxxx>
Subject: Re: [pcp] suitability of PCP for event tracing
From: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Wed, 13 Oct 2010 07:37:02 +1100
Cc: Greg Banks <gnb@xxxxxxxxxxx>, systemtap@xxxxxxxxxxxxxxxxxx, pcp@xxxxxxxxxxx, "Frank Ch. Eigler" <fche@xxxxxxxxxx>
In-reply-to: <258913038.1874461286800426531.JavaMail.root@xxxxxxxxxxxxxxxxxx>
References: <258913038.1874461286800426531.JavaMail.root@xxxxxxxxxxxxxxxxxx>
Reply-to: kenj@xxxxxxxxxxxxxxxx

Thanks for the feedback.

On Mon, 2010-10-11 at 23:33 +1100, Nathan Scott wrote:
> ----- "Ken McDonell" <kenj@xxxxxxxxxxxxxxxx> wrote:
> > It has taken me a while to work through all of the issues here, not
> > to
> > mention more distractions than you could hit with a big stick.
> > 
> > Anyway putting an array of event records inside a pmResult is not
> > going to be too hard - see http://oss.sgi.com/~kenj/event-params.html
> | Event trace data includes the following for each event:
> | timestamp
> | event type
> | event parameters (optional)
> | The event type is really just another event parameter so we will not
> | consider this as a separate data type, and event data consists of a
> | timestamp and one or more parameters.
> IMO, we also need Unique ID (refer X-trace and Dapper papers) and parent ID.
> Preferably not optionally (could be inserted in PCP layers if no underlying
> tracing support for end-to-end style tracing.

I don't think tree-structured traces as in X-trace or Dapper are
sufficiently generic that they warrant explicit support in the PCP
infrastructure.  I've looked at the X-Trace data model, and it maps
simply to the PCP data model (TaskID is simply an event parameter that
is repeated and has the same unique value in all events that are part of
the same tree).

I've updated the proposal document to include some examples, including

> Not sure what the "event types" are you're refering to?  An example trace
> (e.g. something simple - blktrace trace, or strace/ptrace info) encoded as
> PCP results/PDUs would really help me here I think.

I've added an strace example.

> | We each of the data components to be a PCP metric, so we can leverage all
> | of the metadata services to describe the type and semantics of the event
> | data. So each data component would have a PMID (and a corresponding name
> | in the external PMNS).
> I think in practice this will prove a bit optimistic (so much data, 
> classifying
> it all - all parameters, flags, etc - to the tight PCP definitions ... not 
> sure
> this is really feasible, or needed). ...

The number of classifications may be quite small, e.g. the generic data
types in the strace example.  We don't need to capture deep semantics
here, but there is some advantage in capturing shallow semantics.  I
suspect that the tools that process the data are the places where the
deep semantics is needed, and this is likely to be embedded within those
applications rather than the PCP data stream.

> ...
> I think a composite metric type like JSON
> (readable) or perhaps XML might be a good option to provide for tracers.
> Microsoft went with XML in ETW presentation tools FWIW, I think, but I'd lean
> more toward JSON having coded both forms recently - they would have chosen XML
> about 10 years ago and might not choose it a second time ;) given the chance.

This is not precluded by the proposal ... we can encode arbitrary
strings as the "parameters" of an event if that makes most sense.

> | We need to accommodate an array of event records in pmResult to support 
> polled
> | retrieval where each pmFetch may return multiple event records. And the 
> number
> | of parameters per event record may not be fixed, e.g. when the number of
> | parameters depends on the event type.
> ...
> | An event parameter can then be encoded using a pmID (this metric needs to be
> Hmmm ... seems a bit heavyweight to require a PMID per parameter.  Allocating
> PMIDs dynamically is also quite tricky (cgroup stats in Linux PMDA does this,
> so I recently coded something like that ... not alot of fun) and the numbering
> space is limited.

I don't think this would be done dynamically.  The PMDA for a particular
source of event records would have to decide which PMIDs and PMNS names
are required to describe the data it knows about.  I could imagine this
being spartan in some cases, and very rich in others.

> Think I lost the plot shortly after this bit, that diagram sure would help.
> Will talk soon.
> cheers.

<Prev in Thread] Current Thread [Next in Thread>