pcp
[Top] [All Lists]

Re: PCP and flamegraphs/Heatmaps

To: Amer Ather <aather@xxxxxxxxxxx>
Subject: Re: PCP and flamegraphs/Heatmaps
From: "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Date: Tue, 9 Sep 2014 17:15:47 -0400
Cc: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>, pcp@xxxxxxxxxxx, Brendan Gregg <bgregg@xxxxxxxxxxx>, Martin Spier <mspier@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <CAM1aq-G0+gqzYYQAhACfAPiOcwmyhHAjF2wE3vcw6FuUuM6a_w@xxxxxxxxxxxxxx>
References: <CAM1aq-G0+gqzYYQAhACfAPiOcwmyhHAjF2wE3vcw6FuUuM6a_w@xxxxxxxxxxxxxx>
User-agent: Mutt/1.4.2.2i
Hi, Amer -

> [...]  Our plan is to integrate flamegraph with PCP. This allows our
> web tool to trigger profiling on-demand basis on the cloud
> instance. PCP agent (python) will then collect 15-20 seconds worth
> of perf data, process it and save svg file in S3 bucket. PCP agent
> returns success/failure and path of S3 bucket to the web
> client. [...]

It sounds a little bit like the hypothetical pcp agent (pmda?) is
being used only as sort of a remote-process-execution widget, not
feeding metric data in or out through pmcd to pmapi clients.  Do I
have that right?

Likely one can make that work, but one can probably come up with more
pcp-ish ways.

For example, if it's sampled backtrace-based profiling that's being
sought here as the data source, a pmda that returns events or strings
for kernel / userspace backtraces, is a fine fit.  (A pmapi client can
poll for these backtrace samples, and run it through the
flamegraph-building chain to make eye candy out of it, and plop it
somewhere the web server can see, like a pmwebd -R resource directory,
or some S3 volume.)

This new PMDA could be hard-coded (based on invoking perf commands,
consuming their output after a while, etc.).

Or perhaps could be a client of the general systemtap/json PMDA that
dsmith's starting to build.  In this alternative, the only target-side
application-specific part would be a systemtap script/.ko?  that
computes backtraces on demand.  The new general PMDA would relay those
(and whatever other stats the stap script shares) as pcp event-or-atom
metric values.  The rest of the process would be as above (pmapi
client gathering the backtraces, drawing svg's).


- FChE

<Prev in Thread] Current Thread [Next in Thread>