pcp
[Top] [All Lists]

Re: PCP Network Latency PMDA

To: David Smith <dsmith@xxxxxxxxxx>, "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Subject: Re: PCP Network Latency PMDA
From: William Cohen <wcohen@xxxxxxxxxx>
Date: Tue, 12 Aug 2014 15:09:04 -0400
Cc: pcp@xxxxxxxxxxx, Nathan Scott <nathans@xxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <53A35C00.1070703@xxxxxxxxxx>
References: <53A34A47.3060008@xxxxxxxxxx> <y0mk38caa4t.fsf@xxxxxxxx> <53A353C8.8030704@xxxxxxxxxx> <53A35C00.1070703@xxxxxxxxxx>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0
On 06/19/2014 05:54 PM, David Smith wrote:
> On 06/19/2014 04:19 PM, William Cohen wrote:
>> On 06/19/2014 04:59 PM, Frank Ch. Eigler wrote:
>>> William Cohen <wcohen@xxxxxxxxxx> writes:
>>>
>>>> I have been looking at a PMDA that provides information about how long
>>>> it takes for packets to make their way from userspace to the network
>>>> device and from network device to userspace.  [...]
>>>
>>> Neat.
>>>
>>>> network.interface.in.latency instance "devname"
>>>> network.interface.out.latency instance "devname"
>>>> [...]
>>>> (latency_sum[t]-latency_sum[t-delta])/(packets[t]-packets[t-delta])
>>>
>>> This sounds like just the sort of thing that we'd use dsmith's richer
>>> pcp-systemtap interface widgetry, currently being designed, and this
>>> computation could be one of its examples.  Keep the ideas coming, and
>>> work with dsmith.  (I would not suggest building just this into a
>>> separate PMDA.)
>>>
>>>
>>> - FChE
>>>
>>
>> Hi Frank,
>>
>> Is the a pointer to the pcp/systemtap integration that David Smith
>> is working on?
> 
> At this point most of it is in my head. But, here's the original plan
> (although "plan" is probably too strong a word at this point). The plan
> is for stap to provide dynamic values to pcp. This would be a lot like
> pcp's mmv library/pmda, that allows pcp to monitor memory mapped values
> from a program. In this case, systemtap would be providing the values.
> Note that it probably won't end up 100% compatible with pcp's mmv
> library/pmda.
> 
> (In a perfect world, perhaps systemtap's dyninst side could actually use
> pcp's mmv library. I'd hope to make things look the same from the user's
> point of view inside systemtap.)
> 
>> It is possible that similar latency measurements could be done for
>> the block io layer like the stuff that blktrace does.  However, one
>> concern is there could be a huge number of items to track (in the
>> tens of thousands) that might not fit easily in a default sized
>> associative array.
> 
> Here's where things get interesting. I mentioned this "plan" to Nathan,
> and he suggested we might need to go further. If we're going to provide
> real tracing information back to pcp, we're going to need to provide
> timestamp'ed information, read on demand from pcp. This means we'd need
> to buffer it inside systemtap. This might be a phase 2.
> 
> 
> Back to your original idea for a second. Are you at the point where
> you've got a systemtap script that produces the data you want? If so,
> I'd love to see it and use it for a test when I get to that point.
> 

Hi All,

I have been experimenting with the systmetap mmv support David Smith
has been developing in the dsmith/mmv branch of systemtap.  The
attached script is a work-in-progress to measure the amount of time
packets take from getting placed in the queue to the time that the are
actually transmitted.  The list of network devices needs to be passed
into the systemtap script so that it can set up instances for each of
the network devices.  There is probably a better way of getting the
device names to the script, but that is done to keep things simple.

The script creates two metrics for each device: number of packets and
the sum of the latency.  The systemtap script provides these basic
metrics from the start so that the pmda could compute various rate
on different times scale easily using the following formula for a device
instance at two times (t1 and t2):

current_latency/packet = (latency[t2]-latency[t1])/(count[t2]-count[t1])


Rather than looking up the value identifies with mmv_lookup_value()
the value identifiers are cached in a couple of associative arrays.
However, I don't know whether the overhead is a real issue with that.

Next step will be to write the mmv pmda to read this information and
make it available.

Any comments about the current code would be appreciated.

-Will



Attachment: net_xmit_mmv.stp
Description: Text document

<Prev in Thread] Current Thread [Next in Thread>