pcp
[Top] [All Lists]

Verification and validation of performance metric values

To: PCP <pcp@xxxxxxxxxxx>
Subject: Verification and validation of performance metric values
From: William Cohen <wcohen@xxxxxxxxxx>
Date: Thu, 9 Jun 2016 10:24:50 -0400
Delivered-to: pcp@xxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0
Hi,

In the past I have mainly used the performance monitoring events provided by 
the CPU's performance monitoring units.  It is not uncommon for particular 
events monitored by the hardware to be inaccurate  (for example BT243 
Performance Monitor counters May Produce Incorrect Results in the Intel Xeon 
Processor E5 Family Spec Update.  
http://www.intel.com/content/www/us/en/processors/xeon/xeon-e5-family-spec-update.html).
 The hardware may over count or undercount, or in some cases not count events 
at all. Thus, blindly trusting the metrics provided by the performance hardware 
is unwise.  When I started using some of of the performance metric provided by 
PCP I had the same question.  How, are the PCP metrics checked to see if they 
are accurate/reasonable?  For some of the PAPI (http://icl.cs.utk.edu/papi/) 
testsuite there are cross checks to see that values agree with the expected 
values.

For estimating network utilization using PCP the network.interface.baudrate and 
network.interface.speed metrics should provide some indication of the interface 
speed.  However, looking at the numbers produced by pmval below they don't seem 
to be reasonable for 1Gbps ethernet connection of em1.

[wcohen@santana Downloads]$ pmval network.interface.baudrate

metric:    network.interface.baudrate
host:      santana
semantics: discrete instantaneous value
units:     byte / sec
samples:   all

     wlp3s0          lo  virbr0-nic      virbr0         em1 
          ?           ?   316871376           ?  1622366528 
          ?           ?   316871376           ?  1622366528 
^C
[wcohen@santana Downloads]$ pmval network.interface.speed

metric:    network.interface.speed
host:      santana
semantics: discrete instantaneous value
units:     Mbyte / sec
samples:   all

       wlp3s0            lo    virbr0-nic        virbr0           em1 
            ?             ?     7.813E+04             ?     7.813E+06 
            ?             ?     7.813E+04             ?     7.813E+06 
^C

Are there some PCP tests that check that some of the metrics are sane and 
reasonable?

-Will

<Prev in Thread] Current Thread [Next in Thread>