From lberk@redhat.com Sat Nov 1 12:02:36 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 922037F54 for ; Sat, 1 Nov 2014 12:02:36 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 80BF3304032 for ; Sat, 1 Nov 2014 10:02:33 -0700 (PDT) X-ASG-Debug-ID: 1414861351-04cb6c2efc4ae3a0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id URqHEqhBy5qW8MCP (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sat, 01 Nov 2014 10:02:32 -0700 (PDT) X-Barracuda-Envelope-From: lberk@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA1H2Vu7026563 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Sat, 1 Nov 2014 13:02:31 -0400 Received: from toium (vpn-59-179.rdu2.redhat.com [10.10.59.179]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA1H2SlQ011264 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO) for ; Sat, 1 Nov 2014 13:02:30 -0400 From: Lukas Berk To: pcp@oss.sgi.com Subject: PCP Presentation at FSOSS Date: Sat, 01 Nov 2014 13:02:28 -0400 X-ASG-Orig-Subj: PCP Presentation at FSOSS Message-ID: <87ppd6ua57.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1414861352 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hey, Last week a colleague and I presented at Toronto's Free Software and Open Source Symposium (FSOSS) about PCP. I've posted the slide deck if anybody wants to take a look[1]. The overall 'message' of the talk was to show how PCP is very flexible in the data it can assimilate and how it presents data to the user. This was also demonstrated by using several recent/current developments and how they fit in, conceptually, with PCP. My hope is that the talk illustrated how the attendees could solve their performance issues using the facilities PCP provides. Cheers, Lukas [1] - http://fedorapeople.org/~lberk/fsoss-2014.odp From kenj@internode.on.net Sat Nov 1 15:11:44 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 337E47F54 for ; Sat, 1 Nov 2014 15:11:44 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 219408F8037 for ; Sat, 1 Nov 2014 13:11:40 -0700 (PDT) X-ASG-Debug-ID: 1414872695-04cbb070c85054d0001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id JYE26CeFr5wYIMwo for ; Sat, 01 Nov 2014 13:11:35 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AuQJAOw9VVR5LGvJPGdsb2JhbABcgw6BLII20lIEAgKBFRcBAQEBAQYBAQEBODuEAgEBAQQIAh4SHCMMAQMCBgMRBAEBKAcZIAoDCQgCBBMLBYgwyTIBAQEHAQEBAQEdkRAHBoRFBZIaYKVjKS+CSwEBAQ Received: from ppp121-44-107-201.lns20.syd6.internode.on.net (HELO bozohorize) ([121.44.107.201]) by ipmail05.adl6.internode.on.net with ESMTP; 02 Nov 2014 06:41:18 +1030 From: "Ken McDonell" To: "'Frank Ch. Eigler'" Cc: "'pcp developers'" References: <20141031201304.GE1913@redhat.com> <002c01cff56c$6febc830$4fc35890$@internode.on.net> <20141101010427.GF1913@redhat.com> In-Reply-To: <20141101010427.GF1913@redhat.com> Subject: RE: [pcp] qa/518 tweaks on pcpfans.git fche/dev Date: Sun, 2 Nov 2014 07:11:15 +1100 X-ASG-Orig-Subj: RE: [pcp] qa/518 tweaks on pcpfans.git fche/dev Message-ID: <001701cff60f$ff53d460$fdfb7d20$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQIWIQMIOeKcOGylRqOgmw6wi16M5QIkql93Aj2/Pb+bnMRKEA== Content-Language: en-au X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1414872695 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11123 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header G'day Frank. I've had another look at this test. 1. I think the non-determinism Frank observes can be handled in a different way that does not make the test run for longer ... in other places the test is correct if we get the expected outcome for N or N-1 or N+1 iterations. I'll explore this and work with Frank (off the list, as no one else probably cares and Frank has the environment in which the test was failing). 2. I am not sure why the kill's are there at all (thanks for pointing out the extra one that I was not expecting) ... seems to me we can do a better job of filtering the output to ignore the other pmie instances (there is one block of output from pcp -P per pmie instance). I'll take a look at this. 3. Frank's observations about pmie and signals is a bit concerning, although I see evidence of "try TERM and if that does not work try KILL and repeat until successful or timeout" in the pmie init script, so perhaps this is a long standing problem that has been masked by hackery. pmie does have a TERM signal handler and a delayed exit but only after the nanosleep() ... so if we are blocked somewhere else, or don't abandon expression evaluation completely when an I/O returns with EINTR then we could be off in the weeds long enough for some script to believe pmie has not died. Any insight into 3. would be helpful. > -----Original Message----- > From: Frank Ch. Eigler [mailto:fche@redhat.com] > Sent: Saturday, 1 November 2014 12:04 PM > To: Ken McDonell > Cc: 'pcp developers' > Subject: Re: [pcp] qa/518 tweaks on pcpfans.git fche/dev > > ... > I'll try to trace it with something like systemtap. (Even with pmmgr I > encountered cases where a single SIGTERM sent to pmie was blocked/ignored, > so sudo is probably not a necessary component of the > problem.) From fche@redhat.com Sun Nov 2 09:48:47 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id CD7067F56 for ; Sun, 2 Nov 2014 09:48:47 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 5ABD7AC001 for ; Sun, 2 Nov 2014 07:48:44 -0800 (PST) X-ASG-Debug-ID: 1414943319-04cb6c2efa4cc580001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id n3WY9DLjUECL4Dai (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 02 Nov 2014 07:48:40 -0800 (PST) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA2FmZE2030403 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sun, 2 Nov 2014 10:48:35 -0500 Received: from fche.csb (vpn-236-171.phx2.redhat.com [10.3.236.171]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA2FmZHD030329; Sun, 2 Nov 2014 10:48:35 -0500 Received: by fche.csb (Postfix, from userid 2569) id 853FC58186; Sun, 2 Nov 2014 10:48:34 -0500 (EST) To: Dave Brolley Cc: Ken McDonell , "'PCP Mailing List'" Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools From: fche@redhat.com (Frank Ch. Eigler) Date: Sun, 02 Nov 2014 10:48:34 -0500 In-Reply-To: <545110DC.2020104@redhat.com> (Dave Brolley's message of "Wed, 29 Oct 2014 12:07:56 -0400") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1414943320 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 kenj wrote: > [...] One simple solution that might be acceptable for 95% of the > cases would be to rule all of the metadata data differences (except > instance domains) to be unsupported. [...] It'd be interesting to assess how many actual libraries of archives would meet or fail this test. Identical "pmdumplog -d"? A set of outputs from an constantly-configured configured pmlogger may pass OR fail, depending on whether the pmlogger ran long enough to sample the entire set of metrics at least once. (An early SIGTERM might let only the "log once" metrics show up in the .meta file.) Or the order of first-appearance between metrics might be different. Or pmcd might have had a version update, and hinv.*.online metrics may have appeared from one day to the next. - FChE From kenj@internode.on.net Sun Nov 2 13:56:29 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id B79637F56 for ; Sun, 2 Nov 2014 13:56:29 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id A5820304039 for ; Sun, 2 Nov 2014 11:56:26 -0800 (PST) X-ASG-Debug-ID: 1414958180-04cb6c2ef94d15c0001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id fGkvCeV8D6Ke6uZH for ; Sun, 02 Nov 2014 11:56:21 -0800 (PST) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: As4IACyLVlR5LCL3/2dsb2JhbABcgw6BLII20igEAgKBERcBAQEBAX2EAgEBAQQIAh4SHBgGBQwBAwIGAxEEAQEoBxkgDQkIAgQBEgsFiDDJAwEBAQEBBQEBAQEekC4QAgFPBwaERQWSGmChV4QMKS+CSwEBAQ Received: from ppp121-44-34-247.lns20.syd6.internode.on.net (HELO bozohorize) ([121.44.34.247]) by ipmail07.adl2.internode.on.net with ESMTP; 03 Nov 2014 06:26:18 +1030 From: "Ken McDonell" To: "'Frank Ch. Eigler'" , "'Dave Brolley'" Cc: "'PCP Mailing List'" References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> In-Reply-To: Subject: RE: Multi-Volume Archive + Live Data Playback for PCP Client Tools Date: Mon, 3 Nov 2014 06:55:58 +1100 X-ASG-Orig-Subj: RE: Multi-Volume Archive + Live Data Playback for PCP Client Tools Message-ID: <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQHclF1cpR2XxHkXIjv6In84hym83AHe7s6aAjYqx4MCSZJtG5wBjM5g Content-Language: en-au X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1414958180 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11163 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header I should have been clearer in my earlier email ... any metrics that are only in one archive will always be OK ... by "metadata differences" I mean a metric that appears in more than one archive and has some difference in the pmDesc data that describes that metric or the associated PMNS fragment that maps a name to a PMID ... this means - name - PMID - type - indom (identifier of the instance domain) - semantics - units The name to PMID and PMID to name(s) mapping we could handle (I owe Dave a response to his email asking questions about this). type differences we could handle, but I don't think it is worth the effort and there are some combinations that cannot preserve precision. indom and semantics differences are probably impossible to reconcile. We could do units, but it is probably not worth the effort at this stage. Different metadata between archives is sort of expected, as the examples in Frank's mail illustrate. I was referring to differences in metadata for the same metric. In this context, my original brave assertion about this being OK most of the time still stands, I believe. > -----Original Message----- > From: Frank Ch. Eigler [mailto:fche@redhat.com] > Sent: Monday, 3 November 2014 2:49 AM > To: Dave Brolley > Cc: Ken McDonell; 'PCP Mailing List' > Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools > > ... > It'd be interesting to assess how many actual libraries of archives would meet > or fail this test. Identical "pmdumplog -d"? > ... From brolley@redhat.com Mon Nov 3 08:20:06 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 1CCDF7F47 for ; Mon, 3 Nov 2014 08:20:06 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id EF958304032 for ; Mon, 3 Nov 2014 06:20:02 -0800 (PST) X-ASG-Debug-ID: 1415024398-04cbb070c75451f0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 36yhkM5tSQKiNWCH (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 06:19:59 -0800 (PST) X-Barracuda-Envelope-From: brolley@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA3EJstA026126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 3 Nov 2014 09:19:54 -0500 Received: from [10.10.59.254] (vpn-59-254.rdu2.redhat.com [10.10.59.254]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA3EJqpI003822; Mon, 3 Nov 2014 09:19:53 -0500 Message-ID: <54578F87.9000108@redhat.com> Date: Mon, 03 Nov 2014 09:21:59 -0500 From: Dave Brolley User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: Ken McDonell , "'Frank Ch. Eigler'" CC: "'PCP Mailing List'" Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools In-Reply-To: <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415024398 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On 11/02/2014 02:55 PM, Ken McDonell wrote: > I should have been clearer in my earlier email ... any metrics that are only > in one archive will always be OK ... by "metadata differences" I mean a > metric that appears in more than one archive and has some difference in the > pmDesc data that describes that metric or the associated PMNS fragment that > maps a name to a PMID ... this means > - name > - PMID > - type > - indom (identifier of the instance domain) > - semantics > - units > > The name to PMID and PMID to name(s) mapping we could handle (I owe Dave a > response to his email asking questions about this). > > type differences we could handle, but I don't think it is worth the effort > and there are some combinations that cannot preserve precision. > > indom and semantics differences are probably impossible to reconcile. > > We could do units, but it is probably not worth the effort at this stage. > > Different metadata between archives is sort of expected, as the examples in > Frank's mail illustrate. > > I was referring to differences in metadata for the same metric. > > In this context, my original brave assertion about this being OK most of the > time still stands, I believe. > Thanks Ken. This is essentially how I understood things. Dave From fche@redhat.com Mon Nov 3 08:26:47 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id C6C397F47 for ; Mon, 3 Nov 2014 08:26:47 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id B315C304039 for ; Mon, 3 Nov 2014 06:26:47 -0800 (PST) X-ASG-Debug-ID: 1415024806-04cb6c2ef94f0980001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id LV8irEiIo2t91HTJ (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 06:26:46 -0800 (PST) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA3EQi6L012417 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 3 Nov 2014 09:26:44 -0500 Received: from fche.csb (vpn-236-171.phx2.redhat.com [10.3.236.171]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA3EQhXA025199; Mon, 3 Nov 2014 09:26:43 -0500 Received: by fche.csb (Postfix, from userid 2569) id 398A958186; Mon, 3 Nov 2014 09:26:43 -0500 (EST) Date: Mon, 3 Nov 2014 09:26:43 -0500 From: "Frank Ch. Eigler" To: Ken McDonell Cc: "'Dave Brolley'" , "'PCP Mailing List'" Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools Message-ID: <20141103142643.GA3859@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415024806 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - > [...] by "metadata differences" I mean a metric that appears in more > than one archive and has some difference in the pmDesc data that > describes that metric or the associated PMNS fragment that maps a > name to a PMID ... this means - name [...] One glitch with that could be PMDAs whose PMNS is dynamic from run to run (like the papi pmda, which is almost able to be used as a logged data source). The name-to-PMID mapping may vary there (as new PAPI versions come, or host CPU changes, new counters(=metrics) may appear in some odd sequence, so get scrambled PMIDs), but the name & pmDesc (semantics etc.) would remain the same and previous values comparable. I don't know how pervasive this situation would be, but if it's not too hard to support, we should. (e.g., we could track metrics across archives by name rather than pmid.) - FChE From fche@redhat.com Mon Nov 3 15:07:52 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 87F377F56 for ; Mon, 3 Nov 2014 15:07:52 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 66934304051 for ; Mon, 3 Nov 2014 13:07:49 -0800 (PST) X-ASG-Debug-ID: 1415048868-04bdf038cf61fb40001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id fgAhBhEu11s0w8FZ (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 13:07:48 -0800 (PST) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA3L7lai002945 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Mon, 3 Nov 2014 16:07:48 -0500 Received: from fche.csb (vpn-236-171.phx2.redhat.com [10.3.236.171]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA3L7lWQ006894 for ; Mon, 3 Nov 2014 16:07:47 -0500 Received: by fche.csb (Postfix, from userid 2569) id 2CFC858186; Mon, 3 Nov 2014 16:07:47 -0500 (EST) Date: Mon, 3 Nov 2014 16:07:47 -0500 From: "Frank Ch. Eigler" To: pcp developers Subject: RFC: papi pmda auto-enable on fetch Message-ID: <20141103210747.GB3859@redhat.com> X-ASG-Orig-Subj: RFC: papi pmda auto-enable on fetch Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415048868 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - Please have a look over pcpfans.git fche/papi. If it looks generally acceptable, I'm ready to assist lberk in QA'ing the heck out of it. commit 3f7223221fc313441079be7c8c41c3d14a322247 Author: Frank Ch. Eigler Date: Mon Nov 3 15:55:41 2014 -0500 papi pmda: add counter auto-enable on fetch By reinterpreting the papi_info[].metric_enabled as a timestamp rather than a boolean, and a small bit of logic, we get automatic enablement of PAPI metric/counters upon pmfetch. This allows PAPI metrics to be used with plain old pmlogger, pmval, etc., without necessary manual pmstore's. Counters are retired based on a timeout mechanism, assisted by pmaf(3). The papi.control.status metric is enhanced to show the remaining lifespan of the counters, and to make it more safe & reliable. A new papi.control.auto_enable metric is available to set the timeout (default 120 seconds). The Install script runs a papi.control.reset operation to avoid leaving the counters running for even that long after the post-install metric/value checks. The previous pmstore mechanism is left in place unchanged, and overrides the automatic scheme. From kenj@internode.on.net Mon Nov 3 15:47:14 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 497187F57 for ; Mon, 3 Nov 2014 15:47:14 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id BD011AC002 for ; Mon, 3 Nov 2014 13:47:13 -0800 (PST) X-ASG-Debug-ID: 1415051227-04cbb070c8576a30001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id M3h9ZLdY5URBCSxi for ; Mon, 03 Nov 2014 13:47:08 -0800 (PST) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AsQBAIT2V1R20SpqPGdsb2JhbAANT4t3ywuDIAKBPAEBAQEBBgEBAQE4hD4BAQQnETwEARALGAkWDwkDAgECATEUBg0BBwEBvR6VKwEBAQEBAQEDAQEBAQEBARuQPwEBTweESwEEmSyfMYFngTwBAQE Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106]) by ipmail06.adl2.internode.on.net with ESMTP; 04 Nov 2014 08:17:06 +1030 Message-ID: <5457F82D.3050307@internode.on.net> Date: Tue, 04 Nov 2014 08:48:29 +1100 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: 'Dave Brolley' , 'PCP Mailing List' Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> <20141103142643.GA3859@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools In-Reply-To: <20141103142643.GA3859@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1415051227 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11204 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On 04/11/14 01:26, Frank Ch. Eigler wrote: > ... > One glitch with that could be PMDAs whose PMNS is dynamic from run to > run (like the papi pmda, which is almost able to be used as a logged > data source). The name-to-PMID mapping may vary there (as new PAPI > versions come, or host CPU changes, new counters(=metrics) may appear > in some odd sequence, so get scrambled PMIDs), but the name & pmDesc > (semantics etc.) would remain the same and previous values comparable. > > I don't know how pervasive this situation would be, but if it's not > too hard to support, we should. (e.g., we could track metrics across > archives by name rather than pmid.) This is not pervasive at all ... this is the only PMDA I am aware of that behaves like this. And I think I would be lobbying for the PMDA to be different, not the archive handling to be different. There are already services available to a PMDA that allow the instance domain mapping (name <--> id) to be consistent across PMDA restarts. It would only be a small perversion of the pmdaCache* routine usage to allow the papi pmda to maintain a consistent and persistent metric name to low-order bits of the PMID mapping. Note this does not need to be consistent between hosts, just consistent for a single host across repeated PMDA invocations (and possible changes in version and configuration of the PMDA). Returning to the multi-archive work that triggered all of this, where there is a contradiction in the name <---> PMID mapping between archives, we have several cases and options: Case 1 Multiple names map to the same PMID. Options: 1a. Assume this is same metric and the PMID is correct, add both names to the PMNS (the data structure and libpcp routines support this, so for example, given the PMID pmNameAll() will return all the corresponding names). No rewriting of pmDesc or pmResult data structures is required. 1b. Assume these are different metrics. Invent a new (and unique) PMID for the subsequent ones, add all the names and their unique PMIDs to the PMNS. Synthesize a new pmDesc for each metric that gets an invented PMID. Rewrite pmResult data structures to map from the old PMID to an invented PMID, but in the context of the archive in which the name was mapped (need to choose the PMID based on which archive the pmResult came from). Case 2 Multiple PMIDs map to the same name. Options: 2a. Assume these are the same metric. Pick one PMID (probably the first one encountered ... Dave this is the "first" one is the winner part I was cryptically referring to in the earlier mail) and add mappings from all the names to the same PMID into the PMNS. There is only one pmDesc as a result, and in each pmResult any of the loser PMIDs need to be mapped to the winner PMID. 2b. Assume these are different metrics. Invent a new (and unique) name for the subsequent ones, add all the names and their associated PMIDs to the PMNS. No rewriting on pmDesc of pmResult data structures is required. Case 3 N:M mapping between PMIDs and names. Forget it, don't even thing about this one. I was suggesting that initially none of this is supported, i.e. the name <--> PMID mapping was consistent across all archives in the set (Stage 0). Then options 1a and 2b might be suitable for a Stage 1. And Stage 2 might add support for all 4 options above (Frank's suggestion is 1b I think). Note that all of the options above can be supported OUTSIDE libpcp today using (admittedly manual) pmlogrewrite step on one or more archives to remove the inconsistencies before the archive set is processed as a unit. From fche@redhat.com Mon Nov 3 15:58:35 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 4F9487F5A for ; Mon, 3 Nov 2014 15:58:35 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 2D22C304051 for ; Mon, 3 Nov 2014 13:58:35 -0800 (PST) X-ASG-Debug-ID: 1415051913-04cbb070c75773c0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id NcTQdcw0gS0wSCnu (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 13:58:34 -0800 (PST) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA3LwHgm032516 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 3 Nov 2014 16:58:17 -0500 Received: from fche.csb (vpn-236-171.phx2.redhat.com [10.3.236.171]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA3LwHe5021444; Mon, 3 Nov 2014 16:58:17 -0500 Received: by fche.csb (Postfix, from userid 2569) id DE94758186; Mon, 3 Nov 2014 16:58:16 -0500 (EST) Date: Mon, 3 Nov 2014 16:58:16 -0500 From: "Frank Ch. Eigler" To: Ken McDonell Cc: "'Dave Brolley'" , "'PCP Mailing List'" Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools Message-ID: <20141103215816.GC3859@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> <20141103142643.GA3859@redhat.com> <5457F82D.3050307@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5457F82D.3050307@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415051914 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - > >I don't know how pervasive this situation would be, but if it's not > >too hard to support, we should. (e.g., we could track metrics across > >archives by name rather than pmid.) > > This is not pervasive at all ... this is the only PMDA I am aware of > that behaves like this. (Well, the cgroup.* PMNS hierarchy may be similarly affected, but that is due for a revamp.) > And I think I would be lobbying for the PMDA to be different, not > the archive handling to be different. [...] allow the papi pmda to > maintain a consistent and persistent metric name to low-order bits > of the PMID mapping. [...] I see what you mena, but unfortunately, the PAPI event code space (at least 32 bits) is much larger than the PMID low-order bits space, is not densely encoded, and bound to get worse when we address counters with extra option flags, "uncore" counters, and per-process/-cgroup. - FChE From kenj@internode.on.net Mon Nov 3 17:20:14 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id E3F7F7F3F for ; Mon, 3 Nov 2014 17:20:13 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 726FEAC002 for ; Mon, 3 Nov 2014 15:20:10 -0800 (PST) X-ASG-Debug-ID: 1415056804-04cb6c2efc520e60001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id aRQxIGvwjLf3pXoi for ; Mon, 03 Nov 2014 15:20:07 -0800 (PST) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AsQBAPQMWFR20SpqPGdsb2JhbAANT4t3zjECgToCAQEBBgEBAQE4hD4BBThAARALGAkWDwkDAgECATEUBg0BBwEBvXeWGZEQB4RLBZksnzGDIwEBAQ Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106]) by ipmail06.adl2.internode.on.net with ESMTP; 04 Nov 2014 09:39:11 +1030 Message-ID: <54580B6A.1090007@internode.on.net> Date: Tue, 04 Nov 2014 10:10:34 +1100 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: 'Dave Brolley' , 'PCP Mailing List' Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> <20141103142643.GA3859@redhat.com> <5457F82D.3050307@internode.on.net> <20141103215816.GC3859@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools In-Reply-To: <20141103215816.GC3859@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1415056804 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11209 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header G'day Frank. On 04/11/14 08:58, Frank Ch. Eigler wrote: > ... > I see what you mena, but unfortunately, the PAPI event code space (at > least 32 bits) is much larger than the PMID low-order bits space, is > not densely encoded, and bound to get worse when we address counters > with extra option flags, "uncore" counters, and per-process/-cgroup. I was not advocating a fixed mapping ... while there may be more than 2^22 possible event counters, is it really the case that on one machine we could expect that more than 4 million different event codes to be enabled and exported by the PMDA? The pmdaCache* routines would allow a dense allocation of more than 4 million unique ids and maintain a persistent mapping from the PMNS name that the PMDA chooses and those unique ids. Perhaps someone could send me the output from: $ pminfo -dm papi from a machine where the papi PMDA is running and collecting real data. Then restart the PMDA, start a different collection profile and repeat the command above. This might give me some concrete context to understand the issues if I've got it wrong. From lberk@redhat.com Mon Nov 3 18:17:06 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 09CF87F4E for ; Mon, 3 Nov 2014 18:17:06 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id EBCD1304048 for ; Mon, 3 Nov 2014 16:17:02 -0800 (PST) X-ASG-Debug-ID: 1415060218-04cbb070c7580470001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id AIXCPB3j5mHAZnkF (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 16:16:58 -0800 (PST) X-Barracuda-Envelope-From: lberk@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA40GvG1017036 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Mon, 3 Nov 2014 19:16:58 -0500 Received: from toium ([10.15.16.184]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA40GvX8012964 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO); Mon, 3 Nov 2014 19:16:57 -0500 From: Lukas Berk To: "Frank Ch. Eigler" Cc: pcp developers Subject: Re: [pcp] RFC: papi pmda auto-enable on fetch References: <20141103210747.GB3859@redhat.com> X-ASG-Orig-Subj: Re: [pcp] RFC: papi pmda auto-enable on fetch Date: Mon, 03 Nov 2014 19:16:56 -0500 In-Reply-To: <20141103210747.GB3859@redhat.com> (Frank Ch. Eigler's message of "Mon, 3 Nov 2014 16:07:47 -0500") Message-ID: <87k33b251j.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415060218 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 --=-=-= Content-Type: text/plain Hey Frank, "Frank Ch. Eigler" writes: > Please have a look over pcpfans.git fche/papi. If it looks generally > acceptable, I'm ready to assist lberk in QA'ing the heck out of it. Thanks for throwing this together. I like the idea of using the timeout to ensure counters don't continue on for too long. The only nit I have within the code itself; is there a chance you could give the 'auto_afid' variable a more descriptive name? or perhaps a comment to the appropriate man page (I assume pmaf but it wasn't initially clear to me). If you would consider a few doc tweaks (patch attached) I'd appreciate it. I'd be nice if the pminfo help text labled the units for a quick glance without needing the man page. And I added the note that setting papi.control.auto_enable to 0 disables the auto_enabling. Other than that I'd be happy to start throwing together some relavent qa. Cheers, Lukas --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Tweak-pmdapapi-docs.patch >From 95ea7889721068705d7ea4ca9ff0ec1d1c6a1e83 Mon Sep 17 00:00:00 2001 From: Lukas Berk Date: Mon, 3 Nov 2014 18:56:23 -0500 Subject: [PATCH 1/1] Tweak pmdapapi docs Add the unit (seconds) to the papi.control.auto_enable metric and mention how one would disable the auto_enable functionality if needed in the man page. Signed-off-by: Lukas Berk --- src/pmdas/papi/help | 2 +- src/pmdas/papi/pmdapapi.1 | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/src/pmdas/papi/help b/src/pmdas/papi/help index e2d8b6d..3b48ce8 100644 --- a/src/pmdas/papi/help +++ b/src/pmdas/papi/help @@ -34,7 +34,7 @@ @ papi.control.status A string of papi counters current state -@ papi.control.auto_enable Timeout for future auto-enabled counters. +@ papi.control.auto_enable Timeout for future auto-enabled counters in seconds. @ papi.available.num_counters Number of hardware counters available for use diff --git a/src/pmdas/papi/pmdapapi.1 b/src/pmdas/papi/pmdapapi.1 index 868e276..356fb9f 100644 --- a/src/pmdas/papi/pmdapapi.1 +++ b/src/pmdas/papi/pmdapapi.1 @@ -66,7 +66,10 @@ or This automatic activation is temporary, and lasts only a number of seconds governed by the .B papi.control.auto_enable -control value (default 120). +control value (default 120). In the case automatic activation is undesirable, one may +disable it by setting the +.B papi.control.auto_enable +metric to 0. .P Alternately, the .BR pmstore (1) -- 1.9.3 --=-=-=-- From fche@redhat.com Mon Nov 3 20:18:28 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 8F35A7F4E for ; Mon, 3 Nov 2014 20:18:28 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 1DA90AC002 for ; Mon, 3 Nov 2014 18:18:24 -0800 (PST) X-ASG-Debug-ID: 1415067500-04cbb070c7584a10001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id TNfmHYr8t4fxu8tt (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 18:18:20 -0800 (PST) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA42IF1a026809 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 3 Nov 2014 21:18:15 -0500 Received: from fche.csb (vpn-236-171.phx2.redhat.com [10.3.236.171]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA42IF8b004729; Mon, 3 Nov 2014 21:18:15 -0500 Received: by fche.csb (Postfix, from userid 2569) id C191E58186; Mon, 3 Nov 2014 21:18:14 -0500 (EST) Date: Mon, 3 Nov 2014 21:18:14 -0500 From: "Frank Ch. Eigler" To: Ken McDonell Cc: "'Dave Brolley'" , "'PCP Mailing List'" Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools Message-ID: <20141104021814.GD3859@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> <20141103142643.GA3859@redhat.com> <5457F82D.3050307@internode.on.net> <20141103215816.GC3859@redhat.com> <54580B6A.1090007@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54580B6A.1090007@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415067500 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - > [...] I was not advocating a fixed mapping [...] Aha. > The pmdaCache* routines would allow a dense allocation of more than 4 > million unique ids and maintain a persistent mapping from the PMNS name > that the PMDA chooses and those unique ids. OK, that could work. > Perhaps someone could send me the output from: > $ pminfo -dm papi > from a machine where the papi PMDA is running and collecting real data. (Sent under separate cover.) > Then restart the PMDA, start a different collection profile and repeat > the command above. It won't change just based on that. But a different version of PAPI/libpfm, a CPU/kernel change, can easily generate a different enumeration of available events, thus a different and/or reordered PMNS name sets. - FChE From fche@redhat.com Mon Nov 3 20:46:07 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 8E4447F4E for ; Mon, 3 Nov 2014 20:46:07 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 6DC81304062 for ; Mon, 3 Nov 2014 18:46:07 -0800 (PST) X-ASG-Debug-ID: 1415069163-04bdf038d262ff20001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Oy85SFrsM44l2UwD (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 03 Nov 2014 18:46:03 -0800 (PST) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA42k28W004771 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Mon, 3 Nov 2014 21:46:03 -0500 Received: from fche.csb (vpn-236-171.phx2.redhat.com [10.3.236.171]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA42k2O6016473; Mon, 3 Nov 2014 21:46:02 -0500 Received: by fche.csb (Postfix, from userid 2569) id 0FC3658186; Mon, 3 Nov 2014 21:46:02 -0500 (EST) Date: Mon, 3 Nov 2014 21:46:01 -0500 From: "Frank Ch. Eigler" To: Lukas Berk Cc: pcp developers Subject: Re: [pcp] RFC: papi pmda auto-enable on fetch Message-ID: <20141104024601.GE3859@redhat.com> X-ASG-Orig-Subj: Re: [pcp] RFC: papi pmda auto-enable on fetch References: <20141103210747.GB3859@redhat.com> <87k33b251j.fsf@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87k33b251j.fsf@redhat.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415069163 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - > Thanks for throwing this together. I like the idea of using the > timeout to ensure counters don't continue on for too long. No problemo. > The only nit I have within the code itself; is there a chance you > could give the 'auto_afid' variable a more descriptive name? [...] Sure. > If you would consider a few doc tweaks (patch attached) I'd appreciate > it. Sure. (Please feel free to commit such things straight to my branch.) > I'd be nice if the pminfo help text labled the units for a quick > glance without needing the man page. [...] In the case of the papi.control.auto_enable metric, I had that addendum ("in seconds") there too at one point, but then realized that the metric was already self-describing (pminfo -d -> Units: sec). - FChE From kenj@internode.on.net Tue Nov 4 14:10:36 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id D29D37F9D for ; Tue, 4 Nov 2014 14:10:36 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id C08B68F804C for ; Tue, 4 Nov 2014 12:10:33 -0800 (PST) X-ASG-Debug-ID: 1415131827-04cb6c2ef957b390001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id WYvy7vs8mzP62uDi for ; Tue, 04 Nov 2014 12:10:28 -0800 (PST) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ar4BAFQsWVR20SpqPGdsb2JhbAANTodAhDfLT4MgAoE5AQEBAQEGAQEBATiEPgEBBCMEUQEQCxgJFgsCAgkDAgECATEUBg0BBwEBwDN4lRUBAQEBAQEBAQEBAQEBAQEBAQEBGZEQB4J3gVQBBJQ1gVKJMYZ+kiyDIwEBAQ Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106]) by ipmail06.adl2.internode.on.net with ESMTP; 05 Nov 2014 06:18:23 +1030 Message-ID: <54592DDC.8030809@internode.on.net> Date: Wed, 05 Nov 2014 06:49:48 +1100 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: 'Dave Brolley' , 'PCP Mailing List' Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools References: <542C21AE.1010504@redhat.com> <007e01cfe010$7867f090$6937d1b0$@internode.on.net> <545110DC.2020104@redhat.com> <001d01cff6d7$0e9e4a50$2bdadef0$@internode.on.net> <20141103142643.GA3859@redhat.com> <5457F82D.3050307@internode.on.net> <20141103215816.GC3859@redhat.com> <54580B6A.1090007@internode.on.net> <20141104021814.GD3859@redhat.com> X-ASG-Orig-Subj: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools In-Reply-To: <20141104021814.GD3859@redhat.com> Content-Type: multipart/mixed; boundary="------------010601000601010709080000" X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1415131827 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11232 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header This is a multi-part message in MIME format. --------------010601000601010709080000 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 04/11/14 13:18, Frank Ch. Eigler wrote: > Hi, Ken - > >> [...] I was not advocating a fixed mapping [...] > > Aha. > >> The pmdaCache* routines would allow a dense allocation of more than 4 >> million unique ids and maintain a persistent mapping from the PMNS name >> that the PMDA chooses and those unique ids. > > OK, that could work. Attached is a quick script that uses the pmns data Frank sent me and the qa test program pmdacache to demonstrate how a persistent name <--> id map could be managed by the papi PMDA. --------------010601000601010709080000 Content-Type: text/plain; charset=UTF-8; name="papi_pmns_demo" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="papi_pmns_demo" #!/bin/sh # # demo if pmdaCache to manage papi PMNS # here=`pwd` if [ ! -x /var/lib/pcp/testsuite/src/pmdacache ] then if [ -f /var/lib/pcp/testsuite/src/pmdacache.c ] then cd /var/lib/pcp/testsuite/src sudo -u pcpqa make pmdacache if [ ! -x /var/lib/pcp/testsuite/src/pmdacache ] then echo "Error: failed to make pmdacache in /var/lib/pcp/testsuite/src" exit 1 fi else echo "Error: need pmdacache.c in /var/lib/pcp/testsuite/src" exit 1 fi fi sudo -u pcp ./pmdacache -C -S cat < X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 11BE27F90 for ; Tue, 4 Nov 2014 17:55:40 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 001B0304048 for ; Tue, 4 Nov 2014 15:55:36 -0800 (PST) X-ASG-Debug-ID: 1415145334-04cb6c2efb58e220001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id DNP2aCmO4ZOGpLrI (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 04 Nov 2014 15:55:35 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA4NtTlT021172; Tue, 4 Nov 2014 18:55:29 -0500 Date: Tue, 4 Nov 2014 18:55:29 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp developers Message-ID: <785129839.7773835.1415145329525.JavaMail.zimbra@redhat.com> In-Reply-To: <001701cff60f$ff53d460$fdfb7d20$@internode.on.net> References: <20141031201304.GE1913@redhat.com> <002c01cff56c$6febc830$4fc35890$@internode.on.net> <20141101010427.GF1913@redhat.com> <001701cff60f$ff53d460$fdfb7d20$@internode.on.net> Subject: Re: [pcp] qa/518 tweaks on pcpfans.git fche/dev MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] qa/518 tweaks on pcpfans.git fche/dev Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: qa/518 tweaks on pcpfans.git fche/dev Thread-Index: AQIWIQMIOeKcOGylRqOgmw6wi16M5QIkql93Aj2/Pb+bnMRKEG114gej X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1415145335 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11241 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Ken, ----- Original Message ----- > [...] > 3. Frank's observations about pmie and signals is a bit concerning, although > I see evidence of "try TERM and if that does not work try KILL and repeat > until successful or timeout" in the pmie init script, so perhaps this is a > long standing problem that has been masked by hackery. pmie does have a > TERM signal handler and a delayed exit but only after the nanosleep() ... so > if we are blocked somewhere else, or don't abandon expression evaluation > completely when an I/O returns with EINTR then we could be off in the weeds > long enough for some script to believe pmie has not died. > > Any insight into 3. would be helpful. The pmie (init/check) scripts were initially close cousins to the pmlogger scripts - the sigterm-upgrade-to-sigkill there is very likely to be mainly for historical reasons, rather than tackling any pmie specific issue. cheers. -- Nathan From nscott@redhat.com Tue Nov 4 23:40:13 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 92AD77F3F for ; Tue, 4 Nov 2014 23:40:13 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 80CC78F8033 for ; Tue, 4 Nov 2014 21:40:10 -0800 (PST) X-ASG-Debug-ID: 1415166007-04cbb070c564a1a0001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id QZ3l1GhKyBMM1fNX (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 04 Nov 2014 21:40:08 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sA55e76V031743; Wed, 5 Nov 2014 00:40:07 -0500 Date: Wed, 5 Nov 2014 00:40:07 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: David Smith Cc: pcp Message-ID: <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> In-Reply-To: <54512E80.9090302@redhat.com> References: <54512E80.9090302@redhat.com> Subject: Re: [pcp] [RFC] pcp python patch MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp python patch Thread-Index: wdMT5vXXBxXNRbrg0ivYLTeSaJY++w== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1415166008 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11248 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain Hi David, ----- Original Message ----- > First, some background. I'm working on integrating systemtap and pcp, > and I wrote a python PMDA that will be similar to the mmv pmda in that > it will need to monitor a directory for new data. After working on it, I > found out that the pcp python support doesn't really allow adding new > metrics (or clearing metrics) after PMDA.run() has been called. > > So, commit 38983a5 in the pcpfans.git dsmith/dev branch adds a 'refresh > metrics' callback to the pcp python support. This allows a python pmda > to add/remove metrics after PMDA.run() has been called. Looking good. > Here's how you'd use it. In your PMDA.init() function, you'd set the > callback, like this: > > self.set_refresh_metrics(self.__refresh_metrics) > > Then your __refresh_metrics (or whatever you want to call it) function > will get called when needed. I based this code on the mmv pmda, so it > could get called during several PDUs. Then your callback function could > add metrics. New code in the PMDA class will update everything. diff --git a/src/python/pcp/pmda.py b/src/python/pcp/pmda.py index 5d4e71b..51263aa 100644 --- a/src/python/pcp/pmda.py +++ b/src/python/pcp/pmda.py @@ -287,6 +287,7 @@ class PMDA(MetricDispatch): pmdaname = 'pmda' + name helpfile = '%s/%s/help' % (PCP.pmGetConfig('PCP_PMDAS_DIR'), name) MetricDispatch.__init__(self, domain, pmdaname, logfile, helpfile) + self.__refresh_metrics_func = None (see below for more, but I'm not sure we need to keep this handle to the callback routine up at this level as well as the cpmda callback handle? maybe we can simplify this) ## @@ -345,6 +346,26 @@ class PMDA(MetricDispatch): metrics[i] = self._metrictable[i] cpmda.pmda_dispatch(ibuf.raw, numindoms, mbuf.raw, nummetrics) + def __refresh_metrics(self): Hmm, do we need this interface? (there seem to be no callers in the wrapper - is the intention the PMDA would call this? If so a double-underscore (internal) routine is probably not right (for an API we are expecting PMDAs to use). If not explicitly needed, we should keep it simple and drop it (but, maybe I am missing something there?) I was expecting this "refreshing" would happen around the time PDUs exchanges happen with pmcd, which delays it until the last moment (and prevents the situation where unnecessary duplication of "refresh" work from a PMDA is done). Can we set that "refresh needed" flag in cpmda land from the PMDA and delay this work til then? That'd also encapsulate a bit more of the implementation details by pushing that down into the lower layers of the wrapper. + # Call the user's function. + if self.__refresh_metrics_func: + self.__refresh_metrics_func() + + # Now that the user's function has been called, we need to + # refresh all our class internal data. + # + # FIXME: instead of using cpmda.get_need_refresh(), we could + # ask the user to return a '1' from the refresh metrics + # function if a refresh is needed. + if cpmda.get_need_refresh(): + self.pmns_refresh() + cpmda.pmid_oneline_refresh(self._metric_oneline) + cpmda.pmid_longtext_refresh(self._metric_helptext) + cpmda.indom_oneline_refresh(self._indom_oneline) + cpmda.indom_longtext_refresh(self._indom_helptext) + cpmda.pmda_rehash(self._indomtable, self._metrictable) This ended up with lots of calls back down into the cpmda layer from the higher level python code - kinda feels like we may have some unnecessary up/down-calls here? (if that makes sense at all). IOW, can we push most of this down into the cpmda layer, hopefully making the interface exposed to PMDAs a little simpler/clearer? diff --git a/src/python/pmda.c b/src/python/pmda.c index f0c4845..3d6079c 100644 --- a/src/python/pmda.c +++ b/src/python/pmda.c [...] +static PyObject * +pmda_rehash(PyObject *self, PyObject *args, PyObject *keywords) +{ + char *keyword_list[] = {"indoms", "metrics", NULL}; + + if (indom_list) { + Py_DECREF(indom_list); + indom_list = NULL; + } + if (metric_list) { + Py_DECREF(metric_list); + metric_list = NULL; + } + + if (!PyArg_ParseTupleAndKeywords(args, keywords, "OO", keyword_list, + &indom_list, &metric_list)) + return NULL; + + if (indom_list && metric_list) { Do we require both of these (indom_list/metric_list) to be passed in? I guess it may happen that this code is used from PMDAs that do not have indoms (or the dynamic state is such that currently there are only metrics with no associated indoms?) - in those cases, we could be called here with a metric_list only I guess, so the logic in this branch would need some tweaking. #ifdef PyBUF_SIMPLE + static void + pmda_refresh_metrics(void) ... you've got a fair bit of new code there that's only going to be compiled for the python 2.x series (guarded by the above ifdef) - I haven't tried, but I'm punting we'll need to refactor a bit here to make this work for python 3 as well. cheers. -- Nathan From dsmith@redhat.com Wed Nov 5 10:43:36 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 9EFCD7F3F for ; Wed, 5 Nov 2014 10:43:36 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 1FD59AC007 for ; Wed, 5 Nov 2014 08:43:35 -0800 (PST) X-ASG-Debug-ID: 1415205810-04cbb070c7662630001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id kPP1a1mKrOcCjGtw (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 05 Nov 2014 08:43:31 -0800 (PST) X-Barracuda-Envelope-From: dsmith@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA5GhU2D029018 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Wed, 5 Nov 2014 11:43:30 -0500 Received: from t540p.usersys.redhat.com (vpn-61-129.rdu2.redhat.com [10.10.61.129]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA5GhSei023508 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 5 Nov 2014 11:43:29 -0500 Message-ID: <545A53B0.9030500@redhat.com> Date: Wed, 05 Nov 2014 10:43:28 -0600 From: David Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Nathan Scott CC: pcp Subject: Re: [pcp] [RFC] pcp python patch References: <54512E80.9090302@redhat.com> <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch In-Reply-To: <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415205811 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On 11/04/2014 11:40 PM, Nathan Scott wrote: > Hi David, > > ----- Original Message ----- >> First, some background. I'm working on integrating systemtap and pcp, >> and I wrote a python PMDA that will be similar to the mmv pmda in that >> it will need to monitor a directory for new data. After working on it, I >> found out that the pcp python support doesn't really allow adding new >> metrics (or clearing metrics) after PMDA.run() has been called. >> >> So, commit 38983a5 in the pcpfans.git dsmith/dev branch adds a 'refresh >> metrics' callback to the pcp python support. This allows a python pmda >> to add/remove metrics after PMDA.run() has been called. > > Looking good. > >> Here's how you'd use it. In your PMDA.init() function, you'd set the >> callback, like this: >> >> self.set_refresh_metrics(self.__refresh_metrics) >> >> Then your __refresh_metrics (or whatever you want to call it) function >> will get called when needed. I based this code on the mmv pmda, so it >> could get called during several PDUs. Then your callback function could >> add metrics. New code in the PMDA class will update everything. > > diff --git a/src/python/pcp/pmda.py b/src/python/pcp/pmda.py > index 5d4e71b..51263aa 100644 > --- a/src/python/pcp/pmda.py > +++ b/src/python/pcp/pmda.py > @@ -287,6 +287,7 @@ class PMDA(MetricDispatch): > pmdaname = 'pmda' + name > helpfile = '%s/%s/help' % (PCP.pmGetConfig('PCP_PMDAS_DIR'), name) > MetricDispatch.__init__(self, domain, pmdaname, logfile, helpfile) > + self.__refresh_metrics_func = None > > (see below for more, but I'm not sure we need to keep this > handle to the callback routine up at this level as well as > the cpmda callback handle? maybe we can simplify this) > > ## > @@ -345,6 +346,26 @@ class PMDA(MetricDispatch): > metrics[i] = self._metrictable[i] > cpmda.pmda_dispatch(ibuf.raw, numindoms, mbuf.raw, nummetrics) > > + def __refresh_metrics(self): > > > Hmm, do we need this interface? (there seem to be no callers in > the wrapper - is the intention the PMDA would call this? If so > a double-underscore (internal) routine is probably not right > (for an API we are expecting PMDAs to use). If not explicitly > needed, we should keep it simple and drop it (but, maybe I am > missing something there?) You have missed something here. See below. > I was expecting this "refreshing" would happen around the time > PDUs exchanges happen with pmcd, which delays it until the last > moment (and prevents the situation where unnecessary duplication > of "refresh" work from a PMDA is done). Can we set that "refresh > needed" flag in cpmda land from the PMDA and delay this work til > then? That'd also encapsulate a bit more of the implementation > details by pushing that down into the lower layers of the wrapper. > > + # Call the user's function. > + if self.__refresh_metrics_func: > + self.__refresh_metrics_func() > + > + # Now that the user's function has been called, we need to > + # refresh all our class internal data. > + # > + # FIXME: instead of using cpmda.get_need_refresh(), we could > + # ask the user to return a '1' from the refresh metrics > + # function if a refresh is needed. > > + if cpmda.get_need_refresh(): > + self.pmns_refresh() > + cpmda.pmid_oneline_refresh(self._metric_oneline) > + cpmda.pmid_longtext_refresh(self._metric_helptext) > + cpmda.indom_oneline_refresh(self._indom_oneline) > + cpmda.indom_longtext_refresh(self._indom_helptext) > + cpmda.pmda_rehash(self._indomtable, self._metrictable) > > This ended up with lots of calls back down into the cpmda layer > from the higher level python code - kinda feels like we may have > some unnecessary up/down-calls here? (if that makes sense at all). > IOW, can we push most of this down into the cpmda layer, hopefully > making the interface exposed to PMDAs a little simpler/clearer? So here's the deal with this. I probably should have explained this better when I sent the patch. Right now we've got 3 levels: - low (cpmda) - medium (PMDA class) - high (user's class derived from the PMDA class) Up at the high level (the user's class), a user calls add_metric() (for instance), which updates several of the PMDA's class internal variables, then calls the cpmda as needed. So, when a user wants to use the new refresh functionality, he calls PMDA.set_refresh_metrics(my_refresh_function). From the user's point of view, here's what happens. His 'my_refresh_function' gets called when necessary. In that function he can add new metrics or completely clear out the existing metrics and start from scratch adding metrics. The problem is that adding metrics updates a bunch of information at the medium level, the PMDA class itself - all those _metric_* and _indom_* lists above. So, if the low level (cpmda) called directly into the user's derived class (the high level), the PMDA class data would get updated, but neither the user's class or cpmda really knows anything about the PMDA class' internal data. So, the current solution wraps the user's refresh_metrics function with its own function called PMDA.__refresh_metrics(). So, what happens is that the cpmda really calls PMDA.__refresh_metrics() which calls the user's my_refresh_metrics(). If when we get back from the user's my_refresh_metrics() function something has changed that requires a refresh, the PMDA.__refresh_metrics() function sends the cpmda all the PMDA class internal data. Note that the list of PMDA class internal data being sent down to the cpmda in PMDA.__refresh_metrics() is exactly the same list as being sent down in PMDA.run(). If you don't like the PMDA class wrapper around the users' refresh function, another solution would be for the PMDA class to provide a function that would need to be called at the end of the user's refresh function to update everything (kind of like the PMDA.run() function). > > > diff --git a/src/python/pmda.c b/src/python/pmda.c > index f0c4845..3d6079c 100644 > --- a/src/python/pmda.c > +++ b/src/python/pmda.c > [...] > +static PyObject * > +pmda_rehash(PyObject *self, PyObject *args, PyObject *keywords) > +{ > + char *keyword_list[] = {"indoms", "metrics", NULL}; > + > + if (indom_list) { > + Py_DECREF(indom_list); > + indom_list = NULL; > + } > + if (metric_list) { > + Py_DECREF(metric_list); > + metric_list = NULL; > + } > + > + if (!PyArg_ParseTupleAndKeywords(args, keywords, "OO", keyword_list, > + &indom_list, &metric_list)) > + return NULL; > + > + if (indom_list && metric_list) { > > Do we require both of these (indom_list/metric_list) to be passed in? > I guess it may happen that this code is used from PMDAs that do not > have indoms (or the dynamic state is such that currently there are > only metrics with no associated indoms?) - in those cases, we could > be called here with a metric_list only I guess, so the logic in this > branch would need some tweaking. With the current code, calling this with an empty indom list would work fine. Calling it with 'None' would probably fail. > #ifdef PyBUF_SIMPLE > + static void > + pmda_refresh_metrics(void) > > ... you've got a fair bit of new code there that's only going to be > compiled for the python 2.x series (guarded by the above ifdef) - I > haven't tried, but I'm punting we'll need to refactor a bit here to > make this work for python 3 as well. Hmm, from my reading of the python devel code, PyBUF_SIMPLE is defined in python 2.6+. I think the code in the '#else' here is dead code for python < 2.6. -- David Smith dsmith@redhat.com Red Hat http://www.redhat.com 256.217.0141 (direct) 256.837.0057 (fax) From kenj@internode.on.net Wed Nov 5 15:07:05 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 53FAA7F3F for ; Wed, 5 Nov 2014 15:07:05 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id D26F0AC004 for ; Wed, 5 Nov 2014 13:07:04 -0800 (PST) X-ASG-Debug-ID: 1415221612-04cb6c1e6a08430001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id 5PYnqhDwIbrYPDvg for ; Wed, 05 Nov 2014 13:06:53 -0800 (PST) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApUBACGQWlR20Spq/2dsb2JhbAANT4NiWYMGyhaJBwEBAQEBhSlVMAYCBRYLAgsDAgECAVgGAgEBiEq2HXiVW4EtjUeFG4FUBYY3kDuiAViBBgWBQAEBAQ Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106]) by ipmail07.adl2.internode.on.net with ESMTP; 06 Nov 2014 07:36:50 +1030 Message-ID: <545A91B5.6010600@internode.on.net> Date: Thu, 06 Nov 2014 08:08:05 +1100 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - mostly QA Content-Type: text/plain; charset=utf-8 X-ASG-Orig-Subj: pcp updates - mostly QA Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1415221612 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11265 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://git.performancecopilot.org/kenj/pcp.git dev qa/518 | 51 ++++++++++++++++++++++++++++++----------------- qa/518.out | 8 +++---- qa/553 | 2 - qa/763 | 1 qa/841 | 1 qa/843 | 1 qa/985 | 2 - src/pcp/pcp.sh | 1 src/pmcd/src/client.c | 4 +++ src/pmcd/src/dopdus.c | 54 +++++++++++++++++++++++++++++++++++++++++++------- 10 files changed, 94 insertions(+), 31 deletions(-) commit 271e99450d614692db688cfb6ec11f11cb5527c5 Author: Ken McDonell Date: Thu Nov 6 08:06:51 2014 +1100 pmcd: additional diagnostics in the area of user/group authorisation commit 4f090f19d352c941c56874a353fc1121ccf6c8e9 Author: Ken McDonell Date: Thu Nov 6 07:57:17 2014 +1100 pcp(1): add missing sort for -P output With multiple pmie's running, pcp -P could fail because the inputs to one of the (many) join commands is not not sorted. Found in the rework of qa/518. commit b1549077c2e101a73278ffe86249cc5dc6097361 Author: Ken McDonell Date: Thu Nov 6 07:01:44 2014 +1100 qa/518: better checking for the expected number of rule evaluations Change timing to revert to older duration (4sec, not 12sec). Avoid use of pmsignal all together, so pmie exit is guaranteed. Tolerate small differences in the number of rule evaluations. commit 9c6480f97d20e5369d347a5535e859becc2d29c3 Author: Ken McDonell Date: Thu Nov 6 06:58:38 2014 +1100 qa/832: remove pmns before exit commit 9ecebec90605bf25af4ea73d31fd4e4a5cfd5df4 Author: Ken McDonell Date: Thu Nov 6 06:55:50 2014 +1100 qa/841 - remove domain.h.python before exit commit 3dde6954f6f042ef5c11d255d6094ea3cc33c9ab Author: Ken McDonell Date: Thu Nov 6 06:54:02 2014 +1100 qa/763 - remove domain.h.perl and pmns.perl before exit commit 5c13e3fe647822e27b18c862666784e89705b03d Author: Ken McDonell Date: Thu Nov 6 06:47:27 2014 +1100 qa/553 - remove ./gluster.log before exit commit 1bb003afe463f7aeb118b96c7cf451176ff1e218 Author: Ken McDonell Date: Thu Nov 6 06:45:48 2014 +1100 qa/985 - remove ./dmcache.log before exit commit a4393b405e9d3d449d93a40313dfae6d2a2c0d81 Author: Frank Ch. Eigler Date: Fri Oct 31 16:03:37 2014 -0400 qa/518: stretch timing to run more reliably This test case involves a race between pmie running rules a certain number of times, and a separate task waiting (by time) to match the results via /usr/bin/pcp. It was observed as flakey, not always winning the race. This version stretches the pmie time and fine-tunes the waiting interval to give the race a larger (multi-second) window for success. commit 31e5a3e1db23bc1755b39612c11fd3aeb18fc96c Author: Frank Ch. Eigler Date: Fri Oct 31 16:02:03 2014 -0400 qa/518: It was observed that the $sudo pmie invocation didn't always receive the closing signal at this test case, leading it to hang. Now we kill it harder, both by pid and later by name. From nscott@redhat.com Wed Nov 5 17:17:30 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 64DF57F3F for ; Wed, 5 Nov 2014 17:17:30 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 01091AC001 for ; Wed, 5 Nov 2014 15:17:29 -0800 (PST) X-ASG-Debug-ID: 1415229448-04bdf0650d0eb20001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id F6nm5751cgAoUXur (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 05 Nov 2014 15:17:28 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sA5NHRS2001504; Wed, 5 Nov 2014 18:17:27 -0500 Date: Wed, 5 Nov 2014 18:17:27 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: Lukas Berk Cc: pcp@oss.sgi.com Message-ID: <631823046.8750009.1415229447702.JavaMail.zimbra@redhat.com> In-Reply-To: <87ppd6ua57.fsf@redhat.com> References: <87ppd6ua57.fsf@redhat.com> Subject: Re: [pcp] PCP Presentation at FSOSS MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] PCP Presentation at FSOSS Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: PCP Presentation at FSOSS Thread-Index: 3RVkDhdZAB0J/lJKeLooc+YqlfYCAw== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1415229448 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11271 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain Hi Lukas, ----- Original Message ----- > Hey, > > Last week a colleague and I presented at Toronto's Free Software and > Open Source Symposium (FSOSS) about PCP. I've posted the slide deck if > anybody wants to take a look[1]. The overall 'message' of the talk was > to show how PCP is very flexible in the data it can assimilate and > how it presents data to the user. This was also demonstrated by using > several recent/current developments and how they fit in, conceptually, > with PCP. My hope is that the talk illustrated how the attendees could > solve their performance issues using the facilities PCP provides. > That's great (& thanks for the pdf too) - I've put copies of the slides at www.pcp.io/presentations.html as well. cheers. -- Nathan From nscott@redhat.com Wed Nov 5 18:36:58 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id D255F7F3F for ; Wed, 5 Nov 2014 18:36:57 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id B23D38F8039 for ; Wed, 5 Nov 2014 16:36:54 -0800 (PST) X-ASG-Debug-ID: 1415234211-04bdf0650c11840001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id xNAaKikDGeWrQNSv (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 05 Nov 2014 16:36:51 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sA60apXh014115 for ; Wed, 5 Nov 2014 19:36:51 -0500 Date: Wed, 5 Nov 2014 19:36:51 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: pcp Message-ID: <1494245284.8766580.1415234211066.JavaMail.zimbra@redhat.com> In-Reply-To: <291786473.8766572.1415234195881.JavaMail.zimbra@redhat.com> Subject: pcp updates: qa, build MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: qa, build Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: qa, build Thread-Index: d3/fClQJnts+3GUGSYwcUJUBv+XRug== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1415234211 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11275 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.pcp.io/pcp.git dev Ken McDonell (9): qa/985 - remove ./dmcache.log before exit qa/553 - remove ./gluster.log before exit qa/763 - remove domain.h.perl and pmns.perl before exit qa/841 - remove domain.h.python before exit qa/832: remove pmns before exit qa/518: better checking for the expected number of rule evaluations pcp(1): add missing sort for -P output pmcd: additional diagnostics in the area of user/group authorisation qa/944: better handling of multiple users being in the same default group Frank Ch. Eigler (2): qa/518: qa/518: stretch timing to run more reliably Nathan Scott (1): build: update packaging and project forward for next release date CHANGELOG | 3 ++ VERSION.pcp | 2 - build/rpm/fedora.spec | 13 +++--------- build/rpm/pcp.spec.in | 8 ------- debian/changelog | 6 +++++ qa/518 | 51 ++++++++++++++++++++++++++++++----------------- qa/518.out | 8 +++---- qa/553 | 2 - qa/763 | 1 qa/841 | 1 qa/843 | 1 qa/944 | 1 qa/985 | 2 - src/pcp/pcp.sh | 1 src/pmcd/src/client.c | 4 +++ src/pmcd/src/dopdus.c | 54 +++++++++++++++++++++++++++++++++++++++++++------- 16 files changed, 109 insertions(+), 49 deletions(-) From nscott@redhat.com Thu Nov 6 00:34:49 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2FBEB7F3F for ; Thu, 6 Nov 2014 00:34:49 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 0E7EB304043 for ; Wed, 5 Nov 2014 22:34:45 -0800 (PST) X-ASG-Debug-ID: 1415255680-04cbb00e681c2e0001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id R8Hsv24xGlEMNvXy (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 05 Nov 2014 22:34:41 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA66YewF027730; Thu, 6 Nov 2014 01:34:40 -0500 Date: Thu, 6 Nov 2014 01:34:40 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: David Smith Cc: pcp Message-ID: <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> In-Reply-To: <545A53B0.9030500@redhat.com> References: <54512E80.9090302@redhat.com> <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> <545A53B0.9030500@redhat.com> Subject: Re: [pcp] [RFC] pcp python patch MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp python patch Thread-Index: NBa4cZSGmDqGa/KyX1UOH+h2d1ya3w== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1415255681 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11284 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain Hi David, ----- Original Message ----- > On 11/04/2014 11:40 PM, Nathan Scott wrote: > > (for an API we are expecting PMDAs to use). If not explicitly > > needed, we should keep it simple and drop it (but, maybe I am > > missing something there?) > > You have missed something here. See below. That's usually it :) - thanks for the extra explanation. > > of "refresh" work from a PMDA is done). Can we set that "refresh > > needed" flag in cpmda land from the PMDA and delay this work til > > then? That'd also encapsulate a bit more of the implementation > > details by pushing that down into the lower layers of the wrapper. > > > > + # Call the user's function. > > + if self.__refresh_metrics_func: > > + self.__refresh_metrics_func() > > + > > + # Now that the user's function has been called, we need to > > + # refresh all our class internal data. > > + # > > + # FIXME: instead of using cpmda.get_need_refresh(), we could > > + # ask the user to return a '1' from the refresh metrics > > + # function if a refresh is needed. > > > > + if cpmda.get_need_refresh(): > > + self.pmns_refresh() > > + cpmda.pmid_oneline_refresh(self._metric_oneline) > > + cpmda.pmid_longtext_refresh(self._metric_helptext) > > + cpmda.indom_oneline_refresh(self._indom_oneline) > > + cpmda.indom_longtext_refresh(self._indom_helptext) > > + cpmda.pmda_rehash(self._indomtable, self._metrictable) > > > > This ended up with lots of calls back down into the cpmda layer > > from the higher level python code - kinda feels like we may have > > some unnecessary up/down-calls here? (if that makes sense at all). > > IOW, can we push most of this down into the cpmda layer, hopefully > > making the interface exposed to PMDAs a little simpler/clearer? > > So here's the deal with this. I probably should have explained this > better when I sent the patch. > > Right now we've got 3 levels: > > - low (cpmda) > - medium (PMDA class) > - high (user's class derived from the PMDA class) > > Up at the high level (the user's class), a user calls add_metric() (for > instance), which updates several of the PMDA's class internal variables, > then calls the cpmda as needed. Yeah, but :) ... its a bit more convoluted than that isn't it? e.g. the dictionary "self._metric_helptext" at the medium level is also open for business down at the low level, via "pmid_longtext_dict" - isn't it? If this is as I remember it, the boundaries are blurred a bit here for convenient access between middle and lower levels - we may be able to exploit that a bit to simplify the new API? If its not as I remember, just disregard this & carry on. :) The only other thing I wanted to revisit a bit is this subtlety... > > I was expecting this "refreshing" would happen around the time > > PDUs exchanges happen with pmcd, which delays it until the last > > moment (and prevents the situation where unnecessary duplication Its not 100% clear to me from... > So, when a user wants to use the new refresh functionality, he calls > PMDA.set_refresh_metrics(my_refresh_function). From the user's point of > view, here's what happens. His 'my_refresh_function' gets called when > necessary. ... what the definition of "when necessary" is? Is it at the time we are responding to PDUs from pmcd, or is it the time the PMDA has realised "something changed" (i.e. up at the high level)? Heh, it's a twisty maze following all these callbacks - sorry. If its the latter situation, we'll sometimes end up doing unnecessary refreshing - so if that can be delayed until PDU processing time (eg via that existing just-set-a-flag-and-refresh-me-later mechanism), that will help reduce CPU time for some situations. > > If you don't like the PMDA class wrapper around the users' refresh > function, another solution would be for the PMDA class to provide a > function that would need to be called at the end of the user's refresh > function to update everything (kind of like the PMDA.run() function). > I've not got a strong opinion really - stick with what you have? sounds simpler. > > With the current code, calling this with an empty indom list would work > fine. Calling it with 'None' would probably fail. Okie doke, something worth double-checking anyway I guess (since "fail" here means "generate an exception and exit(3) the PMDA process", a long running daemon). > > #ifdef PyBUF_SIMPLE > > + static void > > + pmda_refresh_metrics(void) > > > > ... you've got a fair bit of new code there that's only going to be > > compiled for the python 2.x series (guarded by the above ifdef) - I > > haven't tried, but I'm punting we'll need to refactor a bit here to > > make this work for python 3 as well. > > Hmm, from my reading of the python devel code, PyBUF_SIMPLE is defined > in python 2.6+. I think the code in the '#else' here is dead code for > python < 2.6. Ah, yes, right you are - I was thinking PyBUF_SIMPLE was removed in 3.x but on closer inspection it is still there & yep, looks like dead code. I do vaguely remember some py3 porting issues around buffers - but this looks unrelated. cheers. -- Nathan From nscott@redhat.com Thu Nov 6 02:14:25 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 84AED7F3F for ; Thu, 6 Nov 2014 02:14:25 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 632BF8F8033 for ; Thu, 6 Nov 2014 00:14:22 -0800 (PST) X-ASG-Debug-ID: 1415261660-04cb6c1e6b22d30001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id c2X9FRPYseocwnvk (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 06 Nov 2014 00:14:20 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA68EJun026826 for ; Thu, 6 Nov 2014 03:14:19 -0500 Date: Thu, 6 Nov 2014 03:14:19 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: pcp Message-ID: <384411614.8875153.1415261659884.JavaMail.zimbra@redhat.com> In-Reply-To: <1388625474.8859393.1415259838057.JavaMail.zimbra@redhat.com> Subject: pcp updates: qa, build MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: qa, build Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: qa, build Thread-Index: hMEMixhBLkZgsp2GUDijz0NcNGQzSw== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1415261660 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11286 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.pcp.io/pcp.git dev Nathan Scott (3): qa: fix cleanup trap in qa/898 from leaving tmpfile remnants qa: fix cleanup trap in qa/067 from leaving tmpfile remnants build: default to a more professionally named webapps directory build/rpm/fedora.spec | 6 ++++-- debian/pcp-webapi.dirs | 1 + man/html/guide.html | 8 ++++---- man/man1/pmwebd.1 | 6 +++--- qa/067 | 8 ++++---- qa/661 | 9 ++++++--- qa/898 | 8 +++----- src/pmwebapi/GNUmakefile | 1 + src/pmwebapi/pmwebd.options | 2 +- 9 files changed, 27 insertions(+), 22 deletions(-) From myllynen@redhat.com Thu Nov 6 05:00:58 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 61C5D7F51 for ; Thu, 6 Nov 2014 05:00:58 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id D7DC4AC007 for ; Thu, 6 Nov 2014 03:00:54 -0800 (PST) X-ASG-Debug-ID: 1415271649-04cb6c1e6a3fa80001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id GdFrUbULSibkUyVI (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 06 Nov 2014 03:00:50 -0800 (PST) X-Barracuda-Envelope-From: myllynen@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA6B0lb5011818 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Thu, 6 Nov 2014 06:00:48 -0500 Received: from mmyllyne.csb (vpn-203-21.tlv.redhat.com [10.35.203.21]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA6B0jOH011470 for ; Thu, 6 Nov 2014 06:00:46 -0500 Message-ID: <545B54DD.8060205@redhat.com> Date: Thu, 06 Nov 2014 13:00:45 +0200 From: Marko Myllynen Reply-To: myllynen@redhat.com Organization: Red Hat User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: PCP Subject: [PATCH] Clarify the web services sections in the quick guide Content-Type: text/plain; charset=ISO-8859-1 X-ASG-Orig-Subj: [PATCH] Clarify the web services sections in the quick guide Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415271650 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, below is an attempt to clarify the web services sections in the Quick Reference Guide a bit now that we have first builds available. --- man/html/guide.html | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) diff --git a/man/html/guide.html b/man/html/guide.html index db78c6c..a143655 100644 --- a/man/html/guide.html +++ b/man/html/guide.html @@ -475,7 +475,10 @@ This example shows a PMIE script, checks its syntax, runs it against an archive,

PCP Web Services

-

A daemon for exporting PCP metrics using a REST web service (over HTTP/JSON) is also available. Use this for viewing or monitoring PCP metrics in a web browser - several web interfaces are becoming available (also via the pcp-webapi package) to make this a reality. + +

Performance Metrics Web Daemon

+ +

Performance Metrics Web Daemon (pmwebd(1)) is a front-end to both PMCD and PCP archives, providing a REST web service (over HTTP/JSON) suitable for use by web-based tools wishing to access performance data over HTTP. Custom applications can access all the available PCP information using this method, including custom metrics generated by custom PMDAs.

   To install the PCP web service on Fedora/RHEL:
@@ -490,17 +493,12 @@ This example shows a PMIE script, checks its syntax, runs it against an archive,
- -

Performance Metrics Web Daemon

- -

Performance Metrics Web Daemon (pmwebd(1)) is a front-end to both PMCD and PCP archives, providing a JSON interface suitable for use by web-based tools wishing to access performance data over HTTP. Custom applications can access all the available PCP information using this method, including custom metrics generated by custom PMDAs. - -

Web Interface for Performance Metrics

+

User Web Interface for Performance Metrics

-

Several browser interfaces for accessing PCP performance metrics are available. These web interfaces make PCP metrics available via your choice of Grafana or Graphite. +

Several browser interfaces for accessing PCP performance metrics are also available. These web interfaces make PCP metrics available via your choice of Grafana or Graphite. -

Install the PCP web services daemon as described above, and the contents of the pcp-webjs package (platform-independent HTML and Javascript) into /usr/share/pcp/webapps, then point a browser toward http://localhost:44323. +

After installing the PCP web services daemon as described above, install the pcp-webjs package and then just point a browser toward http://localhost:44323.

Customizing and Extending PCP

-- 1.7.1 -- Marko Myllynen From chandana@desilva.id.au Thu Nov 6 17:01:57 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 6D6A97F3F for ; Thu, 6 Nov 2014 17:01:57 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 4C729304032 for ; Thu, 6 Nov 2014 15:01:54 -0800 (PST) X-ASG-Debug-ID: 1415314912-04bdf0650f9f910001-S8gJnT Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) by cuda.sgi.com with ESMTP id 1m6MNa3Amfyd7Fz1 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 06 Nov 2014 15:01:52 -0800 (PST) X-Barracuda-Envelope-From: chandana@desilva.id.au X-Barracuda-Apparent-Source-IP: 204.13.248.72 Received: from ec2-54-252-74-219.ap-southeast-2.compute.amazonaws.com ([54.252.74.219] helo=mail.desilva.id.au) by mho-02-ewr.mailhop.org with esmtpa (Exim 4.72) (envelope-from ) id 1XmW3r-0006Co-MK for pcp@oss.sgi.com; Thu, 06 Nov 2014 23:01:51 +0000 Received: from [192.168.19.83] (unknown [175.45.83.34]) by mail.desilva.id.au (Postfix) with ESMTPSA id A41DB24190 for ; Thu, 6 Nov 2014 23:01:49 +0000 (UTC) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 54.252.74.219 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+qfh8DZLeRShDkfB7hBbZEkXnECCQ69Sg= Message-ID: <1415314907.2202.17.camel@tardis> Subject: Makepkgs does not work from source tarball From: Chandana De Silva X-ASG-Orig-Subj: Makepkgs does not work from source tarball Reply-To: chandana@desilva.id.au To: pcp@oss.sgi.com Date: Fri, 07 Nov 2014 10:01:47 +1100 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4 (3.10.4-4.fc20) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mho-02-ewr.mailhop.org[204.13.248.72] X-Barracuda-Start-Time: 1415314912 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11311 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- I just downloaded the source tarball for 3.10 $ ./Makepkgs Error: can only be run from within the PCP git repository Is this intentional ? From nscott@redhat.com Thu Nov 6 17:32:28 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 7E1487F3F for ; Thu, 6 Nov 2014 17:32:28 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 1BB69AC006 for ; Thu, 6 Nov 2014 15:32:24 -0800 (PST) X-ASG-Debug-ID: 1415316743-04bdf0650fa76a0001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id fLdrhvoiVwqg3pDP (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 06 Nov 2014 15:32:23 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA6NWMjn030898; Thu, 6 Nov 2014 18:32:22 -0500 Date: Thu, 6 Nov 2014 18:32:21 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: chandana@desilva.id.au Cc: pcp@oss.sgi.com Message-ID: <10189230.9465033.1415316741674.JavaMail.zimbra@redhat.com> In-Reply-To: <1415314907.2202.17.camel@tardis> References: <1415314907.2202.17.camel@tardis> Subject: Re: [pcp] Makepkgs does not work from source tarball MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] Makepkgs does not work from source tarball Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: Makepkgs does not work from source tarball Thread-Index: G2J9DOuhI/3eje/3ceSy5vcT6aYmjA== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1415316743 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11312 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header Hi Chandana, ----- Original Message ----- > I just downloaded the source tarball for 3.10 > > $ ./Makepkgs > Error: can only be run from within the PCP git repository > > Is this intentional ? Yep. A few releases back we changed the build to depend on using git for some steps that form part of the build - this (use of git) became required when using Makepkgs as a result. The source tarball can be built using configure/make (this is what the individual distro builds do). cheers. -- Nathan From nscott@redhat.com Thu Nov 6 23:02:30 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 3AF8E7F3F for ; Thu, 6 Nov 2014 23:02:30 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 29B30304032 for ; Thu, 6 Nov 2014 21:02:27 -0800 (PST) X-ASG-Debug-ID: 1415336539-04cbb00e66aa350001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id 2RJGTlcVxln9w1Ub (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 06 Nov 2014 21:02:20 -0800 (PST) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sA752JjJ017435 for ; Fri, 7 Nov 2014 00:02:19 -0500 Date: Fri, 7 Nov 2014 00:02:19 -0500 (EST) From: Nathan Scott Reply-To: Nathan Scott To: pcp Message-ID: <1691519215.9520620.1415336539691.JavaMail.zimbra@redhat.com> In-Reply-To: <1234051244.9520509.1415336475810.JavaMail.zimbra@redhat.com> Subject: pcp updates: docs, qa, proc, perfevent pmda MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: docs, qa, proc, perfevent pmda Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: docs, qa, proc, perfevent pmda Thread-Index: ScRZO0Nu0gHYw2uNhjSb5x5ntiJlYw== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1415336540 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11321 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.pcp.io/pcp.git dev Joe White (13): Added perfevent PMDA to build structure. Lock file path is now obtained from the environment. Add man page for config file. Don't create DSO build. perfevent is now built as a seperate RPM. Added environment variables to proc pmda. Fixed -A flag. Add cpusallowed metric to linux_proc Added tests for perfevent PMDA Added missing test configuration file. Added another missing file. Forgot to update the expected output with the new test ids. Don't use bash extensions in a borne shell script. Ensure tests are independent of the number of CPUs present on the test host. Allow the test harness to work properly if the perfevent PMDA is not built. Nathan Scott (2): qa: remove a superflous trap in ds390log test perfevent pmda: updates, latest code/build/qa conventions mainly Marko Myllynen (1): docs: clarify web services sections in the quick guide build/rpm/GNUmakefile | 3 build/rpm/fedora.spec | 43 build/rpm/pcp.spec.in | 57 configure | 245 +- configure.ac | 52 man/html/guide.html | 16 man/man1/GNUmakefile | 3 man/man1/perfalloc.1 | 416 ++- man/man1/pmdaperfevent.1 | 196 + man/man5/perfevent.conf.5 | 76 qa/756 | 37 qa/756.out | 77 qa/757 | 215 + qa/757.out | 40 qa/960 | 5 qa/GNUmakefile | 2 qa/GNUmakefile.install | 7 qa/common.filter | 3 qa/group | 2 qa/perfevent/GNUmakefile | 92 qa/perfevent/GNUmakefile.install | 21 qa/perfevent/config/syntax_error.txt | 7 qa/perfevent/config/test_config.txt | 14 qa/perfevent/config/test_event_programming.txt | 7 qa/perfevent/config/test_init.txt | 5 qa/perfevent/config/test_lots_of_counters.txt | 19 qa/perfevent/config/test_node_rr.txt | 10 qa/perfevent/config/test_rapl.txt | 27 qa/perfevent/fakefs.tar.gz |binary qa/perfevent/fakefs/proc/cpuinfo | 600 ++--- qa/perfevent/fakefs/sys/devices/system/node/node0/cpulist | 2 qa/perfevent/fakefs/sys/devices/system/node/node1/cpulist | 2 qa/perfevent/mock_pfm.c | 219 + qa/perfevent/mock_pfm.h | 16 qa/perfevent/mockperfinterface.c | 33 qa/perfevent/perf_event_test.c | 612 +++++ qa/perfevent/perfevent.conf | 7 qa/perfevent/rapl_test.c | 49 qa/perfevent/runtest | 47 qa/perfevent/threadtest.c | 34 src/include/builddefs.in | 4 src/pmdas/GNUmakefile | 2 src/pmdas/linux_proc/help | 2 src/pmdas/linux_proc/pmda.c | 31 src/pmdas/linux_proc/proc_pid.c | 41 src/pmdas/linux_proc/proc_pid.h | 13 src/pmdas/linux_proc/root_proc | 2 src/pmdas/perfevent/ChangeLog | 28 src/pmdas/perfevent/GNUmakefile | 108 src/pmdas/perfevent/Install | 45 src/pmdas/perfevent/Remove | 38 src/pmdas/perfevent/architecture.c | 260 ++ src/pmdas/perfevent/architecture.h | 52 src/pmdas/perfevent/configparser.h | 62 src/pmdas/perfevent/configparser.l | 251 ++ src/pmdas/perfevent/domain.h | 4 src/pmdas/perfevent/help | 20 src/pmdas/perfevent/perfalloc.c | 193 + src/pmdas/perfevent/perfevent.conf | 243 ++ src/pmdas/perfevent/perfinterface.c | 645 +++++ src/pmdas/perfevent/perfinterface.h | 56 src/pmdas/perfevent/perflock.c | 75 src/pmdas/perfevent/perflock.h | 26 src/pmdas/perfevent/perfmanager.c | 287 ++ src/pmdas/perfevent/perfmanager.h | 41 src/pmdas/perfevent/pmda.c | 579 ++++ src/pmdas/perfevent/pmns | 22 src/pmdas/perfevent/rapl-interface.c | 343 ++ src/pmdas/perfevent/rapl-interface.h | 37 src/pmdas/perfevent/root | 10 src/pmdas/perfevent/unittest/config/syntax_error.txt | 14 src/pmdas/perfevent/unittest/config/test_config.txt | 28 src/pmdas/perfevent/unittest/config/test_event_programming.txt | 14 src/pmdas/perfevent/unittest/config/test_init.txt | 10 src/pmdas/perfevent/unittest/config/test_lots_of_counters.txt | 38 src/pmdas/perfevent/unittest/config/test_node_rr.txt | 20 src/pmdas/perfevent/unittest/config/test_rapl.txt | 54 src/pmdas/perfevent/unittest/fakefs/proc/cpuinfo | 600 ++--- src/pmdas/perfevent/unittest/fakefs/sys/devices/system/node/node0/cpulist | 2 src/pmdas/perfevent/unittest/fakefs/sys/devices/system/node/node1/cpulist | 2 src/pmdas/perfevent/unittest/mock_pfm.c | 438 +-- src/pmdas/perfevent/unittest/mock_pfm.h | 32 src/pmdas/perfevent/unittest/mockperfinterface.c | 66 src/pmdas/perfevent/unittest/perf_event_test.c | 1188 +++++----- src/pmdas/perfevent/unittest/rapl_test.c | 98 src/pmdas/perfevent/unittest/runtest | 94 src/pmdas/perfevent/unittest/threadtest.c | 68 87 files changed, 7495 insertions(+), 2109 deletions(-) From tdm@sgi.com Fri Nov 7 09:38:55 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 74CF47F3F for ; Fri, 7 Nov 2014 09:38:55 -0600 (CST) Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id 4D33F8F8033; Fri, 7 Nov 2014 07:38:52 -0800 (PST) Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11]) by estes.americas.sgi.com (Postfix) with ESMTP id 2B5727001E78; Fri, 7 Nov 2014 09:38:52 -0600 (CST) Message-ID: <545CE78C.5000102@sgi.com> Date: Fri, 07 Nov 2014 09:38:52 -0600 From: Troy McCorkell User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Cc: Troy McCorkell Subject: oss.sgi.com - maintenance downtime Friday 11/14/2014 at 10:00 CST USA Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On Friday November 14, 2014 at 10:00 CST USA oss.sgi.com will be unavailable for a short period of time to perform network maintenance. The outage is expected to last approximately 15 minutes. From dsmith@redhat.com Fri Nov 7 10:09:13 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id DF52F7F3F for ; Fri, 7 Nov 2014 10:09:12 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 1A2DFAC002 for ; Fri, 7 Nov 2014 08:09:04 -0800 (PST) X-ASG-Debug-ID: 1415376540-04bdf0650d1113f0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id FDHhmjNrgvBamm3k (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 07 Nov 2014 08:09:01 -0800 (PST) X-Barracuda-Envelope-From: dsmith@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sA7G901O010563 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Fri, 7 Nov 2014 11:09:00 -0500 Received: from t540p.usersys.redhat.com (dhcp-10-15-1-100.hsv.redhat.com [10.15.1.100]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sA7G8xmg024973 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 7 Nov 2014 11:09:00 -0500 Message-ID: <545CEE9A.5060007@redhat.com> Date: Fri, 07 Nov 2014 10:08:58 -0600 From: David Smith User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Nathan Scott CC: pcp Subject: Re: [pcp] [RFC] pcp python patch References: <54512E80.9090302@redhat.com> <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> <545A53B0.9030500@redhat.com> <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch In-Reply-To: <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1415376541 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On 11/06/2014 12:34 AM, Nathan Scott wrote: > Hi David, > > ----- Original Message ----- >> On 11/04/2014 11:40 PM, Nathan Scott wrote: >>> (for an API we are expecting PMDAs to use). If not explicitly >>> needed, we should keep it simple and drop it (but, maybe I am >>> missing something there?) >> >> You have missed something here. See below. > > That's usually it :) - thanks for the extra explanation. > >>> of "refresh" work from a PMDA is done). Can we set that "refresh >>> needed" flag in cpmda land from the PMDA and delay this work til >>> then? That'd also encapsulate a bit more of the implementation >>> details by pushing that down into the lower layers of the wrapper. >>> >>> + # Call the user's function. >>> + if self.__refresh_metrics_func: >>> + self.__refresh_metrics_func() >>> + >>> + # Now that the user's function has been called, we need to >>> + # refresh all our class internal data. >>> + # >>> + # FIXME: instead of using cpmda.get_need_refresh(), we could >>> + # ask the user to return a '1' from the refresh metrics >>> + # function if a refresh is needed. >>> >>> + if cpmda.get_need_refresh(): >>> + self.pmns_refresh() >>> + cpmda.pmid_oneline_refresh(self._metric_oneline) >>> + cpmda.pmid_longtext_refresh(self._metric_helptext) >>> + cpmda.indom_oneline_refresh(self._indom_oneline) >>> + cpmda.indom_longtext_refresh(self._indom_helptext) >>> + cpmda.pmda_rehash(self._indomtable, self._metrictable) >>> >>> This ended up with lots of calls back down into the cpmda layer >>> from the higher level python code - kinda feels like we may have >>> some unnecessary up/down-calls here? (if that makes sense at all). >>> IOW, can we push most of this down into the cpmda layer, hopefully >>> making the interface exposed to PMDAs a little simpler/clearer? >> >> So here's the deal with this. I probably should have explained this >> better when I sent the patch. >> >> Right now we've got 3 levels: >> >> - low (cpmda) >> - medium (PMDA class) >> - high (user's class derived from the PMDA class) >> >> Up at the high level (the user's class), a user calls add_metric() (for >> instance), which updates several of the PMDA's class internal variables, >> then calls the cpmda as needed. > > Yeah, but :) ... its a bit more convoluted than that isn't it? e.g. > the dictionary "self._metric_helptext" at the medium level is also > open for business down at the low level, via "pmid_longtext_dict" - > isn't it? If this is as I remember it, the boundaries are blurred a > bit here for convenient access between middle and lower levels - we > may be able to exploit that a bit to simplify the new API? > > If its not as I remember, just disregard this & carry on. :) I've taken another look at this. I see where my confusion came from. We have 3 distinct classes of dictionaries, each treated differently. During PMDA.run(), the PMDA class passes down to cpmda several dictionaries: (1) self._metric_names (used by cpmda.pmns_refresh) (2) self._metric_oneline (used by cpmda.pmid_oneline_refresh) (2) self._metric_helptext (used by cpmda.pmid_longtext_refresh) (2) self._indom_oneline (used by cpmda.indom_oneline_refresh) (2) self._indom_helptext (used by cpmda.indom_longtext_refresh) (3) self._indomtable (used by cpmda.pmda_dispatch) (3) self._metrictable (used by cpmda.pmda_dispatch) Dictionaries marked by (1) are passed down, used once when needed, then discarded. Dictionaries marked by (2) are passed down and kept, used when needed, never discarded. Dictionaries marked by (3) aren't passed down and kept. Instead the PMDA class creates a string buffer for each dictionary and passes that down to cpmda. This string buffer is used once and then discarded. I was thinking all dictionaries were treated as being in class (1) (used once then discarded). That's why I was passing down everything again in PMDA.__refresh_metrics(). Since I don't need to pass down items again in class (2), that simplifies PMDA.__refresh_metrics() to: ==== def __refresh_metrics(self): # Call the user's function. if self.__refresh_metrics_func: self.__refresh_metrics_func() if cpmda.get_need_refresh(): self.pmns_refresh() cpmda.pmda_rehash(self._indomtable, self._metrictable) ==== We actually could go further here, if we can change the cpmda interface. I'm not sure how set in stone it is. I'm not talking about a change to the PMDA class interface here, just the cpmda interface. If we change cpmda.pmda_dispatch() to take the indomtable and metrictable dictionaries directly (instead of using string buffers), we could keep them around for use later by pmda_rehash() - basically moving them to class (2). If we go that far, then all dictionaries are in class (2) (passed down and kept) - except for _metric_names in class (1). If we decide to move it to class (2), then all dictionaries are treated the same. At that point, we don't need a wrapper around the user's refresh_metrics() function in the PMDA class at all, since the cpmda already has all the information it needs (pointers to all the dictionaries) and knows when to do the refreshing. > The only other thing I wanted to revisit a bit is this subtlety... > >>> I was expecting this "refreshing" would happen around the time >>> PDUs exchanges happen with pmcd, which delays it until the last >>> moment (and prevents the situation where unnecessary duplication I'm afraid you'll have decide on/help me figure out pcp subtleties... Right now the user's refresh_metrics callback function gets called down in cpmda at similar places as the mmv pmda calls mmv_reload_maybe(). I'll need help in figuring out if that is causing unnecessary duplication. Assuming the user's refresh_metrics callback is smart enough to not change anything when it doesn't need to, then cpmda shouldn't really refresh anything either. >>> ... you've got a fair bit of new code there that's only going to be >>> compiled for the python 2.x series (guarded by the above ifdef) - I >>> haven't tried, but I'm punting we'll need to refactor a bit here to >>> make this work for python 3 as well. >> >> Hmm, from my reading of the python devel code, PyBUF_SIMPLE is defined >> in python 2.6+. I think the code in the '#else' here is dead code for >> python < 2.6. > > Ah, yes, right you are - I was thinking PyBUF_SIMPLE was removed in 3.x > but on closer inspection it is still there & yep, looks like dead code. > I do vaguely remember some py3 porting issues around buffers - but this > looks unrelated. We should probably just delete the whole '#else' code to avoid confusing others in the future. -- David Smith dsmith@redhat.com Red Hat http://www.redhat.com 256.217.0141 (direct) 256.837.0057 (fax) From minnus@buffalo.edu Fri Nov 7 14:35:14 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 79A8B7F3F for ; Fri, 7 Nov 2014 14:35:14 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 48819304032 for ; Fri, 7 Nov 2014 12:35:14 -0800 (PST) X-ASG-Debug-ID: 1415392509-04bdf0650f120690001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve6.acsu.buffalo.edu [128.205.6.4]) by cuda.sgi.com with ESMTP id nsIWxQaQKywK608d for ; Fri, 07 Nov 2014 12:35:09 -0800 (PST) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.4 Received: from localmailA.acsu.buffalo.edu (localmaila.acsu.buffalo.edu [128.205.5.196]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id E191F1050A for ; Fri, 7 Nov 2014 15:35:08 -0500 (EST) Received: from localmailA.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id DE1E8EB43 for ; Fri, 7 Nov 2014 15:35:08 -0500 (EST) Received: from localmailA.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailA.acsu.buffalo.edu (Postfix) with ESMTP id 78BBFEB3B for ; Fri, 7 Nov 2014 15:35:07 -0500 (EST) Received: from smtp.buffalo.edu (smtp1.acsu.buffalo.edu [128.205.5.253]) by localmailA.acsu.buffalo.edu (Prefixe) with ESMTP id 5ED93EB3A for ; Fri, 7 Nov 2014 15:35:07 -0500 (EST) Received: from prince.ccr.buffalo.edu (prince.ccr.buffalo.edu [128.205.40.45]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 54DCE21BE for ; Fri, 7 Nov 2014 15:35:07 -0500 (EST) Message-ID: <545D2CFA.1050600@buffalo.edu> Date: Fri, 07 Nov 2014 15:35:06 -0500 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: pcp Subject: Dynamic Proc PMDA metrics Content-Type: multipart/mixed; boundary="------------040503090407020609020107" X-ASG-Orig-Subj: Dynamic Proc PMDA metrics X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve6.acsu.buffalo.edu[128.205.6.4] X-Barracuda-Start-Time: 1415392509 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11342 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 HTML_MESSAGE BODY: HTML included in message This is a multi-part message in MIME format. --------------040503090407020609020107 Content-Type: multipart/alternative; boundary="------------030903090205040300060002" --------------030903090205040300060002 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I think I have a pretty good version of the dynamic proc metrics implemented as they relate to anything with a PID instance. My tree is here: https://github.com/ubccr/pcp/tree/dynamic_proc src/pmdas/linux_proc/GNUmakefile | 2 +- src/pmdas/linux_proc/help | 140 ---------- src/pmdas/linux_proc/help_text.h | 121 +++++++++ src/pmdas/linux_proc/pmda.c | 13 +- src/pmdas/linux_proc/proc_dynamic.c | 477 +++++++++++++++++++++++++++++++++++ src/pmdas/linux_proc/root_proc | 129 +--------- 6 files changed, 614 insertions(+), 268 deletions(-) create mode 100644 src/pmdas/linux_proc/help_text.h create mode 100644 src/pmdas/linux_proc/proc_dynamic.c If this all looks generally ok, I'll submit my hotprocs changes on top of this. I modeled it on the basic structure of linux/interrupts.c. A couple things: 1. I ran the existing QA tests (./check -g pmda.proc) and everything seemed to be OK, except for the ordering of the metrics in 022 and 943. I don't think this can be fixed since it would mean some non-dynamic metrics would have to come in the middle. Let me know if there is a way/desire to fix the ordering. 2. I needed to use pmdaTreeRebuildHash while the interrupts code didn't. Joe pointed out that this is likely due to that fact that the linux pmda runs as a dso and proc does not. So there may be some other code that handles the dso case differently. I couldn't find this documented anywhere. 3. Couldn't find any precedent for handling long form help text for dynamic pmdas. So I just generated a header file from the existing help file and used that. Let me know if you want something different. Attached is the perl script I wrote for that in case it is generally useful. 4. I left a few commented code sections that will show where hotprocs will hook in. Will add another root node, and all other stuff ( help, etc ) will be shared. If this looks ok, it will probably take another day to update hotprocs. Let me know if there is anything that looks wrong or needs to be changed. Thanks Martins --------------030903090205040300060002 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit
Hi,

    I think I have a pretty good version of the dynamic proc metrics implemented as they relate to anything with a PID instance.  My tree is here:

https://github.com/ubccr/pcp/tree/dynamic_proc

 src/pmdas/linux_proc/GNUmakefile    |    2 +-
 src/pmdas/linux_proc/help           |  140 ----------
 src/pmdas/linux_proc/help_text.h    |  121 +++++++++
 src/pmdas/linux_proc/pmda.c         |   13 +-
 src/pmdas/linux_proc/proc_dynamic.c |  477 +++++++++++++++++++++++++++++++++++
 src/pmdas/linux_proc/root_proc      |  129 +---------
 6 files changed, 614 insertions(+), 268 deletions(-)
 create mode 100644 src/pmdas/linux_proc/help_text.h
 create mode 100644 src/pmdas/linux_proc/proc_dynamic.c


If this all looks generally ok, I'll submit my hotprocs changes on top of this.  I modeled it on the basic structure of linux/interrupts.c.  A couple things:

1. I ran the existing QA tests (./check -g pmda.proc) and everything seemed to be OK, except for the ordering of the metrics in 022 and 943.  I don't think this can be fixed since it would mean some non-dynamic metrics would have to come in the middle.  Let me know if there is a way/desire to fix the ordering.

2.  I needed to use pmdaTreeRebuildHash while the interrupts code didn't.  Joe pointed out that this is likely due to that fact that the linux pmda runs as a dso and proc does not.  So there may be some other code that handles the dso case differently.  I couldn't find this documented anywhere.

3.  Couldn't find any precedent for handling long form help text for dynamic pmdas.  So I just generated a header file from the existing help file and used that.  Let me know if you want something different.  Attached is the perl script I wrote for that in case it is generally useful.

4.  I left a few commented code sections that will show where hotprocs will hook in.  Will add another root node, and all other stuff ( help, etc ) will be shared.

If this looks ok, it will probably take another day to update hotprocs.  Let me know if there is anything that looks wrong or needs to be changed.

Thanks

Martins 

  


--------------030903090205040300060002--

--------------040503090407020609020107
Content-Type: text/plain; charset=UTF-8;
 name="convhelp.txt"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="convhelp.txt"

IyEvdXNyL2Jpbi9wZXJsCgoKb3BlbiBIRUxQLCAnPCcsICJoZWxwLm9yaWciIHx8IGRpZSAi
Q2FuJ3Qgb3BlbiBoZWxwIjsKCm15ICRjdXJfbWV0cmljOwpteSAkc2hvcnRfdGV4dDsKbXkg
JGxvbmdfdGV4dDsKCm15ICVoZWxwID0gKCk7Cgp3aGlsZSAoPEhFTFA+KSB7CgogICAgbXkg
JGxpbmUgPSAkXzsKCiAgICBpZiAoJGxpbmUgPX4gL14jLiovICl7ICNjb21tZW50CgluZXh0
OwogICAgfQogICAgZWxzaWYgKCRsaW5lID1+IC9eXHMqJC8gKSB7IyBza2lwIGJsYW5rcwoJ
bmV4dDsKICAgIH0KICAgIGVsc2lmICggJGxpbmUgPX4gL15cQCAoXFMqKSAoLiopJC8gKXsK
CSRjdXJfbWV0cmljID0gJDE7Cgkkc2hvcnRfdGV4dCA9ICQyOwoKCSRjdXJfbWV0cmljID1+
IHMvXnByb2NcLi8vOyAjIHJlbW92ZSBsZWFkaW5nIHByb2MuCgoJJHNob3J0X3RleHQgPX4g
cy8iLy9nOyAjIGdldCByaWQgb2YgYW55IHF1b3RlcywgaGFja2lzaAoKCSRoZWxweyRjdXJf
bWV0cmljfT17fTsKCSRoZWxweyRjdXJfbWV0cmljfS0+eyduYW1lJ309JGN1cl9tZXRyaWM7
CgkkaGVscHskY3VyX21ldHJpY30tPnsnc2hvcnQnfT0kc2hvcnRfdGV4dDsKICAgIH0KICAg
IGVsc2UgeyAjIHNob3VsZCBiZSBhbGwgbG9uZyB0ZXh0IG5vdwoKCSRsb25nX3RleHQgPSAk
bGluZTsKCgkkbG9uZ190ZXh0ID1+IHMvIi8vZzsgIyBnZXQgcmlkIG9mIGFueSBxdW90ZXMs
IGhhY2tpc2gKCWNob21wKCRsb25nX3RleHQpOwoKCSRoZWxweyRjdXJfbWV0cmljfS0+eyds
b25nJ30gPSAkaGVscHskY3VyX21ldHJpY30tPnsnbG9uZyd9IC4gJGxvbmdfdGV4dCAuIlxc
biI7CiAgICB9Cn0KCnByaW50ICJ0eXBlZGVmIHN0cnVjdCB7XG5jaGFyICpuYW1lO1xuY2hh
ciAqc2hvcnRoZWxwO1xuY2hhciAqbG9uZ2hlbHA7XG59IGhlbHBfdGV4dF90O1xuIjsKCnBy
aW50ICJoZWxwX3RleHRfdCBoZWxwX3RleHRbXSA9IHtcbiI7Cgpmb3JlYWNoIG15ICRrZXkg
KGtleXMgJWhlbHApewogICAgbXkgJG5hbWUgPSAkaGVscHska2V5fS0+eyduYW1lJ307CiAg
ICBteSAkc2hvcnQgPSAkaGVscHska2V5fS0+eydzaG9ydCd9OwogICAgbXkgJGxvbmcgPSAk
aGVscHska2V5fS0+eydsb25nJ307CiAgICBwcmludCAieyAubmFtZSA9IFwiJG5hbWVcIiwg
ICAgICAgICAgICAuc2hvcnRoZWxwID0gXCIkc2hvcnRcIiwgICAgICAgIC5sb25naGVscCA9
IFwiJGxvbmdcIiB9LFxuIjsKfQoKcHJpbnQgIn07IjsK
--------------040503090407020609020107--

From michele@acksyn.org  Sun Nov  9 15:19:35 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id A17B77F67
	for ; Sun,  9 Nov 2014 15:19:35 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id EE255AC004
	for ; Sun,  9 Nov 2014 13:19:34 -0800 (PST)
X-ASG-Debug-ID: 1415567968-04cb6c1e6c161070001-S8gJnT
Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id 65JVu2vSHD2yW9hr for ; Sun, 09 Nov 2014 13:19:28 -0800 (PST)
X-Barracuda-Envelope-From: michele@acksyn.org
X-Barracuda-Apparent-Source-IP: 5.9.7.26
Received: from localhost (localhost [127.0.0.1])
	by palahniuk.acksyn.org (Postfix) with ESMTP id 0D14026539;
	Sun,  9 Nov 2014 16:19:28 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h=
	content-transfer-encoding:content-type:content-type:mime-version
	:x-mailer:message-id:date:date:subject:subject:from:from
	:received:received; s=2010; t=1415567966; bh=kdCQiR23GlEzn1oTAjH
	LXgUuHXM4CI/ukl7SKbu0mo0=; b=RQH2qJWu+wPpDplNVumi75Zf1D2Np0YECGH
	8A85bNAZvVR9BbUKnEKTCkjPOV18SUhe992wyZ0DTUE0HZUDQcj6mdWvO32QzVqL
	fxQaa4NvR+C4GYCbrEuTeJBOMabisrY2IdrLdIJB64F1SL3PkplTLIck03gJSu2h
	nf7XfgAE=
Received: from palahniuk.acksyn.org ([127.0.0.1])
	by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id fG5XS2jNyn6U; Sun,  9 Nov 2014 16:19:26 -0500 (EST)
Received: from localhost (host54-190-dynamic.24-79-r.retail.telecomitalia.it [79.24.190.54])
	by palahniuk.acksyn.org (Postfix) with ESMTPSA id 456D3243AC;
	Sun,  9 Nov 2014 16:19:25 -0500 (EST)
From: Michele Baldessari 
To: pcp@oss.sgi.com
Cc: Michele Baldessari 
Subject: [PATCH] Fix manpages sections
Date: Sun,  9 Nov 2014 22:19:16 +0100
X-ASG-Orig-Subj: [PATCH] Fix manpages sections
Message-Id: <1415567956-5906-1-git-send-email-michele@acksyn.org>
X-Mailer: git-send-email 2.1.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26]
X-Barracuda-Start-Time: 1415567968
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11409
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature

- There is no man4/ folder in PCP. Fix all the links pointing to
  the wrong section.
- Fix up pmwebd man section
- Fix pmParseInterval man section pcp-free
- Fix pmconfig man section in pmgetoptions
---
 man/html/lab.importdata.html |  2 +-
 man/html/pcpintro.html       | 10 +++++-----
 man/man1/pmatop.1            |  2 +-
 man/man1/pmchart.1           |  8 ++++----
 man/man1/pmdumptext.1        |  2 +-
 man/man1/pmquery.1           |  4 ++--
 man/man1/pmview.1            |  6 +++---
 man/man1/sheet2pcp.1         |  2 +-
 man/man3/pmgetoptions.3      |  2 +-
 man/man3/pmwebapi.3          |  2 +-
 src/pcp/free/pcp-free.1      |  2 +-
 11 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/man/html/lab.importdata.html b/man/html/lab.importdata.html
index c80b53ca8844..8250b592ab21 100644
--- a/man/html/lab.importdata.html
+++ b/man/html/lab.importdata.html
@@ -159,7 +159,7 @@ a name – the rules must conform to those for th=
e Performance Metrics
 Name Space (PMNS), namely a number of components separated by periods
 "." with each component starting with an alphabetic followed
 by zero or more characters drawn from the alphabetics, digits and the
-underscore "_" character (see pmns(4) for more
+underscore "_" character (see pmns(5) for more
 details). For our metrics we'll use the prefix mover. and so the
 first metric will be known as mover.nfile.
 

diff --git a/man/html/pcpintro.html b/man/html/pcpintro.html
index dc1d20a0767d..3cb9e2c65d70 100644
--- a/man/html/pcpintro.html
+++ b/man/html/pcpintro.html
@@ -149,7 +149,7 @@ where number is an integer or floating point c=
onstant (parsed using st
 

         
PMCD and Archive Versions

 

-
Since PCP version 2, version information has been associated with =
pmcd(1) and PCP archives.  The version number is used in a number of =
ways, but most noticeably for the distributed pmns(4).  In PCP version 1,=
 the client applications would load the PMNS from the default PMNS file b=
ut in PCP version 2, the client applications extract the PMNS information=
 from pmcd or a PCP archive.  Thus in PCP version 2, the version n=
umber is used to determine if the PMNS to use is from the default local f=
ile or from the actual current source of the metrics.
+
Since PCP version 2, version information has been associated with =
pmcd(1) and PCP archives.  The version number is used in a number of =
ways, but most noticeably for the distributed pmns(5).  In PCP version 1,=
 the client applications would load the PMNS from the default PMNS file b=
ut in PCP version 2, the client applications extract the PMNS information=
 from pmcd or a PCP archive.  Thus in PCP version 2, the version n=
umber is used to determine if the PMNS to use is from the default local f=
ile or from the actual current source of the metrics.
 
 

 
@@ -193,7 +193,7 @@ where number is an integer or floating point c=
onstant (parsed using st
 	
-	    
+	    

         
Environment
PMDA_LOCAL_SAMPLE

 	    
If set, then a context established with the type of PM_CONTE=
XT_LOCAL will have access to the =E2=80=98=E2=80=98sample=E2=80=99=E2=80=99=
 PMDA if this optional PMDA has been installed locally.

 	
PMIECONF_PATH
If set, pmieconf(1) will form its pmieconf(4) =
specification (set of parameterized pmie(1) rules) using all valid=
 pmieconf files found below each subdirectory in this colon-separa=
ted list of subdirectories.  If not set, the default is $PCP_VAR_DIR/conf=
ig/pmieconf.
If set, pmieconf(1) will form its pmieconf(5) =
specification (set of parameterized pmie(1) rules) using all valid=
 pmieconf files found below each subdirectory in this colon-separa=
ted list of subdirectories.  If not set, the default is $PCP_VAR_DIR/conf=
ig/pmieconf.

 

 
 

 
@@ -202,7 +202,7 @@ where number is an integer or floating point c=
onstant (parsed using st
 
 

 

-	    
+	    
@@ -229,7 +229,7 @@ where number is an integer or floating point c=
onstant (parsed using st
 	
-	    
+	    
@@ -239,7 +239,7 @@ where number is an integer or floating point c=
onstant (parsed using st
 

 	
/etc/pcp.conf
Configuration file for the PCP runtime environment, see p=
cp.conf(4).
Configuration file for the PCP runtime environment, see p=
cp.conf(5).

 	
$PCP_RC_DIR/pcp

 	    
Script for starting and stopping pmcd(1).

 	
$PCP_PMCDCONF_PATH
$PCP_VAR_DIR/config/pmcd/rc.local

 	    
Local script for controlling PCP boot, shutdown and restart =
actions.

 	
$PCP_VAR_DIR/pmns/root
The ASCII $PCP_LOG_DIR/NOTICES/pmns(4) exported by pmcd(1) by default.  This PMNS is be the super set of all other PMNS=
 files installed in $PCP_VAR_DIR/pmns.
The ASCII $PCP_LOG_DIR/NOTICES/pmns(5) exported by pmcd(1) by default.  This PMNS is be the super set of all other PMNS=
 files installed in $PCP_VAR_DIR/pmns.

 	
$PCP_LOG_DIR/NOTICES

 	    
In addition to the pmcd(1) and PMDA activity, may be =
used to log alarms and notices from pmie(1) via pmpost(1). =


 	
$PCP_VAR_DIR/config/pmlogger/control

         
PCP Environment

 

-
Environment variables with the prefix PCP_ are used to parameterize t=
he file and directory names used by PCP.  On each installation, the file =
/etc/pcp.conf contains the local values for these variables.  The =
$PCP_CONF variable may be used to specify an alternative configura=
tion file, as described in pcp.conf(4).
+
Environment variables with the prefix PCP_ are used to parameterize t=
he file and directory names used by PCP.  On each installation, the file =
/etc/pcp.conf contains the local values for these variables.  The =
$PCP_CONF variable may be used to specify an alternative configura=
tion file, as described in pcp.conf(5).
 

 

 
diff --git a/man/man1/pmatop.1 b/man/man1/pmatop.1
index 3cbbef5f1140..7d10c7035229 100644
--- a/man/man1/pmatop.1
+++ b/man/man1/pmatop.1
@@ -208,5 +208,5 @@ pmatop will continue with a next sample.
 .BR pmval (1),
 .BR PMAPI (3),
 and
-.BR pcp.conf (4).
+.BR pcp.conf (5).
=20
diff --git a/man/man1/pmchart.1 b/man/man1/pmchart.1
index 3cd2a40dd4b4..8632d2f45b7b 100644
--- a/man/man1/pmchart.1
+++ b/man/man1/pmchart.1
@@ -627,7 +627,7 @@ The
 variable may be used to specify an alternative
 configuration file,
 as described in
-.BR pcp.conf (4).
+.BR pcp.conf (5).
 .PP
 Of particular note, the
 .B $PCP_XCONFIRM_PROG
@@ -649,7 +649,7 @@ application.
 .BR pmval (1),
 .BR pmcd (1),
 .BR pminfo (1),
-.BR pcp.conf (4),
-.BR pcp.env (4)
+.BR pcp.conf (5),
+.BR pcp.env (5)
 and
-.BR pmns (4).
+.BR pmns (5).
diff --git a/man/man1/pmdumptext.1 b/man/man1/pmdumptext.1
index 5fffaa3fffd0..04bdc5f01704 100644
--- a/man/man1/pmdumptext.1
+++ b/man/man1/pmdumptext.1
@@ -408,7 +408,7 @@ The
 variable may be used to specify an alternative
 configuration file,
 as described in
-.BR pcp.conf (4).
+.BR pcp.conf (5).
 .SH SEE ALSO
 .BR pmchart (1),
 .BR pmtime (1),
diff --git a/man/man1/pmquery.1 b/man/man1/pmquery.1
index 9644e8443114..b09fafda77cc 100644
--- a/man/man1/pmquery.1
+++ b/man/man1/pmquery.1
@@ -231,7 +231,7 @@ terminal with an ssh session connected to the request=
ed host.
 .B pmquery
 is an excellent choice of utility for the "PCP_XCONFIRM_PROG"
 Performance Co-Pilot configuration parameter (refer to
-.BR pcp.conf (4)
+.BR pcp.conf (5)
 for details).
 .PP
 Note that PCP_XCONFIRM_PROG will be automatically set to
@@ -249,4 +249,4 @@ zero on success.
 .BR pmchart (1),
 .BR xconfirm (1),
 .BR xmessage (1),
-.BR pcp.conf (4).
+.BR pcp.conf (5).
diff --git a/man/man1/pmview.1 b/man/man1/pmview.1
index 3f0ceb7f0706..15750d117df9 100644
--- a/man/man1/pmview.1
+++ b/man/man1/pmview.1
@@ -547,7 +547,7 @@ The
 variable may be used to specify an alternative
 configuration file,
 as described in
-.BR pcp.conf (4).
+.BR pcp.conf (5).
 .SH SEE ALSO
 .BR dkvis (1),
 .BR insight (1),
@@ -569,8 +569,8 @@ as described in
 .BR X (1),
 .BR xconfirm (1),
 .BR xlv_vis (1),
-.BR pcp.conf (4),
-.BR pmview (4),
+.BR pcp.conf (5),
+.BR pmview (5),
 .BR environ (5)
 and
 .BR pmlaunch (5).
diff --git a/man/man1/sheet2pcp.1 b/man/man1/sheet2pcp.1
index 2230ee85e9fc..4a5bd4e2d765 100644
--- a/man/man1/sheet2pcp.1
+++ b/man/man1/sheet2pcp.1
@@ -106,7 +106,7 @@ tag supports the following optional attributes:
 .IX Item "pmid"
 The Performance Metrics Identifier (\s-1PMID\s0), specified as 3 numbers
 separated by a periods (.) to
-set the \fBdomain\fR, \fBcluster\fR and \fBitem\fR fields of the \s-1PMI=
D\s0, see \fB\s-1PMNS\s0\fR(4)
+set the \fBdomain\fR, \fBcluster\fR and \fBitem\fR fields of the \s-1PMI=
D\s0, see \fB\s-1PMNS\s0\fR(5)
 for more details of PMIDs.  If omitted, the \s-1PMID\s0 will be automati=
cally
 assigned by \fBpmiAddMetric\fR(3).
 The value \fB\s-1PM_ID_NULL\s0\fR may be used to explicitly nominate
diff --git a/man/man3/pmgetoptions.3 b/man/man3/pmgetoptions.3
index a713b2b801b6..b14e55eb8b10 100644
--- a/man/man3/pmgetoptions.3
+++ b/man/man3/pmgetoptions.3
@@ -474,7 +474,7 @@ function.
 .BR getopt (3),
 .BR getopt_long (3),
 .BR pmNewContext (3),
-.BR pmconfig (3),
+.BR pmconfig (1),
 .BR pmprintf (3),
 .BR pmflush (3)
 and
diff --git a/man/man3/pmwebapi.3 b/man/man3/pmwebapi.3
index 20fd4355b408..97081d5fce80 100644
--- a/man/man3/pmwebapi.3
+++ b/man/man3/pmwebapi.3
@@ -314,7 +314,7 @@ Same, with a slightly different result encoding.
=20
 .BR PCPIntro (1),
 .BR PCPIntro (3),
-.BR pmwebd (3),
+.BR pmwebd (1),
 .nh
 .BR http://graphite.readthedocs.org/
 .hy
diff --git a/src/pcp/free/pcp-free.1 b/src/pcp/free/pcp-free.1
index 2a8b4f80d4a5..565c4272140d 100644
--- a/src/pcp/free/pcp-free.1
+++ b/src/pcp/free/pcp-free.1
@@ -66,6 +66,6 @@ The shared memory column should be ignored; it is obsol=
ete.
 .BR pcp (1),
 .BR free(1),
 .BR PCPIntro (1),
-.BR pmParseInterval (1)
+.BR pmParseInterval (3)
 and
 .BR environ (5).
--=20
2.1.0

From kenj@internode.on.net  Sun Nov  9 16:38:49 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 1B9AB7F67
	for ; Sun,  9 Nov 2014 16:38:49 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id E098B8F8033
	for ; Sun,  9 Nov 2014 14:38:45 -0800 (PST)
X-ASG-Debug-ID: 1415572719-04bdf0650e185730001-S8gJnT
Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id aoRf298e5HdBXmmL for ; Sun, 09 Nov 2014 14:38:40 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.143
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AsABAN/rX1R20SpqPGdsb2JhbAANToNiWYMGhDfEK4h+AQEBAQEGAQEBATiEZ1UwBgIFFgsCCwMCAQIBMScGAgEBiEq2XXiVRYEtkAWCYYFUBYY6kEqIVYcBkjlYgksBAQE
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail05.adl6.internode.on.net with ESMTP; 10 Nov 2014 09:08:39 +1030
Message-ID: <545FED4E.5080800@internode.on.net>
Date: Mon, 10 Nov 2014 09:40:14 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: pcp updates - assorted
Content-Type: text/plain; charset=utf-8
X-ASG-Orig-Subj: pcp updates - assorted
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143]
X-Barracuda-Start-Time: 1415572720
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11410
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Changes committed to git://git.performancecopilot.org/kenj/pcp.git dev

 qa/003                 |    1 
 qa/752.out             |  228 ++++++++++++++++++++++++-------------------------
 qa/944                 |    1 
 qa/src/rtimetest.c     |   90 ++++++++++++++++++-
 src/include/buildrules |    2 
 5 files changed, 203 insertions(+), 119 deletions(-)

commit d1ab4c41b9be162a3245a1d350bfd5cf2e180f9d
Author: Ken McDonell 
Date:   Mon Nov 10 09:39:19 2014 +1100

    qa/src/rtimetest.c: improve diags, accept command line args
    
    On the trail of the "Next Tuesday" bug ...

commit ecc90bf9f0b23e9bc1a2445cb8eacd5638df26d7
Author: Ken McDonell 
Date:   Mon Nov 10 09:37:24 2014 +1100

    buildrules: need rm -f before ln -s
    
    In the build this was hidden because the alternate named python and
    perl files did not exist.
    
    But the same rule is used in qa/src and there the targets may exist
    and it was only a matter of time before this bit somone ... like me
    a couple of days ago.

commit 5e32d07b38272fda1bce37f24449b352cec9eac7
Author: Ken McDonell 
Date:   Fri Nov 7 06:46:52 2014 +1100

    qa/752: tidy up output from rtimetest.c
    
    Previous commit made the formatting harder to read which was the exact
    opposite of the intention.
    
    The "Tuesday" bug remains unresolved at this point in time.

commit 9b736c050261be3ae9abff857004449a3cd63659
Author: Ken McDonell 
Date:   Fri Nov 7 06:36:18 2014 +1100

    qa/003: hinv.node.online not always available, add to filter

commit db2bd86aa051f8c7d47836002b7d2ab1d291c930
Author: Ken McDonell 
Date:   Thu Nov 6 09:48:56 2014 +1100

    qa/944: better handling of multiple users being in the same default group
    
    For example on grundy, need to handle case when the target is _not_ first
    in the list, ... so filter
     12(games),1000(kenj),1001(trev)
    to produce
     GROUPID(GROUPNAME),...
    not
     12(games), GROUPID(GROUPNAME),...

From nscott@redhat.com  Sun Nov  9 17:28:09 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 4B4817F60
	for ; Sun,  9 Nov 2014 17:28:09 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 3A77B304032
	for ; Sun,  9 Nov 2014 15:28:06 -0800 (PST)
X-ASG-Debug-ID: 1415575665-04cb6c1e6d163e00001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id 4LZ9FdHQmNYBNxsQ (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 09 Nov 2014 15:27:46 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sA9NRjDm012599
	for ; Sun, 9 Nov 2014 18:27:45 -0500
Date: Sun, 9 Nov 2014 18:27:45 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <1912154843.10770665.1415575665397.JavaMail.zimbra@redhat.com>
In-Reply-To: <1906858742.10770643.1415575645946.JavaMail.zimbra@redhat.com>
Subject: pcp updates: docs, qa
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: docs, qa
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: docs, qa
Thread-Index: SjOlVNiTijuvrN52RY54W4IcSpQ2Fw==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1415575666
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Barracuda-BRTS-Status: 1
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11411
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Ken McDonell (4):
      qa/003: hinv.node.online not always available, add to filter
      qa/752: tidy up output from rtimetest.c
      buildrules: need rm -f before ln -s
      qa/src/rtimetest.c: improve diags, accept command line args

Nathan Scott (2):
      qa: extend http request filter for 780/782 determinism
      qa: extend http request filter for qa/660 determinism

Michele Baldessari (1):
      docs: fix man page sections

 man/html/lab.importdata.html |    2 
 man/html/pcpintro.html       |   10 -
 man/man1/pmatop.1            |    2 
 man/man1/pmchart.1           |    8 -
 man/man1/pmdumptext.1        |    2 
 man/man1/pmquery.1           |    4 
 man/man1/pmview.1            |    6 -
 man/man1/sheet2pcp.1         |    2 
 man/man3/pmgetoptions.3      |    2 
 man/man3/pmwebapi.3          |    2 
 qa/003                       |    1 
 qa/660                       |   20 +++
 qa/660.out.4                 |    2 
 qa/660.out.46                |    2 
 qa/752.out                   |  228 +++++++++++++++++++++----------------------
 qa/common.webapi             |   15 +-
 qa/src/rtimetest.c           |   90 ++++++++++++++++
 src/include/buildrules       |    2 
 src/pcp/free/pcp-free.1      |    2 
 19 files changed, 254 insertions(+), 148 deletions(-)

From kenj@internode.on.net  Sun Nov  9 18:08:43 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 811807F67
	for ; Sun,  9 Nov 2014 18:08:43 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 6FB1B8F8033
	for ; Sun,  9 Nov 2014 16:08:43 -0800 (PST)
X-ASG-Debug-ID: 1415578117-04cbb00e681524d0001-S8gJnT
Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id IRpDPmZ8VfRFxl9L for ; Sun, 09 Nov 2014 16:08:38 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.143
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ar0BAP4AYFR20SpqPGdsb2JhbAANTodBhDfIXoRPAQEBAQEGAQEBATiEZxVANgIFFgsCCwMCAQIBMRoNCAEBvzx4lRwsgS2SZoFUBasojWuDIwEBAQ
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail05.adl6.internode.on.net with ESMTP; 10 Nov 2014 10:38:37 +1030
Message-ID: <54600264.2080602@internode.on.net>
Date: Mon, 10 Nov 2014 11:10:12 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: PCP 
Subject: qa/843 - memory_python, failure
Content-Type: text/plain; charset=utf-8; format=flowed
X-ASG-Orig-Subj: qa/843 - memory_python, failure
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143]
X-Barracuda-Start-Time: 1415578118
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11413
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Is there a minimum Python version required for qa/843 and the 
memory_python PMDA?

The test is failing on i586 CentOS 5.10 with this python

kenj@vm04:~/src/pcp/qa$ python -V
Python 2.4.3

and with this in the PMDA's log file.

Log for pmdamemory_python on vm04.localdomain started Sun Nov  9 
09:25:29 2014

[Sun Nov  9 09:25:34] pmdamemory_python(6079) Info: _metric_names value 
is '', not 'memory_python.memory_valid'
Traceback (most recent call last):
   File 
"/home/kenj/src/pcp/qa/pmdas/memory_python/pmdamemory_python.python", 
line 55, in memory_fetch
     raise KeyError(msg)
KeyError: "_metric_names value is '', not 'memory_python.memory_valid'"

From nscott@redhat.com  Sun Nov  9 18:16:48 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 375E67F67
	for ; Sun,  9 Nov 2014 18:16:48 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 23BFA8F8035
	for ; Sun,  9 Nov 2014 16:16:48 -0800 (PST)
X-ASG-Debug-ID: 1415578605-04cb6c1e6a164e40001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id 7fS5k3iDWGJVHDOh (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 09 Nov 2014 16:16:46 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAA0Gg0I014992;
	Sun, 9 Nov 2014 19:16:42 -0500
Date: Sun, 9 Nov 2014 19:16:41 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Ken McDonell 
Cc: PCP 
Message-ID: <1652357511.10779967.1415578601559.JavaMail.zimbra@redhat.com>
In-Reply-To: <54600264.2080602@internode.on.net>
References: <54600264.2080602@internode.on.net>
Subject: Re: [pcp] qa/843 - memory_python, failure
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] qa/843 - memory_python, failure
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: qa/843 - memory_python, failure
Thread-Index: b80bqJ7yl8q0AJT4BFqLkGfK9tv/5Q==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415578606
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11413
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi Ken,

----- Original Message -----
> Is there a minimum Python version required for qa/843 and the
> memory_python PMDA?
> 
> The test is failing on i586 CentOS 5.10 with this python
> 
> kenj@vm04:~/src/pcp/qa$ python -V
> Python 2.4.3

There's a minimum python version for all of PCP - 2.6 - so I'm
guessing this machine has an old pcp-python package installed,
from an older PCP version?

The test is also missing a does-the-python-pmda-module-exist at
all check (notrun) - I'll add that in.

cheers.

--
Nathan

From kenj@internode.on.net  Sun Nov  9 20:44:13 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 88A787F67
	for ; Sun,  9 Nov 2014 20:44:13 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 23E86AC007
	for ; Sun,  9 Nov 2014 18:44:12 -0800 (PST)
X-ASG-Debug-ID: 1415587450-04bdf0650d18e3e0001-S8gJnT
Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id k7dXw6MeV5UejFRT for ; Sun, 09 Nov 2014 18:44:11 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.143
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Ar4BAJQlYFR20SpqPGdsb2JhbAANTodBhDfIXoMgAoEwAQEBAQEGAQEBATiEPQEBAQMBIwQRQAEMBAsYAgIFFgsCAgkDAgECATEUBg0BBwEBiDS3JXiVHgEBAQEBAQEBAgEBAQEBAQEbgS2PaAeCd4FUAQSrKI1rgyMBAQE
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail05.adl6.internode.on.net with ESMTP; 10 Nov 2014 13:14:09 +1030
Message-ID: <546026D9.9050100@internode.on.net>
Date: Mon, 10 Nov 2014 13:45:45 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: PCP 
Subject: Re: [pcp] qa/843 - memory_python, failure
References: <54600264.2080602@internode.on.net> <1652357511.10779967.1415578601559.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] qa/843 - memory_python, failure
In-Reply-To: <1652357511.10779967.1415578601559.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143]
X-Barracuda-Start-Time: 1415587450
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11417
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

On 10/11/14 11:16, Nathan Scott wrote:
> Hi Ken,
>
> ----- Original Message -----
>> Is there a minimum Python version required for qa/843 and the
>> memory_python PMDA?
>>
>> The test is failing on i586 CentOS 5.10 with this python
>>
>> kenj@vm04:~/src/pcp/qa$ python -V
>> Python 2.4.3
>
> There's a minimum python version for all of PCP - 2.6 - so I'm
> guessing this machine has an old pcp-python package installed,
> from an older PCP version?

Sort of yes and no ... has 3.10 pcp-python from an earlier build, but no 
pcp-python from the latest 3.10 build.

kenj@vm04:~/src/pcp/qa$ rpm -qa | grep -E '^(pcp-3|python-pcp)'
python-pcp-3.10.0-7
pcp-3.10.0-19

And python 2.7 is sort of installed here ...  python -> python2.4 but

kenj@vm04:~/src/pcp/qa$ python2.7 -V
Python 2.7.3
kenj@vm04:~/src/pcp/qa$ which python2.7
/usr/bin/python2.7
kenj@vm04:~/src/pcp/qa$ rpm -q -f /usr/bin/python2.7
file /usr/bin/python2.7 is not owned by any package

I have no idea how it got like this, let me see if I can clean the 
installation up.

>
> The test is also missing a does-the-python-pmda-module-exist at
> all check (notrun) - I'll add that in.

Ta.

From kvivekananthan@zoho.com  Mon Nov 10 00:07:09 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_FONT_FACE_BAD,
	HTML_MESSAGE autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 3DE6C7F62
	for ; Mon, 10 Nov 2014 00:07:09 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 1D86C304039
	for ; Sun,  9 Nov 2014 22:07:06 -0800 (PST)
X-ASG-Debug-ID: 1415599621-04bdf0650e193750001-S8gJnT
Received: from sender1.zohomail.com (sender1.zohomail.com [74.201.84.154]) by cuda.sgi.com with ESMTP id 9lF8avZUVVz651io (version=TLSv1 cipher=RC4-MD5 bits=128 verify=NO) for ; Sun, 09 Nov 2014 22:07:02 -0800 (PST)
X-Barracuda-Envelope-From: kvivekananthan@zoho.com
X-Barracuda-Apparent-Source-IP: 74.201.84.154
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; 
  s=zapps768; d=zoho.com; 
  h=date:from:to:message-id:subject:mime-version:content-type:user-agent; 
  b=I7T2csuvc6whWvdGXGlfcJxr82EM4er2oy2nypK10r7Wa2TB4Gj2DdZPpitE/fm2IMRqPSKOkJdQ
    wUYll4IlZlmBmbmAEtkvzPuxLIeI2H62hwdpXdSaK8ZsnPqGWCnl  
Received: from mail.zoho.com by mx.zohomail.com
	with SMTP id 141559962129551.11984090534895; Sun, 9 Nov 2014 22:07:01 -0800 (PST)
Date: Mon, 10 Nov 2014 11:37:01 +0530
From: kvivekananthan 
To:  
Message-ID: <1499850047f.1099d424c320050.6651429955964838422@zoho.com>
Subject: Reg : pmdumptext command not installing
MIME-Version: 1.0
X-ASG-Orig-Subj: Reg : pmdumptext command not installing
Content-Type: multipart/alternative; 
	boundary="----=_Part_416669_323650818.1415599621246"
X-Priority: MEDIUM
User-Agent: Zoho Mail
X-Mailer: Zoho Mail
X-Barracuda-Connect: sender1.zohomail.com[74.201.84.154]
X-Barracuda-Start-Time: 1415599622
X-Barracuda-Encrypted: RC4-MD5
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.61
X-Barracuda-Spam-Status: No, SCORE=0.61 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_FONT_FACE_BAD, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11421
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 HTML_MESSAGE           BODY: HTML included in message
	0.61 HTML_FONT_FACE_BAD     BODY: HTML font face is not a word

------=_Part_416669_323650818.1415599621246
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit

Hi,

   I'm trying to install pcp in a freebsd 9.2 server. pmdumptext command is not installing be default. Is there any option to mention in configure to enable it.


Regards,
Vivekananthan. K



------=_Part_416669_323650818.1415599621246
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: 7bit


Hi,

   I'm trying to install pcp in a freebsd 9.2 server. pmdumptext command is not installing be default. Is there any option to mention in configure to enable it.

Regards,
Vivekananthan. K

------=_Part_416669_323650818.1415599621246--

From nscott@redhat.com  Mon Nov 10 00:16:43 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 07AB67F62
	for ; Mon, 10 Nov 2014 00:16:43 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 88E52AC034
	for ; Sun,  9 Nov 2014 22:16:42 -0800 (PST)
X-ASG-Debug-ID: 1415600199-04cbb00e6915dd80001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id fOTBLfGxGwl7IBLr (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 09 Nov 2014 22:16:40 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAA6GdWW031109;
	Mon, 10 Nov 2014 01:16:39 -0500
Date: Mon, 10 Nov 2014 01:16:38 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: kvivekananthan 
Cc: pcp@oss.sgi.com
Message-ID: <763726291.10878298.1415600198740.JavaMail.zimbra@redhat.com>
In-Reply-To: <1499850047f.1099d424c320050.6651429955964838422@zoho.com>
References: <1499850047f.1099d424c320050.6651429955964838422@zoho.com>
Subject: Re: [pcp] Reg : pmdumptext command not installing
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Reg : pmdumptext command not installing
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Reg : pmdumptext command not installing
Thread-Index: pc0OUfQ/0aH+UpzEfyrBRZJEHOlRfg==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1415600200
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11421
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...



----- Original Message -----
> Hi,
> 
> I'm trying to install pcp in a freebsd 9.2 server. pmdumptext command is not
> installing be default. Is there any option to mention in configure to enable
> it.

pmdumptext uses the Qt console libraries - you'll need to have a Qt development
environment installed, and pmdumptext (and pmchart, pmtime, ...) will then build
automatically.

Info here - https://www.freebsd.org/doc/en/books/porters-handbook/using-qt.html
(you'll need qt >= 4.4 to build the PCP tools that rely on it, incl pmdumptext)

cheers.

--
Nathan

From nscott@redhat.com  Mon Nov 10 00:27:05 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 3651D7F62
	for ; Mon, 10 Nov 2014 00:27:05 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id A7930AC038
	for ; Sun,  9 Nov 2014 22:27:04 -0800 (PST)
X-ASG-Debug-ID: 1415600821-04cbb00e6715e2a0001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id LwRNE8pNcZg8zjJM (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 09 Nov 2014 22:27:02 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAA6R0Ai026324;
	Mon, 10 Nov 2014 01:27:00 -0500
Date: Mon, 10 Nov 2014 01:26:59 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Martins Innus 
Cc: pcp 
Message-ID: <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
In-Reply-To: <545D2CFA.1050600@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: acR6pVAtHRBlSwlKgqqljOQ9nimP9w==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1415600822
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11422
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...



----- Original Message -----
> Hi,
> 
> I think I have a pretty good version of the dynamic proc metrics implemented
> as they relate to anything with a PID instance. My tree is here:
> 
> https://github.com/ubccr/pcp/tree/dynamic_proc

Great, thanks Martins.  I've started using this code as the base for that
cgroups metric rework too.

> If this all looks generally ok, I'll submit my hotprocs changes on top of
> this.

Yes please.

> I modeled it on the basic structure of linux/interrupts.c. A couple
> things:
> 
> 1. I ran the existing QA tests (./check -g pmda.proc) and everything seemed
> to be OK, except for the ordering of the metrics in 022 and 943. I don't
> think this can be fixed since it would mean some non-dynamic metrics would
> have to come in the middle. Let me know if there is a way/desire to fix the
> ordering.

I have a fix for 022 & will do 943 - they're fixable using sort(1) to produce
deterministic output.

I'm also seeing 580 fail, looks probably related:

$ diff 580.out 580.out.bad
7,9c7
< pm*InDom: inst=[1] iname= reverse lookup: inst=[1]
< pmGetInDom:
<    [1] 
---
> Error: Unknown or illegal metric identifier

I'll dig into it later in the week if its not failing elsewhere and/or not
reproducible.

> 2. I needed to use pmdaTreeRebuildHash while the interrupts code didn't. Joe
> pointed out that this is likely due to that fact that the linux pmda runs as
> a dso and proc does not. So there may be some other code that handles the
> dso case differently. I couldn't find this documented anywhere.

Yeah, thats like it - pmcd will do it (internally via libpcp) and that is
probably fixing things up by luck for those interrupt metrics too.

> 3. Couldn't find any precedent for handling long form help text for dynamic
> pmdas. So I just generated a header file from the existing help file and
> used that. Let me know if you want something different. Attached is the perl
> script I wrote for that in case it is generally useful.

We might be able to do some remapping from hotproc -> proc variants, but I
don't think there's any precedence there I can point you towards - I'll can
take a look into that too if you like.

> 4. I left a few commented code sections that will show where hotprocs will
> hook in. Will add another root node, and all other stuff ( help, etc ) will
> be shared.

Perfect.

> If this looks ok, it will probably take another day to update hotprocs. Let
> me know if there is anything that looks wrong or needs to be changed.

Its looking good to me, thanks Martins!

cheers.

--
Nathan

From kenj@internode.on.net  Mon Nov 10 13:42:00 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id D98817F54
	for ; Mon, 10 Nov 2014 13:42:00 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id A90ED304043
	for ; Mon, 10 Nov 2014 11:42:00 -0800 (PST)
X-ASG-Debug-ID: 1415648517-04bdf0650d1c0f90001-S8gJnT
Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id DyaUbmgHDyFYTAp5 for ; Mon, 10 Nov 2014 11:41:57 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.131
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApgBAF4UYVR20Spq/2dsb2JhbAANT4NiWYMGyHWJEgEBAQEBhSlVMAYCBRYLAgsDAgECAVgGAgEBiEq2THiWMIEtkAWCYYFUBZcEog9YgksBAQE
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail07.adl2.internode.on.net with ESMTP; 11 Nov 2014 06:11:57 +1030
Message-ID: <5461155A.9050608@internode.on.net>
Date: Tue, 11 Nov 2014 06:43:22 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: pcp updates - small debian build change + QA
Content-Type: text/plain; charset=utf-8
X-ASG-Orig-Subj: pcp updates - small debian build change + QA
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131]
X-Barracuda-Start-Time: 1415648517
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11441
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Changes committed to git://git.performancecopilot.org/kenj/pcp.git dev

 Makepkgs                   |    1 +
 qa/188                     |    2 +-
 qa/635                     |   11 +++++++++++
 qa/722                     |    3 ++-
 qa/842                     |    2 +-
 qa/admin/pcp-qa-summary    |    6 +++++-
 qa/src/rtimetest.c         |    7 ++++---
 qa/valgrind-suppress-3.5.0 |   10 ++++++++++
 8 files changed, 35 insertions(+), 7 deletions(-)

commit 57f58207e132b54ce2ec74a5b2048f7724534034
Author: Ken McDonell 
Date:   Tue Nov 11 06:41:41 2014 +1100

    qa/722: ensure the pmafm folder is removed

commit d7f6018a069be11eb4b92e777554652cb8baa233
Author: Ken McDonell 
Date:   Tue Nov 11 06:40:35 2014 +1100

    qa/635: extra diags to .full
    
    Trying to track down a rogue failure.

commit c9779728b37d08ff27e5c6c0f4b9300025ccab50
Author: Ken McDonell 
Date:   Tue Nov 11 06:37:40 2014 +1100

    Makepkgs: debian build change
    
    Blow build/deb away and recreate the directory.
    
    Ensures we don't have .deb packages for multiple releases when the
    release version changes or stale packages when the package guards
    change.

commit 90de65fb74089ff6f0ade98bfc25923e0cfa2acf
Author: Ken McDonell 
Date:   Tue Nov 11 06:35:50 2014 +1100

    qa/admin/pcp-qa-summary: fix -f arithmetic
    
    Failure % should be calculated basewd on the number of hosts with
    some QA results, not the number of hosts with some QA failures.
    
    Problem was not visible until we got a significant number of hosts
    with 0 failures ... 8^)>

commit e83c4a46d7886cfeeee0e2db25f0caae54d4fb08
Author: Ken McDonell 
Date:   Tue Nov 11 06:35:02 2014 +1100

    qa/src/rtimetest.c: fix argc logic error when -D... present

commit 5a79e94c77fdfe411abf4b71c5bc751220a5b8d6
Author: Ken McDonell 
Date:   Mon Nov 10 15:54:16 2014 +1100

    qa/188: glibc-2.18- also produces linux-style 188.out

commit 5d32d62f112d5711654fe23b8d2345253d43440e
Author: Ken McDonell 
Date:   Mon Nov 10 15:15:46 2014 +1100

    qa/valgrind-suppress-3.5.0: more strangeness to suppress on CentOS 5.10

commit a6bcedffd15b3be94fe506e66f6c18a36108bd8e
Author: Ken McDonell 
Date:   Mon Nov 10 15:15:03 2014 +1100

    qa/842: fix typo in notrun guard

From kenj@internode.on.net  Mon Nov 10 14:00:33 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id AAD327F3F
	for ; Mon, 10 Nov 2014 14:00:33 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 8A5988F8035
	for ; Mon, 10 Nov 2014 12:00:30 -0800 (PST)
X-ASG-Debug-ID: 1415649628-04bdf0650e1c30e0001-S8gJnT
Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id jd54tg9krTf2eCZT for ; Mon, 10 Nov 2014 12:00:28 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.131
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApUBAOcXYVR20Spq/2dsb2JhbAANT9RpgyACgT0BAQEBAYUAAQEEJxFAEQsYCRYPCQMCAQIBRRMIAQG+b5cCAQEIAgEfkRwWhDUBBJ9ZkSKIGIMjAQEB
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail07.adl2.internode.on.net with ESMTP; 11 Nov 2014 06:30:27 +1030
Message-ID: <546119B1.6050306@internode.on.net>
Date: Tue, 11 Nov 2014 07:01:53 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: Re: [pcp] Dynamic Proc PMDA metrics
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
In-Reply-To: <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131]
X-Barracuda-Start-Time: 1415649628
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11442
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

On 10/11/14 17:26, Nathan Scott wrote:
> ...
>> 3. Couldn't find any precedent for handling long form help text for dynamic
>> pmdas. So I just generated a header file from the existing help file and
>> used that. Let me know if you want something different. Attached is the perl
>> script I wrote for that in case it is generally useful.
>
> We might be able to do some remapping from hotproc -> proc variants, but I
> don't think there's any precedence there I can point you towards - I'll can
> take a look into that too if you like.
>

Looks like we've made a mistake when the pmdadynamicFoo stuff was added. 
  Based on a quick code inspection, I think this prototype

typedef int  (*pmdaUpdateText)(pmdaExt *, pmID, int, char **);

should have included another int parameter to allow the "type" from 
pmdaDynamicLookupText() to be passed through to the callback routine 
registered via pmdaDynamicPMNS() that I'm guessing your PMDA is using in 
its own version of pmdaText() (rather than the default one).

This means you can't get indom helptext either for a dynamic indom that 
is associated with a dynamic metric (and otherwise unknown to the PMDA).

From nscott@redhat.com  Mon Nov 10 20:40:10 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id E26727F3F
	for ; Mon, 10 Nov 2014 20:40:09 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 7F5C4AC00C
	for ; Mon, 10 Nov 2014 18:40:06 -0800 (PST)
X-ASG-Debug-ID: 1415673601-04bdf0650e1ed2a0001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id UaYBtux4meNyFM4j (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Nov 2014 18:40:02 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAB2e1kk016937;
	Mon, 10 Nov 2014 21:40:01 -0500
Date: Mon, 10 Nov 2014 21:40:01 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: David Smith 
Cc: pcp 
Message-ID: <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com>
In-Reply-To: <545CEE9A.5060007@redhat.com>
References: <54512E80.9090302@redhat.com> <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> <545A53B0.9030500@redhat.com> <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> <545CEE9A.5060007@redhat.com>
Subject: Re: [pcp] [RFC] pcp python patch
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp python patch
Thread-Index: e2/j/9RAJf+OgO8c6XlGobFwlB4/zg==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415673601
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11457
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

Hi David,

----- Original Message -----
> [...]
> We have 3 distinct classes of dictionaries, each treated differently.
> During PMDA.run(), the PMDA class passes down to cpmda several dictionaries:
> 
> (1) self._metric_names (used by cpmda.pmns_refresh)
> (2) self._metric_oneline (used by cpmda.pmid_oneline_refresh)
> (2) self._metric_helptext (used by cpmda.pmid_longtext_refresh)
> (2) self._indom_oneline (used by cpmda.indom_oneline_refresh)
> (2) self._indom_helptext (used by cpmda.indom_longtext_refresh)
> (3) self._indomtable (used by cpmda.pmda_dispatch)
> (3) self._metrictable (used by cpmda.pmda_dispatch)
> 
> Dictionaries marked by (1) are passed down, used once when needed, then
> discarded.
> 
> Dictionaries marked by (2) are passed down and kept, used when needed,
> never discarded.
> 
> Dictionaries marked by (3) aren't passed down and kept. Instead the PMDA
> class creates a string buffer for each dictionary and passes that down
> to cpmda. This string buffer is used once and then discarded.
> 
> I was thinking all dictionaries were treated as being in class (1) (used
> once then discarded). That's why I was passing down everything again in
> PMDA.__refresh_metrics(). Since I don't need to pass down items again in
> class (2), that simplifies PMDA.__refresh_metrics() to:
> 
> ====
>     def __refresh_metrics(self):
>         # Call the user's function.
>         if self.__refresh_metrics_func:
>             self.__refresh_metrics_func()
> 
>         if cpmda.get_need_refresh():
>             self.pmns_refresh()
>             cpmda.pmda_rehash(self._indomtable, self._metrictable)
> ====

Yeah, that's much neater - nice!

> We actually could go further here, if we can change the cpmda interface.
> I'm not sure how set in stone it is. I'm not talking about a change to
> the PMDA class interface here, just the cpmda interface. If we change
> cpmda.pmda_dispatch() to take the indomtable and metrictable
> dictionaries directly (instead of using string buffers), we could keep
> them around for use later by pmda_rehash() - basically moving them to
> class (2).

That would be neat indeed.  The cpmda interface is very "internal" - the
python API (which sits atop it) used by PMDAs is really the interface we
need to preserve.  So, I think it'd be OK to make a change like you are
describing here - I can't see that ever affecting anyone.  It may still
be possible to preserve back-compat using the varargs/keyword interfaces
- we could support two calling conventions for this API, one with the
dictionaries, the other with the string buffers.  In fact, it might even
be enough to do a type-check on the passed-in arguments inside the cpmda
code, and switch behaviour based on what we get given.

But, that may be going too far for back-compat since I really can't see
how this change would affecting anyone, in reality, and it may just end
up with us keeping dead code around.  Maybe write the code and you could
then make a judgement call based on how good/bad it gets when preserving
the existing interface/semantics?

> If we go that far, then all dictionaries are in class (2) (passed down
> and kept) - except for _metric_names in class (1). If we decide to move
> it to class (2), then all dictionaries are treated the same. At that
> point, we don't need a wrapper around the user's refresh_metrics()
> function in the PMDA class at all, since the cpmda already has all the
> information it needs (pointers to all the dictionaries) and knows when
> to do the refreshing.

Sounds good to me.

> > The only other thing I wanted to revisit a bit is this subtlety...
> > 
> >>> I was expecting this "refreshing" would happen around the time
> >>> PDUs exchanges happen with pmcd, which delays it until the last
> >>> moment (and prevents the situation where unnecessary duplication
> 
> I'm afraid you'll have decide on/help me figure out pcp subtleties...
> 
> Right now the user's refresh_metrics callback function gets called down
> in cpmda at similar places as the mmv pmda calls mmv_reload_maybe().
> I'll need help in figuring out if that is causing unnecessary
> duplication. Assuming the user's refresh_metrics callback is smart
> enough to not change anything when it doesn't need to, then cpmda
> shouldn't really refresh anything either.

Yep, that'll be fine & is optimal as you have it, I believe.

> > Ah, yes, right you are - I was thinking PyBUF_SIMPLE was removed in 3.x
> > but on closer inspection it is still there & yep, looks like dead code.
> > I do vaguely remember some py3 porting issues around buffers - but this
> > looks unrelated.
> 
> We should probably just delete the whole '#else' code to avoid confusing
> others in the future.

Yes please.

cheers.

--
Nathan

From nscott@redhat.com  Tue Nov 11 00:41:11 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id B003F7F37
	for ; Tue, 11 Nov 2014 00:41:11 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 737D4304032
	for ; Mon, 10 Nov 2014 22:41:08 -0800 (PST)
X-ASG-Debug-ID: 1415688063-04bdf0650d1fc810001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id J6TXIokURiHhvG9V (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Nov 2014 22:41:04 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAB6exus019639;
	Tue, 11 Nov 2014 01:40:59 -0500
Date: Tue, 11 Nov 2014 01:40:58 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Ken McDonell , Martins Innus 
Cc: pcp@oss.sgi.com
Message-ID: <587637728.11787602.1415688058653.JavaMail.zimbra@redhat.com>
In-Reply-To: <546119B1.6050306@internode.on.net>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <546119B1.6050306@internode.on.net>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: mtX+6401WI4BWKTe63aMuDrJyicLhQ==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415688064
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11462
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi guys,

----- Original Message -----
> On 10/11/14 17:26, Nathan Scott wrote:
> > ...
> Looks like we've made a mistake when the pmdadynamicFoo stuff was added.
>   Based on a quick code inspection, I think this prototype
> 
> typedef int  (*pmdaUpdateText)(pmdaExt *, pmID, int, char **);
> 
> should have included another int parameter to allow the "type" from
> pmdaDynamicLookupText() to be passed through to the callback routine
> registered via pmdaDynamicPMNS() that I'm guessing your PMDA is using in
> its own version of pmdaText() (rather than the default one).

For dynamic metrics we can't use the libpcp_pmda pmdaText (aka pmdaGetHelp,
ie the compiled/mapped help text files).

There's a bit of missing functionality here that Martins found: we provide
no way for PMDAs with dynamic metrics to get library help for their metric
text.  This is fundamentally because the compiled text files have PMIDs in
'em, and we're using dynamic namespace entries.  IOW, at the time the help
files would usually be compiled (eg ./Install time) the pmns is not known,
so newhelp(1) has no chance.

Maybe we should have an option for libpcp_pmda to handle non-compiled help
text entries (ie. the plain text files with metric names instead of PMIDs)?
Not sure how much that would help in general, but it probably would in this
case where we could map the two sets of metrics to one set of help texts.

> This means you can't get indom helptext either for a dynamic indom that
> is associated with a dynamic metric (and otherwise unknown to the PMDA).

Not sure this is a real issue - the (custom) help text callbacks have to
deal with indoms (if present & help exists) themselves - and can do this
by simply calling pmdaText like proc does now.  The dynamic code is all
about dynamic metric names, indoms are not really covered (and nor should
they be, I think?).

cheers.

--
Nathan

From minnus@buffalo.edu  Tue Nov 11 09:32:10 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id AC7647F3F
	for ; Tue, 11 Nov 2014 09:32:10 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 9A97C8F8050
	for ; Tue, 11 Nov 2014 07:32:07 -0800 (PST)
X-ASG-Debug-ID: 1415719926-04cb6c1e6c1fe650001-S8gJnT
Received: from mtareserve1.acsu.buffalo.edu (mtareserve6.acsu.buffalo.edu [128.205.6.4]) by cuda.sgi.com with ESMTP id 0Yse8V5I4YEdxw6N for ; Tue, 11 Nov 2014 07:32:06 -0800 (PST)
X-Barracuda-Envelope-From: minnus@buffalo.edu
X-Barracuda-Apparent-Source-IP: 128.205.6.4
Received: from localmailC.acsu.buffalo.edu (localmailc.acsu.buffalo.edu [128.205.5.204])
	by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 010BEF2;
	Tue, 11 Nov 2014 10:32:06 -0500 (EST)
Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1])
	by localhost (Postfix) with SMTP id F0978D9B4;
	Tue, 11 Nov 2014 10:32:05 -0500 (EST)
Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1])
	by localmailC.acsu.buffalo.edu (Postfix) with ESMTP id 50AC4D9A4;
	Tue, 11 Nov 2014 10:32:05 -0500 (EST)
Received: from smtp.buffalo.edu (smtp3.acsu.buffalo.edu [128.205.5.226])
	by localmailC.acsu.buffalo.edu (Prefixe) with ESMTP id 4083FD9A2;
	Tue, 11 Nov 2014 10:32:05 -0500 (EST)
Received: from prince.ccr.buffalo.edu (prince.ccr.buffalo.edu [128.205.40.45])
	(Authenticated sender: minnus@buffalo.edu)
	by smtp.buffalo.edu (Postfix) with ESMTPSA id 358482915;
	Tue, 11 Nov 2014 10:32:05 -0500 (EST)
Message-ID: <54622BF4.4080806@buffalo.edu>
Date: Tue, 11 Nov 2014 10:32:04 -0500
From: Martins Innus 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] Dynamic Proc PMDA metrics
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
In-Reply-To: <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-PM-EL-Spam-Prob: : 8%
X-Barracuda-Connect: mtareserve6.acsu.buffalo.edu[128.205.6.4]
X-Barracuda-Start-Time: 1415719926
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11474
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Nathan,

On 11/10/14 1:26 AM, Nathan Scott wrote:
>
>
> I have a fix for 022 & will do 943 - they're fixable using sort(1) to produce
> deterministic output.
>
> I'm also seeing 580 fail, looks probably related:
>
> $ diff 580.out 580.out.bad
> 7,9c7
> < pm*InDom: inst=[1] iname= reverse lookup: inst=[1]
> < pmGetInDom:
> <    [1] 
> ---
>> Error: Unknown or illegal metric identifier
> I'll dig into it later in the week if its not failing elsewhere and/or not
> reproducible.
>
I think I'm ok with fixing everything but this.  I don't understand why 
it is failing. I put some debugging into the indom.c test and all 
queries into the proc pmda, even non-dynamic elements, return bogus 
information from pmLookupName:

************

[vagrant@pcptest src]$ ./indom -h unix: -i 1 proc.psinfo.pid
proc.psinfo.pid:
running proc.psinfo.pid
pmID (d,c,i): 511, 3, 0
failed 2
Error: Unknown or illegal metric identifier

*************

pmLookupName returns successfully (non-error), but all proc queries have 
the domain,cluster,item values of 511,3,0.

But, strangely, the following works fine:

*************

[vagrant@pcptest src]$ pmval -h unix: -i 1 proc.psinfo.pid

metric:    proc.psinfo.pid
host:      pcptest
semantics: discrete instantaneous value
units:     none
samples:   all

           1

*************

My guess is that I'm not building the tree properly, but I'm not sure 
what is missing.

Thanks

Martins

From bahaihough@gmail.com  Tue Nov 11 11:30:30 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,
	T_DKIM_INVALID autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id BC6867F5E
	for ; Tue, 11 Nov 2014 11:30:30 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 9FB61304048
	for ; Tue, 11 Nov 2014 09:30:30 -0800 (PST)
X-ASG-Debug-ID: 1415727029-04cb6c1e6a20b210001-S8gJnT
Received: from mail-qa0-f48.google.com (mail-qa0-f48.google.com [209.85.216.48]) by cuda.sgi.com with ESMTP id Laldqbhd0LHCOmnK (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Tue, 11 Nov 2014 09:30:29 -0800 (PST)
X-Barracuda-Envelope-From: bahaihough@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.216.48
X-Barracuda-IPDD: Level1 [gmail.com/209.85.216.48]
Received: by mail-qa0-f48.google.com with SMTP id x12so7227528qac.7
        for ; Tue, 11 Nov 2014 09:30:29 -0800 (PST)
X-Barracuda-IPDD: Level1 [gmail.com/209.85.216.48]
X-Barracuda-IPDD: Level1 [gmail.com/209.85.216.48]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=X6Ai02B37QXEVTlVd1mzHahYp9IFs9EWXgqFy6SKc0o=;
        b=gwcRo6UVli3jVgVkTaJcDv4OFT35uS5OzlkMEkFMXkKAkG36OY1VXLTDknHqfNWRtV
         mZqU/MF6g2bXixGv9WaQmbtU/XnGXks/SdBHxJnghBG3VkFkxaEm8123Cmmmu2nsC1Js
         2rz0bfoy02vdfwK4xSBKInyjZ4j4jSwwTtkpNk1kJrMgPA4n8RolhqTYNWDY5UWoRsWv
         k7igcnc/WlrC4Ha15hdhNbV91XL+dHp/VDdJWsCx2Pv4Rp8yGGZyrJhAavVVW8lH62iW
         IczZAS3IgVIb5rebIkrMEHLkzjCXYYsW/erqCUebA0yojzrfVXQu0aRUKVPBAkMQXKiy
         0PsA==
MIME-Version: 1.0
X-Received: by 10.229.7.133 with SMTP id d5mr54234556qcd.24.1415727028985;
 Tue, 11 Nov 2014 09:30:28 -0800 (PST)
Received: by 10.96.122.227 with HTTP; Tue, 11 Nov 2014 09:30:28 -0800 (PST)
Date: Tue, 11 Nov 2014 12:30:28 -0500
Message-ID: 
Subject: PCP-GUI from src
From: Brad Hough 
X-ASG-Orig-Subj: PCP-GUI from src
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a1135ec94a96ac8050798a29d
X-Barracuda-Connect: mail-qa0-f48.google.com[209.85.216.48]
X-Barracuda-Start-Time: 1415727029
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.20
X-Barracuda-Spam-Status: No, SCORE=0.20 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA584, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11477
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 HTML_MESSAGE           BODY: HTML included in message
	0.20 BSF_SC0_SA584          Custom Rule SA584

--001a1135ec94a96ac8050798a29d
Content-Type: text/plain; charset=UTF-8

I'm new to PCP.  I'm trying to install from source.  I wanted to have
access to the pcp-gui tools also, but they don't seem to build by default.
What am I missing?  I'm trying to build on a Centos 6.6 box and have cloned
from git://oss.sgi.com/pcp.git.  Is pcp-gui still a separate repository?
Beacause I don't see anything at git://oss.sgi.com/pcp/pcp-gui.git.

Thanks,
Brad Hough

--001a1135ec94a96ac8050798a29d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


I'm new to PCP.=C2=A0 I'm trying to install from s=
ource.=C2=A0 I wanted to have access to the pcp-gui tools also, but they do=
n't seem to build by default. What am I missing?=C2=A0 I'm trying t=
o build on a Centos 6.6 box and have cloned from=C2=A0git://oss.sgi.com/pcp.git.=C2=A0 Is pcp-gui still a s=
eparate repository?=C2=A0 Beacause I don't see anything at=C2=A0git://oss.sgi.com/pcp/pc=
p-gui.git.

=
Thanks,
Brad Hough


--001a1135ec94a96ac8050798a29d--

From kenj@internode.on.net  Tue Nov 11 14:05:15 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id D187B7F61
	for ; Tue, 11 Nov 2014 14:05:15 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 61436AC003
	for ; Tue, 11 Nov 2014 12:05:11 -0800 (PST)
X-ASG-Debug-ID: 1415736305-04cb6c1e6b212c10001-S8gJnT
Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id eXgAG85l7xhElA7E for ; Tue, 11 Nov 2014 12:05:06 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.129
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: As0DAMpqYlR20SpqPGdsb2JhbAANT4NiVAXBJmGKHYdTAoEwAQEBAQEGAQEBATiEPgEBBDhAEQsYCRYPCQMCAQIBMRQTCAEBiEUFt2yXTAEBCAIBH453giQWhDUBBJcElxOKfFgBgkoBAQE
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail06.adl2.internode.on.net with ESMTP; 12 Nov 2014 06:35:05 +1030
Message-ID: <54626C55.6040704@internode.on.net>
Date: Wed, 12 Nov 2014 07:06:45 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: Re: [pcp] PCP-GUI from src
References: 
X-ASG-Orig-Subj: Re: [pcp] PCP-GUI from src
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129]
X-Barracuda-Start-Time: 1415736306
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11483
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

On 12/11/14 04:30, Brad Hough wrote:
> I'm new to PCP.  I'm trying to install from source.  I wanted to have
> access to the pcp-gui tools also, but they don't seem to build by
> default. What am I missing?  I'm trying to build on a Centos 6.6 box and
> have cloned from git://oss.sgi.com/pcp.git
> .  Is pcp-gui still a separate repository?
> Beacause I don't see anything at git://oss.sgi.com/pcp/pcp-gui.git
> .

All the gui tools have been merged back into the main repository.

But there is configure magic to conditionally not build some components 
if the required prereqs are not installed on the build system.

I'd guess it is the qt-devel package that you're missing (we need Qt4), 
but if you want to find _all_ the prereqs needed to build all the PCP 
pieces and run QA, try the script:

	qa/admin/check-vm

From minnus@buffalo.edu  Tue Nov 11 15:11:32 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 7B3807F6A
	for ; Tue, 11 Nov 2014 15:11:32 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 5AED6304048
	for ; Tue, 11 Nov 2014 13:11:32 -0800 (PST)
X-ASG-Debug-ID: 1415740288-04cbb00e6921a400001-S8gJnT
Received: from mtareserve1.acsu.buffalo.edu (mtareserve6.acsu.buffalo.edu [128.205.6.4]) by cuda.sgi.com with ESMTP id 02ZkHFU6CIBUTlxI for ; Tue, 11 Nov 2014 13:11:28 -0800 (PST)
X-Barracuda-Envelope-From: minnus@buffalo.edu
X-Barracuda-Apparent-Source-IP: 128.205.6.4
Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208])
	by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 02CEF30A4;
	Tue, 11 Nov 2014 16:11:28 -0500 (EST)
Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1])
	by localhost (Postfix) with SMTP id F06F930BE2;
	Tue, 11 Nov 2014 16:11:27 -0500 (EST)
Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1])
	by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id 5725030BCA;
	Tue, 11 Nov 2014 16:11:26 -0500 (EST)
Received: from smtp.buffalo.edu (smtp1.acsu.buffalo.edu [128.205.5.253])
	by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id 4888230BC9;
	Tue, 11 Nov 2014 16:11:26 -0500 (EST)
Received: from prince.ccr.buffalo.edu (prince.ccr.buffalo.edu [128.205.40.45])
	(Authenticated sender: minnus@buffalo.edu)
	by smtp.buffalo.edu (Postfix) with ESMTPSA id 3B9832557;
	Tue, 11 Nov 2014 16:11:26 -0500 (EST)
Message-ID: <54627B7D.6040906@buffalo.edu>
Date: Tue, 11 Nov 2014 16:11:25 -0500
From: Martins Innus 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] Dynamic Proc PMDA metrics
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
In-Reply-To: <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-PM-EL-Spam-Prob: : 8%
X-Barracuda-Connect: mtareserve6.acsu.buffalo.edu[128.205.6.4]
X-Barracuda-Start-Time: 1415740288
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11486
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Nathan,

     OK. My current work on laying hotproc on top of dynamic proc is here:

https://github.com/ubccr/pcp/tree/dynamic_hotproc

Right now the functionality should be the exact same as my initial 
submission except that it works with the new dynamic proc framework.  
Actually it was pretty easy to merge.  This seems like a good way to 
handle these types of pmdas.  I have not done any of the fixes you 
suggested in your initial review of hotproc, so its not ready for a 
merge yet.

You can ignore all of this until I am done. I just wanted to get this 
out there since I don't know where in your cgroup work you are.  I think 
the only conflict might be in proc_pid.c with my addition of an extra 
parameter into some of the existing methods as I mentioned before.


A couple of other comments below.

On 11/10/14 1:26 AM, Nathan Scott wrote:
>> 3. Couldn't find any precedent for handling long form help text for dynamic
>> pmdas. So I just generated a header file from the existing help file and
>> used that. Let me know if you want something different. Attached is the perl
>> script I wrote for that in case it is generally useful.
> We might be able to do some remapping from hotproc -> proc variants, but I
> don't think there's any precedence there I can point you towards - I'll can
> take a look into that too if you like.
OK, this seems to be working OK as I have implemented it.  Feel free to 
make other suggestions if this looks difficult to maintain.
>> 4. I left a few commented code sections that will show where hotprocs will
>> hook in. Will add another root node, and all other stuff ( help, etc ) will
>> be shared.
This has been now filled in and seems to work in the limited testing I 
have done at the end of the day.  I ran the existing pmda.proc QA tests 
and the results are the same as before.  The 2 tests you fixed with sort 
and the third I am stumped at the moment as I mentioned in my other email.

One thing I noticed and am trying to track down:

In the:

static void
refresh_metrictable(pmdaMetric *source, pmdaMetric *dest, int id)

callback that I provide.  The domain doesn't appear to be set in the 
"source" at the time this call is made.  I check, and it is "0". But 
everything seems to work.  I assume this gets set at some later time?

Thanks

Martins

From nscott@redhat.com  Tue Nov 11 23:27:55 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id B8E6F7F9D
	for ; Tue, 11 Nov 2014 23:27:55 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id A76078F8035
	for ; Tue, 11 Nov 2014 21:27:55 -0800 (PST)
X-ASG-Debug-ID: 1415770070-04cbb00e69230420001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id EIxX82ZRLgF9eSpI (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 11 Nov 2014 21:27:50 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAC5Rmex027335;
	Wed, 12 Nov 2014 00:27:48 -0500
Date: Wed, 12 Nov 2014 00:27:47 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Martins Innus 
Cc: pcp 
Message-ID: <268589342.12441210.1415770067959.JavaMail.zimbra@redhat.com>
In-Reply-To: <54622BF4.4080806@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54622BF4.4080806@buffalo.edu>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: 3LhcAha3Es5NkDV+gmr9jrEY7kUMOA==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415770070
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11502
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...



----- Original Message -----
> [...]
> I think I'm ok with fixing everything but this.  I don't understand why

Great, thanks!

> it is failing. I put some debugging into the indom.c test and all
> queries into the proc pmda, even non-dynamic elements, return bogus
> information from pmLookupName:
> [...]
> My guess is that I'm not building the tree properly, but I'm not sure
> what is missing.

Odd - I'll look into it.  It may be something to do with the ordering of
PDUs (pmval and most tools will ask for pmdesc and other things - names/
pmids, etc before asking for instances) - IIRC this test program asks in
a different order, or only for instance PDUs, or something like that.

cheers.

--
Nathan

From nscott@redhat.com  Tue Nov 11 23:34:19 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 00F387F9D
	for ; Tue, 11 Nov 2014 23:34:19 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id CE7B08F8037
	for ; Tue, 11 Nov 2014 21:34:18 -0800 (PST)
X-ASG-Debug-ID: 1415770457-04cbb00e692306d0001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id leblORAWjZGJWnk8 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 11 Nov 2014 21:34:17 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAC5YFMv000733;
	Wed, 12 Nov 2014 00:34:15 -0500
Date: Wed, 12 Nov 2014 00:34:15 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Martins Innus 
Cc: pcp 
Message-ID: <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com>
In-Reply-To: <54627B7D.6040906@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54627B7D.6040906@buffalo.edu>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: 0bTZEQfg56ZAVSYkJWorIr8i3STCkQ==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1415770457
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11502
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi Martins,

----- Original Message -----
> [...]
> You can ignore all of this until I am done. I just wanted to get this
> out there since I don't know where in your cgroup work you are.  I think
> the only conflict might be in proc_pid.c with my addition of an extra
> parameter into some of the existing methods as I mentioned before.

Ah - OK - thanks for getting it out.  cgroups is progressing nicely, I'm
hoping end-of-this-week-ish it should be all revised.  So far, the rework
and refactoring is proving a neat improvement.

> 
> One thing I noticed and am trying to track down:
> 
> In the:
> 
> static void
> refresh_metrictable(pmdaMetric *source, pmdaMetric *dest, int id)
> 
> callback that I provide.  The domain doesn't appear to be set in the
> "source" at the time this call is made.  I check, and it is "0". But
> everything seems to work.  I assume this gets set at some later time?

Yeah, hmm, can't remember exactly how this works - but the domain number
usually gets stamped into the static metrictable at pmdaInit time.  ISTR
some need to pass the domain number around, in order to stamp it into
the dynamic metric PMIDs as well ... but its a vague memory.  This may be
an area where the dynamic metrics libpcp_pmda code could be improved too,
BTW - not sure, but keep that in mind (might be easier to pull the domain
number out of the pmdaExt structure and writing it into the source, if it
is missing at the time the callback is called?  I don't think that could
harm any existing PMDA code)

cheers.

--
Nathan

From tdm@sgi.com  Wed Nov 12 09:02:37 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id A45F37F61
	for ; Wed, 12 Nov 2014 09:02:37 -0600 (CST)
Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 33E4FAC001;
	Wed, 12 Nov 2014 07:02:34 -0800 (PST)
Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11])
	by estes.americas.sgi.com (Postfix) with ESMTP id F1C0D70033BB;
	Wed, 12 Nov 2014 09:02:33 -0600 (CST)
Message-ID: <54637689.2060701@sgi.com>
Date: Wed, 12 Nov 2014 09:02:33 -0600
From: Troy McCorkell 
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: Troy McCorkell , pcp@oss.sgi.com
Subject: Re: oss.sgi.com - maintenance downtime Friday 11/14/2014 at 10:00
 CST USA
References: <545CE78C.5000102@sgi.com>
In-Reply-To: <545CE78C.5000102@sgi.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Second reminder.

On 11/07/2014 09:38 AM, Troy McCorkell wrote:
>
> On Friday November 14, 2014 at 10:00 CST USA
> oss.sgi.com will be unavailable for a short period of time to perform 
> network maintenance.
>
> The outage is expected to last approximately 15 minutes.
>

From minnus@buffalo.edu  Wed Nov 12 14:53:36 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 5FE267F3F
	for ; Wed, 12 Nov 2014 14:53:36 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 4DFF9304066
	for ; Wed, 12 Nov 2014 12:53:33 -0800 (PST)
X-ASG-Debug-ID: 1415825611-04cb6c1e6d24d480001-S8gJnT
Received: from mtareserve1.acsu.buffalo.edu (mtareserve6.acsu.buffalo.edu [128.205.6.4]) by cuda.sgi.com with ESMTP id aO9tY42hxOge6xHS for ; Wed, 12 Nov 2014 12:53:31 -0800 (PST)
X-Barracuda-Envelope-From: minnus@buffalo.edu
X-Barracuda-Apparent-Source-IP: 128.205.6.4
Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208])
	by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id E08D5BA5A;
	Wed, 12 Nov 2014 15:53:30 -0500 (EST)
Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1])
	by localhost (Postfix) with SMTP id D93AA31F32;
	Wed, 12 Nov 2014 15:53:30 -0500 (EST)
Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1])
	by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id B71BA31F1A;
	Wed, 12 Nov 2014 15:53:28 -0500 (EST)
Received: from smtp.buffalo.edu (smtp2.acsu.buffalo.edu [128.205.5.254])
	by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id A819C31F19;
	Wed, 12 Nov 2014 15:53:28 -0500 (EST)
Received: from prince.ccr.buffalo.edu (prince.ccr.buffalo.edu [128.205.40.45])
	(Authenticated sender: minnus@buffalo.edu)
	by smtp.buffalo.edu (Postfix) with ESMTPSA id 809722B20;
	Wed, 12 Nov 2014 15:53:28 -0500 (EST)
Message-ID: <5463C8C8.5010700@buffalo.edu>
Date: Wed, 12 Nov 2014 15:53:28 -0500
From: Martins Innus 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] Dynamic Proc PMDA metrics
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54627B7D.6040906@buffalo.edu> <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
In-Reply-To: <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-PM-EL-Spam-Prob: : 8%
X-Barracuda-Connect: mtareserve6.acsu.buffalo.edu[128.205.6.4]
X-Barracuda-Start-Time: 1415825611
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11526
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Nathan,

     Sorry for the long email below.  I have become completely confused 
with dynamic metrics and it seems like I'm trying to use the API in a 
way that was not intended.  #2 below is the most troubling.  You likely 
want to junk all my dynamic proc code until we get this figured out.

I'd be happy to jump on the phone if my explanations don't make sense.  
It took me a while to wind through this, so maybe I've got it all wrong.


On 11/12/14 12:34 AM, Nathan Scott wrote:
> Hi Martins,
>
> ----- Original Message -----
>> [...]
>> You can ignore all of this until I am done. I just wanted to get this
>> out there since I don't know where in your cgroup work you are.  I think
>> the only conflict might be in proc_pid.c with my addition of an extra
>> parameter into some of the existing methods as I mentioned before.
> Ah - OK - thanks for getting it out.  cgroups is progressing nicely, I'm
> hoping end-of-this-week-ish it should be all revised.  So far, the rework
> and refactoring is proving a neat improvement.

Just realized this is incomplete. Sorry, will need some more work and 
you shouldn't use it.

>> One thing I noticed and am trying to track down:
>>
>> In the:
>>
>> static void
>> refresh_metrictable(pmdaMetric *source, pmdaMetric *dest, int id)
>>
>> callback that I provide.  The domain doesn't appear to be set in the
>> "source" at the time this call is made.  I check, and it is "0". But
>> everything seems to work.  I assume this gets set at some later time?
> Yeah, hmm, can't remember exactly how this works - but the domain number
> usually gets stamped into the static metrictable at pmdaInit time.  ISTR
> some need to pass the domain number around, in order to stamp it into
> the dynamic metric PMIDs as well ... but its a vague memory.  This may be
> an area where the dynamic metrics libpcp_pmda code could be improved too,
> BTW - not sure, but keep that in mind (might be easier to pull the domain
> number out of the pmdaExt structure and writing it into the source, if it
> is missing at the time the callback is called?  I don't think that could
> harm any existing PMDA code)
>
Still no progress on this, except that with a dso pmda, the domain 
number is correct at this point. Illustrated below in the interrupts case.



While researching this, I've run into a few issues that stem from 
calling pmdaDynamicPMNS multiple times:

1. In the "size_metrictable" callback there is no context as to which 
dynamic tree is being queried if we use the same callback for all trees. 
So this is going to way overestimate the storage space needed, since we 
need to return the size of the largest sub tree if we use a common 
callback.  The alternative is one callback per sub tree as far as I can see.



2. This is a larger problem. In dynamic.c, for the function:

static pmdaMetric *
dynamic_metric_table(int index, pmdaMetric *offset)

     When running through the initial metric table, the only check that 
is done is to see if the cluster matches when doing "mtabupdate" and no 
check is done against the item or name.  This will cause multiple calls 
for the same metric if we have clusters that span multiple trees (since 
pmdaDynamicMetricTable calls mtabupdate for each pmdaDynamicPMNS call we 
made), like we have in proc with CLUSTER_PID_STATUS.

     I don't see a solution to this problem.  The initial metric table 
has no name to compare against.  When we call pmdaDynamicPMNS we don't 
provide a list of item ids that are valid. So neither of those checks 
can be done.

     The only solution I can think of would be to allow mtabupdate to 
return a success code if the metric was added, and only then would 
dynamic_metric_table update its internal status.  The callback would 
know whether the cluster/id combo was a valid dynamic metric.  This 
would break any code that uses this interface.

     I think there is a likelihood of buffer overflow here now.  If the 
number of metrics in extra clusters is greater than the extra size 
introduced by the padding due to #1 above, i think there will be memory 
corruption.

Strangely, I haven't found any negative impacts from this. My guess is 
somewhere there are just multiple definitions of the same metric taking 
up space?  Or maybe this has something to do with the one QA failure I 
couldn't debug.




3. In general, its not clear to me the proper way to generate the 
dynamic metrics from the initial "templates". For example, take the 
existing case in the linux pmda. For interrupts we have:

     /* kernel.percpu.interrupts.line[] */
     { NULL, { PMDA_PMID(CLUSTER_INTERRUPT_LINES, 0), PM_TYPE_U32,
     CPU_INDOM, PM_SEM_COUNTER, PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) }, },

     /* kernel.percpu.interrupts.[] */
     { NULL, { PMDA_PMID(CLUSTER_INTERRUPT_OTHER, 0), PM_TYPE_U32,
     CPU_INDOM, PM_SEM_COUNTER, PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) }, },

These are used as templates to create the dynamic metrics. But 
interrupts.c provides in "size_metrictable" the total number of dynamic 
metrics, so we end up with at least 2 extra metrics which as far as I 
can tell are not used.

 From pmcd.log:

interrupts size_metrictable: 2 total x 11 trees
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae5f0) 
metric ID dup: 60.49.0 -> 60.49.1
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae610) 
metric ID dup: 60.49.0 -> 60.49.2
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae630) 
metric ID dup: 60.49.0 -> 60.49.3
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae650) 
metric ID dup: 60.49.0 -> 60.49.4
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae670) 
metric ID dup: 60.49.0 -> 60.49.5
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae690) 
metric ID dup: 60.49.0 -> 60.49.6
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae6b0) 
metric ID dup: 60.49.0 -> 60.49.7
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae6d0) 
metric ID dup: 60.49.0 -> 60.49.8
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae6f0) 
metric ID dup: 60.49.0 -> 60.49.9
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae710) 
metric ID dup: 60.49.0 -> 60.49.10
interrupts refresh_metrictable: (0x7f7c092010a0 -> 0x7f7c0d2ae730) 
metric ID dup: 60.49.0 -> 60.49.11
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae750) 
metric ID dup: 60.50.0 -> 60.50.1
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae770) 
metric ID dup: 60.50.0 -> 60.50.2
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae790) 
metric ID dup: 60.50.0 -> 60.50.3
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae7b0) 
metric ID dup: 60.50.0 -> 60.50.4
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae7d0) 
metric ID dup: 60.50.0 -> 60.50.5
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae7f0) 
metric ID dup: 60.50.0 -> 60.50.6
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae810) 
metric ID dup: 60.50.0 -> 60.50.7
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae830) 
metric ID dup: 60.50.0 -> 60.50.8
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae850) 
metric ID dup: 60.50.0 -> 60.50.9
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae870) 
metric ID dup: 60.50.0 -> 60.50.10
interrupts refresh_metrictable: (0x7f7c092010c0 -> 0x7f7c0d2ae890) 
metric ID dup: 60.50.0 -> 60.50.11
[Wed Nov 12 16:08:24] pmcd(13810) Debug: pmdaRehash: PMDA linux DSO: 
successful rebuild

But:

[vagrant@pcptest testsuite]$ pminfo -M kernel.percpu.interrupts
kernel.percpu.interrupts.MCP PMID: 60.50.10 = 251709450 = 0xf00c80a
kernel.percpu.interrupts.MCE PMID: 60.50.9 = 251709449 = 0xf00c809
kernel.percpu.interrupts.THR PMID: 60.50.8 = 251709448 = 0xf00c808
kernel.percpu.interrupts.TRM PMID: 60.50.7 = 251709447 = 0xf00c807
kernel.percpu.interrupts.TLB PMID: 60.50.6 = 251709446 = 0xf00c806
kernel.percpu.interrupts.CAL PMID: 60.50.5 = 251709445 = 0xf00c805
kernel.percpu.interrupts.RES PMID: 60.50.4 = 251709444 = 0xf00c804
kernel.percpu.interrupts.IWI PMID: 60.50.3 = 251709443 = 0xf00c803
kernel.percpu.interrupts.PMI PMID: 60.50.2 = 251709442 = 0xf00c802
kernel.percpu.interrupts.SPU PMID: 60.50.1 = 251709441 = 0xf00c801
kernel.percpu.interrupts.LOC PMID: 60.50.0 = 251709440 = 0xf00c800
kernel.percpu.interrupts.line20 PMID: 60.49.9 = 251708425 = 0xf00c409
kernel.percpu.interrupts.line19 PMID: 60.49.8 = 251708424 = 0xf00c408
kernel.percpu.interrupts.line16 PMID: 60.49.7 = 251708423 = 0xf00c407
kernel.percpu.interrupts.line15 PMID: 60.49.6 = 251708422 = 0xf00c406
kernel.percpu.interrupts.line14 PMID: 60.49.5 = 251708421 = 0xf00c405
kernel.percpu.interrupts.line12 PMID: 60.49.4 = 251708420 = 0xf00c404
kernel.percpu.interrupts.line9 PMID: 60.49.3 = 251708419 = 0xf00c403
kernel.percpu.interrupts.line8 PMID: 60.49.2 = 251708418 = 0xf00c402
kernel.percpu.interrupts.line1 PMID: 60.49.1 = 251708417 = 0xf00c401
kernel.percpu.interrupts.line0 PMID: 60.49.0 = 251708416 = 0xf00c400

So both of the .11 and one of the .10 metrics is orphaned?  Does that 
cause any issue?  Or since the pmns doesn't reference them, no harm 
except memory used.  If that's the case,  for my issue #2 above I can 
just set dummy cluster or item values for the metrics that are sent 
through twice(or more), but that's pretty wasteful and I'm sure will 
have other unintended consequences.

Thanks

Martins

From lberk@redhat.com  Wed Nov 12 17:09:05 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 10C4F7F3F
	for ; Wed, 12 Nov 2014 17:09:05 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id F31168F8033
	for ; Wed, 12 Nov 2014 15:09:01 -0800 (PST)
X-ASG-Debug-ID: 1415833734-04cb6c1e6b252120001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id uMmT1SxkIEJu1494 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 12 Nov 2014 15:08:55 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sACN8s0b012676
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 12 Nov 2014 18:08:54 -0500
Received: from toium (dhcp-10-15-16-139.yyz.redhat.com [10.15.16.139])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sACN8rBq028314
	(version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO)
	for ; Wed, 12 Nov 2014 18:08:53 -0500
From: Lukas Berk 
To: pcp@oss.sgi.com
Subject: pcp updates - pmdapapi update
Date: Wed, 12 Nov 2014 18:08:52 -0500
X-ASG-Orig-Subj: pcp updates - pmdapapi update
Message-ID: <87oascow3f.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1415833735
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi,

The PAPI pmda has gone through a fair bit of work.  The changes can be
found at: git://sourceware.org/git/pcpfans.git lberk/papi or gitweb:
https://www.sourceware.org/git/gitweb.cgi?p=pcpfans.git;a=shortlog;h=refs/heads/lberk/papi

This iteration features much improved internals, additional qa,
dynamically generated pmns and auto_enable functionality (complimentary
to the standard, and expected control.{enable,disable} metrics).  As
discussed on the list I've added qa cases which take the traditional
route of using dbpmda, as well as a case which uses -hlocal: to track
stateful changes.  qa/967 in particular tests the auto_enable work with
dbpmda, ensuring the it can be turned off (if desired), in addition to
timeout's as described by the accompanied documentation.  If anybody has
suggestions or constructive criticism I'd be happy to work those in as
well.

One area I'd appreciate second opinions on, is how to deal with the
variable number of metrics available on different hardware.  I've
attempted to keep qa tests to the 'basic' counters (TOT_INS as an
example) for the most part.  However, in the case of testing for
ECNFLCT errors with/without multiplexing, this may require enumerating
through metrics with a less common overlap in hardware.  As a starting
point I've simply used 799 with the counters that would have caused an
error on my machine, however I'm aware this is not the case for others.
Thoughts appreciated.

diffstat and commit logs attached below,

Cheers,

Lukas

build/rpm/fedora.spec     |    8 
man/man1/pmdapapi.1       |   22 
qa/799                    |  385 ++
qa/799.out                |  153 +
qa/813                    |  118 
qa/813.out                |   84 
qa/903                    |    6 
qa/914                    |   23 
qa/914.out                |   43 
qa/967                    |  172 +
qa/967.out                |  186 +
qa/group                  |   12 
src/pmdas/papi/Install    |    8 
src/pmdas/papi/help       |  116 
src/pmdas/papi/papi.c     | 6928 ++++++++++++++++++++--------------------------
src/pmdas/papi/pmdapapi.1 |  116 
src/pmdas/papi/pmns       | 1631 ++++------
17 files changed, 4910 insertions(+), 5101 deletions(-)

commit b6f75582968a4d3e0ec377af95675127ece7da6c
Author: Lukas Berk 
Date:   Wed Nov 12 09:16:46 2014 -0500

    Update qa/914 and respective output
    
    We've added new metrics to the papi.control group.  Updating the
    testcase to reflect these changes.

commit b89820cb1c54b66333c9b7706089ca1fe96b80e9
Author: Lukas Berk 
Date:   Wed Nov 5 23:03:53 2014 -0500

    Add qa/967 to test the pmdapapi auto_enable functionality
    
    In this testcase we take two dbpmda runs;
    
    First, we run a check that the existing enable/disable functionality
    works normally when papi.control.auto_enable is disabled (set to 0)
    
    Second, we check that the auto_enable timeout works by setting it to
    a low timeout (5 seconds), fetching a value, checking that
    papi.control.status reports the correct timeout and a valid papi
    VALUE.  After which we wait just over 5 seconds and re-fetch the
    papi.control.status value, which (should) show the metric is no
    longer being counted and that papi has stopped.
    
    This second test implicitly checks the functionality of dynamically
    starting a counter.

commit fcee1e2342ed3788a30ec91951051288b5f12aa2
Author: Lukas Berk 
Date:   Wed Nov 5 12:31:57 2014 -0500

    Add helppath variable
    
    helppath is used in various locations throughout the pmda, dropped
    by accident when merging the branches.

commit a6cbff6790ba51932f59b6d6a8e6544debf23e7f
Author: Frank Ch. Eigler 
Date:   Mon Nov 3 21:48:55 2014 -0500

    papi pmda: tweak auto_afid name & commentary

commit 9b4654d00873cdb03c611a282574018634596397
Author: Lukas Berk 
Date:   Mon Nov 3 21:47:34 2014 -0500

    Tweak pmdapapi docs
    
    Add the unit (seconds) to the papi.control.auto_enable metric and
    mention how one would disable the auto_enable functionality if needed
    in the man page.
    
    Signed-off-by: Lukas Berk 

commit 2f3729268714c8bce74906242350d0277410de82
Author: Frank Ch. Eigler 
Date:   Mon Nov 3 15:55:41 2014 -0500

    papi pmda: add counter auto-enable on fetch
    
    By reinterpreting the papi_info[].metric_enabled as a timestamp rather
    than a boolean, and a small bit of logic, we get automatic enablement
    of PAPI metric/counters upon pmfetch.  This allows PAPI metrics to be
    used with plain old pmlogger, pmval, etc., without necessary manual
    pmstore's.  Counters are retired based on a timeout mechanism,
    assisted by pmaf(3).  The papi.control.status metric is enhanced to
    show the remaining lifespan of the counters, and to make it more safe
    & reliable.
    
    A new papi.control.auto_enable metric is available to set the timeout
    (default 120 seconds).  The Install script runs a papi.control.reset
    operation to avoid leaving the counters running for even that long
    after the post-install metric/value checks.
    
    The previous pmstore mechanism is left in place unchanged, and
    overrides the automatic scheme.

commit 61111a6b50224db18e6a9e5da211e4558ec933f4
Author: Lukas Berk 
Date:   Wed Nov 5 11:49:57 2014 -0500

    Add pmdaText call in papi_text
    
    Somewhere along the way, rebasing ate the call

commit 4ae54fc9bcf2a56bace14e7166b126fccbf31284
Author: Lukas Berk 
Date:   Wed Nov 5 10:40:47 2014 -0500

    Update qa/799 with -hlocal: test against real values
    
    799 will now grab 'real' values from the current papi install.
    This testcase now also includes installing and uninstalling
    the pmda.

commit ce8a1aecca93352d56af89de4de96a951ee1fc40
Author: Lukas Berk 
Date:   Wed Nov 5 10:34:18 2014 -0500

    Add dbpmda testcase for pmdapapi
    
    Add an additional testcase for pmdapapi, ensure a value can be
    enabled, checked, and disabled through dbpmda

commit 27f2f6ee11dc7d596fb5304bb1d7bf9ed81636c4
Author: Frank Ch. Eigler 
Date:   Tue Oct 14 17:16:53 2014 -0400

    papi/pmda: collapse metric addition/removal path
    
    The add- and remove-metric operations are almost identical, with
    respect to the need to save previous counter values, pause, then
    restart with newborn/surviving metrics.  With the recent
    metric_enabled flag, the two code paths can be effectively merged.
    
    Gone are add_metric(), remove_metric(), setup_eventset(), stop_papi(),
    and the number_of_active_counters global.  Left notes for a few more
    candidates.
    
    Welcome refresh_metrics() and a collapsed-case .enable/.disable
    pmstore handler, a trivial .reset handler.
    
    Tested by hand, running pmstore ops to enable/disable multiple
    individual counters, including erroneous names, while monitoring the
    valid counters for consistent progress.  Mechanical testing
    forthcoming.

commit 6de78cfc602149252a7d048192be165ea54a88e2
Author: Lukas Berk 
Date:   Tue Oct 14 14:16:49 2014 -0400

    Make status_string[] variable static
    
    Avoid any type of memory corruption bug by ensuring the variable
    won't already be unwound.

commit 5a4ac8644992dd28bdfcf7bcab6f2f324df1aef8
Author: Lukas Berk 
Date:   Tue Oct 14 11:27:40 2014 -0400

    Fix a papi_info.position bug in remove_metric
    
    Include a new papi.info var (metric_enabled) to track if the metric
    was enabled.  This leaves us to simply assign the position in the
    order we (re)add the metric to the eventset

commit aed29825b29a913b6558a4c609f04aed85e85a3a
Author: Lukas Berk 
Date:   Thu Oct 9 16:16:07 2014 -0400

    Tidy up setup_eventset()
    
    Pick a style of retval assignment/comparison and stick with it.
    Report papi errors in a uniform way

commit 9390154052416c87d9b7153cb20d89a0d412de93
Author: Lukas Berk 
Date:   Thu Oct 9 16:02:56 2014 -0400

    Refactor the eventset setup which is done in papi_internal_init and remove_metric
    
    We were only partially setting up the eventset in remove_metric (due
    to working around papi bug).  Refactor this and ensure it's done properly
    both times.

commit 57deba3642bb8a29f01706d8f7e83bfad482c717
Author: Lukas Berk 
Date:   Thu Oct 9 13:39:24 2014 -0400

    Refactor {add,remove}_metric functions
    
    We can refactor the functions to put stopping and saving the values
    in its own function, while making the return values consistent.

commit b1fe23bd105de11de1d35c1b9cd24c68817d1ece
Author: Lukas Berk 
Date:   Mon Oct 6 19:06:18 2014 -0400

    Reduce the string manipulation functions usage in papi_internal_init
    
    By reusing the same data structures and sizes that papi defines for
    symbols we can directly memcpy the values in without having to loop
    so many times and tokenizing everything.

commit f649a825fa3a0984d1eba2abaea7a637f31ca0f9
Author: Lukas Berk 
Date:   Mon Oct 6 12:20:59 2014 -0400

    Switch to local, stack allocated string for papi.control.status metric
    
    Instead of realloc'ing multiple times a call, lets instead, allocate
    one string (once) and simplify the process

commit 95792977870c1b681d88d04480c78b0c69e6298d
Author: Lukas Berk 
Date:   Mon Oct 6 09:44:56 2014 -0400

    Enable papi multiplexing
    
    Enabling multiplexing will make the pmda a bit more flexible for counting
    metrics for longer periods of time and for hardware with fewer hardware
    counters

commit d74f1ed49d2c4824262ba2edbfe23931f5b0f79c
Author: Lukas Berk 
Date:   Fri Oct 3 15:20:44 2014 -0400

    Remove check_event_exists function
    
    If the event requested isn't in already in papi_info, add_metric
    and remove metric won't get called at all anyways, this check is
    redundant

commit 85e84ac0eb4a23d96a45b5135e18718d1e16398b
Author: Lukas Berk 
Date:   Fri Oct 3 15:08:31 2014 -0400

    Shift papi_event_code variable into papi_event_type_t var in papi_info
    
    We can reduce the number of variables we need and simply use the data
    structures already there.

commit e3ef5fa7990901b13006abec8194fd9f8eea4edb
Author: Lukas Berk 
Date:   Thu Oct 2 15:02:56 2014 -0400

    Add skeleton of new pmdapapi qa test - qa/799
    
    Base for new pmdapapi test

commit 23a5321803f6d5169f0953ddf8efd83d0ae8fcd6
Author: Lukas Berk 
Date:   Thu Oct 2 12:04:37 2014 -0400

    Make adding metrics to the current eventset more robust
    
    If we attempt to add an event that cannot be added (for example,
    if two metrics can't be counted at the same time), we should make
    the attempt, but still restart papi and run the other metrics if
    possible.  If debugging is turned on, output the error message to
    the log file.

commit 767e9f831b19da0f5ff385d633649baf9399656b
Author: Lukas Berk 
Date:   Wed Oct 1 19:13:59 2014 -0400

    Change papi.control.status to include metric values and fix papi_children
    
    Several smaller nits.  papi.control.status should now include the current
    values being counted.  We also now use a __pmnsTree var to keep track
    of the dynamic metrics we've added.  This makes both papi_children much
    eaiser to manage as well as papi_name_lookup.

commit c96c3a12d8702a2b452fff7a393af07b143641b6
Author: Lukas Berk 
Date:   Tue Sep 23 12:17:17 2014 -0400

    Refactor bits of {add,remove}_metric and small cleanups
    
    We can move the check to see if the event specified is valid outside
    the function and reduce the repeated code.  Also remove a loop that
    we no longer need.

commit cd7c33f08b8c757fdd48db5d3f4c26392506ad99
Author: Lukas Berk 
Date:   Mon Sep 22 16:16:59 2014 -0400

    papi_state returns a bitmask, so do a bitwise comparison, not direct compare.
    
    instead of a direct comparison with PAPI_ we should just do a bitwise
    AND to confirm the state

commit 93bed08ea89700c946c85c0ad92dfbba4d80d2d3
Author: Lukas Berk 
Date:   Mon Sep 22 14:50:03 2014 -0400

    Remove papi_string_status and inline the former function
    
    The aforementioned function was incorrect in using a case statement
    and assuming only one would be hit.  The papi state is a bitmask,
    and we should be able to return more than one condition.  It is
    also potentially useful to output the last known values of each active
    metric.

commit 90707b3d01de01592e796f3c45127261dfbf2fc7
Author: Lukas Berk 
Date:   Mon Sep 22 14:48:45 2014 -0400

    Add checks around papi_children realloc statements
    
    realloc's need a check to ensure the allocation returned with no errors.
    If an error was found, throw __pmNoMem

commit 86249c03ec18e4a8a7ebd5cbb2f7d773d4f13d32
Author: Lukas Berk 
Date:   Fri Sep 19 18:07:05 2014 -0400

    Remove internal enable and disable strings for papi metrics
    
    We shouldn't require the use of these strings to pass back and forth.

commit 068f07ee2e87a28446fb0fe123d62cb5b049ab9f
Author: Lukas Berk 
Date:   Fri Sep 19 14:14:18 2014 -0400

    Implement papi.control.reset metric
    
    papi.control.reset will zero the current papi values being counted
    one must use papi.control.disable if they'd like the values to no
    longer be in the active eventset

commit e8875700ab017c10bf655e8649233d97eaf730e9
Author: Lukas Berk 
Date:   Thu Sep 18 21:18:40 2014 -0400

    Remove remove second function paramter from remove_metric
    
    Both parameters being passed were values from papi_info, if we
    instead pass the specific position within the papi_info array,
    we can determine the other values we need without passing extra
    arguments

commit 85e9d659d660ca1c4683546bec67fe9b0f310d6e
Author: Lukas Berk 
Date:   Thu Sep 18 18:36:10 2014 -0400

    Assign papi help text in papi_internal_init and only lookup values in papi_text
    
    Assigning the values of {short,long}_descr can be done within papi_internal_init,
    which allows us to simply recall the values in papi_text.

commit f4ecd6d059fa3b746ec4b3e71342c57e1ea07c4e
Author: Lukas Berk 
Date:   Wed Sep 17 18:30:20 2014 -0400

    Drop unneeded parameter from check_papi_state
    
    Drop unneeded parameter from the check_papi_state() function, and
    don't mix up papi error values and pcp error values that we could
    return.  Stick to papi.

commit 4445ecb274827a9bf3370147b6af5f5ed7c6e7cd
Author: Lukas Berk 
Date:   Wed Sep 17 11:59:46 2014 -0400

    Rename papi_pmid function to papi_name_lookup as requested
    
    Rename function to be more descriptive

commit 8866f8e5ad64ac377a31f76aa981350c2fdb0e27
Author: Lukas Berk 
Date:   Wed Sep 17 11:51:19 2014 -0400

    Update the papi_fetchcallback mechanism for passing metric data
    
    No longer loop through all the available events (as we generate the
    pmns dynamically now), just use the idp->item that we're passed

commit 16f79091795539f5abce73fe9ef5a728674796c4
Author: Lukas Berk 
Date:   Tue Sep 16 11:46:56 2014 -0400

    Add papi_children callback for generating the pmns dynamically
    
    Order the papi metrics under papi.system.metric (expecting expansion
    of metrics to capture pid/tid specifics later) and dynamically
    generate the list of metrics based on what's actually available on
    the current hardware.

commit 7d21807bf0a41234a273b6784c2f00b5cc8349a7
Author: Lukas Berk 
Date:   Wed Sep 10 10:38:15 2014 -0400

    Make use of strtok in papi_pmid
    
    Use strtok in pmda_papi function for a cleaner function

commit b5601e2944328174dc37f22da355187ba7fbc5d5
Author: Lukas Berk 
Date:   Tue Sep 9 18:58:45 2014 -0400

    Move PAPI_event_info_t var into papi_info for metric descriptions
    
    By moving the PAPI_event_info_t varible into the papi_info struct
    we can ensure the references to the descriptions are persistent.

commit c23abe1f5f47feb3512fbd4a3e78060b9f907bc0
Author: Lukas Berk 
Date:   Tue Sep 9 18:26:11 2014 -0400

    First step towards a fully dynamic papipmda (including pmns)
    
    Remove the papi_info[].pmns_position variable and set_pmns_position
    fuction.  Likewise, only add papi metrics to papi_info if they're
    actually present on the current hardware.  In the future we might
    need to add in another papi.'system'.metric to make the pmns fully
    automatically generated and match with the various userspace programs

commit 7cfc4eeb7a6cf0741da71ea33d7696009201cc1b
Author: Lukas Berk 
Date:   Mon Sep 8 12:22:33 2014 -0400

    Rework saving previous papi values when adding/removing metrics
    
    Previously, when we added or removed a metric that we were
    tracking, we'd lose the previous values and all metrics would
    restart from zero.  By adding a prev_metric value in papi_info,
    we can systematically save the values, and add it to the atom->ull
    in the fetchcallback

commit f8a41d7eda518a8b4f6d980f0b78e52ecbe39472
Author: Lukas Berk 
Date:   Mon Sep 8 10:10:40 2014 -0400

    Be more consistent with internal use of variable names
    
    Remove all mentions of 'sts' and consistenly use retval

commit 82bb1916134d1a8895e626e1ed83f70e08de2975
Author: Lukas Berk 
Date:   Mon Sep 8 09:45:47 2014 -0400

    Account for control and available clusters in papi_text
    
    papi_text was returning results from PAPI_enum_event which matched
    the pcp item number, even when the metrics were in different cluters
    (control and available, respectively).

commit 5c8637452f2696afb5bccd2005841a2a0c8695d1
Author: Lukas Berk 
Date:   Fri Sep 5 15:34:56 2014 -0400

    Add debug warning and return PM_ERR_CONV to papi.control.enable
    
    Currently papi.control.enable will just silently fail if presented
    with metrics it can't add, fix that.
    
    *src/pmdas/papi/papi.c - add debug warning and return the error in
    		         papi_store for enable

commit 1ad8894cd9ab9e78a6e09dea615d346a1d82b4a8
Author: Lukas Berk 
Date:   Tue Sep 23 12:17:17 2014 -0400

    Refactor bits of {add,remove}_metric and small cleanups
    
    We can move the check to see if the event specified is valid outside
    the function and reduce the repeated code.  Also remove a loop that
    we no longer need.

commit d45b7a5a69b0bb78d35769a1f88dac41e35609ab
Author: Lukas Berk 
Date:   Mon Sep 22 16:16:59 2014 -0400

    papi_state returns a bitmask, so do a bitwise comparison, not direct compare.
    
    instead of a direct comparison with PAPI_ we should just do a bitwise
    AND to confirm the state

commit 785be8d59b21da97ce8ec460a858774eb9884c7f
Author: Lukas Berk 
Date:   Mon Sep 22 14:50:03 2014 -0400

    Remove papi_string_status and inline the former function
    
    The aforementioned function was incorrect in using a case statement
    and assuming only one would be hit.  The papi state is a bitmask,
    and we should be able to return more than one condition.  It is
    also potentially useful to output the last known values of each active
    metric.

commit 08d6fd7ed266ab1cd22b6eeac23e0ed526a3cf9b
Author: Lukas Berk 
Date:   Mon Sep 22 14:48:45 2014 -0400

    Add checks around papi_children realloc statements
    
    realloc's need a check to ensure the allocation returned with no errors.
    If an error was found, throw __pmNoMem

commit 234c037734ec2f62b5cfb721591b503584d95aa7
Author: Lukas Berk 
Date:   Fri Sep 19 18:07:05 2014 -0400

    Remove internal enable and disable strings for papi metrics
    
    We shouldn't require the use of these strings to pass back and forth.

commit 5242fa14be6e4e14e266f4d96bf730068fc77497
Author: Lukas Berk 
Date:   Fri Sep 19 14:14:18 2014 -0400

    Implement papi.control.reset metric
    
    papi.control.reset will zero the current papi values being counted
    one must use papi.control.disable if they'd like the values to no
    longer be in the active eventset

commit 4e50f21254b62781dfed420634aeb19401b878ba
Author: Lukas Berk 
Date:   Thu Sep 18 21:18:40 2014 -0400

    Remove remove second function paramter from remove_metric
    
    Both parameters being passed were values from papi_info, if we
    instead pass the specific position within the papi_info array,
    we can determine the other values we need without passing extra
    arguments

commit 55c19eb5d1e82fb9044bb6c23343eda936a1b795
Author: Lukas Berk 
Date:   Thu Sep 18 18:36:10 2014 -0400

    Assign papi help text in papi_internal_init and only lookup values in papi_text
    
    Assigning the values of {short,long}_descr can be done within papi_internal_init,
    which allows us to simply recall the values in papi_text.

commit 20adda001f11c425000ae46698d9c046dfc8c996
Author: Lukas Berk 
Date:   Wed Sep 17 18:30:20 2014 -0400

    Drop unneeded parameter from check_papi_state
    
    Drop unneeded parameter from the check_papi_state() function, and
    don't mix up papi error values and pcp error values that we could
    return.  Stick to papi.

commit c2041d73e3be8fc913fdca84900dca6766213b13
Author: Lukas Berk 
Date:   Wed Sep 17 11:59:46 2014 -0400

    Rename papi_pmid function to papi_name_lookup as requested
    
    Rename function to be more descriptive

commit 074c27041db9840051478a6b3c5f0cf68725b8aa
Author: Lukas Berk 
Date:   Wed Sep 17 11:51:19 2014 -0400

    Update the papi_fetchcallback mechanism for passing metric data
    
    No longer loop through all the available events (as we generate the
    pmns dynamically now), just use the idp->item that we're passed

commit 67b79c8130881c6ec5d2f669adaa6fbdae637419
Author: Lukas Berk 
Date:   Tue Sep 16 11:46:56 2014 -0400

    Add papi_children callback for generating the pmns dynamically
    
    Order the papi metrics under papi.system.metric (expecting expansion
    of metrics to capture pid/tid specifics later) and dynamically
    generate the list of metrics based on what's actually available on
    the current hardware.

commit 9ce332107ea024e77f3765f2deb36b8f9ff438a4
Author: Lukas Berk 
Date:   Wed Sep 10 10:38:15 2014 -0400

    Make use of strtok in papi_pmid
    
    Use strtok in pmda_papi function for a cleaner function

commit f5ee043d7ea06888e133ffab917ac33239780316
Author: Lukas Berk 
Date:   Tue Sep 9 18:58:45 2014 -0400

    Move PAPI_event_info_t var into papi_info for metric descriptions
    
    By moving the PAPI_event_info_t varible into the papi_info struct
    we can ensure the references to the descriptions are persistent.

commit bc5ab4ca60a0152c74ef345125ce78f56284c908
Author: Lukas Berk 
Date:   Tue Sep 9 18:26:11 2014 -0400

    First step towards a fully dynamic papipmda (including pmns)
    
    Remove the papi_info[].pmns_position variable and set_pmns_position
    fuction.  Likewise, only add papi metrics to papi_info if they're
    actually present on the current hardware.  In the future we might
    need to add in another papi.'system'.metric to make the pmns fully
    automatically generated and match with the various userspace programs

commit dcb2d35c53edbbefeba46b9c332915d090226d8f
Author: Lukas Berk 
Date:   Mon Sep 8 12:22:33 2014 -0400

    Rework saving previous papi values when adding/removing metrics
    
    Previously, when we added or removed a metric that we were
    tracking, we'd lose the previous values and all metrics would
    restart from zero.  By adding a prev_metric value in papi_info,
    we can systematically save the values, and add it to the atom->ull
    in the fetchcallback

commit c52ebc9d02d82c13196b9f9145055e83eafaa4b2
Author: Lukas Berk 
Date:   Mon Sep 8 10:10:40 2014 -0400

    Be more consistent with internal use of variable names
    
    Remove all mentions of 'sts' and consistenly use retval

commit f37259dd1cf8675c4b02d08d69e2a15dd0587969
Author: Lukas Berk 
Date:   Mon Sep 8 09:45:47 2014 -0400

    Account for control and available clusters in papi_text
    
    papi_text was returning results from PAPI_enum_event which matched
    the pcp item number, even when the metrics were in different cluters
    (control and available, respectively).

commit c618fade0c63f843d9af42120520a4ac44adaa0c
Author: Lukas Berk 
Date:   Fri Sep 5 15:34:56 2014 -0400

    Add debug warning and return PM_ERR_CONV to papi.control.enable
    
    Currently papi.control.enable will just silently fail if presented
    with metrics it can't add, fix that.
    
    *src/pmdas/papi/papi.c - add debug warning and return the error in
    		         papi_store for enable

commit 5640a0d81a9ba23a914efc4eff92a2e299c95f4e
Author: Lukas Berk 
Date:   Wed Nov 5 10:43:02 2014 -0500

    tweak qa/903 to account for lower papi metric output
    
    pmdapapi only creates metrics dynamically based on the system hardware.
    This means the number of metrics by default is much lower, tweak the
    awk statement to reflect that

commit f9aa390ae8440ae3691c8e44f2d2637839de488f
Author: Lukas Berk 
Date:   Wed Nov 5 10:40:47 2014 -0500

    Update qa/799 with -hlocal: test against real values
    
    799 will now grab 'real' values from the current papi install.
    This testcase now also includes installing and uninstalling
    the pmda.

commit 50de988dcdac24021f937d24e33f6857c19ef8c7
Author: Lukas Berk 
Date:   Wed Nov 5 23:03:53 2014 -0500

    Add qa/967 to test the pmdapapi auto_enable functionality
    
    In this testcase we take two dbpmda runs;
    
    First, we run a check that the existing enable/disable functionality
    works normally when papi.control.auto_enable is disabled (set to 0)
    
    Second, we check that the auto_enable timeout works by setting it to
    a low timeout (5 seconds), fetching a value, checking that
    papi.control.status reports the correct timeout and a valid papi
    VALUE.  After which we wait just over 5 seconds and re-fetch the
    papi.control.status value, which (should) show the metric is no
    longer being counted and that papi has stopped.
    
    This second test implicitly checks the functionality of dynamically
    starting a counter.

commit 9acbf877443e918cdc3bc6a4136dca9a5bc4ab10
Author: Lukas Berk 
Date:   Wed Nov 5 12:31:57 2014 -0500

    Add helppath variable
    
    helppath is used in various locations throughout the pmda, dropped
    by accident when merging the branches.

commit 67b1360e43ba136d34f661602afd6bf65b6cde17
Author: Frank Ch. Eigler 
Date:   Mon Nov 3 21:48:55 2014 -0500

    papi pmda: tweak auto_afid name & commentary

commit f9791a2814e055c497c90dfc4913c9121d2639ef
Author: Lukas Berk 
Date:   Mon Nov 3 21:47:34 2014 -0500

    Tweak pmdapapi docs
    
    Add the unit (seconds) to the papi.control.auto_enable metric and
    mention how one would disable the auto_enable functionality if needed
    in the man page.
    
    Signed-off-by: Lukas Berk 

commit 940a38a7e32c3a287a226755a96d5a1df1da287b
Author: Frank Ch. Eigler 
Date:   Mon Nov 3 15:55:41 2014 -0500

    papi pmda: add counter auto-enable on fetch
    
    By reinterpreting the papi_info[].metric_enabled as a timestamp rather
    than a boolean, and a small bit of logic, we get automatic enablement
    of PAPI metric/counters upon pmfetch.  This allows PAPI metrics to be
    used with plain old pmlogger, pmval, etc., without necessary manual
    pmstore's.  Counters are retired based on a timeout mechanism,
    assisted by pmaf(3).  The papi.control.status metric is enhanced to
    show the remaining lifespan of the counters, and to make it more safe
    & reliable.
    
    A new papi.control.auto_enable metric is available to set the timeout
    (default 120 seconds).  The Install script runs a papi.control.reset
    operation to avoid leaving the counters running for even that long
    after the post-install metric/value checks.
    
    The previous pmstore mechanism is left in place unchanged, and
    overrides the automatic scheme.

commit 92faaee4bc303f3be12bf62578dea2ce20a6f531
Author: Lukas Berk 
Date:   Wed Nov 5 11:49:57 2014 -0500

    Add pmdaText call in papi_text
    
    Somewhere along the way, rebasing ate the call

commit 5fda674535f7a9ee60abd9a16d1944fa1bbf944d
Author: Lukas Berk 
Date:   Wed Nov 5 10:43:02 2014 -0500

    tweak qa/903 to account for lower papi metric output
    
    pmdapapi only creates metrics dynamically based on the system hardware.
    This means the number of metrics by default is much lower, tweak the
    awk statement to reflect that

commit a6e8ac926ad750deaafbf1710e94b67d53abc1e2
Author: Lukas Berk 
Date:   Wed Nov 5 10:40:47 2014 -0500

    Update qa/799 with -hlocal: test against real values
    
    799 will now grab 'real' values from the current papi install.
    This testcase now also includes installing and uninstalling
    the pmda.

commit 4368b37fce067e7854d724adb630e027a0dad3dd
Author: Lukas Berk 
Date:   Wed Nov 5 10:34:18 2014 -0500

    Add dbpmda testcase for pmdapapi
    
    Add an additional testcase for pmdapapi, ensure a value can be
    enabled, checked, and disabled through dbpmda

commit 2cc4519ce51c6d3177d1d2c7d1b2520b001534bb
Author: Frank Ch. Eigler 
Date:   Tue Oct 14 17:16:53 2014 -0400

    papi/pmda: collapse metric addition/removal path
    
    The add- and remove-metric operations are almost identical, with
    respect to the need to save previous counter values, pause, then
    restart with newborn/surviving metrics.  With the recent
    metric_enabled flag, the two code paths can be effectively merged.
    
    Gone are add_metric(), remove_metric(), setup_eventset(), stop_papi(),
    and the number_of_active_counters global.  Left notes for a few more
    candidates.
    
    Welcome refresh_metrics() and a collapsed-case .enable/.disable
    pmstore handler, a trivial .reset handler.
    
    Tested by hand, running pmstore ops to enable/disable multiple
    individual counters, including erroneous names, while monitoring the
    valid counters for consistent progress.  Mechanical testing
    forthcoming.

commit 7b73c86f0ca5bf105d43fa05cb797a47691d25b7
Author: Lukas Berk 
Date:   Tue Oct 14 14:16:49 2014 -0400

    Make status_string[] variable static
    
    Avoid any type of memory corruption bug by ensuring the variable
    won't already be unwound.

commit 5d2d7bf25e27b6b93f73e9fa58e17e62c6d8b5dc
Author: Lukas Berk 
Date:   Tue Oct 14 11:27:40 2014 -0400

    Fix a papi_info.position bug in remove_metric
    
    Include a new papi.info var (metric_enabled) to track if the metric
    was enabled.  This leaves us to simply assign the position in the
    order we (re)add the metric to the eventset

commit 436be6ab6e10fd28387b328c463b280eccab805e
Author: Lukas Berk 
Date:   Thu Oct 9 16:16:07 2014 -0400

    Tidy up setup_eventset()
    
    Pick a style of retval assignment/comparison and stick with it.
    Report papi errors in a uniform way

commit 0ac8189e9e930156a601e3e43348ccfd3a25b828
Author: Lukas Berk 
Date:   Thu Oct 9 16:02:56 2014 -0400

    Refactor the eventset setup which is done in papi_internal_init and remove_metric
    
    We were only partially setting up the eventset in remove_metric (due
    to working around papi bug).  Refactor this and ensure it's done properly
    both times.

commit 4db29959e3c3874e17696fdbbb221aa7dce24fc2
Author: Lukas Berk 
Date:   Thu Oct 9 13:39:24 2014 -0400

    Refactor {add,remove}_metric functions
    
    We can refactor the functions to put stopping and saving the values
    in its own function, while making the return values consistent.

commit 90c80fcee53c23f90675c49f853c313e4af2b2c4
Author: Lukas Berk 
Date:   Mon Oct 6 19:06:18 2014 -0400

    Reduce the string manipulation functions usage in papi_internal_init
    
    By reusing the same data structures and sizes that papi defines for
    symbols we can directly memcpy the values in without having to loop
    so many times and tokenizing everything.

commit 5e1e9f444320fcaf44e51449d6c67f22f5b2e1c7
Author: Lukas Berk 
Date:   Mon Oct 6 12:20:59 2014 -0400

    Switch to local, stack allocated string for papi.control.status metric
    
    Instead of realloc'ing multiple times a call, lets instead, allocate
    one string (once) and simplify the process

commit fcfdaae5c6e7929d4a2726cc17675135ab709d15
Author: Lukas Berk 
Date:   Mon Oct 6 09:44:56 2014 -0400

    Enable papi multiplexing
    
    Enabling multiplexing will make the pmda a bit more flexible for counting
    metrics for longer periods of time and for hardware with fewer hardware
    counters

commit 0dffa0f01026c9de820303c588f57b4dc5bfbb06
Author: Lukas Berk 
Date:   Fri Oct 3 15:30:56 2014 -0400

    Drop unneeded (ignored) gid == 0 check in permission_check

commit 9b0c5f70b1aaa8045f5e4aa8dba79581d4d694f8
Author: Lukas Berk 
Date:   Fri Oct 3 15:20:44 2014 -0400

    Remove check_event_exists function
    
    If the event requested isn't in already in papi_info, add_metric
    and remove metric won't get called at all anyways, this check is
    redundant

commit 9af90b09393f846206f3e8267b15bd0e06d8a0fa
Author: Lukas Berk 
Date:   Fri Oct 3 15:08:31 2014 -0400

    Shift papi_event_code variable into papi_event_type_t var in papi_info
    
    We can reduce the number of variables we need and simply use the data
    structures already there.

commit 0bcd7a6aafefe44f44cf782e96c67f6de563af55
Author: Lukas Berk 
Date:   Thu Oct 2 15:02:56 2014 -0400

    Add skeleton of new pmdapapi qa test - qa/799
    
    Base for new pmdapapi test

commit f5bcc3ce37867bf7a9cf05f1ebe2f32337dedda2
Author: Lukas Berk 
Date:   Thu Oct 2 12:04:37 2014 -0400

    Make adding metrics to the current eventset more robust
    
    If we attempt to add an event that cannot be added (for example,
    if two metrics can't be counted at the same time), we should make
    the attempt, but still restart papi and run the other metrics if
    possible.  If debugging is turned on, output the error message to
    the log file.

commit 6a8050b4883b7c15414ead420427677edb75a319
Author: Lukas Berk 
Date:   Wed Oct 1 19:13:59 2014 -0400

    Change papi.control.status to include metric values and fix papi_children
    
    Several smaller nits.  papi.control.status should now include the current
    values being counted.  We also now use a __pmnsTree var to keep track
    of the dynamic metrics we've added.  This makes both papi_children much
    eaiser to manage as well as papi_name_lookup.

commit d7d093586d32bc40aed373d03a77158aa765b8a7
Author: Lukas Berk 
Date:   Wed Oct 1 19:09:55 2014 -0400

    Update qa files for new metrics and added portion to pmns
    
    A new portion of the papi pmns includes 'system' and will extend
    to possible process names when that functionality is included.
    We also need to update qa for implementation of papi.control.reset

commit d7bd289a94a253c0b853701d0c2dfdcf95625822
Author: Lukas Berk 
Date:   Tue Sep 23 12:17:17 2014 -0400

    Refactor bits of {add,remove}_metric and small cleanups
    
    We can move the check to see if the event specified is valid outside
    the function and reduce the repeated code.  Also remove a loop that
    we no longer need.

commit a1d3d402501714c439eb30bd556a4aa1ea3db488
Author: Lukas Berk 
Date:   Mon Sep 22 16:16:59 2014 -0400

    papi_state returns a bitmask, so do a bitwise comparison, not direct compare.
    
    instead of a direct comparison with PAPI_ we should just do a bitwise
    AND to confirm the state

commit 4dfde46964686817313315795374eb9e89c0ab82
Author: Lukas Berk 
Date:   Mon Sep 22 14:50:03 2014 -0400

    Remove papi_string_status and inline the former function
    
    The aforementioned function was incorrect in using a case statement
    and assuming only one would be hit.  The papi state is a bitmask,
    and we should be able to return more than one condition.  It is
    also potentially useful to output the last known values of each active
    metric.

commit 54077d685ea674bad70e0f8589320bb54b6144f9
Author: Lukas Berk 
Date:   Mon Sep 22 14:48:45 2014 -0400

    Add checks around papi_children realloc statements
    
    realloc's need a check to ensure the allocation returned with no errors.
    If an error was found, throw __pmNoMem

commit dc2d9243fd63ff1eb1cf79ae90ed15a4306d2f8d
Author: Lukas Berk 
Date:   Fri Sep 19 18:07:05 2014 -0400

    Remove internal enable and disable strings for papi metrics
    
    We shouldn't require the use of these strings to pass back and forth.

commit 848198e02672e30d8014525aa2d566862d738389
Author: Lukas Berk 
Date:   Fri Sep 19 14:14:18 2014 -0400

    Implement papi.control.reset metric
    
    papi.control.reset will zero the current papi values being counted
    one must use papi.control.disable if they'd like the values to no
    longer be in the active eventset

commit e311cf812980d53cf8c343de9a8ad22afe095c05
Author: Lukas Berk 
Date:   Thu Sep 18 21:18:40 2014 -0400

    Remove remove second function paramter from remove_metric
    
    Both parameters being passed were values from papi_info, if we
    instead pass the specific position within the papi_info array,
    we can determine the other values we need without passing extra
    arguments

commit 114d4cef2502a044b6b595eb3d66d5eb3f73752b
Author: Lukas Berk 
Date:   Thu Sep 18 18:36:10 2014 -0400

    Assign papi help text in papi_internal_init and only lookup values in papi_text
    
    Assigning the values of {short,long}_descr can be done within papi_internal_init,
    which allows us to simply recall the values in papi_text.

commit 2943337143a52d8bd4feb19feae08a21f86595a0
Author: Lukas Berk 
Date:   Wed Sep 17 18:30:20 2014 -0400

    Drop unneeded parameter from check_papi_state
    
    Drop unneeded parameter from the check_papi_state() function, and
    don't mix up papi error values and pcp error values that we could
    return.  Stick to papi.

commit a031cee209de761818005d04a503c52e444d2519
Author: Lukas Berk 
Date:   Wed Sep 17 11:59:46 2014 -0400

    Rename papi_pmid function to papi_name_lookup as requested
    
    Rename function to be more descriptive

commit 9ce845cc0bb12b063d0bb5852bd662171165864c
Author: Lukas Berk 
Date:   Wed Sep 17 11:51:19 2014 -0400

    Update the papi_fetchcallback mechanism for passing metric data
    
    No longer loop through all the available events (as we generate the
    pmns dynamically now), just use the idp->item that we're passed

commit 3d14302269500a17c6055440afd181aefa4cad5b
Author: Lukas Berk 
Date:   Tue Sep 16 11:46:56 2014 -0400

    Add papi_children callback for generating the pmns dynamically
    
    Order the papi metrics under papi.system.metric (expecting expansion
    of metrics to capture pid/tid specifics later) and dynamically
    generate the list of metrics based on what's actually available on
    the current hardware.

commit 318a354e0d9e821dd52844c6023898ee596e8a86
Author: Lukas Berk 
Date:   Wed Sep 10 10:38:15 2014 -0400

    Make use of strtok in papi_pmid
    
    Use strtok in pmda_papi function for a cleaner function

commit cc30f7703a387be0726f221752b78b779d3c5742
Author: Lukas Berk 
Date:   Tue Sep 9 18:58:45 2014 -0400

    Move PAPI_event_info_t var into papi_info for metric descriptions
    
    By moving the PAPI_event_info_t varible into the papi_info struct
    we can ensure the references to the descriptions are persistent.

commit cd2b5939dff78976cd6ba454a66538c6bb99f95a
Author: Lukas Berk 
Date:   Tue Sep 9 18:26:11 2014 -0400

    First step towards a fully dynamic papipmda (including pmns)
    
    Remove the papi_info[].pmns_position variable and set_pmns_position
    fuction.  Likewise, only add papi metrics to papi_info if they're
    actually present on the current hardware.  In the future we might
    need to add in another papi.'system'.metric to make the pmns fully
    automatically generated and match with the various userspace programs

commit 8fdbf4961ccc62aab580e049c60f2457e41a68fb
Author: Lukas Berk 
Date:   Mon Sep 8 12:22:33 2014 -0400

    Rework saving previous papi values when adding/removing metrics
    
    Previously, when we added or removed a metric that we were
    tracking, we'd lose the previous values and all metrics would
    restart from zero.  By adding a prev_metric value in papi_info,
    we can systematically save the values, and add it to the atom->ull
    in the fetchcallback

commit e47e0c36dba0f6e02f4a06922d74c415f7d21a1c
Author: Lukas Berk 
Date:   Mon Sep 8 10:10:40 2014 -0400

    Be more consistent with internal use of variable names
    
    Remove all mentions of 'sts' and consistenly use retval

commit f93083d541d90ca266684aae7bfe7130cb6fb511
Author: Lukas Berk 
Date:   Mon Sep 8 09:45:47 2014 -0400

    Account for control and available clusters in papi_text
    
    papi_text was returning results from PAPI_enum_event which matched
    the pcp item number, even when the metrics were in different cluters
    (control and available, respectively).

commit 99288a413cc5a4d6bddc424794dc05131c24bc88
Author: Lukas Berk 
Date:   Fri Sep 5 15:34:56 2014 -0400

    Add debug warning and return PM_ERR_CONV to papi.control.enable
    
    Currently papi.control.enable will just silently fail if presented
    with metrics it can't add, fix that.
    
    *src/pmdas/papi/papi.c - add debug warning and return the error in
    		         papi_store for enable

commit b0cfaa1b91ef349d7ad78656bb4da57582bad434
Author: Lukas Berk 
Date:   Tue Sep 23 12:17:17 2014 -0400

    Refactor bits of {add,remove}_metric and small cleanups
    
    We can move the check to see if the event specified is valid outside
    the function and reduce the repeated code.  Also remove a loop that
    we no longer need.

commit da33dbaf8266e0374ebb3717103a4b1bedae89af
Author: Lukas Berk 
Date:   Mon Sep 22 16:16:59 2014 -0400

    papi_state returns a bitmask, so do a bitwise comparison, not direct compare.
    
    instead of a direct comparison with PAPI_ we should just do a bitwise
    AND to confirm the state

commit 7068e138a6c58aa72a787a2ddf1f484f4811b6e0
Author: Lukas Berk 
Date:   Mon Sep 22 14:50:03 2014 -0400

    Remove papi_string_status and inline the former function
    
    The aforementioned function was incorrect in using a case statement
    and assuming only one would be hit.  The papi state is a bitmask,
    and we should be able to return more than one condition.  It is
    also potentially useful to output the last known values of each active
    metric.

commit 0df3c625640676f678d20fbce479cf20aa00c17c
Author: Lukas Berk 
Date:   Mon Sep 22 14:48:45 2014 -0400

    Add checks around papi_children realloc statements
    
    realloc's need a check to ensure the allocation returned with no errors.
    If an error was found, throw __pmNoMem

commit 19863941c33e91728196d65407e894612bbb2d2b
Author: Lukas Berk 
Date:   Fri Sep 19 18:07:05 2014 -0400

    Remove internal enable and disable strings for papi metrics
    
    We shouldn't require the use of these strings to pass back and forth.

commit ab114bf05c3dc4b451ce622d8b3c4ea8e04ce368
Author: Lukas Berk 
Date:   Fri Sep 19 14:14:18 2014 -0400

    Implement papi.control.reset metric
    
    papi.control.reset will zero the current papi values being counted
    one must use papi.control.disable if they'd like the values to no
    longer be in the active eventset

commit ce7caf388e0acdc5a8a1cb088430f89f91e8dcca
Author: Lukas Berk 
Date:   Thu Sep 18 21:18:40 2014 -0400

    Remove remove second function paramter from remove_metric
    
    Both parameters being passed were values from papi_info, if we
    instead pass the specific position within the papi_info array,
    we can determine the other values we need without passing extra
    arguments

commit 916a7e0eeb7974428a6348eecb9641fd125bc6a8
Author: Lukas Berk 
Date:   Thu Sep 18 18:36:10 2014 -0400

    Assign papi help text in papi_internal_init and only lookup values in papi_text
    
    Assigning the values of {short,long}_descr can be done within papi_internal_init,
    which allows us to simply recall the values in papi_text.

commit 07e3fb0f357f13e9767f4959dbde3470d3986f6b
Author: Lukas Berk 
Date:   Wed Sep 17 18:30:20 2014 -0400

    Drop unneeded parameter from check_papi_state
    
    Drop unneeded parameter from the check_papi_state() function, and
    don't mix up papi error values and pcp error values that we could
    return.  Stick to papi.

commit ffa8067e5093943bdcbe2ee5b4b037e7081dc2fb
Author: Lukas Berk 
Date:   Wed Sep 17 11:59:46 2014 -0400

    Rename papi_pmid function to papi_name_lookup as requested
    
    Rename function to be more descriptive

commit 206caabeb687b85ec8d70a3eafc669d642e72616
Author: Lukas Berk 
Date:   Wed Sep 17 11:51:19 2014 -0400

    Update the papi_fetchcallback mechanism for passing metric data
    
    No longer loop through all the available events (as we generate the
    pmns dynamically now), just use the idp->item that we're passed

commit 132d86d035a29879b23c8e4dfde731371457a76a
Author: Lukas Berk 
Date:   Tue Sep 16 12:29:09 2014 -0400

    Add example workflow for pmdapapi to man pages
    
    pmdapapi makes use of pmstore to control what papi metrics are currently
    being tracked and which aren't.  document this workflow in the manpage

commit 27e498c4bc13c0740036b8a0bf7bdce5a457ec15
Author: Lukas Berk 
Date:   Tue Sep 16 11:46:56 2014 -0400

    Add papi_children callback for generating the pmns dynamically
    
    Order the papi metrics under papi.system.metric (expecting expansion
    of metrics to capture pid/tid specifics later) and dynamically
    generate the list of metrics based on what's actually available on
    the current hardware.

commit caab47c9c91504f952454a1a167469f846719f0a
Author: Lukas Berk 
Date:   Wed Sep 10 10:38:15 2014 -0400

    Make use of strtok in papi_pmid
    
    Use strtok in pmda_papi function for a cleaner function

commit a0701fbae9119630eec0e3ef18b1b365b499a928
Author: Lukas Berk 
Date:   Tue Sep 9 18:58:45 2014 -0400

    Move PAPI_event_info_t var into papi_info for metric descriptions
    
    By moving the PAPI_event_info_t varible into the papi_info struct
    we can ensure the references to the descriptions are persistent.

commit 4346727ebc4cf342e50d71d725dbdf66275efeaa
Author: Lukas Berk 
Date:   Tue Sep 9 18:26:11 2014 -0400

    First step towards a fully dynamic papipmda (including pmns)
    
    Remove the papi_info[].pmns_position variable and set_pmns_position
    fuction.  Likewise, only add papi metrics to papi_info if they're
    actually present on the current hardware.  In the future we might
    need to add in another papi.'system'.metric to make the pmns fully
    automatically generated and match with the various userspace programs

commit f007cbe732d1ed3e958869464338b29b47665731
Author: Lukas Berk 
Date:   Mon Sep 8 12:22:33 2014 -0400

    Rework saving previous papi values when adding/removing metrics
    
    Previously, when we added or removed a metric that we were
    tracking, we'd lose the previous values and all metrics would
    restart from zero.  By adding a prev_metric value in papi_info,
    we can systematically save the values, and add it to the atom->ull
    in the fetchcallback

commit 12ace0443b7d8b9c79501eda0642ec09ad6fbee6
Author: Lukas Berk 
Date:   Mon Sep 8 10:10:40 2014 -0400

    Be more consistent with internal use of variable names
    
    Remove all mentions of 'sts' and consistenly use retval

commit 6e25eaf9a166b1a9b0fe606d192c9d41ba3d8b6e
Author: Lukas Berk 
Date:   Mon Sep 8 09:45:47 2014 -0400

    Account for control and available clusters in papi_text
    
    papi_text was returning results from PAPI_enum_event which matched
    the pcp item number, even when the metrics were in different cluters
    (control and available, respectively).

commit 6a8ed1d7313d6767d76605773c7b1258f8f041dc
Author: Lukas Berk 
Date:   Fri Sep 5 15:34:56 2014 -0400

    Add debug warning and return PM_ERR_CONV to papi.control.enable
    
    Currently papi.control.enable will just silently fail if presented
    with metrics it can't add, fix that.
    
    *src/pmdas/papi/papi.c - add debug warning and return the error in
    		         papi_store for enable

commit 3559d567e886dd3526d568bc13a1ce001281e657
Author: Lukas Berk 
Date:   Wed Nov 5 10:43:02 2014 -0500

    tweak qa/903 to account for lower papi metric output
    
    pmdapapi only creates metrics dynamically based on the system hardware.
    This means the number of metrics by default is much lower, tweak the
    awk statement to reflect that

commit e595b254124b53cf9625b74c586a6ff6da75dade
Author: Lukas Berk 
Date:   Wed Nov 5 10:40:47 2014 -0500

    Update qa/799 with -hlocal: test against real values
    
    799 will now grab 'real' values from the current papi install.
    This testcase now also includes installing and uninstalling
    the pmda.

commit 24d6f6426f349020def7679f04850badfec39a8a
Author: Lukas Berk 
Date:   Wed Nov 5 10:34:18 2014 -0500

    Add dbpmda testcase for pmdapapi
    
    Add an additional testcase for pmdapapi, ensure a value can be
    enabled, checked, and disabled through dbpmda

commit 80ca1858f7dc284724c62de52829e9e41e9f0b24
Author: Frank Ch. Eigler 
Date:   Tue Oct 14 17:16:53 2014 -0400

    papi/pmda: collapse metric addition/removal path
    
    The add- and remove-metric operations are almost identical, with
    respect to the need to save previous counter values, pause, then
    restart with newborn/surviving metrics.  With the recent
    metric_enabled flag, the two code paths can be effectively merged.
    
    Gone are add_metric(), remove_metric(), setup_eventset(), stop_papi(),
    and the number_of_active_counters global.  Left notes for a few more
    candidates.
    
    Welcome refresh_metrics() and a collapsed-case .enable/.disable
    pmstore handler, a trivial .reset handler.
    
    Tested by hand, running pmstore ops to enable/disable multiple
    individual counters, including erroneous names, while monitoring the
    valid counters for consistent progress.  Mechanical testing
    forthcoming.

commit 7cf0569611c895b32b4b7fa8ce1c6f4625286ae6
Author: Lukas Berk 
Date:   Tue Oct 14 14:16:49 2014 -0400

    Make status_string[] variable static
    
    Avoid any type of memory corruption bug by ensuring the variable
    won't already be unwound.

commit 324ec32f62a81de3fb019bcdcd2d6f6ec34764ec
Author: Lukas Berk 
Date:   Tue Oct 14 11:27:40 2014 -0400

    Fix a papi_info.position bug in remove_metric
    
    Include a new papi.info var (metric_enabled) to track if the metric
    was enabled.  This leaves us to simply assign the position in the
    order we (re)add the metric to the eventset

commit 05c56e4f18deaa97e283260069c0bda7da2b80e9
Author: Lukas Berk 
Date:   Thu Oct 9 16:16:07 2014 -0400

    Tidy up setup_eventset()
    
    Pick a style of retval assignment/comparison and stick with it.
    Report papi errors in a uniform way

commit d351dac6de615ce4f99efb92494f2aeef50e6cb6
Author: Lukas Berk 
Date:   Thu Oct 9 16:02:56 2014 -0400

    Refactor the eventset setup which is done in papi_internal_init and remove_metric
    
    We were only partially setting up the eventset in remove_metric (due
    to working around papi bug).  Refactor this and ensure it's done properly
    both times.

commit 11b2714ece3e22989d475454218b9368e98246ec
Author: Lukas Berk 
Date:   Thu Oct 9 13:39:24 2014 -0400

    Refactor {add,remove}_metric functions
    
    We can refactor the functions to put stopping and saving the values
    in its own function, while making the return values consistent.

commit b27a384f8c4d86cf5d58ad1b53de82ddcfe63277
Author: Lukas Berk 
Date:   Mon Oct 6 19:06:18 2014 -0400

    Reduce the string manipulation functions usage in papi_internal_init
    
    By reusing the same data structures and sizes that papi defines for
    symbols we can directly memcpy the values in without having to loop
    so many times and tokenizing everything.

commit 2bb4cd668abe689bff93664a38f62b77aa632ad9
Author: Lukas Berk 
Date:   Mon Oct 6 12:20:59 2014 -0400

    Switch to local, stack allocated string for papi.control.status metric
    
    Instead of realloc'ing multiple times a call, lets instead, allocate
    one string (once) and simplify the process

commit dea6f80d9e4461b3ca97a662fdd8ae676aeee041
Author: Lukas Berk 
Date:   Mon Oct 6 09:44:56 2014 -0400

    Enable papi multiplexing
    
    Enabling multiplexing will make the pmda a bit more flexible for counting
    metrics for longer periods of time and for hardware with fewer hardware
    counters

commit caf9f5d7b18d17a21984afce469fd8d7cc9cd6a2
Author: Lukas Berk 
Date:   Fri Oct 3 15:30:56 2014 -0400

    Drop unneeded (ignored) gid == 0 check in permission_check

commit 30cee046ce73ee6b4b80245ab5a70c458e682993
Author: Lukas Berk 
Date:   Fri Oct 3 15:20:44 2014 -0400

    Remove check_event_exists function
    
    If the event requested isn't in already in papi_info, add_metric
    and remove metric won't get called at all anyways, this check is
    redundant

commit 0ba97b404e3a812272b2bc245a3b0e391f364a83
Author: Lukas Berk 
Date:   Fri Oct 3 15:08:31 2014 -0400

    Shift papi_event_code variable into papi_event_type_t var in papi_info
    
    We can reduce the number of variables we need and simply use the data
    structures already there.

commit f40f041ba7d2914e16144a146aa826bed0fa9ce1
Author: Lukas Berk 
Date:   Thu Oct 2 15:02:56 2014 -0400

    Add skeleton of new pmdapapi qa test - qa/799
    
    Base for new pmdapapi test

commit e383b905052e126952d98240b03b41c6e6c730f4
Author: Lukas Berk 
Date:   Thu Oct 2 12:04:37 2014 -0400

    Make adding metrics to the current eventset more robust
    
    If we attempt to add an event that cannot be added (for example,
    if two metrics can't be counted at the same time), we should make
    the attempt, but still restart papi and run the other metrics if
    possible.  If debugging is turned on, output the error message to
    the log file.

commit 82d970256663064938536677687de2e76f100273
Author: Lukas Berk 
Date:   Wed Oct 1 19:13:59 2014 -0400

    Change papi.control.status to include metric values and fix papi_children
    
    Several smaller nits.  papi.control.status should now include the current
    values being counted.  We also now use a __pmnsTree var to keep track
    of the dynamic metrics we've added.  This makes both papi_children much
    eaiser to manage as well as papi_name_lookup.

commit 5896a58858bc295e0d84af61c8174d40946e9cf6
Author: Lukas Berk 
Date:   Wed Oct 1 19:09:55 2014 -0400

    Update qa files for new metrics and added portion to pmns
    
    A new portion of the papi pmns includes 'system' and will extend
    to possible process names when that functionality is included.
    We also need to update qa for implementation of papi.control.reset

commit 2015402d494b0f83a7fcfc2844ab41c33d5394e5
Author: Lukas Berk 
Date:   Tue Sep 23 12:17:17 2014 -0400

    Refactor bits of {add,remove}_metric and small cleanups
    
    We can move the check to see if the event specified is valid outside
    the function and reduce the repeated code.  Also remove a loop that
    we no longer need.

commit 206b7a39d7f8a1e2f1608d29c964a30b25432584
Author: Lukas Berk 
Date:   Mon Sep 22 16:16:59 2014 -0400

    papi_state returns a bitmask, so do a bitwise comparison, not direct compare.
    
    instead of a direct comparison with PAPI_ we should just do a bitwise
    AND to confirm the state

commit 0392873d660054e111fa7b6a6caddfe3ba0afec0
Author: Lukas Berk 
Date:   Mon Sep 22 14:50:03 2014 -0400

    Remove papi_string_status and inline the former function
    
    The aforementioned function was incorrect in using a case statement
    and assuming only one would be hit.  The papi state is a bitmask,
    and we should be able to return more than one condition.  It is
    also potentially useful to output the last known values of each active
    metric.

commit bce0c258922fdbd9cf2ce5e4ce53f3956131af4f
Author: Lukas Berk 
Date:   Mon Sep 22 14:48:45 2014 -0400

    Add checks around papi_children realloc statements
    
    realloc's need a check to ensure the allocation returned with no errors.
    If an error was found, throw __pmNoMem

commit a2199cd045712baa71254a513666d8fa1a5c849e
Author: Lukas Berk 
Date:   Fri Sep 19 18:07:05 2014 -0400

    Remove internal enable and disable strings for papi metrics
    
    We shouldn't require the use of these strings to pass back and forth.

commit 1c03e37ce1b2e791accba04099dbb03c3f285800
Author: Lukas Berk 
Date:   Fri Sep 19 14:14:18 2014 -0400

    Implement papi.control.reset metric
    
    papi.control.reset will zero the current papi values being counted
    one must use papi.control.disable if they'd like the values to no
    longer be in the active eventset

commit 777c17d03c7ea9eb199a3cef717e9ab10b3b446c
Author: Lukas Berk 
Date:   Thu Sep 18 21:18:40 2014 -0400

    Remove remove second function paramter from remove_metric
    
    Both parameters being passed were values from papi_info, if we
    instead pass the specific position within the papi_info array,
    we can determine the other values we need without passing extra
    arguments

commit 777ef784fcb2b4fde2eaa00c18f11f0cfade2f74
Author: Lukas Berk 
Date:   Thu Sep 18 18:36:10 2014 -0400

    Assign papi help text in papi_internal_init and only lookup values in papi_text
    
    Assigning the values of {short,long}_descr can be done within papi_internal_init,
    which allows us to simply recall the values in papi_text.

commit 32f8a2b75100704f7483acc6f1ab4447653f4fa2
Author: Lukas Berk 
Date:   Wed Sep 17 18:30:20 2014 -0400

    Drop unneeded parameter from check_papi_state
    
    Drop unneeded parameter from the check_papi_state() function, and
    don't mix up papi error values and pcp error values that we could
    return.  Stick to papi.

commit 500c02b58947d4f2fd8162c46d7d06fe5bfa84e6
Author: Lukas Berk 
Date:   Wed Sep 17 11:59:46 2014 -0400

    Rename papi_pmid function to papi_name_lookup as requested
    
    Rename function to be more descriptive

commit c20b649ad6b0a71b39d71d4df42f43a748d8e499
Author: Lukas Berk 
Date:   Wed Sep 17 11:51:19 2014 -0400

    Update the papi_fetchcallback mechanism for passing metric data
    
    No longer loop through all the available events (as we generate the
    pmns dynamically now), just use the idp->item that we're passed

commit 97b802e0414eb6de909226ebe02a8926e6d7813d
Author: Lukas Berk 
Date:   Tue Sep 16 12:29:09 2014 -0400

    Add example workflow for pmdapapi to man pages
    
    pmdapapi makes use of pmstore to control what papi metrics are currently
    being tracked and which aren't.  document this workflow in the manpage

commit d1c1bbf599f5ed9c975c4df89e19987e76e23fd0
Author: Lukas Berk 
Date:   Tue Sep 16 11:46:56 2014 -0400

    Add papi_children callback for generating the pmns dynamically
    
    Order the papi metrics under papi.system.metric (expecting expansion
    of metrics to capture pid/tid specifics later) and dynamically
    generate the list of metrics based on what's actually available on
    the current hardware.

commit cbf66cd8e33ca6751107a41a8af399cef841c1cf
Author: Lukas Berk 
Date:   Wed Sep 10 10:38:15 2014 -0400

    Make use of strtok in papi_pmid
    
    Use strtok in pmda_papi function for a cleaner function

commit 7bee26c385707808cb3dd28dbf41d02cdcb94beb
Author: Lukas Berk 
Date:   Tue Sep 9 18:58:45 2014 -0400

    Move PAPI_event_info_t var into papi_info for metric descriptions
    
    By moving the PAPI_event_info_t varible into the papi_info struct
    we can ensure the references to the descriptions are persistent.

commit bfff8127f524dfa8a84183326ef9aa551d21a1c6
Author: Lukas Berk 
Date:   Tue Sep 9 18:26:11 2014 -0400

    First step towards a fully dynamic papipmda (including pmns)
    
    Remove the papi_info[].pmns_position variable and set_pmns_position
    fuction.  Likewise, only add papi metrics to papi_info if they're
    actually present on the current hardware.  In the future we might
    need to add in another papi.'system'.metric to make the pmns fully
    automatically generated and match with the various userspace programs

commit f17a856a6e095b4c7fe528fd8bcd237c4a6e5f6a
Author: Lukas Berk 
Date:   Mon Sep 8 12:22:33 2014 -0400

    Rework saving previous papi values when adding/removing metrics
    
    Previously, when we added or removed a metric that we were
    tracking, we'd lose the previous values and all metrics would
    restart from zero.  By adding a prev_metric value in papi_info,
    we can systematically save the values, and add it to the atom->ull
    in the fetchcallback

commit 374ccd7cb0839b1303e93cc1ec40e7be0bf819e0
Author: Lukas Berk 
Date:   Mon Sep 8 10:10:40 2014 -0400

    Be more consistent with internal use of variable names
    
    Remove all mentions of 'sts' and consistenly use retval

commit 215a7292a5b68a3dd2a8d40b6085dd306bc261da
Author: Lukas Berk 
Date:   Mon Sep 8 09:45:47 2014 -0400

    Account for control and available clusters in papi_text
    
    papi_text was returning results from PAPI_enum_event which matched
    the pcp item number, even when the metrics were in different cluters
    (control and available, respectively).

commit 19f099f9ce57d5e7c95cd2ba73cc47fa19b5e179
Author: Lukas Berk 
Date:   Fri Sep 5 15:34:56 2014 -0400

    Add debug warning and return PM_ERR_CONV to papi.control.enable
    
    Currently papi.control.enable will just silently fail if presented
    with metrics it can't add, fix that.
    
    *src/pmdas/papi/papi.c - add debug warning and return the error in
    		         papi_store for enable

commit fd3a9845a32ee4208933dacf557a2094cfd192cf
Author: Lukas Berk 
Date:   Tue Sep 23 12:17:17 2014 -0400

    Refactor bits of {add,remove}_metric and small cleanups
    
    We can move the check to see if the event specified is valid outside
    the function and reduce the repeated code.  Also remove a loop that
    we no longer need.

commit 474e56808984577e1ca82f0ad6dc345d3746ac37
Author: Lukas Berk 
Date:   Mon Sep 22 16:16:59 2014 -0400

    papi_state returns a bitmask, so do a bitwise comparison, not direct compare.
    
    instead of a direct comparison with PAPI_ we should just do a bitwise
    AND to confirm the state

commit de04a53f8de921df2e1f619c4247ad606b601ebe
Author: Lukas Berk 
Date:   Mon Sep 22 14:50:03 2014 -0400

    Remove papi_string_status and inline the former function
    
    The aforementioned function was incorrect in using a case statement
    and assuming only one would be hit.  The papi state is a bitmask,
    and we should be able to return more than one condition.  It is
    also potentially useful to output the last known values of each active
    metric.

commit 580f2ad7963de915d960b19c956baea9fab0705b
Author: Lukas Berk 
Date:   Mon Sep 22 14:48:45 2014 -0400

    Add checks around papi_children realloc statements
    
    realloc's need a check to ensure the allocation returned with no errors.
    If an error was found, throw __pmNoMem

commit 16b12fa51182875e53e5bad433a8e0f534b02ade
Author: Lukas Berk 
Date:   Fri Sep 19 18:07:05 2014 -0400

    Remove internal enable and disable strings for papi metrics
    
    We shouldn't require the use of these strings to pass back and forth.

commit 25298203c1e4eff8ea2372354058e54c26891518
Author: Lukas Berk 
Date:   Fri Sep 19 14:14:18 2014 -0400

    Implement papi.control.reset metric
    
    papi.control.reset will zero the current papi values being counted
    one must use papi.control.disable if they'd like the values to no
    longer be in the active eventset

commit 9e96e2eea29afc1a08c473e124cf72ac767a5f30
Author: Lukas Berk 
Date:   Thu Sep 18 21:18:40 2014 -0400

    Remove remove second function paramter from remove_metric
    
    Both parameters being passed were values from papi_info, if we
    instead pass the specific position within the papi_info array,
    we can determine the other values we need without passing extra
    arguments

commit 9da54f9c9eaf89d4742891566f895d0d39d58f3f
Author: Lukas Berk 
Date:   Thu Sep 18 18:36:10 2014 -0400

    Assign papi help text in papi_internal_init and only lookup values in papi_text
    
    Assigning the values of {short,long}_descr can be done within papi_internal_init,
    which allows us to simply recall the values in papi_text.

commit 2d3196a704f325444f1ea533a809bbfadac4fcdc
Author: Lukas Berk 
Date:   Wed Sep 17 18:30:20 2014 -0400

    Drop unneeded parameter from check_papi_state
    
    Drop unneeded parameter from the check_papi_state() function, and
    don't mix up papi error values and pcp error values that we could
    return.  Stick to papi.

commit 8d679bb5f262605f5e482305984d1bf97e1356ac
Author: Lukas Berk 
Date:   Wed Sep 17 11:59:46 2014 -0400

    Rename papi_pmid function to papi_name_lookup as requested
    
    Rename function to be more descriptive

commit 3e0ba1b4cd6aa0c859a6c40381e6674a2b4e8972
Author: Lukas Berk 
Date:   Wed Sep 17 11:51:19 2014 -0400

    Update the papi_fetchcallback mechanism for passing metric data
    
    No longer loop through all the available events (as we generate the
    pmns dynamically now), just use the idp->item that we're passed

commit 24e2881a0e9d9d624587d92bd667dcc4cee1fda5
Author: Lukas Berk 
Date:   Tue Sep 16 12:29:09 2014 -0400

    Add example workflow for pmdapapi to man pages
    
    pmdapapi makes use of pmstore to control what papi metrics are currently
    being tracked and which aren't.  document this workflow in the manpage

commit 810cfad58be60f0c24d920e85ee532f60edf40ba
Author: Lukas Berk 
Date:   Tue Sep 16 11:51:42 2014 -0400

    Add spec file condition on architecture and distro for pmdapapi
    
    Don't build pmda papi if on a version prior to rhel6 or on
    s390/s390x.  The papi-devel package doesn't exist there.

commit 8006bf0c6fc17cff7e0ecd577e2d8f5940dd8ea1
Author: Lukas Berk 
Date:   Tue Sep 16 11:46:56 2014 -0400

    Add papi_children callback for generating the pmns dynamically
    
    Order the papi metrics under papi.system.metric (expecting expansion
    of metrics to capture pid/tid specifics later) and dynamically
    generate the list of metrics based on what's actually available on
    the current hardware.

commit 54420037c538883c8a88d53eebbf5a4da05e00c4
Author: Lukas Berk 
Date:   Wed Sep 10 10:38:15 2014 -0400

    Make use of strtok in papi_pmid
    
    Use strtok in pmda_papi function for a cleaner function

commit bc72dc53be4952234de9013ed7ff2bd276cec033
Author: Lukas Berk 
Date:   Tue Sep 9 18:58:45 2014 -0400

    Move PAPI_event_info_t var into papi_info for metric descriptions
    
    By moving the PAPI_event_info_t varible into the papi_info struct
    we can ensure the references to the descriptions are persistent.

commit 75e26486fc52a5fad8feb75ec31daf5c4f45cea2
Author: Lukas Berk 
Date:   Tue Sep 9 18:26:11 2014 -0400

    First step towards a fully dynamic papipmda (including pmns)
    
    Remove the papi_info[].pmns_position variable and set_pmns_position
    fuction.  Likewise, only add papi metrics to papi_info if they're
    actually present on the current hardware.  In the future we might
    need to add in another papi.'system'.metric to make the pmns fully
    automatically generated and match with the various userspace programs

commit 2b92cc4e3e34992ac9eb25d613dcbcc53d78e4ea
Author: Lukas Berk 
Date:   Mon Sep 8 12:22:33 2014 -0400

    Rework saving previous papi values when adding/removing metrics
    
    Previously, when we added or removed a metric that we were
    tracking, we'd lose the previous values and all metrics would
    restart from zero.  By adding a prev_metric value in papi_info,
    we can systematically save the values, and add it to the atom->ull
    in the fetchcallback

commit 159bf50dac441f4ac6383a9bd4632f1b34fc8cd5
Author: Lukas Berk 
Date:   Mon Sep 8 10:10:40 2014 -0400

    Be more consistent with internal use of variable names
    
    Remove all mentions of 'sts' and consistenly use retval

commit 78a5f96ac6714eafba4faa4cbaad982883620bd7
Author: Lukas Berk 
Date:   Mon Sep 8 09:45:47 2014 -0400

    Account for control and available clusters in papi_text
    
    papi_text was returning results from PAPI_enum_event which matched
    the pcp item number, even when the metrics were in different cluters
    (control and available, respectively).

commit bc295db9a4e7b2bf5382b5b2cae3851b24829eac
Author: Lukas Berk 
Date:   Fri Sep 5 15:34:56 2014 -0400

    Add debug warning and return PM_ERR_CONV to papi.control.enable
    
    Currently papi.control.enable will just silently fail if presented
    with metrics it can't add, fix that.
    
    *src/pmdas/papi/papi.c - add debug warning and return the error in
    		         papi_store for enable

From nscott@redhat.com  Thu Nov 13 00:02:19 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 26E3F7F3F
	for ; Thu, 13 Nov 2014 00:02:19 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 9553FAC00D
	for ; Wed, 12 Nov 2014 22:02:15 -0800 (PST)
X-ASG-Debug-ID: 1415858530-04cb6c1e6d25d920001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id a2QtWbBc20jtomTg (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 12 Nov 2014 22:02:10 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAD6280F018991;
	Thu, 13 Nov 2014 01:02:08 -0500
Date: Thu, 13 Nov 2014 01:02:08 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Martins Innus 
Cc: pcp 
Message-ID: <1453660478.13246038.1415858528343.JavaMail.zimbra@redhat.com>
In-Reply-To: <5463C8C8.5010700@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54627B7D.6040906@buffalo.edu> <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com> <5463C8C8.5010700@buffalo.edu>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: FFCCSZQxeRXXv3ZNrmzx863MOKnnPw==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415858530
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11543
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi Martins,

----- Original Message -----
> Nathan,
> 
>      Sorry for the long email below.  I have become completely confused
> with dynamic metrics and it seems like I'm trying to use the API in a
> way that was not intended.  #2 below is the most troubling.  You likely
> want to junk all my dynamic proc code until we get this figured out.

No problem.

> 
> Just realized this is incomplete. Sorry, will need some more work and
> you shouldn't use it.

S'ok (I've been tied up with QA duties and cgroups, nearly done and I'll
be able to help you out more here).

> Still no progress on this, except that with a dso pmda, the domain
> number is correct at this point. Illustrated below in the interrupts case.

Looks like a bug - in the cgroups case, the domain# is explicitly
stamped into the PMID for the new metric table entry each time, so
I suspect we've never seen this before as a result.

> While researching this, I've run into a few issues that stem from
> calling pmdaDynamicPMNS multiple times:
> 
> 1. In the "size_metrictable" callback there is no context as to which
> dynamic tree is being queried if we use the same callback for all trees.

(*nod* - use different callbacks?)  Thats the old cgroups model anyway.
The PMID cluster# was the point of division used there - each cluster is
managed separately IIRC.

> So this is going to way overestimate the storage space needed, since we
> need to return the size of the largest sub tree if we use a common
> callback.  The alternative is one callback per sub tree as far as I can see.

Yep.  And that's OK - there's not many proc.* sub-trees (4/5?).

> 2. This is a larger problem. In dynamic.c, for the function:
> 
> static pmdaMetric *
> dynamic_metric_table(int index, pmdaMetric *offset)
> 
>      When running through the initial metric table, the only check that
> is done is to see if the cluster matches when doing "mtabupdate" and no
> check is done against the item or name.  This will cause multiple calls
> for the same metric if we have clusters that span multiple trees (since

If we take away "clusters spanning trees" this all gets simpler I think?
Am I oversimplifying things there?  (apologies if so - I'll look at it
more closely as soon as I get this cgroup stuff finished).

> 3. In general, its not clear to me the proper way to generate the
> dynamic metrics from the initial "templates". For example, take the

Yeah, the code here is pretty freaky and non-obvious as I recall.

>      /* kernel.percpu.interrupts.line[] */
>      { NULL, { PMDA_PMID(CLUSTER_INTERRUPT_LINES, 0), PM_TYPE_U32,
>      CPU_INDOM, PM_SEM_COUNTER, PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) }, },
>  
>      /* kernel.percpu.interrupts.[] */
>      { NULL, { PMDA_PMID(CLUSTER_INTERRUPT_OTHER, 0), PM_TYPE_U32,
>      CPU_INDOM, PM_SEM_COUNTER, PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) }, },
> 
> These are used as templates to create the dynamic metrics. But
> interrupts.c provides in "size_metrictable" the total number of dynamic
> metrics, so we end up with at least 2 extra metrics which as far as I
> can tell are not used.

That was not the original intention, and sounds like a bug - it was
intended this would be allocating the correct amount of space, but
it looks like there's a few extras.  It'd be good to dig more deeply
into that and understand why.

> So both of the .11 and one of the .10 metrics is orphaned?  Does that
> cause any issue?  Or since the pmns doesn't reference them, no harm
> except memory used.  If that's the case,  for my issue #2 above I can

Yeah, I think that's the case.

> just set dummy cluster or item values for the metrics that are sent
> through twice(or more), but that's pretty wasteful and I'm sure will
> have other unintended consequences.

Not that I've seen to date, but yep, we should certainly fix that up.

cheers.

--
Nathan

From nscott@redhat.com  Thu Nov 13 00:47:48 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 1A4C97F3F
	for ; Thu, 13 Nov 2014 00:47:48 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id E587B304066
	for ; Wed, 12 Nov 2014 22:47:47 -0800 (PST)
X-ASG-Debug-ID: 1415861264-04cb6c1e6b25f030001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id YWVi4ZSP8EkAnOLm (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 12 Nov 2014 22:47:45 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAD6lis5032750;
	Thu, 13 Nov 2014 01:47:44 -0500
Date: Thu, 13 Nov 2014 01:47:44 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Lukas Berk 
Cc: pcp@oss.sgi.com
Message-ID: <462023254.13251879.1415861264683.JavaMail.zimbra@redhat.com>
In-Reply-To: <87oascow3f.fsf@redhat.com>
References: <87oascow3f.fsf@redhat.com>
Subject: Re: [pcp] pcp updates - pmdapapi update
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] pcp updates - pmdapapi update
Content-Type: multipart/mixed; 
	boundary="----=_Part_13251874_859187345.1415861264678"
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates - pmdapapi update
Thread-Index: tOjqqkosFUWxxumJxyXycczOuCveQA==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1415861265
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11544
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

------=_Part_13251874_859187345.1415861264678
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Hi Lukas,

----- Original Message -----
> [...]
> The PAPI pmda has gone through a fair bit of work.  The changes can be

Fabulous - awesome effort esp. on the QA front.  I ran out of time to
review it all today (maybe someone else will?), but did sneak a quick
background QA run in on RHEL 6 today.  I'm seeing a few new failures
there - see attached .bad files - any ideas on possible root causes?

Only other general piece of advice I can offer would be "release early,
release often" - the first commit here is >1 month old, and it probably
coulda been merged right away?  *shrug* ... either way is fine, but I'd
go for quicker, smaller merges every day.

> [...]
> commit bc295db9a4e7b2bf5382b5b2cae3851b24829eac
> Author: Lukas Berk 
> Date:   Fri Sep 5 15:34:56 2014 -0400
> 
>     Add debug warning and return PM_ERR_CONV to papi.control.enable
>     
>     Currently papi.control.enable will just silently fail if presented
>     with metrics it can't add, fix that.
>     
>     *src/pmdas/papi/papi.c - add debug warning and return the error in
>     		         papi_store for enable
> 

cheers.

--
Nathan

------=_Part_13251874_859187345.1415861264678
Content-Type: application/octet-stream; name=967.out.bad
Content-Disposition: attachment; filename=967.out.bad
Content-Transfer-Encoding: base64

UUEgb3V0cHV0IGNyZWF0ZWQgYnkgOTY3Cj09PSBEYWVtb24gUE1EQSBwYXBpIGF1dG9fZW5hYmxl
IGRpc2FibGVkIHRlc3QgPT09CmRicG1kYT4gb3BlbiBwaXBlIC92YXIvbGliL3BjcC9wbWRhcy9w
YXBpL3BtZGFwYXBpIC1kIDEyNgpTdGFydCBwbWRhcGFwaSBQTURBOiAvdmFyL2xpYi9wY3AvcG1k
YXMvcGFwaS9wbWRhcGFwaSAtZCAxMjYKZGJwbWRhPiBnZXRkZXNjIG9uCmRicG1kYT4gYXR0ciAi
dXNlcm5hbWUiICJyb290IgpBdHRyaWJ1dGU6IHVzZXJuYW1lPXJvb3QKU3VjY2VzcwpkYnBtZGE+
IGF0dHIgMTEgIjAiCkF0dHJpYnV0ZTogdXNlcmlkPTAKU3VjY2VzcwpkYnBtZGE+IGRlc2MgcGFw
aS5zeXN0ZW0uVE9UX0lOUwpQTUlEOiAxMjYuMC4wCiAgICBEYXRhIFR5cGU6IDY0LWJpdCB1bnNp
Z25lZCBpbnQgIEluRG9tOiBQTV9JTkRPTV9OVUxMIEFERFIKICAgIFNlbWFudGljczogY291bnRl
ciAgVW5pdHM6IG5vbmUKZGJwbWRhPiBkZXNjIHBhcGkuY29udHJvbC5zdGF0dXMKUE1JRDogMTI2
LjEuMwogICAgRGF0YSBUeXBlOiBzdHJpbmcgIEluRG9tOiBQTV9JTkRPTV9OVUxMIEFERFIKICAg
IFNlbWFudGljczogaW5zdGFudCAgVW5pdHM6IG5vbmUKZGJwbWRhPiBkZXNjIHBhcGkuY29udHJv
bC5hdXRvX2VuYWJsZQpQTUlEOiAxMjYuMS40CiAgICBEYXRhIFR5cGU6IDMyLWJpdCB1bnNpZ25l
ZCBpbnQgIEluRG9tOiBQTV9JTkRPTV9OVUxMIEFERFIKICAgIFNlbWFudGljczogZGlzY3JldGUg
IFVuaXRzOiBzZWMKZGJwbWRhPiBzdG9yZSBwYXBpLmNvbnRyb2wuYXV0b19lbmFibGUgIjAiClBN
SUQ6IDEyNi4xLjQKR2V0dGluZyBkZXNjcmlwdGlvbi4uLgpTZW5kaW5nIFByb2ZpbGUuLi4KR2V0
dGluZyBSZXN1bHQgU3RydWN0dXJlLi4uCjEyNi4xLjQ6IDEyMCAtPiAwClNlbmRpbmcgUmVzdWx0
Li4uCmRicG1kYT4gZmV0Y2ggcGFwaS5zeXN0ZW0uVE9UX0lOUwpQTUlEKHMpOiAxMjYuMC4wCnBt
UmVzdWx0IGR1bXAgZnJvbSBBRERSIHRpbWVzdGFtcDogMC4wMDAwMDAgVElNRSBudW1wbWlkOiAx
CiAgMTI2LjAuMCAoPG5vbmFtZT4pOiBObyB2YWx1ZXMgcmV0dXJuZWQhCmRicG1kYT4gc3RvcmUg
cGFwaS5jb250cm9sLmVuYWJsZSAiVE9UX0lOUyIKUE1JRDogMTI2LjEuMApHZXR0aW5nIGRlc2Ny
aXB0aW9uLi4uCkdldHRpbmcgUmVzdWx0IFN0cnVjdHVyZS4uLgoxMjYuMS4wOiAiIiAtPiAiVE9U
X0lOUyIKU2VuZGluZyBSZXN1bHQuLi4KZGJwbWRhPiBmZXRjaCBwYXBpLnN5c3RlbS5UT1RfSU5T
ClBNSUQocyk6IDEyNi4wLjAKcG1SZXN1bHQgZHVtcCBmcm9tIEFERFIgdGltZXN0YW1wOiAwLjAw
MDAwMCBUSU1FIG51bXBtaWQ6IDEKICAxMjYuMC4wICg8bm9uYW1lPik6IG51bXZhbDogMSB2YWxm
bXQ6IDEgdmxpc3RbXToKICAgdmFsdWUgTlVNQkVSCmRicG1kYT4gc3RvcmUgcGFwaS5jb250cm9s
LmRpc2FibGUgIlRPVF9JTlMiClBNSUQ6IDEyNi4xLjIKR2V0dGluZyBkZXNjcmlwdGlvbi4uLgpH
ZXR0aW5nIFJlc3VsdCBTdHJ1Y3R1cmUuLi4KMTI2LjEuMjogIiIgLT4gIlRPVF9JTlMiClNlbmRp
bmcgUmVzdWx0Li4uCmRicG1kYT4gZmV0Y2ggcGFwaS5zeXN0ZW0uVE9UX0lOUwpQTUlEKHMpOiAx
MjYuMC4wCnBtUmVzdWx0IGR1bXAgZnJvbSBBRERSIHRpbWVzdGFtcDogMC4wMDAwMDAgVElNRSBu
dW1wbWlkOiAxCiAgMTI2LjAuMCAoPG5vbmFtZT4pOiBObyB2YWx1ZXMgcmV0dXJuZWQhCmRicG1k
YT4gCj09PSBEYWVtb24gUE1EQSBwYXBpIGF1dG9fZW5hYmxlIHRlc3QgPT09CmRicG1kYT4gb3Bl
biBwaXBlIC92YXIvbGliL3BjcC9wbWRhcy9wYXBpL3BtZGFwYXBpIC1kIDEyNgpTdGFydCBwbWRh
cGFwaSBQTURBOiAvdmFyL2xpYi9wY3AvcG1kYXMvcGFwaS9wbWRhcGFwaSAtZCAxMjYKZGJwbWRh
PiBnZXRkZXNjIG9uCmRicG1kYT4gYXR0ciAidXNlcm5hbWUiICJyb290IgpBdHRyaWJ1dGU6IHVz
ZXJuYW1lPXJvb3QKU3VjY2VzcwpkYnBtZGE+IGF0dHIgMTEgIjAiCkF0dHJpYnV0ZTogdXNlcmlk
PTAKU3VjY2VzcwpkYnBtZGE+IGRlc2MgcGFwaS5zeXN0ZW0uVE9UX0lOUwpQTUlEOiAxMjYuMC4w
CiAgICBEYXRhIFR5cGU6IDY0LWJpdCB1bnNpZ25lZCBpbnQgIEluRG9tOiBQTV9JTkRPTV9OVUxM
IEFERFIKICAgIFNlbWFudGljczogY291bnRlciAgVW5pdHM6IG5vbmUKZGJwbWRhPiBzdG9yZSBw
YXBpLmNvbnRyb2wuYXV0b19lbmFibGUgIjUiClBNSUQ6IDEyNi4xLjQKR2V0dGluZyBkZXNjcmlw
dGlvbi4uLgpTZW5kaW5nIFByb2ZpbGUuLi4KR2V0dGluZyBSZXN1bHQgU3RydWN0dXJlLi4uCjEy
Ni4xLjQ6IDEyMCAtPiA1ClNlbmRpbmcgUmVzdWx0Li4uCmRicG1kYT4gZmV0Y2ggcGFwaS5zeXN0
ZW0uVE9UX0lOUwpQTUlEKHMpOiAxMjYuMC4wCnBtUmVzdWx0IGR1bXAgZnJvbSBBRERSIHRpbWVz
dGFtcDogMC4wMDAwMDAgVElNRSBudW1wbWlkOiAxCiAgMTI2LjAuMCAoPG5vbmFtZT4pOiBObyB2
YWx1ZXMgcmV0dXJuZWQhCmRicG1kYT4gZmV0Y2ggcGFwaS5jb250cm9sLnN0YXR1cwpQTUlEKHMp
OiAxMjYuMS4zCnBtUmVzdWx0IGR1bXAgZnJvbSBBRERSIHRpbWVzdGFtcDogMC4wMDAwMDAgVElN
RSBudW1wbWlkOiAxCiAgMTI2LjEuMyAocGFwaS5jb250cm9sLnN0YXR1cyk6IG51bXZhbDogMSB2
YWxmbXQ6IDEgdmxpc3RbXToKICAgdmFsdWUgIlBhcGkgaXMgcnVubmluZywgaGFzIG11bHRpcGxl
eGluZyBlbmFibGVkLCBUT1RfSU5TKDUpIE5VTUJFUiIKZGJwbWRhPiB3YWl0IDcKZGJwbWRhPiBm
ZXRjaCBwYXBpLmNvbnRyb2wuc3RhdHVzClBNSUQocyk6IDEyNi4xLjMKcG1SZXN1bHQgZHVtcCBm
cm9tIEFERFIgdGltZXN0YW1wOiAwLjAwMDAwMCBUSU1FIG51bXBtaWQ6IDEKICAxMjYuMS4zIChw
YXBpLmNvbnRyb2wuc3RhdHVzKTogbnVtdmFsOiAxIHZhbGZtdDogMSB2bGlzdFtdOgogICB2YWx1
ZSAiUGFwaSBpcyBzdG9wcGVkLCBoYXMgbXVsdGlwbGV4aW5nIGVuYWJsZWQsICIKZGJwbWRhPiAK
PT09IERhZW1vbiBQTURBIHBhcGkgYXV0b19lbmFibGUgd2l0aCBwYXBpLmNvbnRyb2wuZW5hYmxl
IHRlc3QgPT09CmRicG1kYT4gb3BlbiBwaXBlIC92YXIvbGliL3BjcC9wbWRhcy9wYXBpL3BtZGFw
YXBpIC1kIDEyNgpTdGFydCBwbWRhcGFwaSBQTURBOiAvdmFyL2xpYi9wY3AvcG1kYXMvcGFwaS9w
bWRhcGFwaSAtZCAxMjYKZGJwbWRhPiBnZXRkZXNjIG9uCmRicG1kYT4gYXR0ciAidXNlcm5hbWUi
ICJyb290IgpBdHRyaWJ1dGU6IHVzZXJuYW1lPXJvb3QKU3VjY2VzcwpkYnBtZGE+IGF0dHIgMTEg
IjAiCkF0dHJpYnV0ZTogdXNlcmlkPTAKU3VjY2VzcwpkYnBtZGE+IGRlc2MgcGFwaS5zeXN0ZW0u
VE9UX0lOUwpQTUlEOiAxMjYuMC4wCiAgICBEYXRhIFR5cGU6IDY0LWJpdCB1bnNpZ25lZCBpbnQg
IEluRG9tOiBQTV9JTkRPTV9OVUxMIEFERFIKICAgIFNlbWFudGljczogY291bnRlciAgVW5pdHM6
IG5vbmUKZGJwbWRhPiBzdG9yZSBwYXBpLmNvbnRyb2wuYXV0b19lbmFibGUgIjUiClBNSUQ6IDEy
Ni4xLjQKR2V0dGluZyBkZXNjcmlwdGlvbi4uLgpTZW5kaW5nIFByb2ZpbGUuLi4KR2V0dGluZyBS
ZXN1bHQgU3RydWN0dXJlLi4uCjEyNi4xLjQ6IDEyMCAtPiA1ClNlbmRpbmcgUmVzdWx0Li4uCmRi
cG1kYT4gc3RvcmUgcGFwaS5jb250cm9sLmVuYWJsZSAiVE9UX0lOUyIKUE1JRDogMTI2LjEuMApH
ZXR0aW5nIGRlc2NyaXB0aW9uLi4uCkdldHRpbmcgUmVzdWx0IFN0cnVjdHVyZS4uLgoxMjYuMS4w
OiAiIiAtPiAiVE9UX0lOUyIKU2VuZGluZyBSZXN1bHQuLi4KZGJwbWRhPiBmZXRjaCBwYXBpLnN5
c3RlbS5UT1RfSU5TClBNSUQocyk6IDEyNi4wLjAKcG1SZXN1bHQgZHVtcCBmcm9tIEFERFIgdGlt
ZXN0YW1wOiAwLjAwMDAwMCBUSU1FIG51bXBtaWQ6IDEKICAxMjYuMC4wICg8bm9uYW1lPik6IG51
bXZhbDogMSB2YWxmbXQ6IDEgdmxpc3RbXToKICAgdmFsdWUgTlVNQkVSCmRicG1kYT4gZmV0Y2gg
cGFwaS5jb250cm9sLnN0YXR1cwpQTUlEKHMpOiAxMjYuMS4zCnBtUmVzdWx0IGR1bXAgZnJvbSBB
RERSIHRpbWVzdGFtcDogMC4wMDAwMDAgVElNRSBudW1wbWlkOiAxCiAgMTI2LjEuMyAocGFwaS5j
b250cm9sLnN0YXR1cyk6IG51bXZhbDogMSB2YWxmbXQ6IDEgdmxpc3RbXToKICAgdmFsdWUgIlBh
cGkgaXMgcnVubmluZywgaGFzIG11bHRpcGxleGluZyBlbmFibGVkLCBUT1RfSU5TKC0xKSBOVU1C
RVIiCmRicG1kYT4gd2FpdCA3CmRicG1kYT4gZmV0Y2ggcGFwaS5jb250cm9sLnN0YXR1cwpQTUlE
KHMpOiAxMjYuMS4zCnBtUmVzdWx0IGR1bXAgZnJvbSBBRERSIHRpbWVzdGFtcDogMC4wMDAwMDAg
VElNRSBudW1wbWlkOiAxCiAgMTI2LjEuMyAocGFwaS5jb250cm9sLnN0YXR1cyk6IG51bXZhbDog
MSB2YWxmbXQ6IDEgdmxpc3RbXToKICAgdmFsdWUgIlBhcGkgaXMgcnVubmluZywgaGFzIG11bHRp
cGxleGluZyBlbmFibGVkLCBUT1RfSU5TKC0xKSBOVU1CRVIiCmRicG1kYT4gc3RvcmUgcGFwaS5j
b250cm9sLmRpc2FibGUgIlRPVF9JTlMiClBNSUQ6IDEyNi4xLjIKR2V0dGluZyBkZXNjcmlwdGlv
bi4uLgpHZXR0aW5nIFJlc3VsdCBTdHJ1Y3R1cmUuLi4KMTI2LjEuMjogIiIgLT4gIlRPVF9JTlMi
ClNlbmRpbmcgUmVzdWx0Li4uCmRicG1kYT4gZmV0Y2ggcGFwaS5jb250cm9sLnN0YXR1cwpQTUlE
KHMpOiAxMjYuMS4zCnBtUmVzdWx0IGR1bXAgZnJvbSBBRERSIHRpbWVzdGFtcDogMC4wMDAwMDAg
VElNRSBudW1wbWlkOiAxCiAgMTI2LjEuMyAocGFwaS5jb250cm9sLnN0YXR1cyk6IG51bXZhbDog
MSB2YWxmbXQ6IDEgdmxpc3RbXToKICAgdmFsdWUgIlBhcGkgaXMgc3RvcHBlZCwgaGFzIG11bHRp
cGxleGluZyBlbmFibGVkLCAiCmRicG1kYT4gCg==
------=_Part_13251874_859187345.1415861264678
Content-Type: application/octet-stream; name=903.out.bad
Content-Disposition: attachment; filename=903.out.bad
Content-Transfer-Encoding: base64

UUEgb3V0cHV0IGNyZWF0ZWQgYnkgOTAzCldhaXRpbmcgZm9yIHBtY2QgdG8gdGVybWluYXRlIC4u
LgoKPT09IFBBUEkgYWdlbnQgaW5zdGFsbGF0aW9uID09PQpZb3Ugd2lsbCBuZWVkIHRvIGNob29z
ZSBhbiBhcHByb3ByaWF0ZSBjb25maWd1cmF0aW9uIGZvciBpbnN0YWxsYXRpb24gb2YKdGhlICJw
YXBpIiBQZXJmb3JtYW5jZSBNZXRyaWNzIERvbWFpbiBBZ2VudCAoUE1EQSkuCgogIGNvbGxlY3Rv
cgljb2xsZWN0IHBlcmZvcm1hbmNlIHN0YXRpc3RpY3Mgb24gdGhpcyBzeXN0ZW0KICBtb25pdG9y
CWFsbG93IHRoaXMgc3lzdGVtIHRvIG1vbml0b3IgbG9jYWwgYW5kL29yIHJlbW90ZSBzeXN0ZW1z
CiAgYm90aAkJY29sbGVjdG9yIGFuZCBtb25pdG9yIGNvbmZpZ3VyYXRpb24gZm9yIHRoaXMgc3lz
dGVtCgpQbGVhc2UgZW50ZXIgYyhvbGxlY3Rvcikgb3IgbShvbml0b3IpIG9yIGIob3RoKSBbYl0g
VXBkYXRpbmcgdGhlIFBlcmZvcm1hbmNlIE1ldHJpY3MgTmFtZSBTcGFjZSAoUE1OUykgLi4uClRl
cm1pbmF0ZSBQTURBIGlmIGFscmVhZHkgaW5zdGFsbGVkIC4uLgpbLi4uaW5zdGFsbCBmaWxlcywg
bWFrZSBvdXRwdXQuLi5dClVwZGF0aW5nIHRoZSBQTUNEIGNvbnRyb2wgZmlsZSwgYW5kIG5vdGlm
eWluZyBQTUNEIC4uLgpTdGFydGluZyBwbWNkIC4uLiAKU3RhcnRpbmcgcG1sb2dnZXIgLi4uIApD
aGVjayBwYXBpIG1ldHJpY3MgaGF2ZSBhcHBlYXJlZCAuLi4gOSBtZXRyaWNzIGFuZCBZIHZhbHVl
cwoKPT09IHJlbW92ZSBQQVBJIGFnZW50ID09PQpDdWxsaW5nIHRoZSBQZXJmb3JtYW5jZSBNZXRy
aWNzIE5hbWUgU3BhY2UgLi4uCnBhcGkgLi4uIGRvbmUKVXBkYXRpbmcgdGhlIFBNQ0QgY29udHJv
bCBmaWxlLCBhbmQgbm90aWZ5aW5nIFBNQ0QgLi4uClsuLi5yZW1vdmluZyBmaWxlcy4uLl0KQ2hl
Y2sgcGFwaSBtZXRyaWNzIGhhdmUgZ29uZSBhd2F5IC4uLiBPSwpXYWl0aW5nIGZvciBwbWNkIHRv
IHRlcm1pbmF0ZSAuLi4KU3RhcnRpbmcgcG1jZCAuLi4gClN0YXJ0aW5nIHBtbG9nZ2VyIC4uLiAK

------=_Part_13251874_859187345.1415861264678
Content-Type: application/octet-stream; name=813.out.bad
Content-Disposition: attachment; filename=813.out.bad
Content-Transfer-Encoding: base64

UUEgb3V0cHV0IGNyZWF0ZWQgYnkgODEzCj09PSBEYWVtb24gUE1EQSBwYXBpIHRlc3QgPT09CmRi
cG1kYT4gb3BlbiBwaXBlIC92YXIvbGliL3BjcC9wbWRhcy9wYXBpL3BtZGFwYXBpIC1kIDEyNgpT
dGFydCBwbWRhcGFwaSBQTURBOiAvdmFyL2xpYi9wY3AvcG1kYXMvcGFwaS9wbWRhcGFwaSAtZCAx
MjYKZGJwbWRhPiBnZXRkZXNjIG9uCmRicG1kYT4gYXR0ciAidXNlcm5hbWUiICJyb290IgpBdHRy
aWJ1dGU6IHVzZXJuYW1lPXJvb3QKU3VjY2VzcwpkYnBtZGE+IGF0dHIgMTEgIjAiCkF0dHJpYnV0
ZTogdXNlcmlkPTAKU3VjY2VzcwpkYnBtZGE+IGRlc2MgcGFwaS5zeXN0ZW0uVE9UX0lOUwpQTUlE
OiAxMjYuMC4wCiAgICBEYXRhIFR5cGU6IDY0LWJpdCB1bnNpZ25lZCBpbnQgIEluRG9tOiBQTV9J
TkRPTV9OVUxMIEFERFIKICAgIFNlbWFudGljczogY291bnRlciAgVW5pdHM6IG5vbmUKZGJwbWRh
PiBkZXNjIHBhcGkuY29udHJvbC5zdGF0dXMKUE1JRDogMTI2LjEuMwogICAgRGF0YSBUeXBlOiBz
dHJpbmcgIEluRG9tOiBQTV9JTkRPTV9OVUxMIEFERFIKICAgIFNlbWFudGljczogaW5zdGFudCAg
VW5pdHM6IG5vbmUKZGJwbWRhPiBkZXNjIHBhcGkuYXZhaWxhYmxlLm51bV9jb3VudGVycwpQTUlE
OiAxMjYuMi4wCiAgICBEYXRhIFR5cGU6IDMyLWJpdCB1bnNpZ25lZCBpbnQgIEluRG9tOiBQTV9J
TkRPTV9OVUxMIEFERFIKICAgIFNlbWFudGljczogZGlzY3JldGUgIFVuaXRzOiBub25lCmRicG1k
YT4gc3RvcmUgcGFwaS5jb250cm9sLmVuYWJsZSAiVE9UX0lOUyIKUE1JRDogMTI2LjEuMApHZXR0
aW5nIGRlc2NyaXB0aW9uLi4uClNlbmRpbmcgUHJvZmlsZS4uLgpHZXR0aW5nIFJlc3VsdCBTdHJ1
Y3R1cmUuLi4KMTI2LjEuMDogIiIgLT4gIlRPVF9JTlMiClNlbmRpbmcgUmVzdWx0Li4uCmRicG1k
YT4gZmV0Y2ggcGFwaS5zeXN0ZW0uVE9UX0lOUwpQTUlEKHMpOiAxMjYuMC4wCnBtUmVzdWx0IGR1
bXAgZnJvbSBBRERSIHRpbWVzdGFtcDogMC4wMDAwMDAgVElNRSBudW1wbWlkOiAxCiAgMTI2LjAu
MCAoPG5vbmFtZT4pOiBudW12YWw6IDEgdmFsZm10OiAxIHZsaXN0W106CiAgIHZhbHVlIE5VTUJF
UgpkYnBtZGE+IHN0b3JlIHBhcGkuY29udHJvbC5kaXNhYmxlICJUT1RfSU5TIgpQTUlEOiAxMjYu
MS4yCkdldHRpbmcgZGVzY3JpcHRpb24uLi4KR2V0dGluZyBSZXN1bHQgU3RydWN0dXJlLi4uCjEy
Ni4xLjI6ICIiIC0+ICJUT1RfSU5TIgpTZW5kaW5nIFJlc3VsdC4uLgpkYnBtZGE+IAo=
------=_Part_13251874_859187345.1415861264678
Content-Type: application/octet-stream; name=799.out.bad
Content-Disposition: attachment; filename=799.out.bad
Content-Transfer-Encoding: base64

UUEgb3V0cHV0IGNyZWF0ZWQgYnkgNzk5CldhaXRpbmcgZm9yIHBtY2QgdG8gdGVybWluYXRlIC4u
LgoKPT09IFBBUEkgYWdlbnQgaW5zdGFsbGF0aW9uID09PQpZb3Ugd2lsbCBuZWVkIHRvIGNob29z
ZSBhbiBhcHByb3ByaWF0ZSBjb25maWd1cmF0aW9uIGZvciBpbnN0YWxsYXRpb24gb2YKdGhlICJw
YXBpIiBQZXJmb3JtYW5jZSBNZXRyaWNzIERvbWFpbiBBZ2VudCAoUE1EQSkuCgogIGNvbGxlY3Rv
cgljb2xsZWN0IHBlcmZvcm1hbmNlIHN0YXRpc3RpY3Mgb24gdGhpcyBzeXN0ZW0KICBtb25pdG9y
CWFsbG93IHRoaXMgc3lzdGVtIHRvIG1vbml0b3IgbG9jYWwgYW5kL29yIHJlbW90ZSBzeXN0ZW1z
CiAgYm90aAkJY29sbGVjdG9yIGFuZCBtb25pdG9yIGNvbmZpZ3VyYXRpb24gZm9yIHRoaXMgc3lz
dGVtCgpQbGVhc2UgZW50ZXIgYyhvbGxlY3Rvcikgb3IgbShvbml0b3IpIG9yIGIob3RoKSBbYl0g
VXBkYXRpbmcgdGhlIFBlcmZvcm1hbmNlIE1ldHJpY3MgTmFtZSBTcGFjZSAoUE1OUykgLi4uClRl
cm1pbmF0ZSBQTURBIGlmIGFscmVhZHkgaW5zdGFsbGVkIC4uLgpbLi4uaW5zdGFsbCBmaWxlcywg
bWFrZSBvdXRwdXQuLi5dClVwZGF0aW5nIHRoZSBQTUNEIGNvbnRyb2wgZmlsZSwgYW5kIG5vdGlm
eWluZyBQTUNEIC4uLgpTdGFydGluZyBwbWNkIC4uLiAKU3RhcnRpbmcgcG1sb2dnZXIgLi4uIApD
aGVjayBwYXBpIG1ldHJpY3MgaGF2ZSBhcHBlYXJlZCAuLi4gOSBtZXRyaWNzIGFuZCBZIHZhbHVl
cwoKPT0gUEFQSSBsaWJyYXJ5IGJlaGF2aW91ciwgYWRkaW5nIGNvbmZsaWN0aW5nIG1ldHJpY3Mg
KHdoZW4gbXVsdGlwbGV4aW5nIGlzIGRpc2FibGVkKSwgYXMgcm9vdApwYXBpLmNvbnRyb2wuZW5h
YmxlIG9sZCB2YWx1ZT0iIiBuZXcgdmFsdWU9IkwxX0lDTSBMMl9EQ00gTDJfSUNNIEwxX1RDTSBM
Ml9UQ00iCnBhcGkuY29udHJvbC5lbmFibGU6IHBtU3RvcmU6IEltcG9zc2libGUgdmFsdWUgb3Ig
c2NhbGUgY29udmVyc2lvbgoKPT0gUEFQSSBsaWJyYXJ5IGJlaGF2aW91ciwgbWV0cmljIHRvdGFs
cywgYXMgcm9vdApFcnJvcjogcGFwaS5zeXN0ZW0uTDFfSUNNOiBVbmtub3duIG1ldHJpYyBuYW1l
CkVycm9yOiBwYXBpLnN5c3RlbS5MMl9EQ006IFVua25vd24gbWV0cmljIG5hbWUKRXJyb3I6IHBh
cGkuc3lzdGVtLkwyX0lDTTogVW5rbm93biBtZXRyaWMgbmFtZQpFcnJvcjogcGFwaS5zeXN0ZW0u
TDFfVENNOiBVbmtub3duIG1ldHJpYyBuYW1lCkVycm9yOiBwYXBpLnN5c3RlbS5MMl9UQ006IFVu
a25vd24gbWV0cmljIG5hbWUKCj09IFBBUEkgbGlicmFyeSBiZWhhdmlvdXIsIHJlbW92aW5nIG1l
dHJpY3MsIGFzIHJvb3QKcGFwaS5jb250cm9sLmRpc2FibGUgb2xkIHZhbHVlPSIobm9uZSkiIG5l
dyB2YWx1ZT0iTDFfSUNNIEwyX0RDTSBMMl9JQ00gTDFfVENNIEwyX1RDTSIKcGFwaS5jb250cm9s
LmRpc2FibGU6IHBtU3RvcmU6IEltcG9zc2libGUgdmFsdWUgb3Igc2NhbGUgY29udmVyc2lvbgoK
PT09IHJlbW92ZSBQQVBJIGFnZW50ID09PQpDdWxsaW5nIHRoZSBQZXJmb3JtYW5jZSBNZXRyaWNz
IE5hbWUgU3BhY2UgLi4uCnBhcGkgLi4uIGRvbmUKVXBkYXRpbmcgdGhlIFBNQ0QgY29udHJvbCBm
aWxlLCBhbmQgbm90aWZ5aW5nIFBNQ0QgLi4uClsuLi5yZW1vdmluZyBmaWxlcy4uLl0KQ2hlY2sg
cGFwaSBtZXRyaWNzIGhhdmUgZ29uZSBhd2F5IC4uLiBPSwpXYWl0aW5nIGZvciBwbWNkIHRvIHRl
cm1pbmF0ZSAuLi4KU3RhcnRpbmcgcG1jZCAuLi4gClN0YXJ0aW5nIHBtbG9nZ2VyIC4uLiAK
------=_Part_13251874_859187345.1415861264678--

From nscott@redhat.com  Thu Nov 13 00:57:29 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 8E3F07F3F
	for ; Thu, 13 Nov 2014 00:57:29 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 2B95FAC008
	for ; Wed, 12 Nov 2014 22:57:29 -0800 (PST)
X-ASG-Debug-ID: 1415861844-04bdf0650c2979f0001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id ON3wkoMMSxHlvB7Q (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 12 Nov 2014 22:57:24 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAD6vOfI002889
	for ; Thu, 13 Nov 2014 01:57:24 -0500
Date: Thu, 13 Nov 2014 01:57:24 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp@oss.sgi.com
Message-ID: <1904256792.13253406.1415861844087.JavaMail.zimbra@redhat.com>
In-Reply-To: <1499787038.13253349.1415861818350.JavaMail.zimbra@redhat.com>
Subject: pcp updates: qa fixes
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: qa fixes
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: qa fixes
Thread-Index: H1+Cu25PHQllcCGuYMyyUWl7z4ZXSQ==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1415861844
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11545
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Nathan Scott (5):
      qa: ensure quoted pmconfig use in all tests (944/1009/1023)
      qa: resolve occassional "join is not sorted" qa/128 failure
      linux pmda: add missing help text for per-node CPU metrics
      linux pmda: fix cgroup hierarchy metric qa fail on ppc64
      libpcp: fix 64bit endian issue in highres event timer code

 qa/1009                        |    3 --
 qa/1023                        |    3 --
 qa/128                         |    1 
 qa/443                         |    1 
 qa/443.out                     |   53 ++++++++++++++++++++---------------------
 qa/944                         |    2 -
 qa/src/eventrec.0              |binary
 qa/src/eventrec.index          |binary
 qa/src/eventrec.meta           |binary
 src/libpcp/src/endian.c        |    8 +++---
 src/pmdas/linux/help           |   13 +++++++++-
 src/pmdas/linux_proc/cgroups.c |   12 +++------
 12 files changed, 52 insertions(+), 44 deletions(-)

From michele@acksyn.org  Thu Nov 13 04:31:29 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 300997F37
	for ; Thu, 13 Nov 2014 04:31:29 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay3.corp.sgi.com (Postfix) with ESMTP id CEC85AC00E
	for ; Thu, 13 Nov 2014 02:31:28 -0800 (PST)
X-ASG-Debug-ID: 1415874683-04bdf0650e29dc60001-S8gJnT
Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id 5BLSVWur3rB2rAuR for ; Thu, 13 Nov 2014 02:31:24 -0800 (PST)
X-Barracuda-Envelope-From: michele@acksyn.org
X-Barracuda-Apparent-Source-IP: 5.9.7.26
Received: from localhost (localhost [127.0.0.1])
	by palahniuk.acksyn.org (Postfix) with ESMTP id D51C025CF6
	for ; Thu, 13 Nov 2014 05:31:22 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h=
	user-agent:content-disposition:content-type:content-type
	:mime-version:message-id:subject:subject:from:from:date:date
	:received:received; s=2010; t=1415874682; bh=5qnP0lCGLwe9owexGFT
	BfWcHJchssK67Xn8mHJQW1RI=; b=Gu6fQEgWwm/NJVEpaNIAjYu8dl11pvl4T1h
	S5K59qunBIl1RrtSacEhwA5NrPfuwqXQDSdVD0px3/tYScWMFrwgJL8wIbQUkKZt
	cZZum5rQxTo6vbhvcxKtDRl+w3T9e15q1LX6UPLYgtln5qf8pVapZIxkKQ4fwiKZ
	Uexp4mIk=
Received: from palahniuk.acksyn.org ([127.0.0.1])
	by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id vTqZkJqJlJRG for ;
	Thu, 13 Nov 2014 05:31:22 -0500 (EST)
Received: from localhost (host54-190-dynamic.24-79-r.retail.telecomitalia.it [79.24.190.54])
	by palahniuk.acksyn.org (Postfix) with ESMTPSA id C1CA820110
	for ; Thu, 13 Nov 2014 05:31:21 -0500 (EST)
Date: Thu, 13 Nov 2014 11:31:21 +0100
From: Michele Baldessari 
To: pcp@oss.sgi.com
Subject: publican fixes
Message-ID: <20141113103120.GA8595@marquez.int.rhx>
X-ASG-Orig-Subj: publican fixes
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2012-12-30)
X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26]
X-Barracuda-Start-Time: 1415874683
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11549
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature

Hi all,

while working on making the a new shiny PCP website, I was stumbling
in building the books with publican (F21 - publican-4.1.3-1).
I've now prepared a git repo with some fixes that build all the
books correctly again (pdf, html, html-single). Here is
the branch with the fixes:
https://github.com/mbaldessari/pcp/tree/publican-fixes

The issue is that I am quite unfamiliar with docbook/publican
so I am not 100% sure about their correctness.

cheers,
Michele
-- 
Michele Baldessari            
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D

From be.nicolas.michel@gmail.com  Thu Nov 13 06:17:01 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,
	T_DKIM_INVALID autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 9740E7F37
	for ; Thu, 13 Nov 2014 06:17:01 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 7424D8F8050
	for ; Thu, 13 Nov 2014 04:16:58 -0800 (PST)
X-ASG-Debug-ID: 1415881017-04cbb00e6726eb30001-S8gJnT
Received: from mail-ob0-f176.google.com (mail-ob0-f176.google.com [209.85.214.176]) by cuda.sgi.com with ESMTP id ARILl92xYXQRe1Jt (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 13 Nov 2014 04:16:57 -0800 (PST)
X-Barracuda-Envelope-From: be.nicolas.michel@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.214.176
X-Barracuda-IPDD: Level1 [gmail.com/209.85.214.176]
Received: by mail-ob0-f176.google.com with SMTP id va2so10584316obc.7
        for ; Thu, 13 Nov 2014 04:16:57 -0800 (PST)
X-Barracuda-IPDD: Level1 [gmail.com/209.85.214.176]
X-Barracuda-IPDD: Level1 [gmail.com/209.85.214.176]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=8anwH0zKcKage754r5p0nkN+VSQWNFQN5q+U++CJz+s=;
        b=e2drSSd19H90S1WeaW8z3op/chN84CClmj6msCSVblNR7c3Hefn0Y+iJZnLjVjmOz7
         +SXov32IwACUak+kpU/Qi+s1Pm6SSE+/fECTYsXQc1iLj9lL/rfCts3VLoPsqquiiU0e
         iVnUKTc+LtwrCjpTjQ/kdOBFAv/WCeBl25OhSUCkIWp86tgIdxvbHOpP3PIzMxV8UOKZ
         FllIQVEArTIrsiH+k48P6fz++G9dNeQj0nSkF42P/orGo1ykFvEXzdWAc5co2nb7sTjn
         RU1uHdysztcgKoiL83yTHmvWT77Ap+uynGEbTih4+hh4/EojMwpL7PbfyD5CE0zC0570
         OKcA==
MIME-Version: 1.0
X-Received: by 10.60.68.108 with SMTP id v12mr1030282oet.69.1415881016948;
 Thu, 13 Nov 2014 04:16:56 -0800 (PST)
Received: by 10.60.62.73 with HTTP; Thu, 13 Nov 2014 04:16:56 -0800 (PST)
Date: Thu, 13 Nov 2014 13:16:56 +0100
Message-ID: 
Subject: Process analysis
From: Nicolas Michel 
X-ASG-Orig-Subj: Process analysis
To: pcp@oss.sgi.com, yves.weber@nrb.be
Content-Type: multipart/alternative; boundary=001a11330d5e0f3c3a0507bc7d6f
X-Barracuda-Connect: mail-ob0-f176.google.com[209.85.214.176]
X-Barracuda-Start-Time: 1415881017
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11551
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11330d5e0f3c3a0507bc7d6f
Content-Type: text/plain; charset=UTF-8

Hello guys,

We want to give a try to PCP with a centralized pmlogger, and web plotter
like grafa or graphite.
It should be great for visualizing the global performance metrics on our
servers.

However, when analyzing performance issues, one also often need to go
deeper and grab performance metrics at the process level (which process
consume I/O or memory or CPU at a given time). I want to know if pcp or
some plugin of pcp can go that deep, and grab performance metrics by
process?
If not, is someone know any software that could do it? (centralized with
history)

Thanks you very much,

-- 
Nicolas MICHEL

--001a11330d5e0f3c3a0507bc7d6f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hello guys,

We want to give a try to PC=
P with a centralized pmlogger, and web plotter like grafa or graphite.
It should be great for visualizing the global performance metrics on =
our servers.

However, when analyzing performance i=
ssues, one also often need to go deeper and grab performance metrics at the=
 process level (which process consume I/O or memory or CPU at a given time)=
. I want to know if pcp or some plugin of pcp can go that deep, and grab pe=
rformance metrics by process?
If not, is someone know any softwar=
e that could do it? (centralized with history)

Tha=
nks you very much,

-- 
Nicolas MICHEL




--001a11330d5e0f3c3a0507bc7d6f--

From fche@redhat.com  Thu Nov 13 11:02:53 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 260237F37
	for ; Thu, 13 Nov 2014 11:02:53 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 151F78F804C
	for ; Thu, 13 Nov 2014 09:02:49 -0800 (PST)
X-ASG-Debug-ID: 1415898165-04cbb00e69279d20001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id YFeSA13nUk0aqJjD (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 13 Nov 2014 09:02:45 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sADH2dbR023936
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL);
	Thu, 13 Nov 2014 12:02:40 -0500
Received: from fche.csb (vpn-235-113.phx2.redhat.com [10.3.235.113])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sADH2d4K010634;
	Thu, 13 Nov 2014 12:02:39 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id BCA5758174; Thu, 13 Nov 2014 12:02:38 -0500 (EST)
To: Nicolas Michel 
Cc: pcp@oss.sgi.com, yves.weber@nrb.be
Subject: Re: Process analysis
References: 
X-ASG-Orig-Subj: Re: Process analysis
From: fche@redhat.com (Frank Ch. Eigler)
Date: Thu, 13 Nov 2014 12:02:38 -0500
In-Reply-To:  (Nicolas Michel's message of "Thu, 13 Nov 2014 13:16:56 +0100")
Message-ID: 
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1415898165
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1


Hi -


be.nicolas.michel wrote:

> We want to give a try to PCP with a centralized pmlogger, and web
> plotter like grafa or graphite.  It should be great for visualizing
> the global performance metrics on our servers.

OK, might want to try the pcp 3.10.0 + the webapps code, which should
handle apprx. all that out-of-the-box.


> However, when analyzing performance issues, one also often need to
> go deeper and grab performance metrics at the process level (which
> process consume I/O or memory or CPU at a given time). [...]

That's a little trickier, for a few reasons.

First, pcp usually relays data in a relatively raw form from the
kernel.  For the "proc.*" metrics, you can get some per-process status
and statistics, but finding the "top" (by whatever metric) would be left
to your application, and about actual I/O etc. trace type traffic, are
not routinely exposed by the kernel.

Second, the proc.* data is relatively large, so it is not normally
turned on for routine background logging.  If you can narrow your
interest to a few metrics of the processes, or perhaps even to a few
particular process instances, then routinely logging them is probably
fine.

Third, if you need to change on-the-fly the logging configuration (so
as to add more proc.* stuff temporarily, or change sampling rates, for
example), there is currently simple no web frontend nor
multiple-pmlogger-affecting mechanism for this.  The pmlc program can
adjust a single running pmlogger to some extent.  Another way could be
to use a pmmgr instance to supervise a stable of pmloggers for the
remote boxes.  pmmgr can recompute pmlogger configuration files from
fragments you modify (to add/subtract proc.* etc.) & restart all the
pmloggers on demand.  pmwebd can present a glued-together view of the
various archive files to the graphical webapps across restarts.

- FChE

From minnus@buffalo.edu  Thu Nov 13 14:01:18 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id B8A3A7F37
	for ; Thu, 13 Nov 2014 14:01:18 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id A5A3F304066
	for ; Thu, 13 Nov 2014 12:01:18 -0800 (PST)
X-ASG-Debug-ID: 1415908873-04cbb00e682809e0001-S8gJnT
Received: from mtareserve1.acsu.buffalo.edu (mtareserve6.acsu.buffalo.edu [128.205.6.4]) by cuda.sgi.com with ESMTP id jnZEosYReHZOx9nA for ; Thu, 13 Nov 2014 12:01:14 -0800 (PST)
X-Barracuda-Envelope-From: minnus@buffalo.edu
X-Barracuda-Apparent-Source-IP: 128.205.6.4
Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208])
	by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id D5EA2B17F;
	Thu, 13 Nov 2014 15:01:13 -0500 (EST)
Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1])
	by localhost (Postfix) with SMTP id CF72930E45;
	Thu, 13 Nov 2014 15:01:13 -0500 (EST)
Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1])
	by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id 75B2630E32;
	Thu, 13 Nov 2014 15:01:12 -0500 (EST)
Received: from smtp.buffalo.edu (smtp1.acsu.buffalo.edu [128.205.5.253])
	by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id 64CEC30E31;
	Thu, 13 Nov 2014 15:01:12 -0500 (EST)
Received: from prince.ccr.buffalo.edu (prince.ccr.buffalo.edu [128.205.40.45])
	(Authenticated sender: minnus@buffalo.edu)
	by smtp.buffalo.edu (Postfix) with ESMTPSA id 590F12D72;
	Thu, 13 Nov 2014 15:01:12 -0500 (EST)
Message-ID: <54650E07.90908@buffalo.edu>
Date: Thu, 13 Nov 2014 15:01:11 -0500
From: Martins Innus 
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] Dynamic Proc PMDA metrics
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54627B7D.6040906@buffalo.edu> <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com> <5463C8C8.5010700@buffalo.edu> <1453660478.13246038.1415858528343.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
In-Reply-To: <1453660478.13246038.1415858528343.JavaMail.zimbra@redhat.com>
Content-Type: multipart/mixed;
 boundary="------------010106040100020205020700"
X-PM-EL-Spam-Prob: : 8%
X-Barracuda-Connect: mtareserve6.acsu.buffalo.edu[128.205.6.4]
X-Barracuda-Start-Time: 1415908873
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11564
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

This is a multi-part message in MIME format.
--------------010106040100020205020700
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Nathan,

On 11/13/14 1:02 AM, Nathan Scott wrote:
>> Still no progress on this, except that with a dso pmda, the domain
>> number is correct at this point. Illustrated below in the interrupts case.
> Looks like a bug - in the cgroups case, the domain# is explicitly
> stamped into the PMID for the new metric table entry each time, so
> I suspect we've never seen this before as a result.
OK.  I haven't dug too far to see where this should happen.
>
>> While researching this, I've run into a few issues that stem from
>> calling pmdaDynamicPMNS multiple times:
>>
>> 1. In the "size_metrictable" callback there is no context as to which
>> dynamic tree is being queried if we use the same callback for all trees.
> (*nod* - use different callbacks?)  Thats the old cgroups model anyway.
> The PMID cluster# was the point of division used there - each cluster is
> managed separately IIRC.
OK.
>
>> So this is going to way overestimate the storage space needed, since we
>> need to return the size of the largest sub tree if we use a common
>> callback.  The alternative is one callback per sub tree as far as I can see.
> Yep.  And that's OK - there's not many proc.* sub-trees (4/5?).
Sounds good. I can do that.
>
>> 2. This is a larger problem. In dynamic.c, for the function:
>>
>> static pmdaMetric *
>> dynamic_metric_table(int index, pmdaMetric *offset)
>>
>>       When running through the initial metric table, the only check that
>> is done is to see if the cluster matches when doing "mtabupdate" and no
>> check is done against the item or name.  This will cause multiple calls
>> for the same metric if we have clusters that span multiple trees (since
> If we take away "clusters spanning trees" this all gets simpler I think?
> Am I oversimplifying things there?  (apologies if so - I'll look at it
> more closely as soon as I get this cgroup stuff finished).
That's exactly correct.  The existing dynamic code assumes clusters 
don't span trees.  This problem goes away if they are distinct.

This assumption is also made in the following methods:

pmdaDynamicLookupPMID
pmdaDynamicLookupText

These will use the first tree that has the matching cluster.

Actually.  Thinking this through. Something like the attached patch 
could fix this issue once we can get the domain # in the source metric. 
I can't really test it until the domain # is there since the search 
fails currently.  Not sure of any performance issues with all those 
searches.

I did find that cgroup.c doesn't call pmdaTreeRebuildHash while testing 
this.  I put a call into the "namespace" function, but it likely doesn't 
mesh with your recent changes.  I just put something in to get it to run 
for now.

>
>> 3. In general, its not clear to me the proper way to generate the
>> dynamic metrics from the initial "templates". For example, take the
> Yeah, the code here is pretty freaky and non-obvious as I recall.
>
>>       /* kernel.percpu.interrupts.line[] */
>>       { NULL, { PMDA_PMID(CLUSTER_INTERRUPT_LINES, 0), PM_TYPE_U32,
>>       CPU_INDOM, PM_SEM_COUNTER, PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) }, },
>>   
>>       /* kernel.percpu.interrupts.[] */
>>       { NULL, { PMDA_PMID(CLUSTER_INTERRUPT_OTHER, 0), PM_TYPE_U32,
>>       CPU_INDOM, PM_SEM_COUNTER, PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) }, },
>>
>> These are used as templates to create the dynamic metrics. But
>> interrupts.c provides in "size_metrictable" the total number of dynamic
>> metrics, so we end up with at least 2 extra metrics which as far as I
>> can tell are not used.
> That was not the original intention, and sounds like a bug - it was
> intended this would be allocating the correct amount of space, but
> it looks like there's a few extras.  It'd be good to dig more deeply
> into that and understand why.
I looked into this a bit more.  Its 2 different issues.  The first is 
just an off by one error in the pmda.  The pmda should just allocate N-1 
new slots for each group, since the original "template" is used for 
metric 0.  This can be fixed in the linux pmda pretty easily. The second 
is another instance of the allocation problem in "size_metrictable".  
Since the trees are unbalanced, it allocates the max of the 2 options. 
And we end up with one extra slot in the smaller tree.
>
>> So both of the .11 and one of the .10 metrics is orphaned?  Does that
>> cause any issue?  Or since the pmns doesn't reference them, no harm
>> except memory used.  If that's the case,  for my issue #2 above I can
> Yeah, I think that's the case.
>
>> just set dummy cluster or item values for the metrics that are sent
>> through twice(or more), but that's pretty wasteful and I'm sure will
>> have other unintended consequences.
> Not that I've seen to date, but yep, we should certainly fix that up.
>
OK.  This should be sorted out if we fix the above.

Martins

--------------010106040100020205020700
Content-Type: text/x-diff;
 name="dynamic_c.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="dynamic_c.patch"

diff --git a/src/libpcp_pmda/src/dynamic.c b/src/libpcp_pmda/src/dynamic.c
index 3d6e73d..daf3603 100644
--- a/src/libpcp_pmda/src/dynamic.c
+++ b/src/libpcp_pmda/src/dynamic.c
@@ -81,6 +81,9 @@ pmdaDynamicPMNS(const char *prefix,
 	free(dynamic);
 	return;
     }
+
+    //fprintf(stderr, "Adding %s at location %d\n", prefix, dynamic_count);
+
     dynamic[dynamic_count].prefix = prefix;
     dynamic[dynamic_count].prefixlen = strlen(prefix);
     dynamic[dynamic_count].nclusters = nclusters;
@@ -103,7 +106,8 @@ pmdaDynamicLookupName(pmdaExt *pmda, const char *name)
     int i;
 
     for (i = 0; i < dynamic_count; i++) {
-	if (strncmp(name, dynamic[i].prefix, dynamic[i].prefixlen) == 0) {
+	if(strncmp(name, dynamic[i].prefix, dynamic[i].prefixlen) == 0) {
+	    //fprintf(stderr, "pmdaDynamicLookupName found %s in %d\n", name, i);
 	    if (dynamic[i].pmnsupdate(pmda, &dynamic[i].pmns))
 		pmdaDynamicMetricTable(pmda);
 	    return dynamic[i].pmns;
@@ -112,18 +116,55 @@ pmdaDynamicLookupName(pmdaExt *pmda, const char *name)
     return NULL;
 }
 
+/*
+ * Verify if a given pmid should be handled by this dynamic instance.
+ * Assumes pmnsupdate has been called so dynamic[index].pmns can't be NULL
+ */
+
+int
+pmdaDynamicCheckPMID( pmID pmid, int index )
+{
+
+    int numfound=0;
+    char **nameset;
+
+    pmdaNameSpace *tree = dynamic[index].pmns;
+    //__pmDumpNameNode(stderr, tree->root, 1);
+    numfound = pmdaTreeName(tree, pmid, &nameset);
+
+    int domain = pmid_domain(pmid);
+    int cluster = pmid_cluster(pmid);
+    int item = pmid_item(pmid);
+
+    fprintf(stderr, "Check: %d.%d.%d In: %d Result: %d\n", domain, cluster, item, index, numfound);
+
+    /* Don't need the names, just seeing if its there */
+    
+    if( numfound > 0 ){
+	free(nameset);
+	return numfound;
+    }
+
+    return 0;
+
+}
+
 pmdaNameSpace *
 pmdaDynamicLookupPMID(pmdaExt *pmda, pmID pmid)
 {
-    int i, j, cluster;
+    int i, j, cluster, sts;
 
     for (i = 0; i < dynamic_count; i++) {
 	cluster = dynamic_pmid_cluster(i, pmid);
 	for (j = 0; j < dynamic[i].nclusters; j++) {
 	    if (cluster == dynamic[i].clusters[j]) {
-		if (dynamic[i].pmnsupdate(pmda, &dynamic[i].pmns))
-		    pmdaDynamicMetricTable(pmda);
-		return dynamic[i].pmns;
+		/* Need to do this in all cases to be able to search the pmns */
+		sts = dynamic[i].pmnsupdate(pmda, &dynamic[i].pmns);
+		if( pmdaDynamicCheckPMID( pmid, i ) ){
+		    if(sts)
+			pmdaDynamicMetricTable(pmda);
+		    return dynamic[i].pmns;
+		}
 	    }
 	}
     }
@@ -139,7 +180,11 @@ pmdaDynamicLookupText(pmID pmid, int type, char **buf, pmdaExt *pmda)
 	int cluster = dynamic_pmid_cluster(i, pmid);
 	for (j = 0; j < dynamic[i].nclusters; j++)
 	    if (cluster == dynamic[i].clusters[j])
-		return dynamic[i].textupdate(pmda, pmid, type, buf);
+		/* Need to do this in all cases to be able to search the pmns */
+                dynamic[i].pmnsupdate(pmda, &dynamic[i].pmns);
+                if( pmdaDynamicCheckPMID( pmid, i ) ){
+		    return dynamic[i].textupdate(pmda, pmid, type, buf);
+		}
     }
     return -ENOENT;
 }
@@ -150,19 +195,25 @@ pmdaDynamicLookupText(pmID pmid, int type, char **buf, pmdaExt *pmda)
  * which needs to be filled in.  All a bit obscure, really.
  */
 static pmdaMetric *
-dynamic_metric_table(int index, pmdaMetric *offset)
+dynamic_metric_table(int index, pmdaMetric *offset, pmdaExt *pmda)
 {
     struct dynamic *dp = &dynamic[index];
     int m, tree_count = dp->extratrees;
+    int sts=0;
 
     for (m = 0; m < dp->nmetrics; m++) {
 	pmdaMetric *mp = &dp->metrics[m];
 	int cluster = dynamic_pmid_cluster(index, mp->m_desc.pmid);
 	int c, gid;
 
-	for (c = 0; c < dp->nclusters; c++)
-	    if (dp->clusters[c] == cluster)
-		break;
+	for (c = 0; c < dp->nclusters; c++){
+	    if (dp->clusters[c] == cluster){
+		/* Need to do this in all cases to be able to search the pmns */
+                sts = dp->pmnsupdate(pmda, &dp->pmns);
+                if( pmdaDynamicCheckPMID( mp->m_desc.pmid, index ) )
+		    break;
+	    }
+	}
 	if (c < dp->nclusters)
 	    for (gid = 0; gid < tree_count; gid++)
 		dp->mtabupdate(mp, offset++, gid + 1);
@@ -209,7 +260,7 @@ fallback:
 	memcpy(mtab, fixed, total * sizeof(pmdaMetric));
 	offset = mtab + total;
 	for (i = 0; i < dynamic_count; i++)
-	    offset = dynamic_metric_table(i, offset);
+	    offset = dynamic_metric_table(i, offset, pmda);
 	if (pmda->e_metrics != fixed)
 	    free(pmda->e_metrics);
 	pmdaRehash(pmda, mtab, resize);

--------------010106040100020205020700--

From michele@acksyn.org  Thu Nov 13 15:41:45 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 1EC0A7F37
	for ; Thu, 13 Nov 2014 15:41:45 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id ACF3CAC00D
	for ; Thu, 13 Nov 2014 13:41:41 -0800 (PST)
X-ASG-Debug-ID: 1415914896-04cb6c1e6c27f820001-S8gJnT
Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id sxqehgz57D6Fl8HS for ; Thu, 13 Nov 2014 13:41:36 -0800 (PST)
X-Barracuda-Envelope-From: michele@acksyn.org
X-Barracuda-Apparent-Source-IP: 5.9.7.26
Received: from localhost (localhost [127.0.0.1])
	by palahniuk.acksyn.org (Postfix) with ESMTP id F251E28FB8
	for ; Thu, 13 Nov 2014 16:41:35 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h=
	user-agent:content-disposition:content-type:content-type
	:mime-version:message-id:subject:subject:from:from:date:date
	:received:received; s=2010; t=1415914895; bh=/M/WWLILAagfq7JxS/F
	52Dj+VIXOMuKMz2VEk2n4eDU=; b=iAeeYcH0yPxzq9REY6H/omXBYib0m7DW7pv
	QXV5nmo1FQsqsfelHi1JXEXwWrXPSfLipCbglM2013w1qyOscclcqrpUr6lvK41n
	ZlFxvVq5jVPN7TL6UVxtUJNCzUloi8k+d7RoeeivIXUaf/MQSdp2iq4WgsUbwXlL
	XBHJ4buU=
Received: from palahniuk.acksyn.org ([127.0.0.1])
	by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id ICRfByUZAZYh for ;
	Thu, 13 Nov 2014 16:41:35 -0500 (EST)
Received: from localhost (host54-190-dynamic.24-79-r.retail.telecomitalia.it [79.24.190.54])
	by palahniuk.acksyn.org (Postfix) with ESMTPSA id 1B13B20110
	for ; Thu, 13 Nov 2014 16:41:35 -0500 (EST)
Date: Thu, 13 Nov 2014 22:41:34 +0100
From: Michele Baldessari 
To: pcp@oss.sgi.com
Subject: New website
Message-ID: <20141113214134.GC8595@marquez.int.rhx>
X-ASG-Orig-Subj: New website
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2012-12-30)
X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26]
X-Barracuda-Start-Time: 1415914896
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11567
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature

Hi all,

so after presenting PCP to a bunch of companies, I got a bit of a
negative feedback about the website. So I gave a shot at making a
complete new one.

You can find the current proposed version here:
http://pcp.acksyn.org (user:pcp pass:pcp123 - the site is protected
to avoid robots indexing a temporary website)

Here's a bit what I've done:
- Migrated existing pages from the old site to the new web-design
- Slightly cleaned up the logo
- Integrated all the man pages on the website
- Added html,html-single,pdf formats for the UAG and PG books
  (This is still to be completed as it's not yet fully integrated)
- Added some new pages (features, team, documentation, etc.)

I've still got a couple of things to finish up, but the main parts are
done. I'd be happy if you have any feedback, be it about visual changes
or content changes.

Thanks,
Michele
-- 
Michele Baldessari            
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D

From wwwrun@oss.sgi.com  Thu Nov 13 15:57:06 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 503DB7F47; Thu, 13 Nov 2014 15:57:06 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 866] hinv.map.scsi uses /proc/scsi/scsi which is deprecated
Date: Thu, 13 Nov 2014 21:57:05 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: normal
X-Bugzilla-Who: michele@acksyn.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: mort@sgi.com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: keywords cc
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1415915826.16C8aBf1.25117"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1415915826.16C8aBf1.25117
Date: Thu, 13 Nov 2014 15:57:06 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=866

Michele Baldessari  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |EasyFix
                 CC|                            |michele@acksyn.org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1415915826.16C8aBf1.25117
Date: Thu, 13 Nov 2014 15:57:06 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 866
        

             

          

            What
            Removed
            Added
          


         

           Keywords
           
                
           
           EasyFix
           
         


         

           CC
           
                
           
           michele@acksyn.org
           
         

      

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1415915826.16C8aBf1.25117--

From wwwrun@oss.sgi.com  Thu Nov 13 15:57:13 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 9363A29DFB; Thu, 13 Nov 2014 15:57:13 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 964] Mising QA for named PMDA
Date: Thu, 13 Nov 2014 21:57:13 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: normal
X-Bugzilla-Who: michele@acksyn.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: keywords cc
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1415915833.3BDb2.25186"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1415915833.3BDb2.25186
Date: Thu, 13 Nov 2014 15:57:13 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=964

Michele Baldessari  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |EasyFix
                 CC|                            |michele@acksyn.org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1415915833.3BDb2.25186
Date: Thu, 13 Nov 2014 15:57:13 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 964
        

             

          

            What
            Removed
            Added
          


         

           Keywords
           
                
           
           EasyFix
           
         


         

           CC
           
                
           
           michele@acksyn.org
           
         

      

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1415915833.3BDb2.25186--

From lberk@redhat.com  Thu Nov 13 17:02:16 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id F0FBB29DF7
	for ; Thu, 13 Nov 2014 17:02:15 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 71336AC002
	for ; Thu, 13 Nov 2014 15:02:12 -0800 (PST)
X-ASG-Debug-ID: 1415919730-04cbb01e5c013a0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id QJ4zELZVtdZEVkyN (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 13 Nov 2014 15:02:11 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sADN2Aqe014746
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Thu, 13 Nov 2014 18:02:10 -0500
Received: from toium (dhcp-10-15-16-139.yyz.redhat.com [10.15.16.139])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sADN290e029801
	(version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO);
	Thu, 13 Nov 2014 18:02:09 -0500
From: Lukas Berk 
To: Nathan Scott 
Cc: pcp@oss.sgi.com
Subject: Re: [pcp] pcp updates - pmdapapi update
References: <87oascow3f.fsf@redhat.com>
	<462023254.13251879.1415861264683.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] pcp updates - pmdapapi update
Date: Thu, 13 Nov 2014 18:02:08 -0500
In-Reply-To: <462023254.13251879.1415861264683.JavaMail.zimbra@redhat.com>
	(Nathan Scott's message of "Thu, 13 Nov 2014 01:47:44 -0500 (EST)")
Message-ID: <87y4repuvj.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1415919731
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hey,

Nathan Scott  writes:
[...]

> Fabulous - awesome effort esp. on the QA front.  I ran out of time to
> review it all today (maybe someone else will?), but did sneak a quick
> background QA run in on RHEL 6 today.  I'm seeing a few new failures
> there - see attached .bad files - any ideas on possible root causes?

Thanks for looking things over.  I've pushed fixes for 3/4 of the
failures (diffstat and change updates below).  For 967 and 813 the issue
was a difference in pmid's for the TOT_INS metric.  This was due to the
dynamic pmns.  Functionally, the tests were identical (and running
properly), to fix this I added an additional regex to match the
papi.system.* pmid's to output them as 126.0.NUMBER.

Testcase 903 was failing partly due to the dynamic pmns as well.
Apparently the number of metrics on that box was much lower, so it
didn't trigger the regex (which would have swapped the number for an
'X').  After testing it on a vm with no papi metrics available, I
lowered the regex to match 7 or greater.  This provides matches for the
5 papi.control metrics, 1 papi.available metric, and at least one actual
papi.system.* metric.

Testcase 799 failed for the same reason I mentioned in my original
email, and I'd be open to advice on how to fix it.  The metrics I used
to force a ECNFLCT (if multiplexing is disable) on my machine, may not
exist on other machines.  Being able to find a combination of metrics
which would cause such an error, programatically, on the host qa
machine, is something I'm not sure how to do yet.

> Only other general piece of advice I can offer would be "release early,
> release often" - the first commit here is >1 month old, and it probably
> coulda been merged right away?  *shrug* ... either way is fine, but I'd
> go for quicker, smaller merges every day.

Understood, I'll try to do so more often.  Diffstat and commit updates
relevant to above posted below.

Cheers,

Lukas

--------------------------------------------------------------------------

 qa/813     |    1 +
 qa/813.out |    6 +++---
 qa/903     |    2 +-
 qa/967     |    1 +
 qa/967.out |   26 +++++++++++++-------------
 5 files changed, 19 insertions(+), 17 deletions(-)

Author: Lukas Berk 
Date:   Thu Nov 13 16:12:37 2014 -0500

    Alter qa/903 awk statement to account for lower possible metric counts
    
    The number of available papi metrics varies based on the system being
    run on.  Previously there would be a pmid for each possible metric, so
    we could set the awk regex much higher.  At this point, limit it to 7
    or greater, (one for each papi.control and one papi.available).

commit b44c3c0decfcfbff9b4ca315bea2f9cf354fdfae
Author: Lukas Berk 
Date:   Thu Nov 13 16:10:42 2014 -0500

    Update qa testcases to account for dynamic papi pmns
    
    The papi.system.TOT_INS metric is used in both qa/813 and qa/967
    testcases.  With the dynamic pmid's used, this metric may change
    based on the hardware it's run on.  Due to this, add a new
    regex to that matches 126.0.NUMBER, instead of a specific pmid

commit 629bc4ccaf3328c50d3d8b87cb176a60e3dcccb6
Author: Lukas Berk 
Date:   Wed Nov 12 18:55:21 2014 -0500

    Add additional qa test that papi.control overrides auto_enable timeout
    
    qa/967 tests that we can disable the auto_enable metric and use pmdapapi
    as previously expected.  We now add that despite having a timeout (for
    the testcase's purposes we use a small one), papi.control.{enable,disable}
    takes higher priority and will allow counters to remain active even after
    the auto_enable timeout has been hit.

From kenj@internode.on.net  Thu Nov 13 17:21:31 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 9229D29DF7
	for ; Thu, 13 Nov 2014 17:21:31 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 62BAD8F804C
	for ; Thu, 13 Nov 2014 15:21:28 -0800 (PST)
X-ASG-Debug-ID: 1415920885-04bdf0616101cd0001-S8gJnT
Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by cuda.sgi.com with ESMTP id vZFnT2KXd2AQtaPV for ; Thu, 13 Nov 2014 15:21:25 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.145
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AsQBAKg8ZVR20SpqPGdsb2JhbAANToNiWYMGhDfFN4kJAQEBAQEGAQEBATiEZ1UwBgIFFgsCCwMCAQIBMScGAgEBiEq5aHiWUoEtkmWBVAWGPJBRohtZgksBAQE
Received: from ppp118-209-42-106.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.42.106])
  by ipmail06.adl6.internode.on.net with ESMTP; 14 Nov 2014 09:51:03 +1030
Message-ID: <54653D47.9030706@internode.on.net>
Date: Fri, 14 Nov 2014 10:22:47 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: pcp updates - mixed bag, some QA, some init script tweaks
Content-Type: text/plain; charset=utf-8
X-ASG-Orig-Subj: pcp updates - mixed bag, some QA, some init script tweaks
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl6.internode.on.net[150.101.137.145]
X-Barracuda-Start-Time: 1415920885
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11571
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Changes committed to git://git.performancecopilot.org/kenj/pcp.git dev

 qa/115                   |   80 ++++++++++++++++++++++++-----------------------
 qa/511                   |   17 +++++++--
 qa/755                   |    1 
 qa/769                   |   60 ++++++++++++++++++++++++++++++++++-
 qa/769.out               |   25 ++++++++++++++
 qa/admin/pcp-qa-summary  |    2 -
 qa/group                 |    1 
 src/pmcd/rc-proc.sh      |   28 +++++++++++++++-
 src/pmie/rc_pmie         |    9 ++++-
 src/pmlogger/rc_pmlogger |    8 ++++
 10 files changed, 181 insertions(+), 50 deletions(-)

commit 34af50f5f72e42dddba6318b9cfc1c07d4957a8c
Author: Ken McDonell 
Date:   Fri Nov 14 10:19:13 2014 +1100

    src/pmlogger/rc_pmlogger: dodge an annoying blank line of output

commit 5c8d4a885831717ebcc9c175451a153161cac291
Author: Ken McDonell 
Date:   Fri Nov 14 09:43:32 2014 +1100

    src/pmie/rc_pmie: dodge an annoying blank line of output

commit a6fd664acf25401df610eacd7e8b8478e5983e56
Author: Ken McDonell 
Date:   Fri Nov 14 09:42:33 2014 +1100

    qa/755: extra diagnostics to help triage failure

commit 0fff189a067b37161890c80f2e953a258fa50c97
Author: Ken McDonell 
Date:   Fri Nov 14 09:41:25 2014 +1100

    qa/admin/pcp-qa-summary: remove diagnostic left in by accident

commit b328d6603558f2be0a693d5129c5886a8993898a
Author: Ken McDonell 
Date:   Fri Nov 14 09:39:39 2014 +1100

    qa/769: improvements
    
    - preserve service state before and after
    - increase coverage
    - output useful diagnostic stuff to $seq.full

commit 8724b65df347291556a32fa198c0c7caca7d8b64
Author: Ken McDonell 
Date:   Fri Nov 14 09:37:56 2014 +1100

    qa/115: revert to previous state
    
    Real fix was in $PCP_SHARE_DIR/lib/rc-proc.sh, not a maze of
    extra conditionals here.

commit 90c8e906e815be636ea3ba00a9caeebe27a8985c
Author: Ken McDonell 
Date:   Fri Nov 14 09:27:58 2014 +1100

    rc-proc.sh: fix minor systemctl-related regression
    
    On some platforms, systemctl does not like our services and
    redirects queries about their state to chkconfig, as in ...
    
      $ systemctl is-enabled pmlogger.service
      pmlogger.service is not a native service, redirecting to /sbin/chkconfig.
      Executing /sbin/chkconfig pmlogger --level=5
      pmlogger  on
      enabled
    
    This is benign, _except_ that it confuses the is_chkconfig_on()
    method and it does not return the true state of the service,
    i.e. 0 for on or enabled, else 1 for off or disabled).
    
    Fix this and add some debugging (under the control of
    $VERBOSE_CONFIG when set to true) to report which of the twisty
    passages is_chkconfig_on() is using to arrive at an answer.

commit 4d1228ce96dfe3fe608f9322b0cae49b9c90c31a
Author: Ken McDonell 
Date:   Thu Nov 13 07:24:39 2014 +1100

    qa/769: (new) check "chkconfig" controls for init scripts and QA work

commit c8dc9298358ae921742df83475f2580ca90e84b0
Author: Ken McDonell 
Date:   Wed Nov 12 06:52:56 2014 +1100

    qa/115: handle missing controls for init scripts
    
    When we don't have the moral equivalent of chkconfig, we can't
    really disable a service, so running the init.d script directly
    as this test does will always start the service.
    
    Add some convoluted logic in the test to detect this case and
    behave accordingly.
    
    Was failing on i486 Gentoo 2.0.3 (vm05).

commit 1fada59ecf5ba6e0e591373be6be24e8f8b6fb08
Author: Ken McDonell 
Date:   Wed Nov 12 06:51:01 2014 +1100

    qa/511: dodge bad xml from sadf
    
    Some combinations of binary data files and sadf version on some
    platforms conspire to generate badly formed XML ... detect and
    handle these cases in the test.

From nscott@redhat.com  Thu Nov 13 17:35:58 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 21AF77F37
	for ; Thu, 13 Nov 2014 17:35:58 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id F16C18F804C
	for ; Thu, 13 Nov 2014 15:35:57 -0800 (PST)
X-ASG-Debug-ID: 1415921756-04cb6c057202210001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id wHafalZlZ0YcbRpd (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 13 Nov 2014 15:35:56 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sADNZuGK025031
	for ; Thu, 13 Nov 2014 18:35:56 -0500
Date: Thu, 13 Nov 2014 18:35:56 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <519658944.14255886.1415921756221.JavaMail.zimbra@redhat.com>
In-Reply-To: <1909930829.14254818.1415921115325.JavaMail.zimbra@redhat.com>
Subject: Looking for unusual /proc/cpuinfo files
MIME-Version: 1.0
X-ASG-Orig-Subj: Looking for unusual /proc/cpuinfo files
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Looking for unusual /proc/cpuinfo files
Thread-Index: QFljzzee6xxF4VKmx0VW6CO3MY+6Fg==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1415921756
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11570
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi all,

The PCP Linux kernel PMDA parses /proc/cpuinfo to extract some
detailed metrics about the hardware.  One problem we have seen
is that each architecture port places different (and sometimes
completely different!) contents in this file, and we just have
to deal with that in userspace as best we can.

This came up the other day on aarch64, where values for some of
the hinv.cpu.* metrics are incorrect but it affects other ports
too.

I'd like to start a QA effort to maintain static copies of some
contents of /proc/cpuinfo & verify that each new PCP release is
able to successfully parse them.  If you have unusual machines,
problematic cpuinfo contents, large CPU count machines - that
sort of thing - please send me a copy of your /proc/cpuinfo and
I'll add it to the set we test going forward.

Thanks!

--
Nathan

From shirshendu@riva.co  Fri Nov 14 00:05:06 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 945C27F50
	for ; Fri, 14 Nov 2014 00:05:06 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 7E0848F8065
	for ; Thu, 13 Nov 2014 22:05:03 -0800 (PST)
X-ASG-Debug-ID: 1415945098-04cb6c05720cff0001-S8gJnT
Received: from mail-qa0-f45.google.com (mail-qa0-f45.google.com [209.85.216.45]) by cuda.sgi.com with ESMTP id AYKF9F1Lp5hQfUbi (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 13 Nov 2014 22:04:58 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.45
Received: by mail-qa0-f45.google.com with SMTP id dc16so11192933qab.32
        for ; Thu, 13 Nov 2014 22:04:57 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=q1NV7XauZLtRQWQqM2tZgbwGTGPNGasRKu7W8eH5HXY=;
        b=h0q2iaOTROVMu+2XTdYgHyGjOeAfp1iBPYlNt36TNnKYPOnPe4riPCU3ZAUDBowkLD
         fpd0jKK3AaD67ipLlC47SUnRABBQzs3KtNW2G5P0bQmwgf9FEfZwF9lbpXzTMhdIVbb8
         r0WKML4zfzFuHNh6OTv6LnYTlPB0EppX+hTE1BXlM2AvfwfkzC/qNM4a8rFBcF+k2f0i
         sg06pE2aGLF6ODVZCOGv4WgbZKlggAFC6+ypJN1xb3FBLpsTSSA+cXLvAmJ9hZxgqODF
         w4jMr5T6rz0EU46x3NRWy9J4uS+6Rcv+DRK/qgCsfFbzbXEA5ij+qf03jdMizxwBBzjv
         FeOw==
X-Gm-Message-State: ALoCoQnPIAWuWdfapWwwAqYdGWYo1Si5Zu1X0Rf7PnJKhy4rBlm+fPLWt/aC2SXprSOpk3Q+fvWN
MIME-Version: 1.0
X-Received: by 10.224.115.82 with SMTP id h18mr8939433qaq.76.1415945097655;
 Thu, 13 Nov 2014 22:04:57 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 13 Nov 2014 22:04:57 -0800 (PST)
In-Reply-To: <20141113214134.GC8595@marquez.int.rhx>
References: <20141113214134.GC8595@marquez.int.rhx>
Date: Fri, 14 Nov 2014 11:34:57 +0530
Message-ID: 
Subject: Re: [Suspected SPAM] [pcp] New website
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [Suspected SPAM] [pcp] New website
To: Michele Baldessari 
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=047d7bdca348917a060507cb6878
X-Barracuda-Connect: mail-qa0-f45.google.com[209.85.216.45]
X-Barracuda-Start-Time: 1415945098
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11580
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--047d7bdca348917a060507cb6878
Content-Type: text/plain; charset=UTF-8

This is really awesome work. I liked it so much.

Please keep up the good work.

Thanks,

Shirshendu

On Fri, Nov 14, 2014 at 3:11 AM, Michele Baldessari 
wrote:

> Hi all,
>
> so after presenting PCP to a bunch of companies, I got a bit of a
> negative feedback about the website. So I gave a shot at making a
> complete new one.
>
> You can find the current proposed version here:
> http://pcp.acksyn.org (user:pcp pass:pcp123 - the site is protected
> to avoid robots indexing a temporary website)
>
> Here's a bit what I've done:
> - Migrated existing pages from the old site to the new web-design
> - Slightly cleaned up the logo
> - Integrated all the man pages on the website
> - Added html,html-single,pdf formats for the UAG and PG books
>   (This is still to be completed as it's not yet fully integrated)
> - Added some new pages (features, team, documentation, etc.)
>
> I've still got a couple of things to finish up, but the main parts are
> done. I'd be happy if you have any feedback, be it about visual changes
> or content changes.
>
> Thanks,
> Michele
> --
> Michele Baldessari            
> C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D
>
> _______________________________________________
> pcp mailing list
> pcp@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/pcp
>

--047d7bdca348917a060507cb6878
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


This is really awesome work. I liked it so much.

<=
/div>
Please keep up the good work.

Thanks,

Shirshendu
On Fri, Nov 14, 2014 at 3:11 AM, Michele Baldes=
sari <michele@acksyn.org> wrote:
Hi all,



so after presenting PCP to a bunch of companies, I got a bit of a

negative feedback about the website. So I gave a shot at making a

complete new one.



You can find the current proposed version here:

http://pcp.acksyn.org (user:pcp pass:pcp123 - the site is protected

to avoid robots indexing a temporary website)



Here's a bit what I've done:

- Migrated existing pages from the old site to the new web-design

- Slightly cleaned up the logo

- Integrated all the man pages on the website

- Added html,html-single,pdf formats for the UAG and PG books

=C2=A0 (This is still to be completed as it's not yet fully integrated)=


- Added some new pages (features, team, documentation, etc.)



I've still got a couple of things to finish up, but the main parts are<=
br>
done. I'd be happy if you have any feedback, be it about visual changes=


or content changes.



Thanks,

Michele

--

Michele Baldessari=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <michele@acksyn.org>

C2A5 9DA3 9961 4FFB E01B=C2=A0 D0BC DDD4 DCCB 7515 5C6D



_______________________________________________

pcp mailing list

pcp@oss.sgi.com

http:=
//oss.sgi.com/mailman/listinfo/pcp





--047d7bdca348917a060507cb6878--

From nscott@redhat.com  Fri Nov 14 00:08:24 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id D289A7F50
	for ; Fri, 14 Nov 2014 00:08:24 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id BEB09304043
	for ; Thu, 13 Nov 2014 22:08:21 -0800 (PST)
X-ASG-Debug-ID: 1415945299-04cbb01e5a0c620001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id 2zHlMtar7OQAF4G2 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 13 Nov 2014 22:08:20 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAE68G68026194;
	Fri, 14 Nov 2014 01:08:16 -0500
Date: Fri, 14 Nov 2014 01:08:16 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Shirshendu Chakrabarti ,
        Michele Baldessari 
Cc: pcp@oss.sgi.com
Message-ID: <12695043.14357779.1415945296071.JavaMail.zimbra@redhat.com>
In-Reply-To: 
References: <20141113214134.GC8595@marquez.int.rhx> 
Subject: Re: [pcp] New website
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] New website
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: New website
Thread-Index: qZlut/5ji3onR0ur5BxvY5LK5eivIA==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1415945300
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11580
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...



----- Original Message -----
> This is really awesome work. I liked it so much.
> 
> Please keep up the good work.
> 

Couldn't agree more - thanks Michele!

cheers.

--
Nathan

From shirshendu@riva.co  Fri Nov 14 00:30:30 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id BD6BC7F37
	for ; Fri, 14 Nov 2014 00:30:30 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id A9C8B8F804C
	for ; Thu, 13 Nov 2014 22:30:30 -0800 (PST)
X-ASG-Debug-ID: 1415946624-04cb6c05730d910001-S8gJnT
Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) by cuda.sgi.com with ESMTP id E98DzEQaRjBRzrxR (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 13 Nov 2014 22:30:25 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.44
Received: by mail-qa0-f44.google.com with SMTP id v10so1961925qac.3
        for ; Thu, 13 Nov 2014 22:30:24 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=Pei5TEwymghDssmQe/ipACxlB17Immj0jHIxlPkR+WE=;
        b=a6P7SO6KFlB6QN6XnxCmPmkmvC+hJV2ygiZZBtXseCXkB+FrdyUnVuyL67ACWwNp8J
         SF/F1Dn5C5HsXxxZxMG6ZKZiIbvW2kBq1sF3rXlCXLW0wGD46uOb25FHYbf3lGfi+Fyw
         atRjTudXh9MsjE1x3kMU7b+za7BjquIP4cigpGddorVNtETiNS37YPguO8N5TrT9iJt0
         WCMnnXXuYsykcHC+18qeZxaaoJKLvO2ryZORTlZqkBX/bkzXke0t96mjsjkARjMPL2Xs
         trVI5M8MVbg9Xqj6GS/poohFIlEylH62DCOLVHtPGS5Y0j6m/3H3CrgdpCpbamT7YNgj
         zzYw==
X-Gm-Message-State: ALoCoQnYSlR4ZG9Dt5ioDoSHP+x8/bM4mBWGCBJ04VT1bEwJ9+5/mBtlWUnpNIXF72U7f646Kv7A
MIME-Version: 1.0
X-Received: by 10.140.30.165 with SMTP id d34mr2208315qgd.55.1415946624460;
 Thu, 13 Nov 2014 22:30:24 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 13 Nov 2014 22:30:24 -0800 (PST)
In-Reply-To: 
References: 
	
Date: Fri, 14 Nov 2014 12:00:24 +0530
Message-ID: 
Subject: Re: [pcp] Process analysis
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Process analysis
To: "Frank Ch. Eigler" 
Cc: Nicolas Michel , yves.weber@nrb.be, pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a113a94949284ed0507cbc3df
X-Barracuda-Connect: mail-qa0-f44.google.com[209.85.216.44]
X-Barracuda-Start-Time: 1415946624
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.10
X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA085, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11581
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	0.00 HTML_MESSAGE           BODY: HTML included in message
	0.10 BSF_SC0_SA085          Custom Rule SA085

--001a113a94949284ed0507cbc3df
Content-Type: text/plain; charset=UTF-8

This can be a probable solution. You can use systemtap to trace the desired
process.

Also, there is a systemtap PMDA which can help in this exercise. Other
solutions for userspace and kernel level tracing for linux:

1. perf events, https://perf.wiki.kernel.org/index.php/Main_Page
2. ktap, http://www.ktap.org/. ktap is more like dtrace in design.
3. LTTng, https://lttng.org/

Please see, AFAIK, there is no known integration with PCP for above
solutions.

Also, as of systemtap 1.2 there are probes in CPython and JVM.
http://fedoraproject.org/wiki/Features/SystemtapStaticProbes

Please do correct me, if I have committed any errors in this reply.

Thanks,

Shirshendu

On Thu, Nov 13, 2014 at 10:32 PM, Frank Ch. Eigler  wrote:

>
> Hi -
>
>
> be.nicolas.michel wrote:
>
> > We want to give a try to PCP with a centralized pmlogger, and web
> > plotter like grafa or graphite.  It should be great for visualizing
> > the global performance metrics on our servers.
>
> OK, might want to try the pcp 3.10.0 + the webapps code, which should
> handle apprx. all that out-of-the-box.
>
>
> > However, when analyzing performance issues, one also often need to
> > go deeper and grab performance metrics at the process level (which
> > process consume I/O or memory or CPU at a given time). [...]
>
> That's a little trickier, for a few reasons.
>
> First, pcp usually relays data in a relatively raw form from the
> kernel.  For the "proc.*" metrics, you can get some per-process status
> and statistics, but finding the "top" (by whatever metric) would be left
> to your application, and about actual I/O etc. trace type traffic, are
> not routinely exposed by the kernel.
>
> Second, the proc.* data is relatively large, so it is not normally
> turned on for routine background logging.  If you can narrow your
> interest to a few metrics of the processes, or perhaps even to a few
> particular process instances, then routinely logging them is probably
> fine.
>
> Third, if you need to change on-the-fly the logging configuration (so
> as to add more proc.* stuff temporarily, or change sampling rates, for
> example), there is currently simple no web frontend nor
> multiple-pmlogger-affecting mechanism for this.  The pmlc program can
> adjust a single running pmlogger to some extent.  Another way could be
> to use a pmmgr instance to supervise a stable of pmloggers for the
> remote boxes.  pmmgr can recompute pmlogger configuration files from
> fragments you modify (to add/subtract proc.* etc.) & restart all the
> pmloggers on demand.  pmwebd can present a glued-together view of the
> various archive files to the graphical webapps across restarts.
>
> - FChE
>
> _______________________________________________
> pcp mailing list
> pcp@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/pcp
>

--001a113a94949284ed0507cbc3df
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


This can be a probable solution. You can use systemtap to =
trace the desired process.

Also, there is a systemtap PM=
DA which can help in this exercise. Other solutions for userspace and kerne=
l level tracing for linux:

2. ktap,=C2=A0http://www.ktap.org/. ktap is more like dtrace in d=
esign.
3. LTTng,=C2=A0https://lttn=
g.org/

Please see, AFAIK, there is no known in=
tegration with PCP for above solutions.=C2=A0

Also=
, as of systemtap 1.2 there are probes in CPython and JVM.=C2=A0http://fedora=
project.org/wiki/Features/SystemtapStaticProbes

Please do correct me, if I have committed any errors in this reply.
=


Thanks,

Shirshendu=C2=A0
=


On Thu, Nov=
 13, 2014 at 10:32 PM, Frank Ch. Eigler <fche@redhat.com> wrot=
e:


Hi -





be.nicolas.michel wrote:



> We want to give a try to PCP with a centralized pmlogger, and web

> plotter like grafa or graphite.=C2=A0 It should be great for visualizi=
ng

> the global performance metrics on our servers.



OK, might want to try the pcp 3.10.0 + the webapps code, which shoul=
d

handle apprx. all that out-of-the-box.





> However, when analyzing performance issues, one also often need to

> go deeper and grab performance metrics at the process level (which

> process consume I/O or memory or CPU at a given time). [...]


That's a little trickier, for a few reasons.



First, pcp usually relays data in a relatively raw form from the

kernel.=C2=A0 For the "proc.*" metrics, you can get some per-proc=
ess status

and statistics, but finding the "top" (by whatever metric) would =
be left

to your application, and about actual I/O etc. trace type traffic, are

not routinely exposed by the kernel.



Second, the proc.* data is relatively large, so it is not normally

turned on for routine background logging.=C2=A0 If you can narrow your

interest to a few metrics of the processes, or perhaps even to a few

particular process instances, then routinely logging them is probably

fine.



Third, if you need to change on-the-fly the logging configuration (so

as to add more proc.* stuff temporarily, or change sampling rates, for

example), there is currently simple no web frontend nor

multiple-pmlogger-affecting mechanism for this.=C2=A0 The pmlc program can<=
br>
adjust a single running pmlogger to some extent.=C2=A0 Another way could be=


to use a pmmgr instance to supervise a stable of pmloggers for the

remote boxes.=C2=A0 pmmgr can recompute pmlogger configuration files from
fragments you modify (to add/subtract proc.* etc.) & restart all the
pmloggers on demand.=C2=A0 pmwebd can present a glued-together view of the<=
br>
various archive files to the graphical webapps across restarts.



- FChE



_______________________________________________

pcp mailing list

pcp@oss.sgi.com

http:=
//oss.sgi.com/mailman/listinfo/pcp





--001a113a94949284ed0507cbc3df--

From nscott@redhat.com  Fri Nov 14 02:13:00 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 139B67F3F
	for ; Fri, 14 Nov 2014 02:13:00 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 003638F8050
	for ; Fri, 14 Nov 2014 00:12:56 -0800 (PST)
X-ASG-Debug-ID: 1415952771-04cbb01e5c0f510001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id sR5D0vtqxDaN4NDx (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 00:12:51 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAE8CoBF005284;
	Fri, 14 Nov 2014 03:12:50 -0500
Date: Fri, 14 Nov 2014 03:12:50 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Lukas Berk 
Cc: pcp@oss.sgi.com
Message-ID: <2048395798.14415026.1415952770785.JavaMail.zimbra@redhat.com>
In-Reply-To: <87y4repuvj.fsf@redhat.com>
References: <87oascow3f.fsf@redhat.com> <462023254.13251879.1415861264683.JavaMail.zimbra@redhat.com> <87y4repuvj.fsf@redhat.com>
Subject: Re: [pcp] pcp updates - pmdapapi update
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] pcp updates - pmdapapi update
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates - pmdapapi update
Thread-Index: 3AgRh8OKpu9EoOMf1JSXKlbNhhab7g==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1415952771
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11583
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

Hi Lukas,

----- Original Message -----
> Hey,
> 
> Nathan Scott  writes:
> 
> > Fabulous - awesome effort esp. on the QA front.  I ran out of time to
> > review it all today (maybe someone else will?), but did sneak a quick
> > background QA run in on RHEL 6 today.  I'm seeing a few new failures
> > there - see attached .bad files - any ideas on possible root causes?
> 
> Thanks for looking things over.  I've pushed fixes for 3/4 of the
> failures (diffstat and change updates below).  For 967 and 813 the issue
> was a difference in pmid's for the TOT_INS metric.  This was due to the
> dynamic pmns.  Functionally, the tests were identical (and running
> properly), to fix this I added an additional regex to match the
> papi.system.* pmid's to output them as 126.0.NUMBER.

OK, sounds good.  Is the plan to fix this more, later? (if I followed
the other discussion with Ken the other day?  these PMID values need to
be as stable as we can make 'em)

> Testcase 903 was failing partly due to the dynamic pmns as well.
> Apparently the number of metrics on that box was much lower, so it
> didn't trigger the regex (which would have swapped the number for an
> 'X').  After testing it on a vm with no papi metrics available, I
> lowered the regex to match 7 or greater.  This provides matches for the
> 5 papi.control metrics, 1 papi.available metric, and at least one actual
> papi.system.* metric.

*nod*, looks good.

> Testcase 799 failed for the same reason I mentioned in my original
> email, and I'd be open to advice on how to fix it.  The metrics I used
> to force a ECNFLCT (if multiplexing is disable) on my machine, may not
> exist on other machines.  Being able to find a combination of metrics
> which would cause such an error, programatically, on the host qa
> machine, is something I'm not sure how to do yet.

Having reviewed the code now, hmm - there's little point to this test is
there?  It seems the error handling policy has become as anticipated for
auto-enable/disable - none at all.  So if an ECNFLCT or some other error
did occur at the point we were concerned about, we're never going to see
it anyway.  IOW, I guess you could just remove this test if we're going
ahead with that (its not a problem in the non-auto-enable case, so maybe
not worth testing in that mode, as you had it).

So, handle_papi_error() can do nothing useful for regular users?  What can
it sensibly do?  I guess it should really log problems during auto-enable/
disable, right?  Otherwise, there's no way to inform anyone that bad stuff
happened in refresh_metrics() - since the return code is ignored in that
situation - and one or more of the counters spontaneously became inactive
or failed to be added into the set?

Ignoring errors is not really good enough, I think - its going to make it
really hard to debug when people do have problems with it.

Anyway - I've merged it all, and marked 799 as retired for now so that it
doesn't fail for others ... feel free to update and turn it into a more
specific test (maybe not for ECNFLCT, but for some other reproducible error
case) - to check error handling for the non-auto-enabled counters?

I'll send a couple more review notes separately, but its all small followup
stuff; I think you've done a good job here Lukas - nice one.

cheers.

--
Nathan

From nscott@redhat.com  Fri Nov 14 02:22:42 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id BFF4C7F3F
	for ; Fri, 14 Nov 2014 02:22:42 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 9BE8E8F8050
	for ; Fri, 14 Nov 2014 00:22:42 -0800 (PST)
X-ASG-Debug-ID: 1415953360-04bdf0615e10ad0001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id PdfsNmAjpDu44MHG (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 00:22:41 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAE8Mc1S024477;
	Fri, 14 Nov 2014 03:22:38 -0500
Date: Fri, 14 Nov 2014 03:22:38 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Michele Baldessari 
Cc: pcp@oss.sgi.com
Message-ID: <560166708.14420145.1415953358213.JavaMail.zimbra@redhat.com>
In-Reply-To: <20141113103120.GA8595@marquez.int.rhx>
References: <20141113103120.GA8595@marquez.int.rhx>
Subject: Re: [pcp] publican fixes
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] publican fixes
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: publican fixes
Thread-Index: TaOJoarWLfg2rveYFqVyRCaw//YgUg==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415953361
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11584
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...



----- Original Message -----
> Hi all,
> 
> while working on making the a new shiny PCP website, I was stumbling
> in building the books with publican (F21 - publican-4.1.3-1).
> I've now prepared a git repo with some fixes that build all the
> books correctly again (pdf, html, html-single). Here is
> the branch with the fixes:
> https://github.com/mbaldessari/pcp/tree/publican-fixes
> 
> The issue is that I am quite unfamiliar with docbook/publican
> so I am not 100% sure about their correctness.

Its the blind leading the blind here. :)  Maybe we can seek Laura's
opinion?  I'll send her a note anyway.

thanks.

--
Nathan

From nscott@redhat.com  Fri Nov 14 02:41:08 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id DF8547F3F
	for ; Fri, 14 Nov 2014 02:41:07 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 6B24EAC005
	for ; Fri, 14 Nov 2014 00:41:07 -0800 (PST)
X-ASG-Debug-ID: 1415954464-04cbb01e5b101b0001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id uTz9kS5wwTgGbgrC (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 00:41:05 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAE8f48a026926
	for ; Fri, 14 Nov 2014 03:41:04 -0500
Date: Fri, 14 Nov 2014 03:41:04 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <319556460.14436082.1415954464312.JavaMail.zimbra@redhat.com>
In-Reply-To: <659079660.14432964.1415954261182.JavaMail.zimbra@redhat.com>
Subject: pcp updates: merges (kenj,michele,lberk)
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: merges (kenj,michele,lberk)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: merges (kenj,michele,lberk)
Thread-Index: JODulbLJ1mbX6FJl5828u8AvWVcPSQ==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1415954465
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11583
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

(this push includes Lukas' latest papi code - git-log didn't pull
it in here though, and I didn't want to rebase to get it...)

Changes committed to git://git.pcp.io/pcp.git dev

Ken McDonell (10):
      qa/511: dodge bad xml from sadf
      qa/115: handle missing controls for init scripts
      qa/769: (new) check "chkconfig" controls for init scripts and QA work
      rc-proc.sh: fix minor systemctl-related regression
      qa/115: revert to previous state
      qa/769: improvements
      qa/admin/pcp-qa-summary: remove diagnostic left in by accident
      qa/755: extra diagnostics to help triage failure
      src/pmie/rc_pmie: dodge an annoying blank line of output
      src/pmlogger/rc_pmlogger: dodge an annoying blank line of output

Michele Baldessari (10):
      Add an abstract section to each book
      Fix building with newer publican version
      Unset tmp path
      Fix GNUMakefile to get books building
      replace all figures/ with images/
      Also build the books in html format
      Remove sgi_termlength conditionals that brake the build
      Build the html docs with the --publish switch
      Fix man page section mentioned in pmdads389log.1
      Also generate html-single format

Nathan Scott (5):
      pmdapapi: small cleanups after review
      qa: mark papi test 799 as retired, while Lukas ponders it some more
      qa: correct sed invocation typo in couple of tests
      build: update gitignore file for pmdapapi docs

 books/PCP_PG/Author_Group.xml                        |   13 
 books/PCP_PG/Book_Info.xml                           |    8 
 books/PCP_PG/GNUmakefile                             |   27 +
 books/PCP_PG/Revision_History.xml                    |   34 ++
 books/PCP_PG/pcp-programmers-guide.ent               |    3 
 books/PCP_PG/pcp-programmers-guide.xml               |   32 --
 books/PCP_PG/publican.cfg                            |    1 
 books/PCP_TCS/Author_Group.xml                       |   13 
 books/PCP_TCS/Book_Info.xml                          |    8 
 books/PCP_TCS/GNUmakefile                            |   27 +
 books/PCP_TCS/Revision_History.xml                   |   14 
 books/PCP_TCS/pcp-tutorials-and-case-studies.ent     |    3 
 books/PCP_TCS/pcp-tutorials-and-case-studies.xml     |    4 
 books/PCP_TCS/publican.cfg                           |    1 
 books/PCP_UAG/Author_Group.xml                       |   13 
 books/PCP_UAG/Book_Info.xml                          |    9 
 books/PCP_UAG/GNUmakefile                            |   29 +
 books/PCP_UAG/Revision_History.xml                   |   19 +
 books/PCP_UAG/pcp-users-and-administrators-guide.ent |    3 
 books/PCP_UAG/pcp-users-and-administrators-guide.xml |   39 +-
 books/PCP_UAG/publican.cfg                           |    1 
 qa/115                                               |   80 ++---
 qa/443                                               |    1 
 qa/443.out                                           |   53 +--
 qa/511                                               |   17 -
 qa/755                                               |    1 
 qa/769                                               |   60 ++++
 qa/769.out                                           |   25 +
 qa/813                                               |    4 
 qa/813.out                                           |    4 
 qa/967                                               |    6 
 qa/967.out                                           |   12 
 qa/admin/pcp-qa-summary                              |    2 
 qa/group                                             |    5 
 src/pmcd/rc-proc.sh                                  |   28 +
 src/pmdas/ds389log/pmdads389log.1                    |    2 
 src/pmdas/papi/.gitignore                            |    1 
 src/pmdas/papi/papi.c                                |  279 +++++++++----------
 src/pmie/rc_pmie                                     |    9 
 src/pmlogger/rc_pmlogger                             |    8 
 40 files changed, 585 insertions(+), 321 deletions(-)

From nscott@redhat.com  Fri Nov 14 02:53:12 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 3C2CF7F3F
	for ; Fri, 14 Nov 2014 02:53:12 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 1A61F304062
	for ; Fri, 14 Nov 2014 00:53:11 -0800 (PST)
X-ASG-Debug-ID: 1415955186-04cb6c057211100001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id YiQZXfWYGjIJ5sGJ (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 00:53:06 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAE8r5nE022496;
	Fri, 14 Nov 2014 03:53:05 -0500
Date: Fri, 14 Nov 2014 03:53:05 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Lukas Berk 
Cc: pcp@oss.sgi.com
Message-ID: <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>
In-Reply-To: <87oascow3f.fsf@redhat.com>
References: <87oascow3f.fsf@redhat.com>
Subject: Re: [pcp] pcp updates - pmdapapi update
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] pcp updates - pmdapapi update
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates - pmdapapi update
Thread-Index: khhgzDkZJ+KvbXTW3bNITYrHj5v7uA==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1415955186
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11584
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

Hi Lukas,

----- Original Message -----
> Hi,
> 
> The PAPI pmda has gone through a fair bit of work.  The changes can be
> found at: git://sourceware.org/git/pcpfans.git lberk/papi or gitweb:
> https://www.sourceware.org/git/gitweb.cgi?p=pcpfans.git;a=shortlog;h=refs/heads/lberk/papi
> 
> This iteration features much improved internals, additional qa,
> [...]

This is looking pretty good - I've updated some minor stuff (please do
a review) but otherwise its all merged now.  There's a few things that
would be good to still get in that I didn't tackle...

- I'm not following why the (static) metrictab has all those entries
  for event counters still?  Can those be removed now?

- The permissions_check() changed to only look for uid==0 - should we
  drop all references to gid* elsewhere?  I think so (slims down some
  data structures, simplifies the attributes callback, etc)

- As per earlier mail, there's no error handling for auto-fu counters.
  Simplest will be, I think, to give refresh_metric() a parameter that
  indicates whether its caller is going to ignore the return code, and
  in that case, handle_papi_error() should really log any errors into
  the pmdapapi.log file.  In the pmStore-enable/disable-metrics case,
  these need not be logged (the client tool will get notified about an
  error in that case, which is better - keep that pmDebug diagnostic
  for both cases though).

Thanks!

--
Nathan

From tdm@sgi.com  Fri Nov 14 09:09:07 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 1606B7F3F
	for ; Fri, 14 Nov 2014 09:09:07 -0600 (CST)
Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10])
	by relay2.corp.sgi.com (Postfix) with ESMTP id DE53A304062;
	Fri, 14 Nov 2014 07:09:03 -0800 (PST)
Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11])
	by estes.americas.sgi.com (Postfix) with ESMTP id B6832700394B;
	Fri, 14 Nov 2014 09:09:03 -0600 (CST)
Message-ID: <54661B0F.1040106@sgi.com>
Date: Fri, 14 Nov 2014 09:09:03 -0600
From: Troy McCorkell 
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: Troy McCorkell , pcp@oss.sgi.com
Subject: Re: oss.sgi.com - maintenance downtime Friday 11/14/2014 at 10:00
 CST USA
References: <545CE78C.5000102@sgi.com> <54637689.2060701@sgi.com>
In-Reply-To: <54637689.2060701@sgi.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Third and final reminder.

On 11/12/2014 09:02 AM, Troy McCorkell wrote:
> Second reminder.
>
> On 11/07/2014 09:38 AM, Troy McCorkell wrote:
>>
>> On Friday November 14, 2014 at 10:00 CST USA
>> oss.sgi.com will be unavailable for a short period of time to perform 
>> network maintenance.
>>
>> The outage is expected to last approximately 15 minutes.
>>
>

From fche@redhat.com  Fri Nov 14 09:27:25 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id D2F5B7F3F
	for ; Fri, 14 Nov 2014 09:27:25 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id C1B268F8040
	for ; Fri, 14 Nov 2014 07:27:25 -0800 (PST)
X-ASG-Debug-ID: 1415978841-04cb6c0573451f0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id TCD4c3T5eADefwrf (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 07:27:22 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAEFRHY0016058
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL);
	Fri, 14 Nov 2014 10:27:18 -0500
Received: from fche.csb (vpn-235-113.phx2.redhat.com [10.3.235.113])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAEFRHc2009833;
	Fri, 14 Nov 2014 10:27:17 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 8B9045816D; Fri, 14 Nov 2014 10:27:16 -0500 (EST)
Date: Fri, 14 Nov 2014 10:27:16 -0500
From: "Frank Ch. Eigler" 
To: Shirshendu Chakrabarti 
Cc: Nicolas Michel , yves.weber@nrb.be,
        pcp@oss.sgi.com
Subject: Re: [pcp] Process analysis
Message-ID: <20141114152716.GI6430@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Process analysis
References:   
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1415978841
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

On Fri, Nov 14, 2014 at 12:00:24PM +0530, Shirshendu Chakrabarti wrote:

> This can be a probable solution. You can use systemtap to trace the
> desired process. [...]  Please see, AFAIK, there is no known
> integration with PCP for above solutions.

systemtap et al. are certainly possible sources of additional data for
PCP.  We're in the process of building out a richer systemtap-pcp
interface.  Do you have any more specific ideas about what new
per-process or intra-kernel statistics you need collected?

- FChE

From tdm@sgi.com  Fri Nov 14 10:40:11 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 9FC5A7F3F
	for ; Fri, 14 Nov 2014 10:40:11 -0600 (CST)
Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 0EEF4AC005
	for ; Fri, 14 Nov 2014 08:40:10 -0800 (PST)
Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11])
	by estes.americas.sgi.com (Postfix) with ESMTP id 89519700396D;
	Fri, 14 Nov 2014 10:40:10 -0600 (CST)
Message-ID: <5466306A.6010804@sgi.com>
Date: Fri, 14 Nov 2014 10:40:10 -0600
From: Troy McCorkell 
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: xfs@oss.sgi.com, pcp@oss.sgi.com
Subject: test - please ignore
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

test email to verify the network maintenance on oss.sgi.com

From tdm@sgi.com  Fri Nov 14 10:56:12 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id AC6F07F3F
	for ; Fri, 14 Nov 2014 10:56:12 -0600 (CST)
Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 386CEAC007;
	Fri, 14 Nov 2014 08:56:12 -0800 (PST)
Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11])
	by estes.americas.sgi.com (Postfix) with ESMTP id 7B62D700397A;
	Fri, 14 Nov 2014 10:56:11 -0600 (CST)
Message-ID: <5466342B.8080205@sgi.com>
Date: Fri, 14 Nov 2014 10:56:11 -0600
From: Troy McCorkell 
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Cc: Troy McCorkell 
Subject: oss.sgi.com - network maintenance complete
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit


The network maintenance on oss.sgi.com is complete.
If you have problems accessing oss.sgi.com, please send me an email.

Thanks,
Troy McCorkell
tdm@sgi.com

From tdm@sgi.com  Fri Nov 14 11:07:58 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id CF8AB7F3F
	for ; Fri, 14 Nov 2014 11:07:58 -0600 (CST)
Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 60AE3AC005
	for ; Fri, 14 Nov 2014 09:07:55 -0800 (PST)
Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11])
	by estes.americas.sgi.com (Postfix) with ESMTP id 5F395700396F;
	Fri, 14 Nov 2014 10:48:38 -0600 (CST)
Message-ID: <54663266.2020103@sgi.com>
Date: Fri, 14 Nov 2014 10:48:38 -0600
From: Troy McCorkell 
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Cc: Troy McCorkell 
Subject: test 2 - please ignore
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

test email to verify network maintenance

From tdm@sgi.com  Fri Nov 14 11:09:37 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD
	autolearn=unavailable version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id D9A077F3F
	for ; Fri, 14 Nov 2014 11:09:37 -0600 (CST)
Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 6A7F7AC005
	for ; Fri, 14 Nov 2014 09:09:37 -0800 (PST)
Received: from [128.162.232.11] (porter.americas.sgi.com [128.162.232.11])
	by estes.americas.sgi.com (Postfix) with ESMTP id C97D9700396E;
	Fri, 14 Nov 2014 10:43:19 -0600 (CST)
Message-ID: <54663127.7020601@sgi.com>
Date: Fri, 14 Nov 2014 10:43:19 -0600
From: Troy McCorkell 
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: xfs@oss.sgi.com, pcp@oss.sgi.com, tdm@sgi.com
Subject: Re: test - please ignore
References: <5466306A.6010804@sgi.com>
In-Reply-To: <5466306A.6010804@sgi.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 11/14/2014 10:40 AM, Troy McCorkell wrote:
> test email to verify the network maintenance on oss.sgi.com

From be.nicolas.michel@gmail.com  Fri Nov 14 11:18:34 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,
	T_DKIM_INVALID autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 2E1187F3F
	for ; Fri, 14 Nov 2014 11:18:34 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id A2F82AC005
	for ; Fri, 14 Nov 2014 09:18:33 -0800 (PST)
X-ASG-Debug-ID: 1415981286-04cbb01e5c72b60001-S8gJnT
Received: from mail-oi0-f45.google.com (mail-oi0-f45.google.com [209.85.218.45]) by cuda.sgi.com with ESMTP id OufhxLXBAAxQn55M (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Fri, 14 Nov 2014 08:08:06 -0800 (PST)
X-Barracuda-Envelope-From: be.nicolas.michel@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.218.45
X-Barracuda-IPDD: Level1 [gmail.com/209.85.218.45]
Received: by mail-oi0-f45.google.com with SMTP id a141so3988737oig.4
        for ; Fri, 14 Nov 2014 08:08:06 -0800 (PST)
X-Barracuda-IPDD: Level1 [gmail.com/209.85.218.45]
X-Barracuda-IPDD: Level1 [gmail.com/209.85.218.45]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        bh=SKG0gYkXkrLuCoPUx3+HOr6lui52ylrA9UJhATHZBU8=;
        b=zd1wvnSkAYyjCuUy1lO82HKtYb8BGnWbq+uH0NSyQzjPfCSefgkULC9QyY1gYStyej
         ezqAOX7/T5CKAic74i7aXRshrX7LLcD3WgrNJgUPEhbctE1CjOcufXyNrm0dbDO2a5ez
         TFkmSUOVTnqvFhLlkoacAB4MZLXuVc9vg5QEnk08RXgDf8mnvMV85izKgNpnMOyyZr5S
         9M3tg43h916u5N23VDsUrPqjdnZjBxRUlmu1Xne6sBziX61Jid+IF6hnBKRQFdzuOGCr
         ASuBq3i2LuXUyLw/f8unr7su0uin5YlWkrZGn5YDWOb6JVCsP1lSwuS4KEovd1W+QQBU
         AyIw==
MIME-Version: 1.0
X-Received: by 10.182.89.161 with SMTP id bp1mr8497634obb.19.1415980967227;
 Fri, 14 Nov 2014 08:02:47 -0800 (PST)
Received: by 10.60.62.73 with HTTP; Fri, 14 Nov 2014 08:02:47 -0800 (PST)
In-Reply-To: <20141114152716.GI6430@redhat.com>
References: 
	
	
	<20141114152716.GI6430@redhat.com>
Date: Fri, 14 Nov 2014 17:02:47 +0100
Message-ID: 
Subject: Re: [pcp] Process analysis
From: Nicolas Michel 
X-ASG-Orig-Subj: Re: [pcp] Process analysis
To: "Frank Ch. Eigler" 
Cc: Shirshendu Chakrabarti , yves.weber@nrb.be, pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=089e013d0e028f776c0507d3c2a2
X-Barracuda-Connect: mail-oi0-f45.google.com[209.85.218.45]
X-Barracuda-Start-Time: 1415981286
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11594
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 HTML_MESSAGE           BODY: HTML included in message

--089e013d0e028f776c0507d3c2a2
Content-Type: text/plain; charset=UTF-8

Thank you all for your ideas. We'll try the native web app ;)

The majority of ideas you propose, if I understand well, makes possible to
measure the stats of predefined processes we have to configure. It is great
when you know in advance which process and what in the process you want to
watch.

Before, in case of a performance problem (a real case I had here), you
first have to identify which process is the cause of the performance
problem.
So I would wanted something more like a top with history, and with I/O
stats (it only keeps says the stats of the tenth hungrier processes). I
think it would be a great plugin, disabled by default of course. And I
would want to check process stats every hour or 2 hours (it is a matter of
reading /proc/* I suppose?). In case of problem, we could change the
precision or even seing it "live" like when connecting to pmcd directly
with pmchart.

I need more a by-process statistics than intra-kernel stats. That "magical"
plugin ;) would able me to tell: it is the process XYZ with PID XXX that
from 01:00pm to 02:00pm used the more the I/O, at a first glance. Then I
would be able to watch that particular process more closely/precisely, with
custom probes if necessary, which systemtap-pcp is for if I understood well.

I know collected  have a feature that can help: you
can have a timeline of processes forking. But collected is not well
packaged and difficult to install/configure (and often crash). And it
doesn't give me I/O, memory and CPU stats by-process.

2014-11-14 16:27 GMT+01:00 Frank Ch. Eigler :

> Hi -
>
> On Fri, Nov 14, 2014 at 12:00:24PM +0530, Shirshendu Chakrabarti wrote:
>
> > This can be a probable solution. You can use systemtap to trace the
> > desired process. [...]  Please see, AFAIK, there is no known
> > integration with PCP for above solutions.
>
> systemtap et al. are certainly possible sources of additional data for
> PCP.  We're in the process of building out a richer systemtap-pcp
> interface.  Do you have any more specific ideas about what new
> per-process or intra-kernel statistics you need collected?
>
> - FChE
>



-- 
Nicolas MICHEL

--089e013d0e028f776c0507d3c2a2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Thank you all for your ideas. We'll try the native web=
 app ;)

The majority of ideas you propose, if I understa=
nd well, makes possible to measure the stats of predefined processes we hav=
e to configure. It is great when you know in advance which process and what=
 in the process you want to watch.

Before, in case=
 of a performance problem (a real case I had here), you first have to ident=
ify which process is the cause of the performance problem.
So I w=
ould wanted something more like a top with history, and with I/O stats (it =
only keeps says the stats of the tenth hungrier processes). I think it woul=
d be a great plugin, disabled by default of course. And I would want to che=
ck process stats every hour or 2 hours (it is a matter of reading /proc/* I=
 suppose?). In case of problem, we could change the precision or even seing=
 it "live" like when connecting to pmcd directly with pmchart.

I need more a by-process statistics than intra-kerne=
l stats. That "magical" plugin ;) would able me to tell: it is th=
e process XYZ with PID XXX that from 01:00pm to 02:00pm used the more the I=
/O, at a first glance. Then I would be able to watch that particular proces=
s more closely/precisely, with custom probes if necessary, which systemtap-=
pcp is for if I understood well.

I know collected have a feature that can help: you can=
 have a timeline of processes forking. But collected is not well packaged a=
nd difficult to install/configure (and often crash). And it doesn't giv=
e me I/O, memory and CPU stats by-process.

2014-11-14 16:27 GMT+01:00 Frank Ch. E=
igler <fche@redhat.com>:
Hi =
-



On Fri, Nov 14, 2014 at 12:00:24PM +0530, Shirshendu Chakrabarti wrote:



> This can be a probable solution. You can use systemtap to trace the
> desired process. [...]=C2=A0 Please see, AFAIK, there is no kno=
wn

> integration with PCP for above solutions.



systemtap et al. are certainly possible sources of additional data f=
or

PCP.=C2=A0 We're in the process of building out a richer systemtap-pcp<=
br>
interface.=C2=A0 Do you have any more specific ideas about what new

per-process or intra-kernel statistics you need collected?



- FChE





-- 
Nicolas MICHEL




--089e013d0e028f776c0507d3c2a2--

From dsmith@redhat.com  Fri Nov 14 15:17:55 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 584577F3F
	for ; Fri, 14 Nov 2014 15:17:55 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay3.corp.sgi.com (Postfix) with ESMTP id E99C9AC002
	for ; Fri, 14 Nov 2014 13:17:51 -0800 (PST)
X-ASG-Debug-ID: 1415999867-04bdf0615f55860001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 6JZGdekIFdB8BYuQ (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 13:17:48 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAELHkHJ021982
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Fri, 14 Nov 2014 16:17:47 -0500
Received: from t540p.usersys.redhat.com (vpn-51-241.rdu2.redhat.com [10.10.51.241])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAELHjXd011790
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Fri, 14 Nov 2014 16:17:46 -0500
Message-ID: <54667179.1060605@redhat.com>
Date: Fri, 14 Nov 2014 15:17:45 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com>	<1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com>	<545A53B0.9030500@redhat.com>	<704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com>	<545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1415999868
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/10/2014 08:40 PM, Nathan Scott wrote:
> Hi David,
> 
> ----- Original Message -----
>> [...]
>> We have 3 distinct classes of dictionaries, each treated differently.
>> During PMDA.run(), the PMDA class passes down to cpmda several dictionaries:
>>
>> (1) self._metric_names (used by cpmda.pmns_refresh)
>> (2) self._metric_oneline (used by cpmda.pmid_oneline_refresh)
>> (2) self._metric_helptext (used by cpmda.pmid_longtext_refresh)
>> (2) self._indom_oneline (used by cpmda.indom_oneline_refresh)
>> (2) self._indom_helptext (used by cpmda.indom_longtext_refresh)
>> (3) self._indomtable (used by cpmda.pmda_dispatch)
>> (3) self._metrictable (used by cpmda.pmda_dispatch)
>>
>> Dictionaries marked by (1) are passed down, used once when needed, then
>> discarded.
>>
>> Dictionaries marked by (2) are passed down and kept, used when needed,
>> never discarded.
>>
>> Dictionaries marked by (3) aren't passed down and kept. Instead the PMDA
>> class creates a string buffer for each dictionary and passes that down
>> to cpmda. This string buffer is used once and then discarded.
>>
>> I was thinking all dictionaries were treated as being in class (1) (used
>> once then discarded). That's why I was passing down everything again in
>> PMDA.__refresh_metrics(). Since I don't need to pass down items again in
>> class (2), that simplifies PMDA.__refresh_metrics() to:
>>
>> ====
>>     def __refresh_metrics(self):
>>         # Call the user's function.
>>         if self.__refresh_metrics_func:
>>             self.__refresh_metrics_func()
>>
>>         if cpmda.get_need_refresh():
>>             self.pmns_refresh()
>>             cpmda.pmda_rehash(self._indomtable, self._metrictable)
>> ====
> 
> Yeah, that's much neater - nice!
> 
>> We actually could go further here, if we can change the cpmda interface.
>> I'm not sure how set in stone it is. I'm not talking about a change to
>> the PMDA class interface here, just the cpmda interface. If we change
>> cpmda.pmda_dispatch() to take the indomtable and metrictable
>> dictionaries directly (instead of using string buffers), we could keep
>> them around for use later by pmda_rehash() - basically moving them to
>> class (2).

So, commit b2e3b5c in the pcpfans.git dsmith/dev branch does things as I
discussed above. That dramatically simplifies the changes to pmda.py (1)
and the refresh_metrics wrapper in pmda.py goes completely away.

(1) Except for one thing. I about went nuts with one problem. So we
stashed list/dictionary references down in cpmda. I'd update the
list/dictionaries in pmda.py and would see newly added metric info, but
if I cleared the metrics, I'd still get the old values. I finally
figured out the difference between 'dict = []' and 'dict.clear()'. The
former assigns a new empty dictionary while the latter clears out the
current dictionary. At the pmda.py level there really isn't any
difference, but the caused problems for cpmda which had a stashed
reference to the original dictionary, not the new one.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From fche@redhat.com  Fri Nov 14 15:21:50 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 6EA347F3F
	for ; Fri, 14 Nov 2014 15:21:50 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 5C178304043
	for ; Fri, 14 Nov 2014 13:21:47 -0800 (PST)
X-ASG-Debug-ID: 1416000105-04cbb01e597ef00001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id gHRsoomwxEmMtOYs (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 14 Nov 2014 13:21:46 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAELLjuY029857
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Fri, 14 Nov 2014 16:21:45 -0500
Received: from fche.csb (vpn-235-113.phx2.redhat.com [10.3.235.113])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAELLj39000696;
	Fri, 14 Nov 2014 16:21:45 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id C32135816D; Fri, 14 Nov 2014 16:21:44 -0500 (EST)
To: Nathan Scott 
Cc: pcp 
Subject: Re: pcp updates: merges (kenj,michele,lberk)
References: <659079660.14432964.1415954261182.JavaMail.zimbra@redhat.com>
	<319556460.14436082.1415954464312.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: pcp updates: merges (kenj,michele,lberk)
From: fche@redhat.com (Frank Ch. Eigler)
Date: Fri, 14 Nov 2014 16:21:44 -0500
In-Reply-To: <319556460.14436082.1415954464312.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Fri, 14 Nov 2014 03:41:04 -0500 (EST)")
Message-ID: 
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416000106
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Nathan Scott  writes:

> [...]
> Nathan Scott (5):
>       pmdapapi: small cleanups after review
> [...]

Please reconsider the removal of some the comment blocks.  Some (the
int return code markups) were put in place because of genuine problems
found in prior reviews; some (the pmaf ones) were put in place as an
aid to diagnose future known problems.

- FChE

From kenj@internode.on.net  Fri Nov 14 15:26:02 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 062497F3F
	for ; Fri, 14 Nov 2014 15:26:02 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id C92798F804C
	for ; Fri, 14 Nov 2014 13:26:01 -0800 (PST)
X-ASG-Debug-ID: 1416000359-04bdf0616155d10001-S8gJnT
Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id XyfCQNFDKytL52fe for ; Fri, 14 Nov 2014 13:26:00 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.143
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgILAPdyZlSQiUxsPGdsb2JhbABbgw6BLoI2UIQ3zRAEAgKBGxcBAQEBAQYBAQEBODuEAgEBAQQIAhkFLiMMAQMCBgMOAwQBAQMCIwMCAhkgCgMJCAIEARILBYgxu22WNQEBAQEGAQEBAR6BLY91BwaCcYFUBZJHZqYpKTCCSwEBAQ
Received: from cpe-144-137-76-108.lnse5.cht.bigpond.net.au (HELO bozohorize) ([144.137.76.108])
  by ipmail05.adl6.internode.on.net with ESMTP; 15 Nov 2014 07:55:58 +1030
From: "Ken McDonell" 
To: "'Nicolas Michel'" ,
	"'Frank Ch. Eigler'" 
Cc: ,
	,
	"'Shirshendu Chakrabarti'" 
References: 			<20141114152716.GI6430@redhat.com> 
In-Reply-To: 
Subject: RE: [pcp] Process analysis
Date: Sat, 15 Nov 2014 08:25:56 +1100
X-ASG-Orig-Subj: RE: [pcp] Process analysis
Message-ID: <009701d00051$95512b20$bff38160$@internode.on.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Outlook 15.0
Thread-Index: AQHIKVF6ipCe5Ss+eFYrdYM0LBK/kAJzhbaSAZAMViECNXgjygHZjiR1nC+6m7A=
Content-Language: en-au
X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143]
X-Barracuda-Start-Time: 1416000359
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.01
X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11602
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header


> -----Original Message-----
> From: pcp-bounces@oss.sgi.com [mailto:pcp-bounces@oss.sgi.com] On
> Behalf Of Nicolas Michel
> Sent: Saturday, 15 November 2014 3:03 AM
> To: Frank Ch. Eigler
> Cc: yves.weber@nrb.be; pcp@oss.sgi.com; Shirshendu Chakrabarti
> Subject: Re: [pcp] Process analysis
>=20
> ...
> So I would wanted something more like a top with history, and with I/O =
stats
> (it only keeps says the stats of the tenth hungrier processes). I =
think it would
> be a great plugin, disabled by default of course. And I would want to =
check
> process stats every hour or 2 hours (it is a matter of reading /proc/* =
I
> suppose?). In case of problem, we could change the precision or even =
seing it
> "live" like when connecting to pmcd directly with pmchart.

This sounds a lot like the use case that the hotproc PMDA is intended to =
address ... the hotproc PMDA (in the original IRIX version) used a =
configuration file to define what characterized an "interesting" process =
and then allowed collection of arbitrary proc metrics for those =
processes that were "interesting".

Soon I expect the new hotproc PMDA will be available, and it would be =
interesting to see how well this matches Nicholas' needs.

From minnus@buffalo.edu  Fri Nov 14 20:28:09 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=MIME_QP_LONG_LINE
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 806EE7F58
	for ; Fri, 14 Nov 2014 20:28:09 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 6E4418F8035
	for ; Fri, 14 Nov 2014 18:28:09 -0800 (PST)
X-ASG-Debug-ID: 1416018484-04cb6c05735c780001-S8gJnT
Received: from mtareserve1.acsu.buffalo.edu (mtareserve7.acsu.buffalo.edu [128.205.6.7]) by cuda.sgi.com with ESMTP id Fc1P20akYg49Mbjm for ; Fri, 14 Nov 2014 18:28:05 -0800 (PST)
X-Barracuda-Envelope-From: minnus@buffalo.edu
X-Barracuda-Apparent-Source-IP: 128.205.6.7
Received: from localmailB.acsu.buffalo.edu (localmailb.acsu.buffalo.edu [128.205.5.200])
	by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 9ADD8483;
	Fri, 14 Nov 2014 21:28:04 -0500 (EST)
Received: from localmailB.acsu.buffalo.edu (localhost [127.0.0.1])
	by localhost (Postfix) with SMTP id 8DC61D0F1;
	Fri, 14 Nov 2014 21:28:04 -0500 (EST)
Received: from localmailB.acsu.buffalo.edu (localhost [127.0.0.1])
	by localmailB.acsu.buffalo.edu (Postfix) with ESMTP id 88ED2D0EE;
	Fri, 14 Nov 2014 21:28:03 -0500 (EST)
Received: from smtp.buffalo.edu (smtp3.acsu.buffalo.edu [128.205.5.226])
	by localmailB.acsu.buffalo.edu (Prefixe) with ESMTP id 5FB1DD0ED;
	Fri, 14 Nov 2014 21:28:03 -0500 (EST)
Received: from [10.0.1.171] (cpe-69-204-8-250.buffalo.res.rr.com [69.204.8.250])
	(Authenticated sender: minnus@buffalo.edu)
	by smtp.buffalo.edu (Postfix) with ESMTPSA id 3B164205E;
	Fri, 14 Nov 2014 21:28:03 -0500 (EST)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (1.0)
Subject: Re: [pcp] Dynamic Proc PMDA metrics
From: Martins Innus 
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
X-Mailer: iPad Mail (11D167)
In-Reply-To: <54650E07.90908@buffalo.edu>
Date: Fri, 14 Nov 2014 21:28:02 -0500
Cc: pcp 
Content-Transfer-Encoding: quoted-printable
Message-Id: <63EB3786-F60E-40C5-AB91-7EC80FD91CA9@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54627B7D.6040906@buffalo.edu> <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com> <5463C8C8.5010700@buffalo.edu> <1453660478.13246038.1415858528343.JavaMail.zimbra@redhat.com> <54650E07.90908@buffalo.edu>
To: Nathan Scott 
X-PM-EL-Spam-Prob: XX: 28%
X-Barracuda-Connect: mtareserve7.acsu.buffalo.edu[128.205.6.7]
X-Barracuda-Start-Time: 1416018484
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=MIME_QP_LONG_LINE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11608
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 MIME_QP_LONG_LINE      RAW: Quoted-printable line longer than 76 chars

Ok, I think I have made some progress.

Current version is here:

https://github.com/ubccr/pcp/tree/dynamic_hotproc

I will be traveling all next week and not have much time to work on this unt=
il I get back.

> On Nov 13, 2014, at 3:01 PM, Martins Innus  wrote:
>=20
> Nathan,
>=20
> On 11/13/14 1:02 AM, Nathan Scott wrote:
>>> Still no progress on this, except that with a dso pmda, the domain
>>> number is correct at this point. Illustrated below in the interrupts cas=
e.
>> Looks like a bug - in the cgroups case, the domain# is explicitly
>> stamped into the PMID for the new metric table entry each time, so
>> I suspect we've never seen this before as a result.
> OK.  I haven't dug too far to see where this should happen.

I had misinterpreted what was going on here. The domain # in the pmid was ok=
.  It was the domain of the instance domain that needed to be stamped in. Al=
l should be fine now.

>>=20
>>>=20
>>>      When running through the initial metric table, the only check that
>>> is done is to see if the cluster matches when doing "mtabupdate" and no
>>> check is done against the item or name.  This will cause multiple calls
>>> for the same metric if we have clusters that span multiple trees (since
>> If we take away "clusters spanning trees" this all gets simpler I think?
>> Am I oversimplifying things there?  (apologies if so - I'll look at it
>> more closely as soon as I get this cgroup stuff finished).
> That's exactly correct.  The existing dynamic code assumes clusters don't s=
pan trees.  This problem goes away if they are distinct.
>=20
> This assumption is also made in the following methods:
>=20
> pmdaDynamicLookupPMID
> pmdaDynamicLookupText
>=20
> These will use the first tree that has the matching cluster.
>=20
> Actually.  Thinking this through. Something like the attached patch could f=
ix this issue once we can get the domain # in the source metric. I can't rea=
lly test it until the domain # is there since the search fails currently.  N=
ot sure of any performance issues with all those searches.
>=20

I now have a fix for this.  I modified dynamic.c significantly to get rid of=
 the checks against cluster and prefix.  They are not used at all, I just le=
ft the prefix in for now since it is used to do the cluster mask.  I'm not s=
ure if that will be needed in the new cgroups anymore.
   All the checks are now done just against the pmns. This allows there to b=
e only one setup call for dynamic proc metrics.

Let me know if you think this is a workable solution or if there is a better=
 way to do it.

Martins=

From fche@redhat.com  Sun Nov 16 09:05:47 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 792DF7F51
	for ; Sun, 16 Nov 2014 09:05:47 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id E1857304043
	for ; Sun, 16 Nov 2014 07:05:41 -0800 (PST)
X-ASG-Debug-ID: 1416150340-04cbb01e5bd2180001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id beNxxn3CQPqgLzfH (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 16 Nov 2014 07:05:40 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAGF5dQx026791
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Sun, 16 Nov 2014 10:05:40 -0500
Received: from fche.csb (vpn-235-113.phx2.redhat.com [10.3.235.113])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAGF5d3a020533
	for ; Sun, 16 Nov 2014 10:05:39 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 28A0C5822C; Sun, 16 Nov 2014 10:05:39 -0500 (EST)
Date: Sun, 16 Nov 2014 10:05:39 -0500
From: "Frank Ch. Eigler" 
To: pcp developers 
Subject: pcp update: pmwebd graphite-dashboard support
Message-ID: <20141116150539.GB11329@redhat.com>
X-ASG-Orig-Subj: pcp update: pmwebd graphite-dashboard support
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416150340
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

While invited to play with another serious graphite installation,
some fixable shortcomings in pmwebd's emulation became apparent.

commit 895f416a3b45a5d7c8e1eee4c1ed1401b6f3699e (HEAD, origin/fche/dev, fche/dev)
Author: Frank Ch. Eigler 
Date:   Sun Nov 16 09:46:49 2014 -0500

    pmwebd: improve /metrics/find emulation for graphite dashboard
    
    Comparing a live graphite installation to pmwebd's emulation, it
    showed that a small tweak to the JSON result formatting was sufficient
    to get the basic graphite-side dashboard working.  Further tweaking
    made it possible to expand wildcards throughout the graphite metric
    string in a compatible manner.  (The grafana interactive metric
    builder is affected, and keeps working.)  A further tweak lets pmwebd
    tolerate the graphite dashboard webapp's tendency to use
    non-/graphite-prefixed URLs to do its work.

From janfrode@tanso.net  Sun Nov 16 14:10:10 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 887FD29DF7
	for ; Sun, 16 Nov 2014 14:10:10 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 5F4198F8035
	for ; Sun, 16 Nov 2014 12:10:07 -0800 (PST)
X-ASG-Debug-ID: 1416168601-04cbb01e5ad7d50001-S8gJnT
Received: from mail-la0-f47.google.com (mail-la0-f47.google.com [209.85.215.47]) by cuda.sgi.com with ESMTP id JZI7Xd6M3pRlHe1t (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Sun, 16 Nov 2014 12:10:02 -0800 (PST)
X-Barracuda-Envelope-From: janfrode@tanso.net
X-Barracuda-Apparent-Source-IP: 209.85.215.47
Received: by mail-la0-f47.google.com with SMTP id gq15so1628227lab.34
        for ; Sun, 16 Nov 2014 12:10:00 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:date:from:to:subject:message-id:mime-version
         :content-type:content-disposition:user-agent;
        bh=NCObwZ/5uHROeLmE9HznNdateNuR0eNZZWwWwKxhNMU=;
        b=R6FlcLTWkBPY1Dw8Dqpcoa/dYpXnei19LsLQqjjDUnOdwFUJUDKJH98lieoSOUBTMo
         3oM3ZXYk2pqLiJmLhXk02wO5XTEFU0bUAkYAPOvlZTOVlRFAOhTX3xp104V/F8/6zzXS
         QK3ahXWfIOn+z6We9MDotFX0dEIQh7CY0DQETd2bVeh93JVLAMVQFUwf0JUVjR8v7EvF
         DfJp0hsM0JhDCdVBZrgKQYOl1wEC2vAxewtRmOXWbLsGsos00zDvZqmOMrx0pmTA1n1x
         jypNVkVPaZihPyWbxwHOVEVYi7YbfzYpSvlnfDlUm+pZTAE+2tSrJOHlmwaDJS3F2U1Z
         h12Q==
X-Gm-Message-State: ALoCoQmdOkF43dlHj9T15suLDnyu24+V4ef3ZjHf6AuwH/m+UYuEJknXYwTS1bF8GbylcTiDXdYz
X-Received: by 10.112.156.138 with SMTP id we10mr5150597lbb.88.1416168600560;
        Sun, 16 Nov 2014 12:10:00 -0800 (PST)
Received: from localhost (120.81-167-109.customer.lyse.net. [81.167.109.120])
        by mx.google.com with ESMTPSA id p9sm9345777laj.8.2014.11.16.12.09.59
        for 
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Sun, 16 Nov 2014 12:10:00 -0800 (PST)
Date: Sun, 16 Nov 2014 21:09:58 +0100
From: Jan-Frode Myklebust 
To: pcp@oss.sgi.com
Subject: My first PMDA, some questions..
Message-ID: <20141116200958.GA8464@mushkin.tanso.net>
X-ASG-Orig-Subj: My first PMDA, some questions..
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
X-Barracuda-Connect: mail-la0-f47.google.com[209.85.215.47]
X-Barracuda-Start-Time: 1416168601
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11665
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

I'm writing my first python pmda to collect 143 global + numthreads*13
metrics from the Unbound recursive DNS server. A sample of the metrics:

thread0.num.queries=565719821
thread0.num.cachehits=537437733
thread0.num.cachemiss=28282088
thread1.num.queries=565719821
thread1.num.cachehits=537437733
thread1.num.cachemiss=28282088
--
total.num.queries=3359533335
total.num.cachehits=2967548455
total.num.cachemiss=391984880
total.requestlist.avg=293.892
total.requestlist.max=4335
total.requestlist.overwritten=85422
total.requestlist.exceeded=528767
num.query.type.TYPE0=1967231
num.query.type.A=2761228065
num.query.type.NS=2494800
num.query.type.MD=3
num.query.type.MF=3

So one set of metrics that are repeated once for each thread the server
is defined to run, and another set of metrics that will always be there.
The numbers are generated by running the command "unbound-control
stats_noreset".

I have a simple python pmda running that sets up and populates the
total.num metrics, and that works fine, but I don't see how I'm
supposed to handle the dynamic numthreads metrics. Any pointers? What I
have now are simple:

        self.values['total.num.queries'] = 0
        self.add_metric(name + '.total.num.queries', pmdaMetric(
                self.pmid(0, 0),
                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
        'Total number of queries from all threads.')


How would I add the "threadX.num.queries" when I don't know how many
threads I will have?

Also, I'm wondering how often the fetch() procedure in the pmda is called,
and if there's any way of controlling that. I don't know how advicable it
will be to be calling the external command for generating all stats every
second..

What's triggering the fetch()? Is it me running "pmval -t 1 total.num.queries",
or will it always be collecting at a given intervall?


   -jf

From nscott@redhat.com  Mon Nov 17 00:29:10 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 6E8717F56
	for ; Mon, 17 Nov 2014 00:29:10 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 58D988F8035
	for ; Sun, 16 Nov 2014 22:29:07 -0800 (PST)
X-ASG-Debug-ID: 1416205741-04cb6c0573de490001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id BwDmn3GDULBFpxMq (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 16 Nov 2014 22:29:02 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAH6T1LS002990;
	Mon, 17 Nov 2014 01:29:01 -0500
Date: Mon, 17 Nov 2014 01:28:59 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: David Smith 
Cc: pcp 
Message-ID: <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com>
In-Reply-To: <54667179.1060605@redhat.com>
References: <54512E80.9090302@redhat.com> <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> <545A53B0.9030500@redhat.com> <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com>
Subject: Re: [pcp] [RFC] pcp python patch
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp python patch
Thread-Index: zlfFpJNJmaEJWiSONDYGhZBaqyXIrA==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1416205742
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11679
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> [...]
> So, commit b2e3b5c in the pcpfans.git dsmith/dev branch does things as I
> discussed above. That dramatically simplifies the changes to pmda.py (1)
> and the refresh_metrics wrapper in pmda.py goes completely away.

Nice.

> (1) Except for one thing. I about went nuts with one problem. So we
> stashed list/dictionary references down in cpmda. I'd update the
> list/dictionaries in pmda.py and would see newly added metric info, but
> if I cleared the metrics, I'd still get the old values. I finally
> figured out the difference between 'dict = []' and 'dict.clear()'. The
> former assigns a new empty dictionary while the latter clears out the
> current dictionary. At the pmda.py level there really isn't any
> difference, but the caused problems for cpmda which had a stashed
> reference to the original dictionary, not the new one.

Argh!  Good find.  Do we need a new API to allow new dictionary object
ref(s) to be pushed down to cpmda, overwriting the old?

cheers.

--
Nathan

From nscott@redhat.com  Mon Nov 17 00:29:34 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 3EAFC7F56
	for ; Mon, 17 Nov 2014 00:29:34 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 03366304043
	for ; Sun, 16 Nov 2014 22:29:30 -0800 (PST)
X-ASG-Debug-ID: 1416205768-04bdf0615fbc550001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id E3BSfip8S7ZzfaG2 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 16 Nov 2014 22:29:28 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAH6TSuC027551;
	Mon, 17 Nov 2014 01:29:28 -0500
Date: Mon, 17 Nov 2014 01:29:27 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Jan-Frode Myklebust 
Cc: pcp@oss.sgi.com
Message-ID: <69304097.15487953.1416205767949.JavaMail.zimbra@redhat.com>
In-Reply-To: <20141116200958.GA8464@mushkin.tanso.net>
References: <20141116200958.GA8464@mushkin.tanso.net>
Subject: Re: [pcp] My first PMDA, some questions..
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] My first PMDA, some questions..
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: My first PMDA, some questions..
Thread-Index: CcRWSbswWHqdEMLBztIw86+mzEUVCw==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1416205768
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.62
X-Barracuda-Spam-Status: No, SCORE=0.62 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=COMMA_SUBJECT, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11679
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.60 COMMA_SUBJECT          Subject is like 'Re: FDSDS, this is a subject'

Hi Jan-Frode,

----- Original Message -----
> I'm writing my first python pmda to collect 143 global + numthreads*13
> metrics from the Unbound recursive DNS server. A sample of the metrics:

Great, welcome!

> [...]
> I have a simple python pmda running that sets up and populates the
> total.num metrics, and that works fine, but I don't see how I'm
> supposed to handle the dynamic numthreads metrics. Any pointers? What I
> have now are simple:

There are two options:
- using an instance domain to represent the dynamic nature of the threads
(see add_indom() in pmdagluster for a python example)
- using dynamic metrics (all python metrics are actually dynamic, but see
below for current restrictions).

>         self.values['total.num.queries'] = 0
>         self.add_metric(name + '.total.num.queries', pmdaMetric(
>                 self.pmid(0, 0),
>                 c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
>                 pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
>         'Total number of queries from all threads.')
> 
> 
> How would I add the "threadX.num.queries" when I don't know how many
> threads I will have?

At what point do you know how many threads you have?  If you can find out
before the call to PMDA.run(), then you can just add them like any other
metric (with the first parameter to add_metric() being "thread%d.xxx".  If
you can only find out after the call to run(), then the current python API
will not work for you.

There is an effort nearing completion to allow metrics to be registered at
any point after PMDA.run() too, via the python API - this will be merged
shortly [http://www.pcp.io/pipermail/pcp/2014-November/005969.html] but is
not in place for you right now.

> Also, I'm wondering how often the fetch() procedure in the pmda is called,
> and if there's any way of controlling that. I don't know how advicable it
> will be to be calling the external command for generating all stats every
> second..
> What's triggering the fetch()? Is it me running "pmval -t 1
> total.num.queries",

(yep)

> or will it always be collecting at a given intervall?

(no)

The client (monitor) tools drive the fetch interval, PMDAs just respond with
the latest values - they typically have no knowledge of the sampling rate.
There can also be multiple clients, sampling at different (unrelated) times.

If you have concerns about running the command frequently, one option is to
cache the results for a short time, and respond with the cached results if
fetches come in too quickly for the PMDA to comfortably handle.  Chapter 2
of the Programmers Guide (http://www.pcp.io/doc/pcp-programmers-guide.pdf)
has a section "2.2.4 - Caching PMDA" with a more details.

cheers.

--
Nathan

From nscott@redhat.com  Mon Nov 17 00:39:48 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id DC4347F5A
	for ; Mon, 17 Nov 2014 00:39:48 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id C546A304043
	for ; Sun, 16 Nov 2014 22:39:48 -0800 (PST)
X-ASG-Debug-ID: 1416206386-04cbb01e59e2c30001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id BgoTXOU16Qg9O9pA (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 16 Nov 2014 22:39:47 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAH6djwS007597;
	Mon, 17 Nov 2014 01:39:45 -0500
Date: Mon, 17 Nov 2014 01:39:44 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Martins Innus 
Cc: pcp 
Message-ID: <159273086.15490672.1416206384938.JavaMail.zimbra@redhat.com>
In-Reply-To: <63EB3786-F60E-40C5-AB91-7EC80FD91CA9@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54627B7D.6040906@buffalo.edu> <2023629481.12445911.1415770455874.JavaMail.zimbra@redhat.com> <5463C8C8.5010700@buffalo.edu> <1453660478.13246038.1415858528343.JavaMail.zimbra@redhat.com> <54650E07.90908@buffalo.edu> <63EB3786-F60E-40C5-AB91-7EC80FD91CA9@buffalo.edu>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: aR7zolYWr8UjYBFLwOf6JncCWxU7nw==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1416206387
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11679
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi Martins,

----- Original Message -----
> > [...]
> > This assumption is also made in the following methods:
> > 
> > pmdaDynamicLookupPMID
> > pmdaDynamicLookupText
> > 
> > These will use the first tree that has the matching cluster.
> > 
> > Actually.  Thinking this through. Something like the attached patch could
> > fix this issue once we can get the domain # in the source metric. I can't

(no patch attached?)

> [...]
> I now have a fix for this.  I modified dynamic.c significantly to get rid of
> the checks against cluster and prefix.  They are not used at all, I just
> left the prefix in for now since it is used to do the cluster mask.  I'm not

OK.

> sure if that will be needed in the new cgroups anymore.

Nope.

>    All the checks are now done just against the pmns. This allows there to be
>    only one setup call for dynamic proc metrics.

Yep - sounds simpler & betterer.

cheers.

--
Nathan

From dsmith@redhat.com  Mon Nov 17 12:56:53 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id AAC8A7F51
	for ; Mon, 17 Nov 2014 12:56:53 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 95F468F8059
	for ; Mon, 17 Nov 2014 10:56:53 -0800 (PST)
X-ASG-Debug-ID: 1416250612-04cbb01e59fad20001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id xU4DAeGAplW885Ao (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 10:56:52 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAHIupJH003743
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 17 Nov 2014 13:56:52 -0500
Received: from t540p.usersys.redhat.com (vpn-52-167.rdu2.redhat.com [10.10.52.167])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHIumYP021094
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Mon, 17 Nov 2014 13:56:51 -0500
Message-ID: <546A44F0.1070001@redhat.com>
Date: Mon, 17 Nov 2014 12:56:48 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <1832581184.7889066.1415166007440.JavaMail.zimbra@redhat.com> <545A53B0.9030500@redhat.com> <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416250612
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/17/2014 12:28 AM, Nathan Scott wrote:
> 
> 
> ----- Original Message -----
>> [...]
>> So, commit b2e3b5c in the pcpfans.git dsmith/dev branch does things as I
>> discussed above. That dramatically simplifies the changes to pmda.py (1)
>> and the refresh_metrics wrapper in pmda.py goes completely away.
> 
> Nice.
> 
>> (1) Except for one thing. I about went nuts with one problem. So we
>> stashed list/dictionary references down in cpmda. I'd update the
>> list/dictionaries in pmda.py and would see newly added metric info, but
>> if I cleared the metrics, I'd still get the old values. I finally
>> figured out the difference between 'dict = []' and 'dict.clear()'. The
>> former assigns a new empty dictionary while the latter clears out the
>> current dictionary. At the pmda.py level there really isn't any
>> difference, but the caused problems for cpmda which had a stashed
>> reference to the original dictionary, not the new one.
> 
> Argh!  Good find. 

This would only have affected anyone who called
clear_metrics()/reset_metrics() or clear_indoms().

> Do we need a new API to allow new dictionary object
> ref(s) to be pushed down to cpmda, overwriting the old?

Hmm, I'm not really seeing a use case for that with the way the PMDA
class is currently written. What is your thought here?

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From fche@redhat.com  Mon Nov 17 14:54:07 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id EFBAE7F51
	for ; Mon, 17 Nov 2014 14:54:07 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id D019A30405F
	for ; Mon, 17 Nov 2014 12:54:07 -0800 (PST)
X-ASG-Debug-ID: 1416257643-04cb6c0571fb140001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id jyzZQbVOOafmDH2j (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 12:54:04 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAHKs2rE005964
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 17 Nov 2014 15:54:03 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHKs2kV018569;
	Mon, 17 Nov 2014 15:54:02 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id C1E9A5822C; Mon, 17 Nov 2014 15:54:01 -0500 (EST)
To: Nathan Scott 
Cc: Lukas Berk , pcp@oss.sgi.com
Subject: Re: pcp updates - pmdapapi update
References: <87oascow3f.fsf@redhat.com>
	<50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: pcp updates - pmdapapi update
From: fche@redhat.com (Frank Ch. Eigler)
Date: Mon, 17 Nov 2014 15:54:01 -0500
In-Reply-To: <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Fri, 14 Nov 2014 03:53:05 -0500 (EST)")
Message-ID: 
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416257643
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1


nathans wrote:

> [...]  - As per earlier mail, there's no error handling for auto-fu
> counters. [...]

Does this cover the case?

diff --git a/src/pmdas/papi/papi.c b/src/pmdas/papi/papi.c
index 15e224150704..86f38146e197 100644
--- a/src/pmdas/papi/papi.c
+++ b/src/pmdas/papi/papi.c
@@ -645,7 +645,9 @@ papi_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
                 if (auto_enable_time) {
                     // auto-enable this metric for a while
                     papi_info[idp->item].metric_enabled = now + auto_enable_time;
-                    refresh_metrics();
+                    sts = refresh_metrics();
+                    if (sts < 0)
+                        return sts;
                 }
                return PMDA_FETCH_NOVALUES;
             }


- FChE

From nscott@redhat.com  Mon Nov 17 16:30:55 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id CE0387F51
	for ; Mon, 17 Nov 2014 16:30:55 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 6BCB2AC007
	for ; Mon, 17 Nov 2014 14:30:52 -0800 (PST)
X-ASG-Debug-ID: 1416263449-04cbb01e5c1032a0001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id 49EMespzKMzNGoh7 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 14:30:50 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHMUnVb026319;
	Mon, 17 Nov 2014 17:30:49 -0500
Date: Mon, 17 Nov 2014 17:30:49 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: "Frank Ch. Eigler" , Lukas Berk 
Cc: pcp@oss.sgi.com
Message-ID: <237142894.496530.1416263449269.JavaMail.zimbra@redhat.com>
In-Reply-To: 
References: <87oascow3f.fsf@redhat.com> <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com> 
Subject: Re: pcp updates - pmdapapi update
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: pcp updates - pmdapapi update
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates - pmdapapi update
Thread-Index: +eMGV1yLgsESHbOZkadLV2/pqbRliA==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1416263450
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11703
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> > [...]  - As per earlier mail, there's no error handling for auto-fu
> > counters. [...]
> 
> Does this cover the case?

There's two parts to this problem.  The first part is that patch (for
auto-enabling), the second part is during disabling - where do those
errors go?  At the moment, they are all silently discarded.  I think
they at least need to be logged in papi.log - that's the best we can
do, I guess? - there's no way to tell the client, which may not even
be running anymore.

Also, once we have logging of errors on auto-disable, we should add
checks to the existing tests to scan for errors in papi.log (just a
"cat" on the papi.log file should do the trick, with some date/time
filtering for the PMDA start/stop messages that are always logged).

cheers.

--
Nathan

From nscott@redhat.com  Mon Nov 17 16:53:12 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 7B2247F51
	for ; Mon, 17 Nov 2014 16:53:12 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 4D704304039
	for ; Mon, 17 Nov 2014 14:53:09 -0800 (PST)
X-ASG-Debug-ID: 1416264784-04bdf06160dfea0001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id 9OkZg707641eIFAd (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 14:53:04 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHMr3js030934;
	Mon, 17 Nov 2014 17:53:03 -0500
Date: Mon, 17 Nov 2014 17:53:03 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: David Smith 
Cc: pcp 
Message-ID: <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com>
In-Reply-To: <546A44F0.1070001@redhat.com>
References: <54512E80.9090302@redhat.com> <545A53B0.9030500@redhat.com> <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com>
Subject: Re: [pcp] [RFC] pcp python patch
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp python patch
Thread-Index: rUBLr1xkPk/X4NUsa53RImGWYYNwSQ==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1416264784
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11704
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> [...]
> This would only have affected anyone who called
> clear_metrics()/reset_metrics() or clear_indoms().
> 
> > Do we need a new API to allow new dictionary object
> > ref(s) to be pushed down to cpmda, overwriting the old?
> 
> Hmm, I'm not really seeing a use case for that with the way the PMDA
> class is currently written. What is your thought here?

Oh, I just had the impression from your earlier mail you weren't
completely satisfied that we'd covered off all the cases ... its
likely I've just misinterpreted that "except for [1]" reference.

cheers.

--
Nathan

From fche@redhat.com  Mon Nov 17 17:03:35 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id B79817F51
	for ; Mon, 17 Nov 2014 17:03:35 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 54796AC006
	for ; Mon, 17 Nov 2014 15:03:35 -0800 (PST)
X-ASG-Debug-ID: 1416265409-04cb6c0571ff470001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id B3GPFOsXohZkVYTu (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 15:03:30 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAHN3TaI025600
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 17 Nov 2014 18:03:29 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHN3TQl006026;
	Mon, 17 Nov 2014 18:03:29 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 9DB285822C; Mon, 17 Nov 2014 18:03:28 -0500 (EST)
Date: Mon, 17 Nov 2014 18:03:28 -0500
From: "Frank Ch. Eigler" 
To: Nathan Scott 
Cc: Lukas Berk , pcp@oss.sgi.com
Subject: Re: pcp updates - pmdapapi update
Message-ID: <20141117230328.GB5700@redhat.com>
X-ASG-Orig-Subj: Re: pcp updates - pmdapapi update
References: <87oascow3f.fsf@redhat.com> <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>  <237142894.496530.1416263449269.JavaMail.zimbra@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <237142894.496530.1416263449269.JavaMail.zimbra@redhat.com>
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416265410
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

> There's two parts to this problem.  The first part is that patch (for
> auto-enabling), the second part is during disabling - where do those
> errors go?  [...]

Yeah, that part is tricky (and really not much better in the pmstore
.disable case); there are just too many possible failure points.  The
log file, as you say, is probably the best bet for now.  In the longer
future, perhaps we can explore further our earlier ideas about
nested/detailed error messages.

- FChE

From dsmith@redhat.com  Mon Nov 17 17:04:04 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 33A787F51
	for ; Mon, 17 Nov 2014 17:04:04 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 22B9730405F
	for ; Mon, 17 Nov 2014 15:04:01 -0800 (PST)
X-ASG-Debug-ID: 1416265439-04cbb01e5a104240001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id yHBmcyOXe45KpI3i (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 15:04:00 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAHN3xNa011396
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 17 Nov 2014 18:03:59 -0500
Received: from t540p.usersys.redhat.com (vpn-52-167.rdu2.redhat.com [10.10.52.167])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHN3vVV026137
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Mon, 17 Nov 2014 18:03:58 -0500
Message-ID: <546A7EDD.9000009@redhat.com>
Date: Mon, 17 Nov 2014 17:03:57 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <545A53B0.9030500@redhat.com> <704052736.8837998.1415255680009.JavaMail.zimbra@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416265440
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/17/2014 04:53 PM, Nathan Scott wrote:
> 
> 
> ----- Original Message -----
>> [...]
>> This would only have affected anyone who called
>> clear_metrics()/reset_metrics() or clear_indoms().
>>
>>> Do we need a new API to allow new dictionary object
>>> ref(s) to be pushed down to cpmda, overwriting the old?
>>
>> Hmm, I'm not really seeing a use case for that with the way the PMDA
>> class is currently written. What is your thought here?
> 
> Oh, I just had the impression from your earlier mail you weren't
> completely satisfied that we'd covered off all the cases ... its
> likely I've just misinterpreted that "except for [1]" reference.

I believe I've covered all the cases. All list/dictionary references are
cached down in cpmda and no longer thrown away after use (a). In the
PMDA class, I made sure all lists/dictionaries are cleared, not recreated.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From nscott@redhat.com  Mon Nov 17 18:41:50 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 5559F7F51
	for ; Mon, 17 Nov 2014 18:41:50 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 35632304048
	for ; Mon, 17 Nov 2014 16:41:46 -0800 (PST)
X-ASG-Debug-ID: 1416271305-04cbb01e59106df0001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id pE1JBtPgmHAGTSrG (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 16:41:45 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAI0fiWc017038;
	Mon, 17 Nov 2014 19:41:44 -0500
Date: Mon, 17 Nov 2014 19:41:44 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Lukas Berk , "Frank Ch. Eigler" 
Cc: pcp@oss.sgi.com
Message-ID: <1779902774.629370.1416271304916.JavaMail.zimbra@redhat.com>
In-Reply-To: <20141117230328.GB5700@redhat.com>
References: <87oascow3f.fsf@redhat.com> <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>  <237142894.496530.1416263449269.JavaMail.zimbra@redhat.com> <20141117230328.GB5700@redhat.com>
Subject: Re: pcp updates - pmdapapi update
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: pcp updates - pmdapapi update
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates - pmdapapi update
Thread-Index: wJHJrBlWLwz+gVzNfm52ZyLoIJ38NA==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1416271305
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11707
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> > There's two parts to this problem.  The first part is that patch (for
> > auto-enabling), the second part is during disabling - where do those
> > errors go?  [...]
> 
> Yeah, that part is tricky (and really not much better in the pmstore
> .disable case);

Hmm, not really - life is *much* simpler with pmStore.  It is the
nature of the async operation being introduced here that it removes
opportunities designed into the protocol to report errors.  Whereas
for pmStore, that's not the case - clients are informed of an error
immediately and can report it straight away.

And of course there's no timeouts to shoot the clients in the foot
without them knowing about it.  But that's going over old ground;
we knew these problems existed - there's always trade-offs to be
made & experimentation to be done.

> [...] there are just too many possible failure points.  The

Well, its not really about how *many* failure points there are...

> [...]
> future, perhaps we can explore further our earlier ideas about
> nested/detailed error messages.

...nor is it a nesting/detail problem.  Its more of an asynchronous,
after-the-fact, noone-is-around-to-hear-the-screams-anymore kind of
problem.

cheers.

--
Nathan

From nscott@redhat.com  Mon Nov 17 22:54:33 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 5E2F67F51
	for ; Mon, 17 Nov 2014 22:54:33 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 4CF01304039
	for ; Mon, 17 Nov 2014 20:54:30 -0800 (PST)
X-ASG-Debug-ID: 1416286465-04cbb01e5910d400001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id 8NMCvxKwyifBbnDg (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 20:54:25 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAI4sNLw025234;
	Mon, 17 Nov 2014 23:54:23 -0500
Date: Mon, 17 Nov 2014 23:54:22 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Martins Innus 
Cc: pcp 
Message-ID: <517956398.717585.1416286462916.JavaMail.zimbra@redhat.com>
In-Reply-To: <54622BF4.4080806@buffalo.edu>
References: <545D2CFA.1050600@buffalo.edu> <1477043892.10880496.1415600819766.JavaMail.zimbra@redhat.com> <54622BF4.4080806@buffalo.edu>
Subject: Re: [pcp] Dynamic Proc PMDA metrics
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Dynamic Proc PMDA metrics
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Dynamic Proc PMDA metrics
Thread-Index: 2C5qwdI8esk63bNYQETGuogn5koxnQ==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1416286465
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11714
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi Martins,

----- Original Message -----
> [...]
> I think I'm ok with fixing everything but this.  I don't understand why
> it is failing. I put some debugging into the indom.c test and all
> queries into the proc pmda, even non-dynamic elements, return bogus
> information from pmLookupName:
> 
> [vagrant@pcptest src]$ ./indom -h unix: -i 1 proc.psinfo.pid
> [...]
> pmLookupName returns successfully (non-error), but all proc queries have
> the domain,cluster,item values of 511,3,0.
> 

OK, this test failure is finally understood.  The root cause was that
qa/src/indom.c makes an unconditional pmLoadNameSpace(3) call, which
puts libpcp into the same mode that pmcd runs it in, where it doesn't
use the distributed namespace calls - instead it answers PMID lookups
itself (so the proc PMDA doesn't get called at all).

This means that pmLookupName calls will return with the dynamic (high)
flag bit set in any metrics from dynamic subtrees, and so subsequent
calls (like the pmLookupDesc on the next line) see an "invalid" PMID.

Anyway, easily fixed - just a QA test update.  With that and a couple
more pending QA test updates we can merge in all the initial work and
the cgroups rework on top of that - arriving shortly.  Lets look into
hotproc QA & get the final hotproc merge done when you're back.

cheers.

--
Nathan

From nscott@redhat.com  Tue Nov 18 00:35:30 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 0B1987F3F
	for ; Tue, 18 Nov 2014 00:35:30 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 80478AC005
	for ; Mon, 17 Nov 2014 22:35:26 -0800 (PST)
X-ASG-Debug-ID: 1416292523-04cb6c057010afd0001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id NnW9pzgzFZnncINa (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2014 22:35:24 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAI6ZMMi017362
	for ; Tue, 18 Nov 2014 01:35:22 -0500
Date: Tue, 18 Nov 2014 01:35:22 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <1468616216.771853.1416292522829.JavaMail.zimbra@redhat.com>
In-Reply-To: <1309338393.770280.1416292315684.JavaMail.zimbra@redhat.com>
Subject: pcp updates: pmdaproc, cgroups, books
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: pmdaproc, cgroups, books
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: pmdaproc, cgroups, books
Thread-Index: xKqaSVBc1fM65K72OT3km7PPZhK3Kw==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1416292523
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11716
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Martins Innus (6):
      dynamic_proc : Initial version for just schedstat
      dynamic_proc : all metrics should now be here
      dynamic_proc : remove some old code
      dynamic_proc : started help cleanup
      dynamic_hotproc : cleanups for submission
      dynamic_proc : add help header

Michele Baldessari (2):
      Remove --publish switch
      Remove all the condition="sgi_termlength:*"

Nathan Scott (2):
      qa: additional qa updates for dynamic proc metrics
      pmda proc: rework existing per-cgroup metrics, and add new ones

 books/PCP_PG/GNUmakefile                             |    2 
 books/PCP_PG/pcp-programmers-guide.xml               |    4 
 books/PCP_TCS/GNUmakefile                            |    2 
 books/PCP_UAG/GNUmakefile                            |    2 
 books/PCP_UAG/pcp-users-and-administrators-guide.xml |   18 
 qa/022                                               |    4 
 qa/022.out.linux                                     |  176 +-
 qa/730                                               |    2 
 qa/730.out                                           | 1474 ++++++++++---------
 qa/731.out                                           | 1244 ++++++++--------
 qa/943                                               |    3 
 qa/943.out                                           |  310 +--
 qa/src/indom.c                                       |    8 
 src/pmdas/linux_proc/GNUmakefile                     |    2 
 src/pmdas/linux_proc/cgroups.c                       | 1426 +++++++-----------
 src/pmdas/linux_proc/cgroups.h                       |  301 +++
 src/pmdas/linux_proc/clusters.h                      |    6 
 src/pmdas/linux_proc/help                            |  140 -
 src/pmdas/linux_proc/help_text.h                     |  121 +
 src/pmdas/linux_proc/indom.h                         |   30 
 src/pmdas/linux_proc/pmda.c                          | 1380 +++++++++++++----
 src/pmdas/linux_proc/proc_dynamic.c                  |  686 +++++++-
 src/pmdas/linux_proc/proc_dynamic.h                  |   19 
 src/pmdas/linux_proc/proc_pid.c                      |   34 
 src/pmdas/linux_proc/proc_pid.h                      |   25 
 src/pmdas/linux_proc/proc_runq.c                     |  112 -
 src/pmdas/linux_proc/proc_runq.h                     |    8 
 src/pmdas/linux_proc/root_proc                       |  336 ++--
 28 files changed, 4688 insertions(+), 3187 deletions(-)

From fche@redhat.com  Tue Nov 18 09:20:09 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id E0EB87F3F
	for ; Tue, 18 Nov 2014 09:20:09 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id D00908F8066
	for ; Tue, 18 Nov 2014 07:20:09 -0800 (PST)
X-ASG-Debug-ID: 1416324003-04cbb01e5c11e9c0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id dMuZACVOkzc7UvFm (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 18 Nov 2014 07:20:04 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAIFK2FU032636
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Tue, 18 Nov 2014 10:20:03 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAIFK2Ub012078;
	Tue, 18 Nov 2014 10:20:02 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id B65695822C; Tue, 18 Nov 2014 10:20:01 -0500 (EST)
To: Nathan Scott 
Cc: Lukas Berk , pcp@oss.sgi.com
Subject: Re: pcp updates - pmdapapi update
References: <87oascow3f.fsf@redhat.com>
	<50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>
	
	<237142894.496530.1416263449269.JavaMail.zimbra@redhat.com>
	<20141117230328.GB5700@redhat.com>
	<1779902774.629370.1416271304916.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: pcp updates - pmdapapi update
From: fche@redhat.com (Frank Ch. Eigler)
Date: Tue, 18 Nov 2014 10:20:01 -0500
In-Reply-To: <1779902774.629370.1416271304916.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Mon, 17 Nov 2014 19:41:44 -0500 (EST)")
Message-ID: 
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416324004
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Barracuda-BRTS-Status: 1
X-Virus-Scanned: by bsmtpd at sgi.com


nathans wrote:

> [...]
>> > There's two parts to this problem.  The first part is that patch (for
>> > auto-enabling), the second part is during disabling - where do those
>> > errors go?  [...]
>> 
>> Yeah, that part is tricky (and really not much better in the pmstore
>> .disable case);
>
> Hmm, not really - life is *much* simpler with pmStore.  It is the
> nature of the async operation being introduced here that it removes
> opportunities designed into the protocol to report errors.  [...]

When one remembers that clients that use the pure auto-enable model
are doing so with the intent to *abandon* counters when they are no
longer needed, it seems natural to ignore errors during cleanup.
Plus, since every counter addition operation involves a complete
counter removal stage, errors only in the latter are going to be rare.

Clients that need to micromanage papi can to some extent do so with
careful, incremental, exclusive pmstore operations, but even then
there's no way to return to them the richer underlying error
indications.  (That's why the nested-errors idea was brought up.
We also lack a warning facility, which could be used to pass
partial-failure type info.)


- FChE

From shirshendu@riva.co  Tue Nov 18 09:49:58 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 7C5EC7F3F
	for ; Tue, 18 Nov 2014 09:49:58 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 19E1CAC007
	for ; Tue, 18 Nov 2014 07:49:54 -0800 (PST)
X-ASG-Debug-ID: 1416325775-04bdf0615efebc0001-S8gJnT
Received: from mail-qc0-f181.google.com (mail-qc0-f181.google.com [209.85.216.181]) by cuda.sgi.com with ESMTP id jtXEcWFpDUc7nTDw (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Tue, 18 Nov 2014 07:49:35 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.181
Received: by mail-qc0-f181.google.com with SMTP id m20so5611166qcx.40
        for ; Tue, 18 Nov 2014 07:49:35 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:date:message-id:subject:from:to
         :content-type;
        bh=kTRyx006XfxGrtvBqm7ParQ/fU+O8ip1JvEpHi521wM=;
        b=TgWTY65sfbTI7/91Vq9cWfrCeKATpF8NqE22N5CumhnZubkzE6zw4hgB2dJEZWlJeb
         98iJzAk1YplB6g9p+Ab/oHLmGRNAhOSJvKtYr3bkFYwzKjLjT3fvC7XdSRtyTWYaAH1e
         FLxjq/h1GYR/4ATMfDEXMcKv2EGjDb9NqdIiMpfgD3JAgN1bh8L9BanV2iVoorNzRU1U
         Si7N3NSaQGXhI509b1WLUolIuCzmyh2+9Wec1y+Dh9U3xSdhhEluf72erGo6SOhmLOyr
         H8UsCJJqQs0Map35p1ELQz2bMqglYwLIScCTSkG/RbtnPi14gP3mw/X3/40tnjBRDosS
         D8Gg==
X-Gm-Message-State: ALoCoQm7iEY5NbvRHRpUXAAaVA9VlmhCDKbZvUII8NKjRELv1kBwFBY+mVHWG8T/7FUmjXFmDCd9
MIME-Version: 1.0
X-Received: by 10.224.65.4 with SMTP id g4mr44852140qai.20.1416325775008; Tue,
 18 Nov 2014 07:49:35 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Tue, 18 Nov 2014 07:49:34 -0800 (PST)
Date: Tue, 18 Nov 2014 21:19:34 +0530
Message-ID: 
Subject: Converting and SQL statement into a PMIE rule for evaluation against
 pmlogger archives
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Converting and SQL statement into a PMIE rule for evaluation against
 pmlogger archives
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2b980b4c5cf0508240a4d
X-Barracuda-Connect: mail-qc0-f181.google.com[209.85.216.181]
X-Barracuda-Start-Time: 1416325775
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11729
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2b980b4c5cf0508240a4d
Content-Type: text/plain; charset=UTF-8

Hi PCP Team,

Is there any way where we can translate a SQL query into a pmie rule which
can be evaluated against a live stream of metrics data or against some
pmlogger created archive.

An example of what I am trying to achieve:

SQL statement:

select attribute_1, attribute_2 from table where attribute_1 = 'some_val1'
and attribute_2 = 'some_val2' group by 'attribute_1';

equivalent metrics, that need to be processed:

..
.
.

probable pmie rule:

result = some_inst (
                              $.
. =
'some_val1' &&
                              $.
. = 'some_val2'
) -> alarm "some message" "%i ";

The intended audience for this use-case is business users and analysts with
basic SQL skill set.

Thanks,

Shirshendu Chakrabarti

--001a11c2b980b4c5cf0508240a4d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi PCP Team,

Is there any way where we =
can translate a SQL query into a pmie rule which can be evaluated against a=
 live stream of metrics data or against some pmlogger created archive.

An example of what I am trying to achieve:
<=
br>
SQL statement:

select attribute_1, a=
ttribute_2 from table where attribute_1 =3D 'some_val1' and attribu=
te_2 =3D 'some_val2' group by 'attribute_1';

=

equivalent metrics, that need to be processed:

<appname>.<table>.<attribute_1>
<app=
name>.<table>.<attribute_2>

pro=
bable pmie rule:

result =3D some_inst (
=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 $<appname>.<table>.<attri=
bute_1>=C2=A0=3D 'some_val1' &&
=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 $<appname>.<table>.<attribute_1>=C2=A0=
=3D 'some_val2'
) -> alarm "some message" &q=
uot;%i ";

The intended audience for thi=
s use-case is business users and analysts with basic SQL skill set.

Thanks,

Shirshendu Chakrabarti


--001a11c2b980b4c5cf0508240a4d--

From fche@redhat.com  Tue Nov 18 11:53:03 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id F15967F3F
	for ; Tue, 18 Nov 2014 11:53:03 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id B4C948F8035
	for ; Tue, 18 Nov 2014 09:53:00 -0800 (PST)
X-ASG-Debug-ID: 1416333176-04bdf06161105410001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id gtEBBfpI00Bn90dH (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 18 Nov 2014 09:52:56 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAIHquZv008385
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Tue, 18 Nov 2014 12:52:56 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAIHqtOK000384;
	Tue, 18 Nov 2014 12:52:55 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 2926F5822C; Tue, 18 Nov 2014 12:52:54 -0500 (EST)
To: Nathan Scott 
Cc: pcp 
Subject: Re: pcp updates: pmdaproc, cgroups, books
References: <1309338393.770280.1416292315684.JavaMail.zimbra@redhat.com>
	<1468616216.771853.1416292522829.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: pcp updates: pmdaproc, cgroups, books
From: fche@redhat.com (Frank Ch. Eigler)
Date: Tue, 18 Nov 2014 12:52:54 -0500
In-Reply-To: <1468616216.771853.1416292522829.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Tue, 18 Nov 2014 01:35:22 -0500 (EST)")
Message-ID: 
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416333176
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1


> [...]
> Nathan Scott (2):
>       pmda proc: rework existing per-cgroup metrics, and add new ones
> [...]

Nice work!  Tried it out; some observations:


#1 - After installation, and a couple 
  % pminfo -f cgroups # and/or cgroup.cpuacct.usage_percpu
iterations, got suddenly got a persistent:

cgroup.cpuacct.usage
    inst [0 or "/"] value 258190128361866
    inst [1 or "/docker"] value 32018287616
    inst [2 or "/system"] value 4970946920425
pmNameIndom: indom=3.21 inst=3: Unknown or illegal instance identifier
    inst [3] value 229040599
pmNameIndom: indom=3.21 inst=4: Unknown or illegal instance identifier
    inst [4] value 53662058907
pmNameIndom: indom=3.21 inst=5: Unknown or illegal instance identifier
    inst [5] value 18446744073709551614
pmNameIndom: indom=3.21 inst=6: Unknown or illegal instance identifier
[...]
cgroup.blkio.dev.io_queued.async
No value(s) available!
cgroup.blkio.dev.io_queued.total
No value(s) available!
[...]

In /var/log/pcp/pmcd/pmcd.log, possibly related:

[Tue Nov 18 12:01:57] pmcd(1840) Error: ClientLoop: error sending Conn ACK PDU to new client IPC protocol failure


#2.  After a restart, tried to trigger the same problem again.  This time,
     results simply stopped changing, and a copy of strace watching for open syscalls
     told the tale:

open("/proc/diskstats", O_RDONLY)       = 5
open("/sys/fs/cgroup/blkio/blkio.io_merged", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.io_queued", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.io_service_bytes", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.io_serviced", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.io_service_time", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.io_wait_time", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.sectors", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/blkio.time", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/docker/blkio.io_merged", O_RDONLY) = -1 EMFILE (Too many open files)
open("/sys/fs/cgroup/blkio/docker/blkio.io_queued", O_RDONLY) = -1 EMFILE (Too many open files)

     And indeed lsof shows many open files, in this case
     /sys/fs/cgroup/.../usage_percpu. Of all the cgroup.* metrics, that seems to
     be the only fd leaker, so this trivial fix may be enough:

--- a/src/pmdas/linux_proc/cgroups.c
+++ b/src/pmdas/linux_proc/cgroups.c
@@ -458,8 +458,10 @@ read_percpuacct_usage(const char *file, const char *name)
     if ((fp = fopen(file, "r")) == NULL)
        return -ENOENT;
     p = fgets(buffer, sizeof(buffer), fp);
-    if (!p)
+    if (!p) {
+        fclose(fp);
        return -ENOMEM;
+    }
 
     for (cpu = 0; ; cpu++) {
        value = strtoull(p, &endp, 0);
@@ -480,6 +482,7 @@ read_percpuacct_usage(const char *file, const char *name)
        percpuacct->usage = value;
        pmdaCacheStore(indom, PMDA_CACHE_ADD, inst, percpuacct);
     }
+    fclose(fp);
     return 0;
 }


#3.  From looking more at strace and the code, it looks as though the pmda might 
     be doing too much work per unit fetch.  For example, again for fetching only
     the cgroup.cpuacct.usage_percpu metric, strace shows:

open("/sys/fs/cgroup/cpu,cpuacct//system/lvm2-lvmetad.service/cpuacct.stat", O_RDONLY) = 185
open("/sys/fs/cgroup/cpu,cpuacct//system/lvm2-lvmetad.service/cpuacct.usage", O_RDONLY) = 185
open("/sys/fs/cgroup/cpu,cpuacct//system/lvm2-lvmetad.service/cpuacct.usage_percpu", O_RDONLY) = 185

     i.e., data for unrelated metrics is being gathered every time.  From looking
     at the code, this seems to occur in other groups of metrics too. 
     This is kind of like .


#4.  Instance domain unawareness.  A variant of #3, when a pcp client is looking for
     info on just one cgroup, the pmda fetches them all anyway:
     % pminfo -f cgroup.cpuacct.usage
     [... pick one indom ...]
     % strace -eopen -f -p `pgrep pmdaproc` &
     % pmval -i /system/pmcd.service cgroup.cpuacct.usage
[...]
open("/sys/fs/cgroup/cpu,cpuacct//system/plymouth-start.service/cpuacct.stat", O_RDONLY) = 969
open("/sys/fs/cgroup/cpu,cpuacct//system/plymouth-start.service/cpuacct.usage", O_RDONLY) = 969
open("/sys/fs/cgroup/cpu,cpuacct//system/plymouth-start.service/cpuacct.usage_percpu", O_RDONLY) = 969
open("/sys/fs/cgroup/cpu,cpuacct//system/dracut-initqueue.service/cpuacct.stat", O_RDONLY) = 970
open("/sys/fs/cgroup/cpu,cpuacct//system/dracut-initqueue.service/cpuacct.usage", O_RDONLY) = 970
open("/sys/fs/cgroup/cpu,cpuacct//system/dracut-initqueue.service/cpuacct.usage_percpu", O_RDONLY) = 970
[...]


#5.  Multiple metrics unawareness (so to speak).  A variant of #3,
     where a pcp client is looking for info on two metrics (without
     indom restrictions).  strace indicates pmda effort is doubled:

     % strace -eopen -f -p `pgrep pmdaproc` &
     % pminfo -f cgroup.cpuacct.usage cgroup.blkio.all.sectors
[...]
open("/proc/cgroups", O_RDONLY)         = 5
open("/proc/mounts", O_RDONLY)          = 5
open("/proc/stat", O_RDONLY)            = 5
open("/sys/fs/cgroup/cpu,cpuacct/cpuacct.stat", O_RDONLY) = 239
open("/sys/fs/cgroup/cpu,cpuacct/cpuacct.usage", O_RDONLY) = 239
open("/sys/fs/cgroup/cpu,cpuacct/cpuacct.usage_percpu", O_RDONLY) = 239
[...]
open("/proc/diskstats", O_RDONLY)       = 5
open("/sys/fs/cgroup/blkio/blkio.io_merged", O_RDONLY) = 285
open("/sys/fs/cgroup/blkio/blkio.io_queued", O_RDONLY) = 285
open("/sys/fs/cgroup/blkio/blkio.io_service_bytes", O_RDONLY) = 285
[...]
open("/sys/fs/cgroup/blkio/docker/blkio.time", O_RDONLY) = 285
open("/proc/cgroups", O_RDONLY)         = 5
open("/proc/mounts", O_RDONLY)          = 5
open("/proc/stat", O_RDONLY)            = 5
open("/sys/fs/cgroup/cpu,cpuacct/cpuacct.stat", O_RDONLY) = 285
open("/sys/fs/cgroup/cpu,cpuacct/cpuacct.usage", O_RDONLY) = 285
open("/sys/fs/cgroup/cpu,cpuacct/cpuacct.usage_percpu", O_RDONLY) = 285
open("/sys/fs/cgroup/cpu,cpuacct/docker/cpuacct.stat", O_RDONLY) = 329
open("/sys/fs/cgroup/cpu,cpuacct/docker/cpuacct.usage", O_RDONLY) = 329
[...]
open("/proc/cgroups", O_RDONLY)         = 5
open("/proc/mounts", O_RDONLY)          = 5
open("/proc/diskstats", O_RDONLY)       = 5
open("/sys/fs/cgroup/blkio/blkio.io_merged", O_RDONLY) = 331
open("/sys/fs/cgroup/blkio/blkio.io_queued", O_RDONLY) = 331
open("/sys/fs/cgroup/blkio/blkio.io_service_bytes", O_RDONLY) = 331
[...]

Note the above traffic all came from one pminfo run, not two.


(Some of these may be cases of false confidence based on testing
primarily against the pcpqa artificial system files as in
linux/cgroups-root*.tgz.  It's clever & useful, but cannot be as
thorough.)


- FChE

From shirshendu@riva.co  Tue Nov 18 12:17:07 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id E3C437F3F
	for ; Tue, 18 Nov 2014 12:17:07 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id C2E838F8033
	for ; Tue, 18 Nov 2014 10:17:07 -0800 (PST)
X-ASG-Debug-ID: 1416334622-04cb6c0570122b40001-S8gJnT
Received: from mail-qc0-f175.google.com (mail-qc0-f175.google.com [209.85.216.175]) by cuda.sgi.com with ESMTP id HDbKR68sWrahqLqd (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Tue, 18 Nov 2014 10:17:02 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.175
Received: by mail-qc0-f175.google.com with SMTP id b13so20519956qcw.34
        for ; Tue, 18 Nov 2014 10:17:00 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:date:message-id:subject:from:to
         :content-type;
        bh=/I6zj75t9Db46UEwZoccmXklNamnL5oUxgesm1CQkho=;
        b=jo6FrWBRivDed+sdzWuFgv9FZeg4xXGtof6j1IDVlCrkf3+kUQgSaQVq08T+v4/yQf
         KlmOASWFNZAkPGu+hwYGgy6IoguIKOq9wmUWD5+hFM/ZfhvETehbmDQohwjBorfa5WKr
         2DTADDGahbKrNFStqN69LpH0oYAS2GP0nlypguUhngDae7bLo9eCdUQUhigeM8jqwpNj
         rhmwBux3O89cxoCHyocqVZCmqGEI8PoMqyOFhIXAxl0hGuVD6xwNYgFWjrBCoa3lq/rk
         Rgzbo3gisDE0lTdXpV/9DW1BwD0XhVHVS3SyYSBbmxHFv5iWxZw40MIzaNu1Mb5SlIua
         FqOw==
X-Gm-Message-State: ALoCoQksBD7IFc27bN4dxyifS/FHvog/GRrxhftByKtEQm15FNW5qj/TGM4Wzcy3GtWo9h96GbQH
MIME-Version: 1.0
X-Received: by 10.224.93.18 with SMTP id t18mr46025703qam.102.1416334620291;
 Tue, 18 Nov 2014 10:17:00 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Tue, 18 Nov 2014 10:17:00 -0800 (PST)
Date: Tue, 18 Nov 2014 23:47:00 +0530
Message-ID: 
Subject: Non-availability of pmwebd on Amazon Linux
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Non-availability of pmwebd on Amazon Linux
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=089e0149c9b4ed14f00508261913
X-Barracuda-Connect: mail-qc0-f175.google.com[209.85.216.175]
X-Barracuda-Start-Time: 1416334622
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11732
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--089e0149c9b4ed14f00508261913
Content-Type: text/plain; charset=UTF-8

Hi PCP Team,

I have bee trying to obtain the pmwebd binary from pcp sources on Amazon
Linux.

I build the rpms but I never see the pmwebd daemon installed.

Please see, I use Fedora 18 for development purposes on my laptop and
pmwebd installs without any issues.

Please Amazon Linux is an rpm based system.

Please do let me know if I am missing something.

Thanks,

Shirshendu Chakrabarti

--089e0149c9b4ed14f00508261913
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi PCP Team,

I have bee trying to obtai=
n the pmwebd binary from pcp sources on Amazon Linux.

<=
div>I build the rpms but I never see the pmwebd daemon installed.

Please see, I use Fedora 18 for development purposes on my =
laptop and pmwebd installs without any issues.

Ple=
ase Amazon Linux is an rpm based system.

Please do=
 let me know if I am missing something.

Thanks,

Shirshendu Chakrabarti=C2=A0


--089e0149c9b4ed14f00508261913--

From kenj@internode.on.net  Tue Nov 18 13:20:54 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id F22A67F3F
	for ; Tue, 18 Nov 2014 13:20:53 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 806DEAC009
	for ; Tue, 18 Nov 2014 11:20:53 -0800 (PST)
X-ASG-Debug-ID: 1416338446-04cbb01e59127e00001-S8gJnT
Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by cuda.sgi.com with ESMTP id jxuGTzedsGwwo3ly for ; Tue, 18 Nov 2014 11:20:47 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.145
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AvoBADaba1R20W7CPGdsb2JhbAANTot5yRODHAKBJwEBAQEBBgEBAQE4hD4BAQQ4QBELGAkWDwkDAgECATEUEwgBAcIAl3sBAQgCAR+RDxaENQEEuV6DJAEBAQ
Received: from ppp118-209-110-194.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.110.194])
  by ipmail06.adl6.internode.on.net with ESMTP; 19 Nov 2014 05:50:45 +1030
Message-ID: <546B9C7F.1050706@internode.on.net>
Date: Wed, 19 Nov 2014 06:22:39 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: Re: [pcp] Non-availability of pmwebd on Amazon Linux
References: 
X-ASG-Orig-Subj: Re: [pcp] Non-availability of pmwebd on Amazon Linux
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl6.internode.on.net[150.101.137.145]
X-Barracuda-Start-Time: 1416338446
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11736
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

On 19/11/14 05:17, Shirshendu Chakrabarti wrote:
> Hi PCP Team,
>
> I have bee trying to obtain the pmwebd binary from pcp sources on Amazon
> Linux.
>
> I build the rpms but I never see the pmwebd daemon installed.

I don't have access to an Amazon Linux system, but it is most likely 
that there are missing RPMS, probably qt-devel.

If you could post the output from check-vm script (in the PCP source 
this lives at qa/admin/check-vm) that would help prove/disprove this 
assertion.

From webmaster@testfreewww5.com  Tue Nov 18 13:45:49 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,T_DKIM_INVALID
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 7EC817F3F
	for ; Tue, 18 Nov 2014 13:45:49 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 42E9D304043
	for ; Tue, 18 Nov 2014 11:45:45 -0800 (PST)
X-ASG-Debug-ID: 1416339937-04bdf0615f10a580001-S8gJnT
Received: from 109 ([109.237.111.90]) by cuda.sgi.com with ESMTP id PXQQfHNnhdchDb4t (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO) for ; Tue, 18 Nov 2014 11:45:39 -0800 (PST)
X-Barracuda-Envelope-From: webmaster@testfreewww5.com
X-Barracuda-Apparent-Source-IP: 109.237.111.90
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=testfreewww5.com; s=dkim;
	h=List-Unsubscribe:Content-Type:MIME-Version:Date:From:Message-ID:Subject:To; bh=oVAIV1PmbSAiKEU63Nbd0TT6kjCBy2MhCWLTC11hIvc=;
	b=cRZHivvNSCqTx3X3byz6xTWn3NQjSHt9QqhAmiYDQMnCtZqTf4JjwUdW8PygdzfgyrKLA5W4RTfvUns2OAnWjrbthuYf4Js/ekNv76QQ8GQsb1oRKXgUnx0yn2+P34+bU5Kgph0qGYplhAOYnZurgHSXJpXq/o4lgTThg+r6A4o=;
Received: from puxas by 109 with local (Exim 4.80)
	(envelope-from )
	id 1XqogS-0007oM-LI; Tue, 18 Nov 2014 14:43:30 -0500
To: "giuseppe.bilotta" ,"php-suhosin-maintainers" ,"jeffrey.m.ahrenholz" ,"condor-debian" ,"abo" ,"tanders" ,"packaging" ,"team" ,"florent.angly" ,"pcp" ,"lipsia" ,"openvas-distro-deb" ,"cutout33" ,"lb1programmer" ,"noblenote.developers" 
Subject: =?windows-1251?B?zuTt7ukg7O7p6ugg9eLg8uDl8iDt4CDj7uQh?=
	=?windows-1251?B?IMzu6SDx7+7x7uEg7O7p6ugg7OD46O37IO/u?=
	=?windows-1251?B?8fLg4ujrICLt4CDz+OgiIOLl8fwgyO3y5fDt?=
	=?windows-1251?B?5fIh?=
X-PHP-Originating-Script: 500:rumail.php
X-ASG-Orig-Subj: =?windows-1251?B?zuTt7ukg7O7p6ugg9eLg8uDl8iDt4CDj7uQh?=
	=?windows-1251?B?IMzu6SDx7+7x7uEg7O7p6ugg7OD46O37IO/u?=
	=?windows-1251?B?8fLg4ujrICLt4CDz+OgiIOLl8fwgyO3y5fDt?=
	=?windows-1251?B?5fIh?=
Message-ID: <292C9AC9C21638FB27226BD4D0FF0FC3@ojnu>
From: =?windows-1251?B?1Ojr6O8=?= 
Date: Tue, 18 Nov 2014 19:43:48 +0000
Organization: =?windows-1251?B?wMrCwA==?=
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_12FB_01D00367.F8AEBB60"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.5931
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157
List-Unsubscribe: 
X-Barracuda-Connect: UNKNOWN[109.237.111.90]
X-Barracuda-Start-Time: 1416339939
X-Barracuda-Encrypted: AES128-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.10
X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE, RDNS_NONE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11736
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 HTML_MESSAGE           BODY: HTML included in message
	0.10 RDNS_NONE              Delivered to trusted network by a host with no rDNS

This is a multi-part message in MIME format.

------=_NextPart_000_12FB_01D00367.F8AEBB60
Content-Type: text/plain;
	charset="windows-1251"
Content-Transfer-Encoding: quoted-printable

=DF =EF=EE=EA=E0=E6=F3 =C2=E0=EC =F1=E2=EE=FE =F4=EE=F0=EC=F3=EB=F3 =F7=E8=
=F1=F2=EE=F2=FB! =CC=E0=F8=E8=ED=E0 =E2=F1=E5=E3=E4=E0 =F7=E8=F1=F2=E0=FF=
! =CF=EE=E4=F0=EE=E1=ED=EE=F1=F2=E8 =E7=E4=E5=F1=FC>>>>>=20

------=_NextPart_000_12FB_01D00367.F8AEBB60
Content-Type: text/html;
	charset="windows-1251"
Content-Transfer-Encoding: quoted-printable







=DF =EF=EE=EA=E0=E6=F3 =C2=E0=EC =F1=E2=EE=FE =F4=EE=F0=
=EC=F3=EB=F3 =F7=E8=F1=F2=EE=F2=FB! =CC=E0=F8=E8=ED=E0 =E2=F1=E5=E3=E4=E0=
 =F7=E8=F1=F2=E0=FF! =CF=EE=E4=F0=EE=E1=ED=EE=F1=
=F2=E8 =E7=E4=E5=F1=FC>>>>>=20



------=_NextPart_000_12FB_01D00367.F8AEBB60--

From nscott@redhat.com  Tue Nov 18 14:08:08 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 0FEDD7F3F
	for ; Tue, 18 Nov 2014 14:08:08 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id F2C3630405F
	for ; Tue, 18 Nov 2014 12:08:04 -0800 (PST)
X-ASG-Debug-ID: 1416341281-04cbb01e5c129d40001-S8gJnT
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id ODx2DFHSsNbN5a1M (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 18 Nov 2014 12:08:02 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAIK7wDv019633;
	Tue, 18 Nov 2014 15:07:58 -0500
Date: Tue, 18 Nov 2014 15:07:57 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Ken McDonell ,
        Shirshendu Chakrabarti 
Cc: pcp@oss.sgi.com
Message-ID: <432584280.1372959.1416341277776.JavaMail.zimbra@redhat.com>
In-Reply-To: <546B9C7F.1050706@internode.on.net>
References:  <546B9C7F.1050706@internode.on.net>
Subject: Re: [pcp] Non-availability of pmwebd on Amazon Linux
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Non-availability of pmwebd on Amazon Linux
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Non-availability of pmwebd on Amazon Linux
Thread-Index: HbvaodRaRege4quZuumw20xG3OUQYg==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1416341282
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11737
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...



----- Original Message -----
> On 19/11/14 05:17, Shirshendu Chakrabarti wrote:
> > Hi PCP Team,
> >
> > I have bee trying to obtain the pmwebd binary from pcp sources on Amazon
> > Linux.
> >
> > I build the rpms but I never see the pmwebd daemon installed.
> 
> I don't have access to an Amazon Linux system, but it is most likely
> that there are missing RPMS, probably qt-devel.

It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 required) package
on the build machine ("libmicrohttpd-dev" package on Debian/Ubuntu).

cheers.

--
Nathan

From wwwrun@oss.sgi.com  Tue Nov 18 14:28:34 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 76DFF7F4E; Tue, 18 Nov 2014 14:28:34 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1072] New: pmlogger output semantic or physical compression
Date: Tue, 18 Nov 2014 20:28:34 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416342514.12dCbe1.8499"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416342514.12dCbe1.8499
Date: Tue, 18 Nov 2014 14:28:34 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1072

            Bug ID: 1072
           Summary: pmlogger output semantic or physical compression
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo todo list item #3:

- pcp todo item 3: pmlogger logging proc.* - data quantity
  - it is desirable to log some textual proc.* bits for analysis, like
    a hypothetical (?) proc.psinfo.environ metric, but this is very large
[...]
  - repeated data values bloat the log file
  - (a sign of this is how gzip-compressible archives are: >>90% common)
  - pmlogger should get a mode to skip same-as-last-value entries
    - (though this would not reduce query load on target pmcd)
  - semantics may permit this sort of change without archive file format
revision
  - or use libz to compress on the fly & decompress on the fly too
    - ideally retaining compatible .index file-offset format

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416342514.12dCbe1.8499
Date: Tue, 18 Nov 2014 14:28:34 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1072
          
        


        

          Summary
          pmlogger output semantic or physical compression
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo todo list item #3:

- pcp todo item 3: pmlogger logging proc.* - data quantity
  - it is desirable to log some textual proc.* bits for analysis, like
    a hypothetical (?) proc.psinfo.environ metric, but this is very large
[...]
  - repeated data values bloat the log file
  - (a sign of this is how gzip-compressible archives are: >>90% common)
  - pmlogger should get a mode to skip same-as-last-value entries
    - (though this would not reduce query load on target pmcd)
  - semantics may permit this sort of change without archive file format
revision
  - or use libz to compress on the fly & decompress on the fly too
    - ideally retaining compatible .index file-offset format

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416342514.12dCbe1.8499--

From wwwrun@oss.sgi.com  Tue Nov 18 14:29:40 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id B8BC57F3F; Tue, 18 Nov 2014 14:29:40 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1073] New: pmlogger on-the-fly one-shot recording
Date: Tue, 18 Nov 2014 20:29:40 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416342580.CfBCCAFE1.8608"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416342580.CfBCCAFE1.8608
Date: Tue, 18 Nov 2014 14:29:40 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1073

            Bug ID: 1073
           Summary: pmlogger on-the-fly one-shot recording
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo
- pcp todo item 4: need to log some metrics on demand, oneshot type
  - to match exact moment of job start/stop
  - something like % pmlc log oneshot _metriclist  perhaps?
  - or deploy job-lifetime-matched temporary pmloggers
    - almost as in % pmlogger --lifetime PID
    - related to earlier idea re. pmval -c CMD in papi context

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416342580.CfBCCAFE1.8608
Date: Tue, 18 Nov 2014 14:29:40 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1073
          
        


        

          Summary
          pmlogger on-the-fly one-shot recording
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo
- pcp todo item 4: need to log some metrics on demand, oneshot type
  - to match exact moment of job start/stop
  - something like % pmlc log oneshot _metriclist  perhaps?
  - or deploy job-lifetime-matched temporary pmloggers
    - almost as in % pmlogger --lifetime PID
    - related to earlier idea re. pmval -c CMD in papi context

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416342580.CfBCCAFE1.8608--

From brolley@redhat.com  Tue Nov 18 16:50:36 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 02A627F3F
	for ; Tue, 18 Nov 2014 16:50:36 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id E57978F8040
	for ; Tue, 18 Nov 2014 14:50:35 -0800 (PST)
X-ASG-Debug-ID: 1416351031-04cb6c057012d120001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id D1rdYXgA9Ava7gKt (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 18 Nov 2014 14:50:32 -0800 (PST)
X-Barracuda-Envelope-From: brolley@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAIMoUW8009400
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Tue, 18 Nov 2014 17:50:30 -0500
Received: from [10.15.16.126] (dhcp-10-15-16-126.yyz.redhat.com [10.15.16.126])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAIMoTXJ007925
	for ; Tue, 18 Nov 2014 17:50:30 -0500
Message-ID: <546BCDC9.1040301@redhat.com>
Date: Tue, 18 Nov 2014 17:52:57 -0500
From: Dave Brolley 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: PCP Mailing List 
Subject: pcp updates: pmwebd graphite-dashboard, pmdapapi, fedora.spec
Content-Type: text/plain; charset=utf-8; format=flowed
X-ASG-Orig-Subj: pcp updates: pmwebd graphite-dashboard, pmdapapi, fedora.spec
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416351032
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Changes committed to git://git.pcp.io/pcp.git dev

Frank Ch. Eigler (2):
       pmwebd: improve /metrics/find emulation for graphite dashboard
       papi pmda: auto-enable error path

Dave Brolley (1):
       papi 5.4.0 rebuild.

  build/rpm/fedora.spec                                | 3
  qa/.gitignore                                        | 1
  qa/661                                               | 3
  qa/661.out                                           | 96 +
  src/pmdas/papi/papi.c                                | 4
  src/pmwebapi/main.cxx                                | 8
  src/pmwebapi/pmgraphite.cxx                          | 118 -
  src/pmwebapi/pmwebapi.h                              | 2
  src/pmwebapi/util.cxx                                | 23

From janfrode@tanso.net  Tue Nov 18 18:00:05 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id B4C1D7F3F
	for ; Tue, 18 Nov 2014 18:00:05 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 862408F8037
	for ; Tue, 18 Nov 2014 16:00:02 -0800 (PST)
X-ASG-Debug-ID: 1416355196-04bdf0615e112ec0001-S8gJnT
Received: from mail-la0-f49.google.com (mail-la0-f49.google.com [209.85.215.49]) by cuda.sgi.com with ESMTP id IMT9mAWy9c8VUx2r (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Tue, 18 Nov 2014 15:59:58 -0800 (PST)
X-Barracuda-Envelope-From: janfrode@tanso.net
X-Barracuda-Apparent-Source-IP: 209.85.215.49
Received: by mail-la0-f49.google.com with SMTP id hs14so1858966lab.22
        for ; Tue, 18 Nov 2014 15:59:56 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-type:content-disposition:in-reply-to
         :user-agent;
        bh=FOH5gvn0BBMI9iTpo/GANnSnTtxD6eIpTZwFJ9X2rxI=;
        b=LBbnlxiQ2C9/++S023QTvKMQa52h/YrQT9rh8B/4oMJDuOGZ4A6FgIrgIPqvJlSSuj
         3qA1gW9omn7+cEBDRbbfWt9DkohcUf9M+C+N1gmP1alqO7rJgX1hZ2qqrtY7Kt6Kispc
         akFxX7dsO+PMInzVcyDqRF0Ew3f4MZm14iTw65Ypht0cx4tMAKNKMVUFl/b++w6P+HBn
         GDee+WGs0sd5WAdk3LV5fdnjOeQ3jz0wYNTrXove+A5VZL06w6ltXNqnkvGmYHnNzq67
         DEjDAjEJn98OR2ihbxdA5SdSyq17miQgnowpl8+384VfHiEgiGb4nJAzgve7UakaHxIL
         GT6A==
X-Gm-Message-State: ALoCoQlcH8vlK4JM1h9RyXmWfUG1NTvd8fu7k6ytc409eIntZTUJX5soOMgrGJfxLCAc0wuuSY8D
X-Received: by 10.152.28.227 with SMTP id e3mr2264127lah.54.1416355196430;
        Tue, 18 Nov 2014 15:59:56 -0800 (PST)
Received: from localhost (120.81-167-109.customer.lyse.net. [81.167.109.120])
        by mx.google.com with ESMTPSA id lh7sm31701lbc.27.2014.11.18.15.59.55
        for 
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Tue, 18 Nov 2014 15:59:55 -0800 (PST)
Date: Wed, 19 Nov 2014 00:59:54 +0100
From: Jan-Frode Myklebust 
To: Nathan Scott 
Cc: pcp@oss.sgi.com
Subject: Re: [pcp] My first PMDA, some questions..
Message-ID: <20141118235954.GA2225@mushkin.tanso.net>
X-ASG-Orig-Subj: Re: [pcp] My first PMDA, some questions..
References: <20141116200958.GA8464@mushkin.tanso.net>
 <69304097.15487953.1416205767949.JavaMail.zimbra@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <69304097.15487953.1416205767949.JavaMail.zimbra@redhat.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-Barracuda-Connect: mail-la0-f49.google.com[209.85.215.49]
X-Barracuda-Start-Time: 1416355197
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Barracuda-BRTS-Status: 1
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-Spam-Score: 0.60
X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=COMMA_SUBJECT
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11745
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.60 COMMA_SUBJECT          Subject is like 'Re: FDSDS, this is a subject'

On Mon, Nov 17, 2014 at 01:29:27AM -0500, Nathan Scott wrote:
> 
> At what point do you know how many threads you have?  

After "unbound" is started, and I guess it might change if someone
changes the unbound configuration. But should in general be pretty
static..


> If you can find out
> before the call to PMDA.run(), then you can just add them like any other
> metric (with the first parameter to add_metric() being "thread%d.xxx".  If
> you can only find out after the call to run(), then the current python API
> will not work for you.

Ok, hmm... not sure which option I'll go for.


> 
> 
> The client (monitor) tools drive the fetch interval, PMDAs just respond with
> the latest values - they typically have no knowledge of the sampling rate.
> There can also be multiple clients, sampling at different (unrelated) times.

So two people running "pmval metric" will by default trigger 2 fetches
per second?? Or will they be smart enough to share one? 1 fetch per
second should be OK, but much more is getting scary..

> 
> If you have concerns about running the command frequently, one option is to
> cache the results for a short time, and respond with the cached results if
> fetches come in too quickly for the PMDA to comfortably handle.  Chapter 2
> of the Programmers Guide (http://www.pcp.io/doc/pcp-programmers-guide.pdf)
> has a section "2.2.4 - Caching PMDA" with a more details.
> 

Ok.


BTW: Here's my first shot -> https://github.com/janfrode/unbound-pmda
based on the gluster pmda.


  -jf

From nscott@redhat.com  Tue Nov 18 22:26:07 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 4AB147F3F
	for ; Tue, 18 Nov 2014 22:26:07 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 3932C304039
	for ; Tue, 18 Nov 2014 20:26:04 -0800 (PST)
X-ASG-Debug-ID: 1416371162-04cbb01e5a138ad0001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id EZ4AZ6T5izB1gORm (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 18 Nov 2014 20:26:02 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAJ4Q1Fr004518;
	Tue, 18 Nov 2014 23:26:01 -0500
Date: Tue, 18 Nov 2014 23:26:01 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: "Frank Ch. Eigler" 
Cc: pcp 
Message-ID: <1249426361.1589601.1416371161222.JavaMail.zimbra@redhat.com>
In-Reply-To: 
References: <1309338393.770280.1416292315684.JavaMail.zimbra@redhat.com> <1468616216.771853.1416292522829.JavaMail.zimbra@redhat.com> 
Subject: Re: pcp updates: pmdaproc, cgroups, books
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: pcp updates: pmdaproc, cgroups, books
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: pmdaproc, cgroups, books
Thread-Index: CZ1Qm8WTVaHUaq9ySf5zgQwnPGJr1Q==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1416371162
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11754
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> > [...]
> > Nathan Scott (2):
> >       pmda proc: rework existing per-cgroup metrics, and add new ones
> > [...]
> 
> Nice work!  Tried it out; some observations:
> 

Thanks.

>      [...] And indeed lsof shows many open files, in this case

Yep, nasty - good find.

> #3.  From looking more at strace and the code, it looks as though the pmda
>      might be doing too much work per unit fetch.
> [...]
> #4.  Instance domain unawareness.  A variant of #3, when a pcp client is
> [...]
> #5.  Multiple metrics unawareness (so to speak).  A variant of #3,

There's subtleties to the protocol you may not be following here
and thats misleading you in interpreting the observed results, I
think.  The instance PDUs are a separate entity to fetch PDUs,
but the PMDA needs to take similar refresh actions to respond to
each.

> (Some of these may be cases of false confidence based on testing
> primarily against the pcpqa artificial system files as in

Nah, at first glance they mostly seem to be misunderstandings about
the PCP protocol, as issued by pminfo/pmval.

> linux/cgroups-root*.tgz.  It's clever & useful, but cannot be as
> thorough.)

Oh, it can be far more thorough than we could ever hope to be with
testing by-hand; if people contribute to the testsuite and build it
up over time with more cgroup scenarios, more output from different
kernel versions, different platforms, etc, etc ... there is no way
testing by-hand will ever beat that kind of coverage.

Almost anything one can test by-hand can be scripted, and in most
cases any by-hand testing is throw-away effort that could have been
(& should have been) contributed to the testsuite.

In this particular case, everything is reproducible and readily QA-
automated.  I imagine you simply have many more active cgroups than
I did on my dev box when I created cgroups-root-001.tgz, and hence
I don't trip that thar fd-limit.  Could you tar up the contents of
/proc/{cgroups,mounts,diskstats,stat} and /sys/fs/cgroup from this
machine (lets call it cgroups-root-002.tgz), and send it through to
me please?

thanks.

--
Nathan

From pcp-announce-bounces@oss.sgi.com  Tue Nov 18 23:35:14 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from oss.sgi.com (localhost [IPv6:::1])
	by oss.sgi.com (Postfix) with ESMTP id B45207F4E;
	Tue, 18 Nov 2014 23:35:14 -0600 (CST)
X-Original-To: pcp-announce@oss.sgi.com
Delivered-To: pcp-announce@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 5C30D7F3F
	for ; Tue, 18 Nov 2014 23:35:13 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id D1BF0AC007
	for ; Tue, 18 Nov 2014 21:35:09 -0800 (PST)
X-ASG-Debug-ID: 1416375304-04cbb01e591470c0001-87ZIJf
Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by
	cuda.sgi.com with ESMTP id o3HylahoNIHAgDL6 (version=TLSv1
	cipher=AES256-SHA bits=256 verify=NO) for ;
	Tue, 18 Nov 2014 21:35:05 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.37
Received: from zmail20.collab.prod.int.phx2.redhat.com
	(zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJ5Z3e2022436;
	Wed, 19 Nov 2014 00:35:03 -0500
Date: Wed, 19 Nov 2014 00:35:03 -0500 (EST)
From: Nathan Scott 
To: pcp-announce 
Message-ID: <1031495503.1600999.1416375303881.JavaMail.zimbra@redhat.com>
In-Reply-To: <546BCDC9.1040301@redhat.com>
References: <546BCDC9.1040301@redhat.com>
MIME-Version: 1.0
X-ASG-Orig-Subj: Maintainer role addition for PCP
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Maintainer role addition for PCP
Thread-Index: 6Uwe9ndSiPyXGi5UlUQDxf66f9u8FA==
X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37]
X-Barracuda-Start-Time: 1416375304
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No,
	SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0
	QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX,
	THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11755
	Rule breakdown below
	pts rule name              description
	---- ----------------------
	--------------------------------------------------
	0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
Subject: [pcp-announce] Maintainer role addition for PCP
X-BeenThere: pcp-announce@oss.sgi.com
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: Nathan Scott 
List-Id: Performance Co-Pilot announcements 
List-Unsubscribe: ,
	
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: ,
	
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pcp-announce-bounces@oss.sgi.com
Sender: pcp-announce-bounces@oss.sgi.com

Hi all,

As you may have noticed (mail below) Dave Brolley has
kindly offered to take on more code review, merge and
release responsibilities.  Dave and I will be doing a
pcp-3.10.1 release together (scheduled for 1 Dec) and
thereafter we're planning to rotate.

Dave has a deep knowledge of PCP protocol low-levels,
having done all the IPv6 support, secure sockets, and
lots of other PCP work.  He also has a great history
of writing tests, running tests, fixing tests and, of
late, even fixing other peoples tests.  (Zen moment!)

He's done a fantastic job at working towards mutually
agreeable solutions whenever consensus is lacking.  So
I hope others will enjoy working with him as much as I
have in getting changes into the tree.

I'm still working on Ken, trying to convince him to be
part of this rotation, but so far he's resisted all of
the requests and bribes I've offered!  ;-)

Thanks Dave!

cheers.


----- Forwarded Message -----
From: "Dave Brolley" 
To: "PCP Mailing List" 
Sent: Wednesday, November 19, 2014 9:52:57 AM
Subject: [pcp] pcp updates: pmwebd graphite-dashboard, pmdapapi, fedora.spec

Changes committed to git://git.pcp.io/pcp.git dev

Frank Ch. Eigler (2):
       pmwebd: improve /metrics/find emulation for graphite dashboard
       papi pmda: auto-enable error path

Dave Brolley (1):
       papi 5.4.0 rebuild.

  build/rpm/fedora.spec                                | 3
  qa/.gitignore                                        | 1
  qa/661                                               | 3
  qa/661.out                                           | 96 +
  src/pmdas/papi/papi.c                                | 4
  src/pmwebapi/main.cxx                                | 8
  src/pmwebapi/pmgraphite.cxx                          | 118 -
  src/pmwebapi/pmwebapi.h                              | 2
  src/pmwebapi/util.cxx                                | 23

_______________________________________________
pcp-announce mailing list
pcp-announce@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/pcp-announce

From shirshendu@riva.co  Wed Nov 19 00:15:36 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id B04747F3F
	for ; Wed, 19 Nov 2014 00:15:36 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 9F52E30404E
	for ; Tue, 18 Nov 2014 22:15:36 -0800 (PST)
X-ASG-Debug-ID: 1416377731-04cb6c0571145810001-S8gJnT
Received: from mail-qg0-f45.google.com (mail-qg0-f45.google.com [209.85.192.45]) by cuda.sgi.com with ESMTP id rXagzjzfFzUZi8dW (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Tue, 18 Nov 2014 22:15:32 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.192.45
Received: by mail-qg0-f45.google.com with SMTP id f51so169688qge.18
        for ; Tue, 18 Nov 2014 22:15:31 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=gTSX9fIlwLY3qqSsmyAKVGwIMu0lKJFysKWhve1FAIQ=;
        b=XS9GfA2hXbwT1FE7TlXC5yBqZ89F5FD9DbW9L9ii550fsANMmLvnvxd180hIj3OfFt
         mCwVqkulqfmrZ8EmPFXDeCmPY8Qu5XnrXmhU7+9gBOnQupeWdTI1YKgXWBnPS+Js19hN
         gYvRkS2irH3RBUzEqbpgsSLRnLmFgDrHX54wLnZVQ37Limtkwp4wa4W4f9FQ/G6MJ9Lx
         2xUrosinFYKuhJbo26K+Z4NX03qWR7aJM1JEVFktozGBWUzQEmQOCK3N7SwFddp2hwAk
         u0wOWBy5NNNmQ3LVmtbFYlmNebvGGANaDKeRpsB8waxZyV4PuSjCN+G6AsGNaePESUzD
         qh/Q==
X-Gm-Message-State: ALoCoQkq1MHbKOih7dzUAh2j2Rk/GVlMeUQNSWQFfD/yZpsLk3pnuy38snGEFfVWNoruroa5n2rH
MIME-Version: 1.0
X-Received: by 10.140.92.215 with SMTP id b81mr32179688qge.5.1416377731645;
 Tue, 18 Nov 2014 22:15:31 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Tue, 18 Nov 2014 22:15:31 -0800 (PST)
In-Reply-To: <432584280.1372959.1416341277776.JavaMail.zimbra@redhat.com>
References: 
	<546B9C7F.1050706@internode.on.net>
	<432584280.1372959.1416341277776.JavaMail.zimbra@redhat.com>
Date: Wed, 19 Nov 2014 11:45:31 +0530
Message-ID: 
Subject: Re: [pcp] Non-availability of pmwebd on Amazon Linux
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Non-availability of pmwebd on Amazon Linux
To: Nathan Scott 
Cc: Ken McDonell , pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a1139126c902f70050830236d
X-Barracuda-Connect: mail-qg0-f45.google.com[209.85.192.45]
X-Barracuda-Start-Time: 1416377732
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11755
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a1139126c902f70050830236d
Content-Type: text/plain; charset=UTF-8

Hi Team,

Thanks for the pointers, I will try both missing packages listed above and
report back.

Thanks,

Shirshendu Chakrabarti

On Wed, Nov 19, 2014 at 1:37 AM, Nathan Scott  wrote:

>
>
> ----- Original Message -----
> > On 19/11/14 05:17, Shirshendu Chakrabarti wrote:
> > > Hi PCP Team,
> > >
> > > I have bee trying to obtain the pmwebd binary from pcp sources on
> Amazon
> > > Linux.
> > >
> > > I build the rpms but I never see the pmwebd daemon installed.
> >
> > I don't have access to an Amazon Linux system, but it is most likely
> > that there are missing RPMS, probably qt-devel.
>
> It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 required) package
> on the build machine ("libmicrohttpd-dev" package on Debian/Ubuntu).
>
> cheers.
>
> --
> Nathan
>

--001a1139126c902f70050830236d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Team,

Thanks for the pointers, I wil=
l try both missing packages listed above and report back.

Thanks,

Shirshendu Chakrabarti
=


On Wed, Nov 19, 2=
014 at 1:37 AM, Nathan Scott <nathans@redhat.com> wrote:




----- Original Message -----

> On 19/11/14 05:17, Shirshendu Chakrabarti wrote:

> > Hi PCP Team,

> >

> > I have bee trying to obtain the pmwebd binary from pcp sources on=
 Amazon

> > Linux.

> >

> > I build the rpms but I never see the pmwebd daemon installed.

>

> I don't have access to an Amazon Linux system, but it is most like=
ly

> that there are missing RPMS, probably qt-devel.



It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 req=
uired) package

on the build machine ("libmicrohttpd-dev" package on Debian/Ubunt=
u).



cheers.



--

Nathan





--001a1139126c902f70050830236d--

From nscott@redhat.com  Wed Nov 19 03:29:49 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 079027F3F
	for ; Wed, 19 Nov 2014 03:29:49 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id D93C78F804C
	for ; Wed, 19 Nov 2014 01:29:45 -0800 (PST)
X-ASG-Debug-ID: 1416389380-04bdf061611428b0001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id CQbW2VrFoiiV7k80 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 01:29:40 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJ9TekD010119;
	Wed, 19 Nov 2014 04:29:40 -0500
Date: Wed, 19 Nov 2014 04:29:40 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: David Smith 
Cc: pcp 
Message-ID: <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com>
In-Reply-To: <546A7EDD.9000009@redhat.com>
References: <54512E80.9090302@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com>
Subject: Re: [pcp] [RFC] pcp python patch
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp python patch
Thread-Index: nPacU9zAGA0iUuzOX22WuOlh/Wm0gQ==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1416389380
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11760
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

Hi David,

----- Original Message -----
> On 11/17/2014 04:53 PM, Nathan Scott wrote:
> > Oh, I just had the impression from your earlier mail you weren't
> > completely satisfied that we'd covered off all the cases ... its
> > likely I've just misinterpreted that "except for [1]" reference.
> 
> I believe I've covered all the cases. All list/dictionary references are
> cached down in cpmda and no longer thrown away after use (a). In the
> PMDA class, I made sure all lists/dictionaries are cleared, not recreated.
> 

OK, good stuff.  Do you want to merge the PMDA API changes at this
stage?

I tried cherry-picking (there's lots of other commits in dsmith/dev):
cec13bfd0297ecc755265ba2db69a86daf32a05c
f2f5a51cde0646fcdf35bc2f60798024c0931c9e
1fc0bbf9d517810e8512fb3a7775b68fc6f64572
... but there was a fair few test failures (I haven't dug deeper yet -
they all look like python PMDAs failing to start or exiting early on,
from a quick glance).

Or, do you want to wait for that JSON PMDA to be closer to complete so
we have at least one use case for the API?  (either that or some form
of specialised API test case will be needed - a real PMDA is probably
easiest, with its tests covering use of the new API).

cheers.

--
Nathan

From nscott@redhat.com  Wed Nov 19 04:41:54 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id B856D7F3F
	for ; Wed, 19 Nov 2014 04:41:54 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id A79ED304039
	for ; Wed, 19 Nov 2014 02:41:51 -0800 (PST)
X-ASG-Debug-ID: 1416393710-04cb6c057314b4a0001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id 3t6wQ6LCwpKxrg9f (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 02:41:50 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJAfnWU024088
	for ; Wed, 19 Nov 2014 05:41:49 -0500
Date: Wed, 19 Nov 2014 05:41:49 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <1858541981.1807300.1416393709898.JavaMail.zimbra@redhat.com>
In-Reply-To: <1484891499.1806617.1416393564672.JavaMail.zimbra@redhat.com>
Subject: pcp updates: proc, qa
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: proc, qa
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: proc, qa
Thread-Index: XXMxqIykVzBB50lytkhHex5Z4E5/hQ==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1416393710
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11761
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Nathan Scott (2):
      build: add .gitignore file for perfevent pmda
      qa: begin a collection of /proc/cpuinfo files for testing

Frank Ch. Eigler (1):
      pmda proc: fix cgroup fd leaks causing too many open files

 qa/linux/GNUmakefile                  |    3 
 qa/linux/cpuinfo-32-4830              |  800 ++++++++++++++++++++++++++++++++
 qa/linux/cpuinfo-aarch64-linux-3.17.0 |   17 
 qa/linux/cpuinfo-amd-32-6132          |  832 ++++++++++++++++++++++++++++++++++
 qa/linux/cpuinfo-ia64-linux-2.6.32    |   76 +++
 src/pmdas/linux_proc/cgroups.c        |    5 
 src/pmdas/perfevent/.gitignore        |    6 
 7 files changed, 1737 insertions(+), 2 deletions(-)

From manoopjoseph@gmail.com  Wed Nov 19 07:19:27 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,
	T_DKIM_INVALID autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id BD5AB7F3F
	for ; Wed, 19 Nov 2014 07:19:27 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 8F4708F8040
	for ; Wed, 19 Nov 2014 05:19:24 -0800 (PST)
X-ASG-Debug-ID: 1416403162-04bdf061601487c0001-S8gJnT
Received: from mail-vc0-f172.google.com (mail-vc0-f172.google.com [209.85.220.172]) by cuda.sgi.com with ESMTP id e2YepJGyH7wJMCXD (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Wed, 19 Nov 2014 05:19:23 -0800 (PST)
X-Barracuda-Envelope-From: manoopjoseph@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.220.172
X-Barracuda-IPDD: Level1 [gmail.com/209.85.220.172]
Received: by mail-vc0-f172.google.com with SMTP id hq11so255796vcb.17
        for ; Wed, 19 Nov 2014 05:19:22 -0800 (PST)
X-Barracuda-IPDD: Level1 [gmail.com/209.85.220.172]
X-Barracuda-IPDD: Level1 [gmail.com/209.85.220.172]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=XBTOp4sHkK6v0A6TvrQP9VaR1pHESD13ofmWrnaw1cw=;
        b=sfeO8koKG3zuMdIj0gz5R+WcqxnmDSvWWy4RDfwYBQyUNaigCtafoSbC067TDE2/JC
         8Us5L/ax4m00uGmXAnC2AQgS0XGAYhsCm7oHz9CO0xQ4Rz4aXy0QW0arP98zGFXMQ+FT
         lpkf2Q3KroiSfdpStn5Ss4zEM3ZtHZ2cBu0QGHLIWn4TXZoMDO5zp8Y0IBi3DP4og1Kv
         /aco6AM6u7EiMfBgAKV4DHfv2snX+pyf+2ey6RUWjA3vQpXFyYWDmCwqTK2rNk6m9Cbd
         xucn41d2mkCpx5VD9W5bMq8dYb0GlQhfrI2Yurgc3LIAZyuldPq2DSkufB6IgvnNmgNK
         imDg==
MIME-Version: 1.0
X-Received: by 10.52.167.129 with SMTP id zo1mr31825993vdb.9.1416403161695;
 Wed, 19 Nov 2014 05:19:21 -0800 (PST)
Received: by 10.31.141.139 with HTTP; Wed, 19 Nov 2014 05:19:21 -0800 (PST)
In-Reply-To: 
References: 
Date: Wed, 19 Nov 2014 18:49:21 +0530
Message-ID: 
Subject: Fwd: PCP framework queries
From: Anoop Joseph Babu 
X-ASG-Orig-Subj: Fwd: PCP framework queries
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=089e0160c0be4fcfc80508360fa8
X-Barracuda-Connect: mail-vc0-f172.google.com[209.85.220.172]
X-Barracuda-Start-Time: 1416403162
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11765
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 HTML_MESSAGE           BODY: HTML included in message

--089e0160c0be4fcfc80508360fa8
Content-Type: text/plain; charset=UTF-8

Hi,

I was referring through the mmv_genstats.c

file in github. Can you elaborate on  mmv_stats_interval_start() Api. How
can I set the interval period and get interval expiry notice.
If possible please answer in
http://stackoverflow.com/questions/27011416/mmv-stats-interval-start-usage-example-in-c


Regards,
Anoop Joseph Babu
+918861450505

--089e0160c0be4fcfc80508360fa8
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable




Hi,
I was referring through the mmv_genstat=
s.c file in github. Can you elaborate on =C2=A0mmv_stats_interval_start=
() Api. How can I set the interval period and get interval expiry notice.If possible please answer in =
http://stackoverflow.com/questions/27011416/mmv-stats-interval-start-usage-=
example-in-c


Regards,
An=
oop Joseph Babu
+918861450505







--089e0160c0be4fcfc80508360fa8--

From dsmith@redhat.com  Wed Nov 19 09:56:55 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id E6A747F3F
	for ; Wed, 19 Nov 2014 09:56:55 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id D4ED230405F
	for ; Wed, 19 Nov 2014 07:56:52 -0800 (PST)
X-ASG-Debug-ID: 1416412608-04cb6c05721557f0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 0TOxX07cXz6uVJ1J (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 07:56:48 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAJFul4d020983
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 19 Nov 2014 10:56:47 -0500
Received: from t540p.usersys.redhat.com (vpn-51-251.rdu2.redhat.com [10.10.51.251])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJFujiC006349
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Wed, 19 Nov 2014 10:56:46 -0500
Message-ID: <546CBDBD.5080605@redhat.com>
Date: Wed, 19 Nov 2014 09:56:45 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com> <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416412608
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/19/2014 03:29 AM, Nathan Scott wrote:
> Hi David,
> 
> ----- Original Message -----
>> On 11/17/2014 04:53 PM, Nathan Scott wrote:
>>> Oh, I just had the impression from your earlier mail you weren't
>>> completely satisfied that we'd covered off all the cases ... its
>>> likely I've just misinterpreted that "except for [1]" reference.
>>
>> I believe I've covered all the cases. All list/dictionary references are
>> cached down in cpmda and no longer thrown away after use (a). In the
>> PMDA class, I made sure all lists/dictionaries are cleared, not recreated.
>>
> 
> OK, good stuff.  Do you want to merge the PMDA API changes at this
> stage?
> 
> I tried cherry-picking (there's lots of other commits in dsmith/dev):
> cec13bfd0297ecc755265ba2db69a86daf32a05c
> f2f5a51cde0646fcdf35bc2f60798024c0931c9e
> 1fc0bbf9d517810e8512fb3a7775b68fc6f64572
> ... but there was a fair few test failures (I haven't dug deeper yet -
> they all look like python PMDAs failing to start or exiting early on,
> from a quick glance).

Hmm, in theory these changes shouldn't effect existing python PMDAs.
I'll try to figure out how to run those tests and see what I get.

Let me do that before we merge the changes.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From shirshendu@riva.co  Wed Nov 19 10:27:53 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id EF1D27F3F
	for ; Wed, 19 Nov 2014 10:27:53 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id DDC5330404E
	for ; Wed, 19 Nov 2014 08:27:50 -0800 (PST)
X-ASG-Debug-ID: 1416414468-04cb6c05711568b0001-S8gJnT
Received: from mail-qa0-f48.google.com (mail-qa0-f48.google.com [209.85.216.48]) by cuda.sgi.com with ESMTP id MThXL6RklEQL5QGi (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Wed, 19 Nov 2014 08:27:49 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.48
Received: by mail-qa0-f48.google.com with SMTP id v10so618293qac.21
        for ; Wed, 19 Nov 2014 08:27:48 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=8at8yzEycMH7CHya00LX0dH607xtu9uKC0OtVpUWrgQ=;
        b=IWS64Iu7Bpfrj0Yd97oaZlXSSAmDJQ4qmc2Vio8u6J2oZSTOuN9aKp7fH4odgfP0fj
         TIlMoIam4ut1f+ywQNMomWcMkhiA5pmijwaX5mmQ5eJF8lO/eXL0eHdZlu3N8JwZ1D5W
         tqpZuTIIM8B8I7jHL6hG0PmeYirQU1mNDix+vKZomPgotksGR6u+Sgqi6r9AUERRCqJc
         F3Hq5oWQcYjb9bFIMXfLDLTmJu9u8ZBL2jPW6r/kx+TpusSzpGVS/Y9t1apknaNTJTGh
         s14x/aQW1uPpBBYxRPSoUu5eN3qv1lYBzkdWxSd2nmCGkAkrWgSe4YxH/V4giTTFsJ7K
         zloA==
X-Gm-Message-State: ALoCoQmxtm8NkfZwm3ieVHJY+b+oEw1eRNn9yUCppFIi3XIEHaK3EVc6z1r9dxgouyk52SiVrbkS
MIME-Version: 1.0
X-Received: by 10.224.65.4 with SMTP id g4mr53334558qai.20.1416414468667; Wed,
 19 Nov 2014 08:27:48 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Wed, 19 Nov 2014 08:27:48 -0800 (PST)
In-Reply-To: 
References: 
	<546B9C7F.1050706@internode.on.net>
	<432584280.1372959.1416341277776.JavaMail.zimbra@redhat.com>
	
Date: Wed, 19 Nov 2014 21:57:48 +0530
Message-ID: 
Subject: Re: [pcp] Non-availability of pmwebd on Amazon Linux
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Non-availability of pmwebd on Amazon Linux
To: Nathan Scott 
Cc: Ken McDonell , pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2b98042ae4e050838b1b0
X-Barracuda-Connect: mail-qa0-f48.google.com[209.85.216.48]
X-Barracuda-Start-Time: 1416414469
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11769
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2b98042ae4e050838b1b0
Content-Type: text/plain; charset=UTF-8

Hi Ken and Nathan,

Please find the pastie at http://pastie.org/private/kj1ow3plpriuuwfx4xpdsg

This pastie is the output of check-vm command.

As was correctly pointed out by both of you, qt-devel and
libmicrohttpd-devel are missing.

Working to fix them. Will keep the team posted.

Thanks,

Shirshendu Chakrabarti

On Wed, Nov 19, 2014 at 11:45 AM, Shirshendu Chakrabarti  wrote:

> Hi Team,
>
> Thanks for the pointers, I will try both missing packages listed above and
> report back.
>
> Thanks,
>
> Shirshendu Chakrabarti
>
> On Wed, Nov 19, 2014 at 1:37 AM, Nathan Scott  wrote:
>
>>
>>
>> ----- Original Message -----
>> > On 19/11/14 05:17, Shirshendu Chakrabarti wrote:
>> > > Hi PCP Team,
>> > >
>> > > I have bee trying to obtain the pmwebd binary from pcp sources on
>> Amazon
>> > > Linux.
>> > >
>> > > I build the rpms but I never see the pmwebd daemon installed.
>> >
>> > I don't have access to an Amazon Linux system, but it is most likely
>> > that there are missing RPMS, probably qt-devel.
>>
>> It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 required)
>> package
>> on the build machine ("libmicrohttpd-dev" package on Debian/Ubuntu).
>>
>> cheers.
>>
>> --
>> Nathan
>>
>
>

--001a11c2b98042ae4e050838b1b0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Ken and Nathan,


=
This pastie is the output of check-vm command.

As =
was correctly pointed out by both of you, qt-devel and libmicrohttpd-devel =
are missing.

Working to fix them. Will keep the te=
am posted.

Thanks,

Shirsh=
endu Chakrabarti

On Wed, Nov 19, 2014 at 11:45 AM, Shirshendu Chakrabarti <shir=
shendu@riva.co> wrote:
Hi Team,

Thanks for the pointers, I will tr=
y both missing packages listed above and report back.

<=
div>Thanks,

Shirshendu Chakrabarti

On Wed, Nov 19, 2014 at 1:37 AM, Nathan Scott <nathan=
s@redhat.com> wrote:
=




----- Original Message -----

> On 19/11/14 05:17, Shirshendu Chakrabarti wrote:

> > Hi PCP Team,

> >

> > I have bee trying to obtain the pmwebd binary from pcp sources on=
 Amazon

> > Linux.

> >

> > I build the rpms but I never see the pmwebd daemon installed.

>

> I don't have access to an Amazon Linux system, but it is most like=
ly

> that there are missing RPMS, probably qt-devel.



It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 req=
uired) package

on the build machine ("libmicrohttpd-dev" package on Debian/Ubunt=
u).



cheers.



--

Nathan








--001a11c2b98042ae4e050838b1b0--

From fche@redhat.com  Wed Nov 19 11:18:40 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 96B677F3F
	for ; Wed, 19 Nov 2014 11:18:40 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 85F3A30405F
	for ; Wed, 19 Nov 2014 09:18:37 -0800 (PST)
X-ASG-Debug-ID: 1416417512-04cbb01e5a1600e0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id XaCxSi3oIworq7K0 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 09:18:33 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAJHIVgt005856
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 19 Nov 2014 12:18:32 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJHIV3e027965
	for ; Wed, 19 Nov 2014 12:18:31 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 689E25822C; Wed, 19 Nov 2014 12:18:30 -0500 (EST)
Date: Wed, 19 Nov 2014 12:18:30 -0500
From: "Frank Ch. Eigler" 
To: pcp developers 
Subject: fetch/desc conflict defeats dynamic pmns, papi pmda case study
Message-ID: <20141119171830.GE5700@redhat.com>
X-ASG-Orig-Subj: fetch/desc conflict defeats dynamic pmns, papi pmda case study
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416417513
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

Looking at lberk's fine commit #7e9b87e49b18 in pcpfans.git
lberk/papi, I was reminded of an earlier idea to make the pmda desc
callback function fully dynamic.  This would avoid having to have a
materialized metrictab[] within the pmda code at all, which would
simplify the code, reduce data size, basically sounds like a win/win.

I even prototyped it (patch below), but it doesn't quite work!  It
supplies "pminfo -d" and such all right, so clients get nice pmDesc
data, but it kills the ability to do fetches!  And that is because
the way pmdaFetch() is implemented in libpcp_pmda: it mandates that
the metric table be materialized into a pmdaMetric table, which it
searches for a pmid->pmDesc entry.  All it wants is the pmDesc, but
instead of calling the pmda->desc callback function to get it, it does
a private lookup on a potentially empty or stale table.

I started prototyping a change to pmdaFetch() (patch below) to fall
back to a call into the pmda desc callback function pointer, but
unfortunately pmdaFetch() is not given enough parameters to find it:
only a pmdaExt*, from which it appears impossible to find the
associated pmdaInterface*.

What shall we do about this?  It seems an unsatisfactory state of
affairs, to both allow pmdas to provide a custom pmid->pmDesc lookup
function, but then ignore it in the fetch path and instead mandate
a previously-supplied lookup table.

- FChE


--- a/src/libpcp_pmda/src/callback.c
+++ b/src/libpcp_pmda/src/callback.c
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2013 Red Hat.
+ * Copyright (c) 2013-2014 Red Hat.
  * Copyright (c) 1995-2000 Silicon Graphics, Inc.  All Rights Reserved.
  * 
  * This library is free software; you can redistribute it and/or modify it
@@ -449,6 +449,7 @@ pmdaFetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda)
     int			inst;
     int			numval;
     pmValueSet		*vset;
+    pmDesc		dpBuf;
     pmDesc		*dp;
     pmdaMetric		*metap;
     pmAtomValue		atom;
@@ -472,7 +473,12 @@ pmdaFetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda)
     extp->res->timestamp.tv_usec = 0;
     extp->res->numpmid = numpmid;
 
+    /* Look up the pmDesc for the incoming pmids in our pmdaMetrics tables,
+       if present.  Fall back to .desc callback if not found (for highly
+       dynamic pmdas). */
     for (i = 0; i < numpmid; i++) {
+        dp = NULL;
+
 	if (pmda->e_flags & PMDA_EXT_FLAG_HASHED)
 	    metap = __pmdaHashedSearch(pmidlist[i], &extp->hashpmids);
 	else if (pmda->e_direct)
@@ -480,8 +486,17 @@ pmdaFetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda)
 	else
 	    metap = __pmdaLinearSearch(pmidlist[i], pmda);
 
-	if (metap) {
+	if (metap != NULL)
 	    dp = &(metap->m_desc);
+        else {
+            if (extp->version.any.desc != NULL) { // <--- XXX: inaccessible!!
+                sts = (*(extp->version.any.desc))(pmidlist[i], &dpBuf, pmda);
+                if (sts >= 0)
+                    dp = &dpBuf;
+            }
+        }
+
+	if (dp != NULL) {
 	    if (dp->indom != PM_INDOM_NULL) {
 		/* count instances in the profile */
 		numval = 0;
@@ -497,7 +512,6 @@ pmdaFetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda)
 	    }
 	}
 	else {
-	    dp = NULL;
 	    /* dynamic name metrics may often vanish, avoid log spam */
 	    if (extp->pmda_interface < PMDA_INTERFACE_4) {
 		char	strbuf[20];


--- a/src/pmdas/papi/papi.c
+++ b/src/pmdas/papi/papi.c
@@ -66,8 +66,6 @@ static int refresh_metrics();
 static void auto_enable_expiry_cb (int, void *);
 
 static char helppath[MAXPATHLEN];
-static pmdaMetric *metrictab;
-static int nummetrics;
 
 static int
 permission_check(int context)
@@ -123,21 +121,6 @@ enlarge_ctxtab(int context)
     }
 }
 
-static void
-expand_metric_tab(int size)
-{
-    pmUnits units = PMDA_PMUNITS(0,0,0,0, PM_TIME_NSEC, 0);
-    size_t need = sizeof(pmdaMetric)*(nummetrics+1);
-    metrictab = realloc(metrictab, need);
-    if (metrictab == NULL)
-	__pmNoMem("metrictab expansion", need, PM_FATAL_ERR);
-
-    metrictab[nummetrics-1].m_desc.pmid = papi_info[size].pmid;
-    metrictab[nummetrics-1].m_desc.type = PM_TYPE_U64;
-    metrictab[nummetrics-1].m_desc.indom = PM_INDOM_NULL;
-    metrictab[nummetrics-1].m_desc.sem = PM_SEM_COUNTER;
-    metrictab[nummetrics-1].m_desc.units = units;
-}
 
 static int
 check_papi_state()
@@ -188,7 +171,7 @@ papi_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
 	if (idp->item >= 0 && idp->item <= number_of_events) {
 	    // the 'case' && 'idp->item' value we get is the pmns_position
 	    if (papi_info[idp->item].position >= 0) {
-		atom->ull = papi_info[idp->item].prev_value + values[papi_info[idp->item].position];
+		atom->ll = papi_info[idp->item].prev_value + values[papi_info[idp->item].position];
                 // if previously auto-enabled, extend the timeout
                 if (papi_info[idp->item].metric_enabled != METRIC_ENABLED_FOREVER &&
                     auto_enable_time)
@@ -543,9 +526,60 @@ papi_store(pmResult *result, pmdaExt *pmda)
 static int
 papi_desc(pmID pmid, pmDesc *desc, pmdaExt *pmda)
 {
-    return pmdaDesc(pmid, desc, pmda);
+    int sts = PM_ERR_PMID; // presume fail; switch statements fall through to this
+    __pmID_int *idp = (__pmID_int *)&(pmid);
+
+    switch (idp->cluster) {
+    case CLUSTER_PAPI:
+        desc->pmid = pmid; // needed?
+        desc->type = PM_TYPE_64;
+        desc->indom = PM_INDOM_NULL;
+        desc->sem = PM_SEM_COUNTER;
+        desc->units = (pmUnits) PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE);
+        sts = 0;
+        break;
+
+    case CLUSTER_CONTROL:
+        switch (idp->item) {
+        case 0: // papi.control.enable
+        case 1: // papi.control.reset
+        case 2: // papi.control.disable
+        case 3: // papi.control.status
+            desc->pmid = pmid; // needed?
+            desc->type = PM_TYPE_STRING;
+            desc->indom = PM_INDOM_NULL;
+            desc->sem = PM_SEM_DISCRETE;
+            desc->units = (pmUnits) PMDA_PMUNITS(0,0,0,0,0,0);
+            sts = 0;
+            break;
+        case 4: // papi.control.auto_enable
+            desc->pmid = pmid; // needed?
+            desc->type = PM_TYPE_U32;
+            desc->indom = PM_INDOM_NULL;
+            desc->sem = PM_SEM_DISCRETE;
+            desc->units = (pmUnits) PMDA_PMUNITS(0,1,0,0,PM_TIME_SEC,0);
+            sts = 0;
+            break;
+        }
+        break;
+
+    case CLUSTER_AVAILABLE:
+        switch (idp->item) {
+        case 0: // .available.num_counters
+            desc->pmid = pmid; // needed?
+            desc->type = PM_TYPE_U32;
+            desc->indom = PM_INDOM_NULL;
+            desc->sem = PM_SEM_DISCRETE;
+            desc->units = (pmUnits) PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE);
+            sts = 0;
+        }
+        break;
+    }
+
+    return sts;
 }
 
+
 static int
 papi_text(int ident, int type, char **buffer, pmdaExt *ep)
 {
@@ -563,9 +597,11 @@ papi_text(int ident, int type, char **buffer, pmdaExt *ep)
 		*buffer = papi_info[pmidp->item].info.long_descr;
 	    return 0;
 	}
+        /* delegate to "help" file */
 	return pmdaText(ident, type, buffer, ep);
     }
     else
+        /* delegate to "help" file */
 	return pmdaText(ident, type, buffer, ep);
 }
 
@@ -625,8 +661,6 @@ papi_internal_init(pmdaInterface *dp)
 		memset(&entry[0], 0, sizeof(entry));
 		papi_info[i].position = -1;
 		papi_info[i].metric_enabled = 0;
-		nummetrics++;
-		expand_metric_tab(i);
 		expand_values(i);
 		i++;
 	    }
@@ -687,39 +721,7 @@ void
 __PMDA_INIT_CALL
 papi_init(pmdaInterface *dp)
 {
-    int i;
     int retval;
-    pmUnits auto_enable_units = PMDA_PMUNITS(0,1,0,0, PM_TIME_SEC, 0);
-    pmUnits other_units = PMDA_PMUNITS(0,0,0,0, PM_TIME_NSEC, 0);
-    nummetrics = 6;
-    metrictab = malloc(nummetrics*sizeof(pmdaMetric));
-    if (metrictab == NULL)
-	__pmNoMem("initial metrictab allocation", (nummetrics*sizeof(pmdaMetric)), PM_FATAL_ERR);
-    for(i = 0; i < 6; i++) {
-	switch (i) {
-	case 4:
-	    metrictab[i].m_desc.pmid = pmid_build(dp->domain, CLUSTER_CONTROL, i);
-	    metrictab[i].m_desc.type = PM_TYPE_U32;
-	    metrictab[i].m_desc.indom = PM_INDOM_NULL;
-	    metrictab[i].m_desc.sem = PM_SEM_DISCRETE;
-	    metrictab[i].m_desc.units = auto_enable_units;
-	    break;
-	case 5:
-	    metrictab[i].m_desc.pmid = pmid_build(dp->domain, CLUSTER_AVAILABLE, 0);
-	    metrictab[i].m_desc.type = PM_TYPE_U32;
-	    metrictab[i].m_desc.indom = PM_INDOM_NULL;
-	    metrictab[i].m_desc.sem = PM_SEM_DISCRETE;
-	    metrictab[i].m_desc.units = other_units;
-	    break;
-	default:
-	    metrictab[i].m_desc.pmid = pmid_build(dp->domain, CLUSTER_CONTROL, i);
-	    metrictab[i].m_desc.type = PM_TYPE_STRING;
-	    metrictab[i].m_desc.indom = PM_INDOM_NULL;
-	    metrictab[i].m_desc.sem = PM_SEM_INSTANT;
-	    metrictab[i].m_desc.units = other_units;
-	    break;
-	}
-    }
 
     if (isDSO) {
 	int	sep = __pmPathSeparator();
@@ -755,7 +757,7 @@ papi_init(pmdaInterface *dp)
     dp->version.four.children = papi_children;
     pmdaSetFetchCallBack(dp, papi_fetchCallBack);
     pmdaSetEndContextCallBack(dp, papi_endContextCallBack);
-    pmdaInit(dp, NULL, 0, metrictab, nummetrics);
+    pmdaInit(dp, NULL, 0, NULL, 0);
 }
 
 static pmLongOptions longopts[] = {
@@ -801,7 +803,6 @@ main(int argc, char **argv)
     free(ctxtab);
     free(papi_info);
     free(values);
-    free(metrictab);
 
     exit(0);
 }

From fche@redhat.com  Wed Nov 19 12:14:47 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 6ADAA7F3F
	for ; Wed, 19 Nov 2014 12:14:47 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id EE77BAC00B
	for ; Wed, 19 Nov 2014 10:14:43 -0800 (PST)
X-ASG-Debug-ID: 1416420881-04cbb01e5a16eb20001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Ypxk1KjBiAgaVGSF (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 10:14:42 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAJIEewT005786
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 19 Nov 2014 13:14:41 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJIEe0W016792;
	Wed, 19 Nov 2014 13:14:40 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 1EA185822C; Wed, 19 Nov 2014 13:14:40 -0500 (EST)
Date: Wed, 19 Nov 2014 13:14:40 -0500
From: "Frank Ch. Eigler" 
To: Nathan Scott 
Cc: pcp 
Subject: Re: pcp updates: pmdaproc, cgroups, books
Message-ID: <20141119181439.GF5700@redhat.com>
X-ASG-Orig-Subj: Re: pcp updates: pmdaproc, cgroups, books
References: <1309338393.770280.1416292315684.JavaMail.zimbra@redhat.com> <1468616216.771853.1416292522829.JavaMail.zimbra@redhat.com>  <1249426361.1589601.1416371161222.JavaMail.zimbra@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1249426361.1589601.1416371161222.JavaMail.zimbra@redhat.com>
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416420882
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

On Tue, Nov 18, 2014 at 11:26:01PM -0500, Nathan Scott wrote:
> [...]
> > #3.  From looking more at strace and the code, it looks as though the pmda
> >      might be doing too much work per unit fetch.
> > [...]
> > #4.  Instance domain unawareness.  A variant of #3, when a pcp client is
> > [...]
> > #5.  Multiple metrics unawareness (so to speak).  A variant of #3,
> 
> There's subtleties to the protocol you may not be following here
> and thats misleading you in interpreting the observed results, I
> think.  The instance PDUs are a separate entity to fetch PDUs,
> but the PMDA needs to take similar refresh actions to respond to
> each.

The wire protocol is not at issue here.  The pmcd / pmda does get told
eventually of the instances / metrics of specific interest to each
client, so by the time the custom pmda fetch callbacks are called,
they has all the info they need.

It's just that the way the linux-proc pmda responds to a fetch(), it
considers it necessary to refresh to world for each incoming call
(src/pmdas/linux_proc/pmda.c:2388).  That's only a little burdensome
on small machines, but will outright fail to scale to hundreds or
thousands of cgroups.

If instead the pmda fetch callback produced pinpoint answers to the
incoming pmid/inst pair by looking up a single value in /proc or
/sys/fs/cgroup, there would be no wasted work.  Many of those caches
(misnamed - they're just ordinary lookup tables here rather than
locality-based accelerators) might not be needed at all.

 
> > linux/cgroups-root*.tgz.  It's clever & useful, but cannot be as
> > thorough.)
> 
> Oh, it can be far more thorough than we could ever hope to be with
> testing by-hand [...]

I was not comparing it to testing-by-hand, but rather to testing
against a real live OS in its unforseeable gloriously messy diversity.


> [...]  I imagine you simply have many more active cgroups than
> I did on my dev box when I created cgroups-root-001.tgz [...]

That could be part of it.  Another part is that a small sequence of
operations against a fixed data set is not going to exercise the code
the same way as long-term operation against a varying data set.

Perhaps we could use a fuzzing-inspired test case which blasts all
PMDAs with a random bunch of traffic for an extended period, just
looking for the preservation of pcp service/uptime.


- FChE

From fche@redhat.com  Wed Nov 19 12:16:09 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 1F28F7F3F
	for ; Wed, 19 Nov 2014 12:16:09 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 0D7798F8059
	for ; Wed, 19 Nov 2014 10:16:06 -0800 (PST)
X-ASG-Debug-ID: 1416420964-04cb6c057217cfa0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id spNBfBf4Ia3PAON5 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 10:16:05 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAJIG3js006272
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 19 Nov 2014 13:16:04 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJIG3LN031499
	for ; Wed, 19 Nov 2014 13:16:03 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id CB7D35822C; Wed, 19 Nov 2014 13:16:02 -0500 (EST)
Date: Wed, 19 Nov 2014 13:16:02 -0500
From: "Frank Ch. Eigler" 
To: pcp developers 
Subject: pcp update: small papi-pmda units tweak
Message-ID: <20141119181602.GG5700@redhat.com>
X-ASG-Orig-Subj: pcp update: small papi-pmda units tweak
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416420965
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

>From fche/papi on pcpfans.git:

commit 395b0982b199cb4c81942cf96f13414591da2072
Author: Frank Ch. Eigler 
Date:   Wed Nov 19 12:50:57 2014 -0500

    papi pmda: tweak metric units
    
    Some of the papi.* metric units can be more precise:
    
    papi.system.* counters      to signed 64-bit and Units: count
    papi.available.num_counters to Units: count
    papi.control.auto_enable    left as is
    papi.control.enable etc.    to Units: none

From nscott@redhat.com  Wed Nov 19 14:11:11 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 913A87F3F
	for ; Wed, 19 Nov 2014 14:11:11 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 71794304062
	for ; Wed, 19 Nov 2014 12:11:11 -0800 (PST)
X-ASG-Debug-ID: 1416427866-04cbb01e5a172c80001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id jIgbB7qLWh9vUFGL (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 12:11:07 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJKB6rF018051;
	Wed, 19 Nov 2014 15:11:06 -0500
Date: Wed, 19 Nov 2014 15:11:05 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: "Frank Ch. Eigler" 
Cc: pcp 
Message-ID: <1666386574.2247920.1416427865663.JavaMail.zimbra@redhat.com>
In-Reply-To: <20141119181439.GF5700@redhat.com>
References: <1309338393.770280.1416292315684.JavaMail.zimbra@redhat.com> <1468616216.771853.1416292522829.JavaMail.zimbra@redhat.com>  <1249426361.1589601.1416371161222.JavaMail.zimbra@redhat.com> <20141119181439.GF5700@redhat.com>
Subject: Re: pcp updates: pmdaproc, cgroups, books
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: pcp updates: pmdaproc, cgroups, books
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: pmdaproc, cgroups, books
Thread-Index: rHmC7vOL1Tm5IaIj6bsOHu6/17us/Q==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1416427866
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11776
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> > [...]  I imagine you simply have many more active cgroups than
> > I did on my dev box when I created cgroups-root-001.tgz [...]
> 
> That could be part of it.  [...]

Could you send through that tarball I asked for please, so I can try
to reproduce and verify the fix here?

thanks.

--
Nathan

From wwwrun@oss.sgi.com  Wed Nov 19 15:57:25 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 060547F4E; Wed, 19 Nov 2014 15:57:25 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1074] New: 3d chart view: opengl pmview and/or pmwebd/webgl
 webapp
Date: Wed, 19 Nov 2014 21:57:24 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416434245.D5442dCF1.3389"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416434245.D5442dCF1.3389
Date: Wed, 19 Nov 2014 15:57:25 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1074

            Bug ID: 1074
           Summary: 3d chart view: opengl pmview and/or pmwebd/webgl
                    webapp
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo todo

- pcp todo item 6: 3d view
  - used because anomaly detection still manual with mk. 1 eyeball
  - old sgi pmview still attractive, especially if considering topology
  - an openscenegraph-based partial reimplementation exists
    - so dies a unity 3d/gl-based one, with browser blob etc.
  - maybe have pmwebd / webgl reimplementation

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416434245.D5442dCF1.3389
Date: Wed, 19 Nov 2014 15:57:25 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1074
          
        


        

          Summary
          3d chart view: opengl pmview and/or pmwebd/webgl webapp
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo todo

- pcp todo item 6: 3d view
  - used because anomaly detection still manual with mk. 1 eyeball
  - old sgi pmview still attractive, especially if considering topology
  - an openscenegraph-based partial reimplementation exists
    - so dies a unity 3d/gl-based one, with browser blob etc.
  - maybe have pmwebd / webgl reimplementation

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416434245.D5442dCF1.3389--

From wwwrun@oss.sgi.com  Wed Nov 19 15:58:19 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id C156C7F4E; Wed, 19 Nov 2014 15:58:19 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1075] New: centralized logging with highly dynamic instances
Date: Wed, 19 Nov 2014 21:58:19 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416434299.CA13daf1.3499"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416434299.CA13daf1.3499
Date: Wed, 19 Nov 2014 15:58:19 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1075

            Bug ID: 1075
           Summary: centralized logging with highly dynamic instances
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 9: centralized logging with dynamic instances
  - need better support for monitoring instances that come & go
    asynchronously
  - need more awareness of pcp discovery / scanning / pmmgr
  - probably also need an instance-invoked mechanism to notify
    central logger of arrival of new node to log (apart from avahi?),
    or cloud/grid-infrastructure apis for instance enumeration

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416434299.CA13daf1.3499
Date: Wed, 19 Nov 2014 15:58:19 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1075
          
        


        

          Summary
          centralized logging with highly dynamic instances
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 9: centralized logging with dynamic instances
  - need better support for monitoring instances that come & go
    asynchronously
  - need more awareness of pcp discovery / scanning / pmmgr
  - probably also need an instance-invoked mechanism to notify
    central logger of arrival of new node to log (apart from avahi?),
    or cloud/grid-infrastructure apis for instance enumeration

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416434299.CA13daf1.3499--

From kenj@internode.on.net  Wed Nov 19 16:00:04 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 2D6D07F3F
	for ; Wed, 19 Nov 2014 16:00:04 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 1C6A1304043
	for ; Wed, 19 Nov 2014 14:00:04 -0800 (PST)
X-ASG-Debug-ID: 1416434398-04cb6c0571183ad0001-S8gJnT
Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id Wnc8zW4rkMuXy3id for ; Wed, 19 Nov 2014 13:59:59 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.143
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: An4CADQSbVR20ZQhPGdsb2JhbAANTYtFyU6DHAKBHQEBAQEBBgEBAQE4hD0BAQEDASdREQsYCRYPCQMCAQIBMRQTCAEBiDS9C5dMAQEIAgEfkQ8WhDUBBLlrgyQBAQE
Received: from ppp118-209-148-33.lns20.mel8.internode.on.net (HELO [192.168.1.100]) ([118.209.148.33])
  by ipmail05.adl6.internode.on.net with ESMTP; 20 Nov 2014 08:29:32 +1030
Message-ID: <546D1339.1000601@internode.on.net>
Date: Thu, 20 Nov 2014 09:01:29 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: Re: [pcp] fetch/desc conflict defeats dynamic pmns, papi pmda case
 study
References: <20141119171830.GE5700@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] fetch/desc conflict defeats dynamic pmns, papi pmda case
 study
In-Reply-To: <20141119171830.GE5700@redhat.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143]
X-Barracuda-Start-Time: 1416434398
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11780
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

G'day Frank.

On 20/11/14 04:18, Frank Ch. Eigler wrote:
> ...

Fair enough observation.  My only defence is that for about 8 years there was no justification for dynamic metrics as all the PMDAs we needed had predefined sets of possible metrics.  Dynamic metrics were added quite late in the game, when it seemed like I could kill 2 birds with almost one stone, giving us derived metrics (long realized as useful) and dynamic metrics (believed at the time to be in the nice to have bucket).

It is not surprising that the libpcp_pmda design (first written in Cuneiform) did not accommodate all the nuances of dynamic metrics that have become more important than derived metrics ... 8^)>

> 
> I started prototyping a change to pmdaFetch() (patch below) to fall
> back to a call into the pmda desc callback function pointer, but
> unfortunately pmdaFetch() is not given enough parameters to find it:
> only a pmdaExt*, from which it appears impossible to find the
> associated pmdaInterface*.
> 
> What shall we do about this?  ...

You could try the patch below ... the e_ext field of pmdaExt is already a pointer to a private helper structure that holds the pmda_interface (version) number that is needed for similar situations ... extending this to also have a back pointer seems safe for everyone.


kenj@bozo:~/src/pcp/src/libpcp_pmda/src$ cat /tmp/patch.pmdainterface
diff --git a/src/libpcp_pmda/src/libdefs.h b/src/libpcp_pmda/src/libdefs.h
index 1d8b1c6..35554c2 100644
--- a/src/libpcp_pmda/src/libdefs.h
+++ b/src/libpcp_pmda/src/libdefs.h
@@ -27,6 +27,7 @@
  * when multiple DSO PMDAs are in use
  */
 typedef struct {
+    pmdaInterface	*dispatch;	/* back pointer to our pmdaInterface */
     int		pmda_interface;
     pmResult	*res;			/* high-water allocation for */
     int		maxnpmids;		/* pmResult for each PMDA */
diff --git a/src/libpcp_pmda/src/open.c b/src/libpcp_pmda/src/open.c
index 340f363..ab554d8 100644
--- a/src/libpcp_pmda/src/open.c
+++ b/src/libpcp_pmda/src/open.c
@@ -920,6 +920,7 @@ __pmdaSetup(pmdaInterface *dispatch, int version, char *name)
 	dispatch->status = PM_ERR_GENERIC;
 	return;
     }
+    extp->dispatch = dispatch;
     extp->pmda_interface = version;
     pmda->e_ext = (void *)extp;
 

From lberk@redhat.com  Wed Nov 19 17:45:29 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 69DE27F3F
	for ; Wed, 19 Nov 2014 17:45:29 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 4ACE7304039
	for ; Wed, 19 Nov 2014 15:45:25 -0800 (PST)
X-ASG-Debug-ID: 1416440724-04cbb01e5c177bc0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id LOjPS7sQ3iXeJ9Mn (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 15:45:24 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAJNjONo014970
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 19 Nov 2014 18:45:24 -0500
Received: from toium ([10.15.16.195])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJNjN0g015820
	(version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO)
	for ; Wed, 19 Nov 2014 18:45:23 -0500
From: Lukas Berk 
To: pcp@oss.sgi.com
Subject: pcp updates - pmdapapi
Date: Wed, 19 Nov 2014 18:45:23 -0500
X-ASG-Orig-Subj: pcp updates - pmdapapi
Message-ID: <87ioiasqjw.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416440724
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi,

Changes committed to: lberk/papi on git://sourceware.org/git/pcpfans.git

A few more updates for pmdapapi, including dynamically generated
metrictab and enabling/disabling multiplexing based on metric value (and
of course updated qa to go with it).  The multiplexing commits
specifically may make it easier to test for ECNFLCT cases.  I'll be
working on adding that next.

I've also gone ahead and cherry-picked Frank's metric unit changes[1] as
it conflicted with my multiplexing commit.  Should be easier to merge
directly from my branch.

Cheers,

Lukas

[1] - http://www.pcp.io/pipermail/pcp/2014-November/006006.html

qa/813.out                |    6 
qa/914.out                |   10 
qa/967.out                |   12 
src/pmdas/papi/help       |    4 
src/pmdas/papi/papi.c     |  596 +++++++---------------------------------------
src/pmdas/papi/pmdapapi.1 |    7 
src/pmdas/papi/pmns       |   17 -
7 files changed, 135 insertions(+), 517 deletions(-)

commit f81da17d7ee108072510e20a89778d2c725aab03
Author: Frank Ch. Eigler 
Date:   Wed Nov 19 12:50:57 2014 -0500

    papi pmda: tweak metric units
    
    Some of the papi.* metric units can be more precise:
    
    papi.system.* counters      to signed 64-bit and Units: count
    papi.available.num_counters to Units: count
    papi.control.auto_enable    left as is
    papi.control.enable etc.    to Units: none
    
    Conflicts:
        src/pmdas/papi/papi.c

commit b4f511d5ada7edf99167cddcf28c9e2df3aa7c86
Author: Lukas Berk 
Date:   Wed Nov 19 14:52:11 2014 -0500

    Update multiplexing metric to papi.control.multiplex
    
    Shorten the metric name, update docs, pmns, qa accordingly

commit 3a5abd38cbaab6ee412818097c2d8f572707c60c
Author: Lukas Berk 
Date:   Wed Nov 19 14:06:14 2014 -0500

    Add papi.control.enable_multiplexing metric to allow value based enablement
    
    qa/914.out - update current testcase
    src/pmdas/papi/help - add help text
    src/pmdas/papi/papi.c - add functionality and new metric
    src/pmdas/papi/pmdapapi.1 - update documentation
    src/pmdas/papi/pmns - update papi.control namespace and add metric

commit 826762970250e21556635e29bccb9ed252f198ce
Author: Lukas Berk 
Date:   Wed Nov 19 09:59:12 2014 -0500

    Remove unused gid references
    
    We only run checks for uid's so remove the gid fields and update
    the variable names accordingly

commit 7e9b87e49b18969c9a05d221f55155c8d78bce8b
Author: Lukas Berk 
Date:   Tue Nov 18 20:31:07 2014 -0500

    Dynamically populate pmdapapi metric table based on available metrics
    
    We should fill in the 'metrictab' variable dynamically, based on the
    metrics we observe during initialization.  This allows us to avoid
    having a giant table of all possible papi metrics.

From nscott@redhat.com  Wed Nov 19 17:49:08 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 11BCF7F4E
	for ; Wed, 19 Nov 2014 17:49:08 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id F077F304032
	for ; Wed, 19 Nov 2014 15:49:07 -0800 (PST)
X-ASG-Debug-ID: 1416440944-04cbb01e5c177cc0001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id wQ20Wl5aHV6WzIuv (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 15:49:05 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJNn4r0027978;
	Wed, 19 Nov 2014 18:49:04 -0500
Date: Wed, 19 Nov 2014 18:49:04 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Jan-Frode Myklebust 
Cc: pcp@oss.sgi.com
Message-ID: <1584858682.2368740.1416440944540.JavaMail.zimbra@redhat.com>
In-Reply-To: <20141118235954.GA2225@mushkin.tanso.net>
References: <20141116200958.GA8464@mushkin.tanso.net> <69304097.15487953.1416205767949.JavaMail.zimbra@redhat.com> <20141118235954.GA2225@mushkin.tanso.net>
Subject: Re: [pcp] My first PMDA, some questions..
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] My first PMDA, some questions..
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: My first PMDA, some questions..
Thread-Index: 8bP7ZaA8hSBrt2ZwlmuPlIYq5ws6gQ==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1416440945
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.62
X-Barracuda-Spam-Status: No, SCORE=0.62 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=COMMA_SUBJECT, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11788
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.60 COMMA_SUBJECT          Subject is like 'Re: FDSDS, this is a subject'

Hi Jan-Frode,

----- Original Message -----
> On Mon, Nov 17, 2014 at 01:29:27AM -0500, Nathan Scott wrote:
> > 
> > At what point do you know how many threads you have?
> 
> After "unbound" is started, and I guess it might change if someone
> changes the unbound configuration. But should in general be pretty
> static..
> 
> 
> > If you can find out
> > before the call to PMDA.run(), then you can just add them like any other
> > metric (with the first parameter to add_metric() being "thread%d.xxx".  If
> > you can only find out after the call to run(), then the current python API
> > will not work for you.
> 
> Ok, hmm... not sure which option I'll go for.
> 

I'd recommend choosing instances, looks like they'd suit well here
from what I've seen so far and they're simpler to work with.

> > 
> > The client (monitor) tools drive the fetch interval, PMDAs just respond
> > with
> > the latest values - they typically have no knowledge of the sampling rate.
> > There can also be multiple clients, sampling at different (unrelated)
> > times.
> 
> So two people running "pmval metric" will by default trigger 2 fetches
> per second??

Yes.  For some PMDAs thats a walk in the park (we have folks sampling
the kernel, mmv, proc, etc PMDAs many times per second, just from one
client).  These rates can be readily achieved with no measurable load,
for those PMDAs.

> Or will they be smart enough to share one? 1 fetch per
> second should be OK, but much more is getting scary..

OK - its up to the PMDA to cap this if it feels it's appropriate for
the domain its extracting data from.

> > the Programmers Guide (http://www.pcp.io/doc/pcp-programmers-guide.pdf)
> > has a section "2.2.4 - Caching PMDA" with a more details.
> > [...]
> 
> BTW: Here's my first shot -> https://github.com/janfrode/unbound-pmda
> based on the gluster pmda.
> 

Good stuff.  Is this a PMDA you would like to include in PCP, or do you
want to maintain/release it separately? (its your choice of course)

One thing I noticed is this PMDA would greatly benefit from a python
version of pmda_pmid_name() from the perl API.  This converts a PMID
(well - just the cluser,item pair) into a metric name using the data
structures that the language binding maintains.

So In your fetch callback, you'd call that first and it gives back the
string metric name.  This'd remove the need for that long if/elseif/
elseif/..etc. you have there.

But, noone has implemented a python version of that API yet - would you
be interested in hacking on that?  (see the data structures maintained
by PMDA.add_metric() in src/python/pcp/pmda.py, if so)

cheers.

--
Nathan

From lberk@redhat.com  Wed Nov 19 17:52:02 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 7C3447F51
	for ; Wed, 19 Nov 2014 17:52:02 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 4D5EC8F804B
	for ; Wed, 19 Nov 2014 15:51:59 -0800 (PST)
X-ASG-Debug-ID: 1416441117-04bdf0616116cfb0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id H5xmsbEvNXCiEtwA (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 15:51:58 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAJNpvjm004734
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 19 Nov 2014 18:51:57 -0500
Received: from toium ([10.15.16.195])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAJNpuQr019964
	(version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO);
	Wed, 19 Nov 2014 18:51:57 -0500
From: Lukas Berk 
To: Nathan Scott 
Cc: pcp@oss.sgi.com
Subject: Re: [pcp] pcp updates - pmdapapi update
References: <87oascow3f.fsf@redhat.com>
	<50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] pcp updates - pmdapapi update
Date: Wed, 19 Nov 2014 18:51:56 -0500
In-Reply-To: <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com>
	(Nathan Scott's message of "Fri, 14 Nov 2014 03:53:05 -0500 (EST)")
Message-ID: <877fyqsq8z.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416441118
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi,

Nathan Scott  writes:
[...]
> This is looking pretty good - I've updated some minor stuff (please do
> a review) but otherwise its all merged now.  There's a few things that
> would be good to still get in that I didn't tackle...

Thanks for merging this. I've worked on the comments below.

> - I'm not following why the (static) metrictab has all those entries
>   for event counters still?  Can those be removed now?

Yes it can, fixed on lberk/papi.

> - The permissions_check() changed to only look for uid==0 - should we
>   drop all references to gid* elsewhere?  I think so (slims down some
>   data structures, simplifies the attributes callback, etc)

I've done this as well on lberk/papi.

> - As per earlier mail, there's no error handling for auto-fu counters.
>   Simplest will be, I think, to give refresh_metric() a parameter that
>   indicates whether its caller is going to ignore the return code, and
>   in that case, handle_papi_error() should really log any errors into
>   the pmdapapi.log file.  In the pmStore-enable/disable-metrics case,
>   these need not be logged (the client tool will get notified about an
>   error in that case, which is better - keep that pmDebug diagnostic
>   for both cases though).

I believe the patch that Frank proposed for this has been merged
upstream.  The rest of the issues have been addressed in my latest pcp
updates mail[1].  

Thanks for reviewing and merging my changes!

Cheers,

Lukas

[1] - http://www.pcp.io/pipermail/pcp/2014-November/006012.html

From nscott@redhat.com  Wed Nov 19 21:02:51 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id D76FE29DF7
	for ; Wed, 19 Nov 2014 21:02:51 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id AA0FA304048
	for ; Wed, 19 Nov 2014 19:02:48 -0800 (PST)
X-ASG-Debug-ID: 1416452563-04bdf0615e171690001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id YUgfOIiNppIFjnYz (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 19:02:44 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAK32h6F018450;
	Wed, 19 Nov 2014 22:02:43 -0500
Date: Wed, 19 Nov 2014 22:02:43 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Lukas Berk 
Cc: pcp@oss.sgi.com
Message-ID: <1707444793.2413172.1416452563526.JavaMail.zimbra@redhat.com>
In-Reply-To: <877fyqsq8z.fsf@redhat.com>
References: <87oascow3f.fsf@redhat.com> <50063157.14443613.1415955185726.JavaMail.zimbra@redhat.com> <877fyqsq8z.fsf@redhat.com>
Subject: Re: [pcp] pcp updates - pmdapapi update
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] pcp updates - pmdapapi update
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates - pmdapapi update
Thread-Index: mSm2OR4xpUO2ZMX0Q5DPWskZTwfuAg==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1416452564
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11797
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

Hi Lukas,

----- Original Message -----
> Nathan Scott  writes:
> [...]
> > This is looking pretty good - I've updated some minor stuff (please do
> > a review) but otherwise its all merged now.  There's a few things that
> > would be good to still get in that I didn't tackle...
> 
> Thanks for merging this. I've worked on the comments below.

Nice.  I had a conflict merging, looks like the overlapping changes
in dev had not yet been merged back into your branch, or something
like that - please check my merge extra carefully in case I missed
any changes?  Thanks!

> > - As per earlier mail, there's no error handling for auto-fu counters.
> >   Simplest will be, I think, to give refresh_metric() a parameter that
> >   indicates whether its caller is going to ignore the return code, and
> >   in that case, handle_papi_error() should really log any errors into
> >   the pmdapapi.log file.  In the pmStore-enable/disable-metrics case,
> >   these need not be logged (the client tool will get notified about an
> >   error in that case, which is better - keep that pmDebug diagnostic
> >   for both cases though).
> 
> I believe the patch that Frank proposed for this has been merged

The first part of the problem was tackled, but the second part remains
(handle_papi_error() needs to log to papi.log - __pmNotiyErr(LOG_ERR,..
- during async operations).

> upstream.  The rest of the issues have been addressed in my latest pcp
> updates mail[1].
> 
> Thanks for reviewing and merging my changes!

No problem at all, Lukas - thanks for taking the time to test everything
too - all the tests are passing here and there's new additions, so I'm a
happy camper.

cheers.

--
Nathan

From nscott@redhat.com  Wed Nov 19 21:03:51 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 76E7129DF7
	for ; Wed, 19 Nov 2014 21:03:51 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 660B48F8035
	for ; Wed, 19 Nov 2014 19:03:48 -0800 (PST)
X-ASG-Debug-ID: 1416452625-04cbb01e5c17bd00001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id FSF1Wo1OOrIoiTT1 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 19:03:46 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAK33jwn008641
	for ; Wed, 19 Nov 2014 22:03:45 -0500
Date: Wed, 19 Nov 2014 22:03:45 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <178707012.2414053.1416452625662.JavaMail.zimbra@redhat.com>
Subject: pcp updates: papi, proc pmdas
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: papi, proc pmdas
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: papi, proc pmdas
Thread-Index: 706lgw+mlXpXps20bNSj0ur8++KglQ==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1416452626
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11797
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Lukas Berk (4):
      Dynamically populate pmdapapi metric table based on available metrics
      Remove unused gid references
      Add papi.control.enable_multiplexing metric to allow value based enablement
      Update multiplexing metric to papi.control.multiplex

Nathan Scott (2):
      qa: update 361 to handle revisions to cgroups metrics
      qa: resolve uninitd memory use in cgroups memory metrics

Frank Ch. Eigler (1):
      papi pmda: tweak metric units

 qa/361                         |   32 +-
 qa/361.out                     |  101 +-----
 qa/730.out                     |  116 +++----
 qa/731.out                     |   40 +-
 qa/813.out                     |    6 
 qa/914.out                     |   10 
 qa/967.out                     |   12 
 src/pmdas/linux_proc/cgroups.c |    7 
 src/pmdas/papi/help            |    4 
 src/pmdas/papi/papi.c          |  596 ++++++-----------------------------------
 src/pmdas/papi/pmdapapi.1      |    7 
 src/pmdas/papi/pmns            |   17 -
 12 files changed, 262 insertions(+), 686 deletions(-)

From nscott@redhat.com  Thu Nov 20 01:33:32 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 65D327F3F
	for ; Thu, 20 Nov 2014 01:33:32 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 45DB58F8033
	for ; Wed, 19 Nov 2014 23:33:29 -0800 (PST)
X-ASG-Debug-ID: 1416468804-04bdf0615e177260001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id 7GFY21xdtxexzsMS (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 19 Nov 2014 23:33:24 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAK7XOI5014876
	for ; Thu, 20 Nov 2014 02:33:24 -0500
Date: Thu, 20 Nov 2014 02:33:24 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <658457925.2467853.1416468804182.JavaMail.zimbra@redhat.com>
Subject: pcp updates: cpuinfo
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: cpuinfo
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: cpuinfo
Thread-Index: 2DzqjZuKRR7cIf4qARdb/hi0LDeEZg==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1416468804
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11804
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Nathan Scott (2):
      qa: exercise the /proc/cpuinfo parsing code
      pmdalinux: fix aarch64 case in the /proc/cpuinfo parsing code

 qa/885                                     |   59 +
 qa/885.out                                 |  977 +++++++++++++++++++++++++++++
 qa/group                                   |    1 
 qa/linux/cpuinfo-1cpu-alpha                |   19 
 qa/linux/cpuinfo-1cpu-g3ibook              |   11 
 qa/linux/cpuinfo-1cpu-powermac             |    9 
 qa/linux/cpuinfo-1cpu-ppc-cPCI405          |    8 
 qa/linux/cpuinfo-1cpu-ppc-pcippc2          |   13 
 qa/linux/cpuinfo-2cpu-umax-s900dp          |   20 
 qa/linux/cpuinfo-32-4830                   |  800 -----------------------
 qa/linux/cpuinfo-32cpu-4830                |  800 +++++++++++++++++++++++
 qa/linux/cpuinfo-32cpu-amd-6132            |  832 ++++++++++++++++++++++++
 qa/linux/cpuinfo-4cpu-alpha                |   19 
 qa/linux/cpuinfo-4cpu-ia64-linux-2.6.32    |   76 ++
 qa/linux/cpuinfo-8cpu-aarch64-linux-3.17.0 |   17 
 qa/linux/cpuinfo-aarch64-linux-3.17.0      |   17 
 qa/linux/cpuinfo-amd-32-6132               |  832 ------------------------
 qa/linux/cpuinfo-ia64-linux-2.6.32         |   76 --
 src/pmdas/linux/proc_cpuinfo.c             |   44 +
 19 files changed, 2901 insertions(+), 1729 deletions(-)

From brolley@redhat.com  Thu Nov 20 09:08:07 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id E8B047F47
	for ; Thu, 20 Nov 2014 09:08:06 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 7454EAC001
	for ; Thu, 20 Nov 2014 07:08:03 -0800 (PST)
X-ASG-Debug-ID: 1416496078-04cbb01e5918f100001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id VbZ9WlCa2ggYH7Wv (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 20 Nov 2014 07:07:59 -0800 (PST)
X-Barracuda-Envelope-From: brolley@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAKF7v1n019376
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Thu, 20 Nov 2014 10:07:57 -0500
Received: from [10.10.54.105] (vpn-54-105.rdu2.redhat.com [10.10.54.105])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAKF7uiu010161
	for ; Thu, 20 Nov 2014 10:07:56 -0500
Message-ID: <546E0461.2080206@redhat.com>
Date: Thu, 20 Nov 2014 10:10:25 -0500
From: Dave Brolley 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: Re: [pcp] [pcp-announce] Maintainer role addition for PCP
References: <546BCDC9.1040301@redhat.com> <1031495503.1600999.1416375303881.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [pcp-announce] Maintainer role addition for PCP
In-Reply-To: <1031495503.1600999.1416375303881.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416496078
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/19/2014 12:35 AM, Nathan Scott wrote:
> Hi all,
>
> As you may have noticed (mail below) Dave Brolley has
> kindly offered to take on more code review, merge and
> release responsibilities.  Dave and I will be doing a
> pcp-3.10.1 release together (scheduled for 1 Dec) and
> thereafter we're planning to rotate.
>
> Dave has a deep knowledge of PCP protocol low-levels,
> having done all the IPv6 support, secure sockets, and
> lots of other PCP work.  He also has a great history
> of writing tests, running tests, fixing tests and, of
> late, even fixing other peoples tests.  (Zen moment!)
>
> He's done a fantastic job at working towards mutually
> agreeable solutions whenever consensus is lacking.  So
> I hope others will enjoy working with him as much as I
> have in getting changes into the tree.
>
Not completely sure that I realize what I've gotten myself into, but 
will do my best to keep up!

Dave

From wwwrun@oss.sgi.com  Thu Nov 20 10:11:55 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id A226B7F4E; Thu, 20 Nov 2014 10:11:55 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1076] New: archive access api across network
Date: Thu, 20 Nov 2014 16:11:55 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416499915.b2c4f1.12986"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416499915.b2c4f1.12986
Date: Thu, 20 Nov 2014 10:11:55 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1076

            Bug ID: 1076
           Summary: archive access api across network
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 10: archive access api across network
  - to let analytics / guis run remotely from archive files
  - already in plans of one flavour of the "grand unified context"
  - pmwebd graphite interface one possible near-term hack for this

See also earlier "grand unified context" threads.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416499915.b2c4f1.12986
Date: Thu, 20 Nov 2014 10:11:55 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1076
          
        


        

          Summary
          archive access api across network
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 10: archive access api across network
  - to let analytics / guis run remotely from archive files
  - already in plans of one flavour of the "grand unified context"
  - pmwebd graphite interface one possible near-term hack for this

See also earlier "grand unified context" threads.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416499915.b2c4f1.12986--

From wwwrun@oss.sgi.com  Thu Nov 20 10:13:35 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id D991F7F4E; Thu, 20 Nov 2014 10:13:35 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1077] New: tacc_stats data importer
Date: Thu, 20 Nov 2014 16:13:35 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416500015.80FE1.13133"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416500015.80FE1.13133
Date: Thu, 20 Nov 2014 10:13:35 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1077

            Bug ID: 1077
           Summary: tacc_stats data importer
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item #11

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416500015.80FE1.13133
Date: Thu, 20 Nov 2014 10:13:35 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1077
          
        


        

          Summary
          tacc_stats data importer
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item #11

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416500015.80FE1.13133--

From wwwrun@oss.sgi.com  Thu Nov 20 10:28:44 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id CBD9A7F4E; Thu, 20 Nov 2014 10:28:44 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1078] New: ganglia=rrd importer
Date: Thu, 20 Nov 2014 16:28:44 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416500924.01ef1.14014"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416500924.01ef1.14014
Date: Thu, 20 Nov 2014 10:28:44 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1078

            Bug ID: 1078
           Summary: ganglia=rrd importer
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item #11
(see also bug #1077).

(Interestingly, ganglia's metrics/ library could perhaps use a pcp client
alternate implementation.)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416500924.01ef1.14014
Date: Thu, 20 Nov 2014 10:28:44 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1078
          
        


        

          Summary
          ganglia=rrd importer
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item #11
(see also bug #1077).

(Interestingly, ganglia's metrics/ library could perhaps use a pcp client
alternate implementation.)

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416500924.01ef1.14014--

From wwwrun@oss.sgi.com  Thu Nov 20 10:31:21 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 95C8C7F4E; Thu, 20 Nov 2014 10:31:21 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1079] New: pmda fetch timeout toleration
Date: Thu, 20 Nov 2014 16:31:21 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501081.e8eD5481.14266"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501081.e8eD5481.14266
Date: Thu, 20 Nov 2014 10:31:21 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1079

            Bug ID: 1079
           Summary: pmda fetch timeout toleration
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 13: proc pmda timeouts
  - well known problem, hits ccr (and other larger servers) good and hard
  - http://oss.sgi.com/bugzilla/show_bug.cgi?id=1036
  - kenj had drafted work for pmda timeout-detecting thread for fetches
    - still need this!
  - pmie-based auto-pmcd-restarting possible too, though one must run with
root/sudo
    - and restarting doesn't guarantee persistency of pmids etc. 
  - elevated-privileged split-pmcd a possible solution too (it could restart
pmda
    processes)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501081.e8eD5481.14266
Date: Thu, 20 Nov 2014 10:31:21 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1079
          
        


        

          Summary
          pmda fetch timeout toleration
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 13: proc pmda timeouts
  - well known problem, hits ccr (and other larger servers) good and hard
  - http://oss.sgi.com/bugzilla/show_bug.cgi?id=1036
  - kenj had drafted work for pmda timeout-detecting thread for fetches
    - still need this!
  - pmie-based auto-pmcd-restarting possible too, though one must run with
root/sudo
    - and restarting doesn't guarantee persistency of pmids etc. 
  - elevated-privileged split-pmcd a possible solution too (it could restart
pmda
    processes)

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501081.e8eD5481.14266--

From wwwrun@oss.sgi.com  Thu Nov 20 10:34:30 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 2882F7F4E; Thu, 20 Nov 2014 10:34:30 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1080] New: pmcd result timestamping
Date: Thu, 20 Nov 2014 16:34:30 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501270.4b4a4c61.14502"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501270.4b4a4c61.14502
Date: Thu, 20 Nov 2014 10:34:30 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1080

            Bug ID: 1080
           Summary: pmcd result timestamping
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 14: pmfetch timestamps
  - especially for slightly tardy pmdas, the pmResult timestamp is
    ambiguous: is it at beginning, end, or middle of process?
  - having too-short inter-fetch timestamp intervals can lead to
    crazy-big rate-converted values
  - maybe just confirm/fix pmcd to return beginning-of-operation timestamps
  - apps can also confirm that timestamp is reasonable, by comparing to
    their own clocks around pmFetch()
  - pmlogger could emit diagnostics for violations

Note src/pmcd/src/dofetch.c:530, __pmtimevalNow() is called -after- all
the subsidiary agent fetches are complete.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501270.4b4a4c61.14502
Date: Thu, 20 Nov 2014 10:34:30 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1080
          
        


        

          Summary
          pmcd result timestamping
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 14: pmfetch timestamps
  - especially for slightly tardy pmdas, the pmResult timestamp is
    ambiguous: is it at beginning, end, or middle of process?
  - having too-short inter-fetch timestamp intervals can lead to
    crazy-big rate-converted values
  - maybe just confirm/fix pmcd to return beginning-of-operation timestamps
  - apps can also confirm that timestamp is reasonable, by comparing to
    their own clocks around pmFetch()
  - pmlogger could emit diagnostics for violations

Note src/pmcd/src/dofetch.c:530, __pmtimevalNow() is called -after- all
the subsidiary agent fetches are complete.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501270.4b4a4c61.14502--

From wwwrun@oss.sgi.com  Thu Nov 20 10:38:00 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id AE8777F4E; Thu, 20 Nov 2014 10:38:00 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1081] New: network peer traffic stats
Date: Thu, 20 Nov 2014 16:38:00 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501480.70dd8881.14754"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501480.70dd8881.14754
Date: Thu, 20 Nov 2014 10:38:00 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1081

            Bug ID: 1081
           Summary: network peer traffic stats
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 15: per-peer stats
  - sometimes need to be able to measure inter-node traffic stats
  - not just on network interface basis, but per-peer 
    e.g., network.traffic.ipv4.perpeer.in.packets ["addr"]
  - also came up in comparisons to ntop
  - this would could generate large indoms
  - potentially generate from userspace (a la tcpdump)
  - or perhaps systemtap (via json / mmv interfaces)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501480.70dd8881.14754
Date: Thu, 20 Nov 2014 10:38:00 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1081
          
        


        

          Summary
          network peer traffic stats
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 15: per-peer stats
  - sometimes need to be able to measure inter-node traffic stats
  - not just on network interface basis, but per-peer 
    e.g., network.traffic.ipv4.perpeer.in.packets ["addr"]
  - also came up in comparisons to ntop
  - this would could generate large indoms
  - potentially generate from userspace (a la tcpdump)
  - or perhaps systemtap (via json / mmv interfaces)

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501480.70dd8881.14754--

From wwwrun@oss.sgi.com  Thu Nov 20 10:39:47 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id E3DF07F51; Thu, 20 Nov 2014 10:39:47 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1082] New: per-proc per-syscall stats
Date: Thu, 20 Nov 2014 16:39:47 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501587.d2A6C1.14890"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501587.d2A6C1.14890
Date: Thu, 20 Nov 2014 10:39:47 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1082

            Bug ID: 1082
           Summary: per-proc per-syscall stats
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 16: syscall stats
  - sometimes need to be able to measure per-process per-syscall stats
    e.g., proc.syscall.FOO ["pid"]   or proc.syscall ["pid:FOO"]
  - naturally matrix-valued
  - probably gather via systemtap or similar

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501587.d2A6C1.14890
Date: Thu, 20 Nov 2014 10:39:47 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1082
          
        


        

          Summary
          per-proc per-syscall stats
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 16: syscall stats
  - sometimes need to be able to measure per-process per-syscall stats
    e.g., proc.syscall.FOO ["pid"]   or proc.syscall ["pid:FOO"]
  - naturally matrix-valued
  - probably gather via systemtap or similar

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501587.d2A6C1.14890--

From wwwrun@oss.sgi.com  Thu Nov 20 10:42:04 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 138E97F51; Thu, 20 Nov 2014 10:42:04 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1083] New: gpfs pmda
Date: Thu, 20 Nov 2014 16:42:03 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501724.0b181.15072"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501724.0b181.15072
Date: Thu, 20 Nov 2014 10:42:04 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1083

            Bug ID: 1083
           Summary: gpfs pmda
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 17: gpfs pmda
  - need it

(I believe this referred to the proprietary
https://en.wikipedia.org/wiki/IBM_General_Parallel_File_System .)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501724.0b181.15072
Date: Thu, 20 Nov 2014 10:42:04 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1083
          
        


        

          Summary
          gpfs pmda
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 17: gpfs pmda
  - need it

(I believe this referred to the proprietary
https://en.wikipedia.org/wiki/IBM_General_Parallel_File_System .)

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501724.0b181.15072--

From wwwrun@oss.sgi.com  Thu Nov 20 10:42:42 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 4D5B37F50; Thu, 20 Nov 2014 10:42:42 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1084] New: simplfied fetch-oriented pmapi
Date: Thu, 20 Nov 2014 16:42:42 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: fche@redhat.com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501762.bf8c1840.15192"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501762.bf8c1840.15192
Date: Thu, 20 Nov 2014 10:42:42 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1084

            Bug ID: 1084
           Summary: simplfied fetch-oriented pmapi
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: fche@redhat.com
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 18: simplified pmapi
  - fche already prototyping in C
  - python pmcc also useful for python clients

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501762.bf8c1840.15192
Date: Thu, 20 Nov 2014 10:42:42 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1084
          
        


        

          Summary
          simplfied fetch-oriented pmapi
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          fche@redhat.com
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 18: simplified pmapi
  - fche already prototyping in C
  - python pmcc also useful for python clients

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501762.bf8c1840.15192--

From wwwrun@oss.sgi.com  Thu Nov 20 10:43:44 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 503407F54; Thu, 20 Nov 2014 10:43:44 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1085] New: more analytics
Date: Thu, 20 Nov 2014 16:43:42 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status bug_severity priority component assigned_to reporter cc
 classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416501824.5D4111.15260"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416501824.5D4111.15260
Date: Thu, 20 Nov 2014 10:43:44 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1085

            Bug ID: 1085
           Summary: more analytics
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: fche@redhat.com
                CC: pcp@oss.sgi.com
    Classification: Unclassified

buffalo item

- pcp todo item 5: more analysis capability
  - if pcp had more native analytics (well in excess of pmdiff), the
    data warehouse aggregations could become secondary and enable
    drilldown to raw but analyzed data
  - example types of calculations: cross-correlations between metrics
  - anomaly detection (big area)
    - fche's hobbyhorse: connecting this to pmlogger to govern archiving

  - also: export to external stats like numpy

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416501824.5D4111.15260
Date: Thu, 20 Nov 2014 10:43:44 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1085
          
        


        

          Summary
          more analytics
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          fche@redhat.com
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
buffalo item

- pcp todo item 5: more analysis capability
  - if pcp had more native analytics (well in excess of pmdiff), the
    data warehouse aggregations could become secondary and enable
    drilldown to raw but analyzed data
  - example types of calculations: cross-correlations between metrics
  - anomaly detection (big area)
    - fche's hobbyhorse: connecting this to pmlogger to govern archiving

  - also: export to external stats like numpy

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416501824.5D4111.15260--

From wwwrun@oss.sgi.com  Thu Nov 20 13:29:08 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id DDCC17F51; Thu, 20 Nov 2014 13:29:08 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1074] 3d chart view: opengl pmview and/or pmwebd/webgl webapp
Date: Thu, 20 Nov 2014 19:29:08 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: kenj@internode.on.net
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: cc
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416511748.1FeE3.25214"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416511748.1FeE3.25214
Date: Thu, 20 Nov 2014 13:29:08 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1074

Ken McDonell  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kenj@internode.on.net

--- Comment #1 from Ken McDonell  ---
The partial implementation of pmview is actually quite close to working.
It is in the source tree (src/pmview).
All of the parser, scene graph construction, scene visualization parts appear
to be working.
There is some issue with the connection to the time controller that is
currently blocking any pmFetch from happening, so the scene is not modulated.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416511748.1FeE3.25214
Date: Thu, 20 Nov 2014 13:29:08 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 1074
        

             

          

            What
            Removed
            Added
          


         

           CC
           
                
           
           kenj@internode.on.net
           
         

      

        

            Comment # 1
              on bug 1074
              from 

        
The partial implementation of pmview is actually quite close to working.
It is in the source tree (src/pmview).
All of the parser, scene graph construction, scene visualization parts appear
to be working.
There is some issue with the connection to the time controller that is
currently blocking any pmFetch from happening, so the scene is not modulated.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416511748.1FeE3.25214--

From janfrode@tanso.net  Thu Nov 20 15:55:26 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id DD0DC7F3F
	for ; Thu, 20 Nov 2014 15:55:25 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 5D97BAC001
	for ; Thu, 20 Nov 2014 13:55:22 -0800 (PST)
X-ASG-Debug-ID: 1416520516-04cbb01e5a19d4a0001-S8gJnT
Received: from mail-lb0-f178.google.com (mail-lb0-f178.google.com [209.85.217.178]) by cuda.sgi.com with ESMTP id WuBCSdRAkXiY1jNA (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 20 Nov 2014 13:55:17 -0800 (PST)
X-Barracuda-Envelope-From: janfrode@tanso.net
X-Barracuda-Apparent-Source-IP: 209.85.217.178
Received: by mail-lb0-f178.google.com with SMTP id f15so1719948lbj.9
        for ; Thu, 20 Nov 2014 13:55:15 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:date:from:to:cc:subject:message-id:references
         :mime-version:content-type:content-disposition:in-reply-to
         :user-agent;
        bh=kfdpfOlv3hDX2rTOLDXOYw29wj8y0vcpbrchb2m7bjk=;
        b=mqm/beEG6R/NUdmHZmdO/Z9c/wJvTHeiJiYLb9i1XnD2d8InHvNmVx/oJLQ5bi1xPt
         VjGX3okb8bzlBLkvCx3JruEEyTwK9QdduyCRrGQORNIwtnDXkxDJ4curLrGTCkhsuF9Z
         rndLJdT6R8jD47XlpElSBeeDSrnRHcVR9g0hSTK1CKFBn2G4JdUgQ5JBqlXSYqhvN5w8
         ozXHTJy8TtWRtUJ3vv1313+bemFOKTau3HTUfdGqw1cxfUtYn3fAXxSBn/tsNPxRWJOn
         44qcDRe09Q7OF5x0JbaIMCZFKjJoBJDKj46vDoUjkmJwdQu/4JQOz0e+Cjjk1ilfmO0a
         8fsg==
X-Gm-Message-State: ALoCoQmppfqA5MhM4s/8Ln4ZseoFUerTatzeNzHdjDIRwHsgq692NMOwah1yRV5i93PkSfCdQoEi
X-Received: by 10.112.72.225 with SMTP id g1mr484143lbv.75.1416520515683;
        Thu, 20 Nov 2014 13:55:15 -0800 (PST)
Received: from localhost (120.81-167-109.customer.lyse.net. [81.167.109.120])
        by mx.google.com with ESMTPSA id r4sm787179lah.23.2014.11.20.13.55.14
        for 
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 20 Nov 2014 13:55:15 -0800 (PST)
Date: Thu, 20 Nov 2014 22:55:14 +0100
From: Jan-Frode Myklebust 
To: Nathan Scott 
Cc: pcp@oss.sgi.com
Subject: Re: [pcp] My first PMDA, some questions..
Message-ID: <20141120215514.GA18343@mushkin.tanso.net>
X-ASG-Orig-Subj: Re: [pcp] My first PMDA, some questions..
References: <20141116200958.GA8464@mushkin.tanso.net>
 <69304097.15487953.1416205767949.JavaMail.zimbra@redhat.com>
 <20141118235954.GA2225@mushkin.tanso.net>
 <1584858682.2368740.1416440944540.JavaMail.zimbra@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <1584858682.2368740.1416440944540.JavaMail.zimbra@redhat.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-Barracuda-Connect: mail-lb0-f178.google.com[209.85.217.178]
X-Barracuda-Start-Time: 1416520516
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.60
X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=COMMA_SUBJECT
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11833
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.60 COMMA_SUBJECT          Subject is like 'Re: FDSDS, this is a subject'

On Wed, Nov 19, 2014 at 06:49:04PM -0500, Nathan Scott wrote:
> 
> Good stuff.  Is this a PMDA you would like to include in PCP, or do you
> want to maintain/release it separately? (its your choice of course)

Would be very cool to get it included with PCP. I'll submit it for
inclusion once I feel it's closer to finished.

> 
> One thing I noticed is this PMDA would greatly benefit from a python
> version of pmda_pmid_name() from the perl API.  This converts a PMID
> (well - just the cluser,item pair) into a metric name using the data
> structures that the language binding maintains.

Right, that would save me some lines :-)

> 
> But, noone has implemented a python version of that API yet - would you
> be interested in hacking on that?  (see the data structures maintained
> by PMDA.add_metric() in src/python/pcp/pmda.py, if so)

Sure, I'll look into it.



  -jf

From wwwrun@oss.sgi.com  Thu Nov 20 16:15:09 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id CD7CB7F50; Thu, 20 Nov 2014 16:15:09 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1083] gpfs pmda
Date: Thu, 20 Nov 2014 22:15:09 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: janfrode@tanso.net
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: cc
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416521709.f7c12.6977"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416521709.f7c12.6977
Date: Thu, 20 Nov 2014 16:15:09 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1083

Jan-Frode Myklebust  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |janfrode@tanso.net

--- Comment #1 from Jan-Frode Myklebust  ---
Do you have any ideas for which metrics should be included in this PMDA?

There's quite a lot available via SNMP,
https://github.com/mcallaway/sdm/blob/master/sdm-mibs/mibs/GPFS-MIB.txt 

Will it be cheating to fetch from SNMP? I guess the same numbers could be
pulled via "mmpmon", but it's probably a good idea to be limiting the
performance collecting to a single node (like they do for SNMP).

Some monitoring links for GPFS:

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20%28GPFS%29/page/Monitoring%20and%20Tools

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416521709.f7c12.6977
Date: Thu, 20 Nov 2014 16:15:09 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 1083
        

             

          

            What
            Removed
            Added
          


         

           CC
           
                
           
           janfrode@tanso.net
           
         

      

        

            Comment # 1
              on bug 1083
              from 

        
Do you have any ideas for which metrics should be included in this PMDA?

There's quite a lot available via SNMP,
https://github.com/mcallaway/sdm/blob/master/sdm-mibs/mibs/GPFS-MIB.txt 

Will it be cheating to fetch from SNMP? I guess the same numbers could be
pulled via "mmpmon", but it's probably a good idea to be limiting the
performance collecting to a single node (like they do for SNMP).

Some monitoring links for GPFS:

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20%28GPFS%29/page/Monitoring%20and%20Tools

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416521709.f7c12.6977--

From kenj@internode.on.net  Thu Nov 20 16:31:40 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 089377F3F
	for ; Thu, 20 Nov 2014 16:31:40 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id DD97A8F804B
	for ; Thu, 20 Nov 2014 14:31:36 -0800 (PST)
X-ASG-Debug-ID: 1416522694-04cbb01e5c19e620001-S8gJnT
Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id 1vQq5IlB30PzfYnu for ; Thu, 20 Nov 2014 14:31:35 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.129
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: An4CAMRqblR20ZQhPGdsb2JhbAANTYtFyQSDHAKBHgEBAQEBBgEBAQE4hD4BAQQ4QBELGAkWDwkDAgECATEUEwgBAcVkl2kBAQgCAR+RDxaENQEEuWuDJAEBAQ
Received: from ppp118-209-148-33.lns20.mel8.internode.on.net (HELO [192.168.1.100]) ([118.209.148.33])
  by ipmail06.adl2.internode.on.net with ESMTP; 21 Nov 2014 09:01:34 +1030
Message-ID: <546E6C3C.9010808@internode.on.net>
Date: Fri, 21 Nov 2014 09:33:32 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: Re: [pcp] fetch/desc conflict defeats dynamic pmns, papi pmda case
 study
References: <20141119171830.GE5700@redhat.com> <546D1339.1000601@internode.on.net>
X-ASG-Orig-Subj: Re: [pcp] fetch/desc conflict defeats dynamic pmns, papi pmda case
 study
In-Reply-To: <546D1339.1000601@internode.on.net>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129]
X-Barracuda-Start-Time: 1416522694
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11834
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

On 20/11/14 09:01, Ken McDonell wrote:
> > ...
>
> You could try the patch below ... the e_ext field of pmdaExt is already a pointer to a private helper structure that holds the pmda_interface (version) number that is needed for similar situations ... extending this to also have a back pointer seems safe for everyone.

I've cleaned this patch up and replaced the pmda_interface field with a 
back pointer to the pmdaInterface struct from where the real (as opposed 
to a copy of the) pmda_interface value can be found.

This breaks nothing (all PMDAs been through QA) and allows Frank or any 
other PMDA builder to get access to the full pmdaInterface struct from 
within any of the callbacks.

Commit comming soon ...

From kenj@internode.on.net  Thu Nov 20 16:53:57 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 2D3657F3F
	for ; Thu, 20 Nov 2014 16:53:57 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 0DF8A30404E
	for ; Thu, 20 Nov 2014 14:53:53 -0800 (PST)
X-ASG-Debug-ID: 1416524030-04cb6c05721aae70001-S8gJnT
Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id ZOENethF7WaFE5fA for ; Thu, 20 Nov 2014 14:53:51 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.129
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: ApECAJhwblR20ZQhPGdsb2JhbAANTYNjWYMGhAPEU4hqAQEBAQEGAQEBATiEZxVAMAYCBRYLAgsDAgECATEOGQYCAQGISr0KeJcZgS2PeIJhgVQFlzWiNlmCSwEBAQ
Received: from ppp118-209-148-33.lns20.mel8.internode.on.net (HELO [192.168.1.100]) ([118.209.148.33])
  by ipmail06.adl2.internode.on.net with ESMTP; 21 Nov 2014 09:23:20 +1030
Message-ID: <546E7156.7080203@internode.on.net>
Date: Fri, 21 Nov 2014 09:55:18 +1100
From: Ken McDonell 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com
Subject: pcp updates
Content-Type: text/plain; charset=utf-8; format=flowed
X-ASG-Orig-Subj: pcp updates
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129]
X-Barracuda-Start-Time: 1416524030
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11834
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

Changes committed to git://git.performancecopilot.org/kenj/pcp.git dev

  qa/admin/check-vm              |   21 +++++++++++++++------
  src/libpcp_pmda/src/callback.c |   14 +++++++-------
  src/libpcp_pmda/src/libdefs.h  |    8 ++++----
  src/libpcp_pmda/src/open.c     |    2 +-
  src/pmcd/rc_pmcd               |    2 +-
  5 files changed, 28 insertions(+), 19 deletions(-)

commit f856e2c17103540b87952e396f3a7de9cec9a66a
Author: Ken McDonell 
Date:   Fri Nov 21 09:46:59 2014 +1100

     libpcp_pmda: change e_ext_t to expose pmdaInterface

     The e_ext_t struct is internal to libpcp_pmda, but it is exposed
     via the e_ext field of the pmdaExt struct that is accessible to
     every callback that a PMDA developer might choose to override.

     This change replaces the pmda_interface field of e_ext_t with
     a back pointer to the PMDA's controlling pmdaInterface struct
     (from which pmda_interface can be found, but more importantly a
     while lot more information is available).

     Motivated by Frank's observation that the existing libpcp_pmda
     does not allow a "fetch" callback to be independent of the
     metrictab, which restricts some dynamic metric use cases where
     the pmDesc is not in a metrictab, but is instantiatd by the PMDA's
     "desc" callback.

commit 9f008b28bb176b95cd2e8cb6ebfc847859e20410
Author: Ken McDonell 
Date:   Fri Nov 21 09:46:10 2014 +1100

     qa/admin/check-vm: cleanup -v output formatting

commit 9c4c70749aea09ff53c212df88e3049e60ea2e90
Author: Ken McDonell 
Date:   Tue Nov 18 07:19:19 2014 +1100

     src/pmcd/rc_pmcd: fix mkdir mode

     If the script finds $PCP_TMP_DIR/pmlogger is missing it recreates
     it, but the mode should be 775 not 777.

From wwwrun@oss.sgi.com  Thu Nov 20 17:40:17 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 7484E7F51; Thu, 20 Nov 2014 17:40:17 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1083] gpfs pmda
Date: Thu, 20 Nov 2014 23:40:17 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: cc
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416526817.f7e27d3.24794"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416526817.f7e27d3.24794
Date: Thu, 20 Nov 2014 17:40:17 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1083

Nathan Scott  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nathans@debian.org

--- Comment #2 from Nathan Scott  ---
> Will it be cheating to fetch from SNMP? 

Heh.  Generally I've found it best to reduce reliance on "extra"
daemons/packages
wherever it possible/feasible (its more admin overhead, more things that can go
wrong, sometimes much less efficient, etc) - if its feasible to extract data
more directly I'd recommend going that route.  It'd be interesting to see how
the code backing that MIB extracts the stats (via API? or running
mmpmon/mmfsadm? or...).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416526817.f7e27d3.24794
Date: Thu, 20 Nov 2014 17:40:17 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 1083
        

             

          

            What
            Removed
            Added
          


         

           CC
           
                
           
           nathans@debian.org
           
         

      

        

            Comment # 2
              on bug 1083
              from 

        
> Will it be cheating to fetch from SNMP? 

Heh.  Generally I've found it best to reduce reliance on "extra"
daemons/packages
wherever it possible/feasible (its more admin overhead, more things that can go
wrong, sometimes much less efficient, etc) - if its feasible to extract data
more directly I'd recommend going that route.  It'd be interesting to see how
the code backing that MIB extracts the stats (via API? or running
mmpmon/mmfsadm? or...).

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416526817.f7e27d3.24794--

From nscott@redhat.com  Thu Nov 20 22:52:07 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id E12E57F3F
	for ; Thu, 20 Nov 2014 22:52:07 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id B2FBC8F8052
	for ; Thu, 20 Nov 2014 20:52:04 -0800 (PST)
X-ASG-Debug-ID: 1416545519-04bdf0615e19f5c0001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id AbVWsCO68RVidkrC (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 20 Nov 2014 20:52:00 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAL4pxsG002191;
	Thu, 20 Nov 2014 23:51:59 -0500
Date: Thu, 20 Nov 2014 23:51:59 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Jan-Frode Myklebust 
Cc: pcp@oss.sgi.com
Message-ID: <1409994965.3149466.1416545519258.JavaMail.zimbra@redhat.com>
In-Reply-To: <20141120215514.GA18343@mushkin.tanso.net>
References: <20141116200958.GA8464@mushkin.tanso.net> <69304097.15487953.1416205767949.JavaMail.zimbra@redhat.com> <20141118235954.GA2225@mushkin.tanso.net> <1584858682.2368740.1416440944540.JavaMail.zimbra@redhat.com> <20141120215514.GA18343@mushkin.tanso.net>
Subject: Re: [pcp] My first PMDA, some questions..
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] My first PMDA, some questions..
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: My first PMDA, some questions..
Thread-Index: hCqlZMgl7XzreO0xGRMDZw39Od1Ycg==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1416545519
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.62
X-Barracuda-Spam-Status: No, SCORE=0.62 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=COMMA_SUBJECT, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11845
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.60 COMMA_SUBJECT          Subject is like 'Re: FDSDS, this is a subject'



----- Original Message -----
> On Wed, Nov 19, 2014 at 06:49:04PM -0500, Nathan Scott wrote:
> > 
> > Good stuff.  Is this a PMDA you would like to include in PCP, or do you
> > want to maintain/release it separately? (its your choice of course)
> 
> Would be very cool to get it included with PCP. I'll submit it for
> inclusion once I feel it's closer to finished.

Sounds good - I've reserved domain number 132 for the unbound PMDA in
the dev branch.

> > 
> > One thing I noticed is this PMDA would greatly benefit from a python
> > version of pmda_pmid_name() from the perl API.  This converts a PMID
> > (well - just the cluser,item pair) into a metric name using the data
> > structures that the language binding maintains.
> 
> Right, that would save me some lines :-)

Heh, yeah - looks like about 250 lines or so less.

> > 
> > But, noone has implemented a python version of that API yet - would you
> > be interested in hacking on that?  (see the data structures maintained
> > by PMDA.add_metric() in src/python/pcp/pmda.py, if so)
> 
> Sure, I'll look into it.

Fabulous, thanks!

--
Nathan

From nscott@redhat.com  Thu Nov 20 22:54:14 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 0E43E7F3F
	for ; Thu, 20 Nov 2014 22:54:14 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id E307D30405F
	for ; Thu, 20 Nov 2014 20:54:10 -0800 (PST)
X-ASG-Debug-ID: 1416545648-04bdf0615e19f680001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id QxUlwj0bWPKbwad9 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 20 Nov 2014 20:54:09 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAL4s8T9012302
	for ; Thu, 20 Nov 2014 23:54:08 -0500
Date: Thu, 20 Nov 2014 23:54:08 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <811259808.3149628.1416545648556.JavaMail.zimbra@redhat.com>
Subject: pcp updates: kenj merge, build
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: kenj merge, build
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.12]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: kenj merge, build
Thread-Index: u+AEkhy1Ywdw/4dob805FltyWRt6sA==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1416545649
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11845
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Ken McDonell (3):
      src/pmcd/rc_pmcd: fix mkdir mode
      qa/admin/check-vm: cleanup -v output formatting
      libpcp_pmda: change e_ext_t to expose pmdaInterface

Nathan Scott (3):
      packaging: add new/old/new pmie config directory into rpm spec
      packaging: move to revised pmie/pmlogger permissions schemes in rpm spec
      pmns: reserve a domain number for the unbound PMDA

 build/rpm/fedora.spec          |   15 +++++++++------
 qa/admin/check-vm              |   21 +++++++++++++++------
 src/libpcp_pmda/src/callback.c |   14 +++++++-------
 src/libpcp_pmda/src/libdefs.h  |    8 ++++----
 src/libpcp_pmda/src/open.c     |    2 +-
 src/pmcd/rc_pmcd               |    2 +-
 src/pmns/stdpmid.pcp           |    1 +
 7 files changed, 38 insertions(+), 25 deletions(-)

From wwwrun@oss.sgi.com  Fri Nov 21 13:48:08 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 9B7497F51; Fri, 21 Nov 2014 13:48:08 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1083] gpfs pmda
Date: Fri, 21 Nov 2014 19:48:08 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: janfrode@tanso.net
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: 
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416599288.6D2d3.9686"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416599288.6D2d3.9686
Date: Fri, 21 Nov 2014 13:48:08 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1083

--- Comment #3 from Jan-Frode Myklebust  ---
Ok, I've asked on the gpfsug if anybody has any ideas for how the snmp agent
gathers its metrics:

http://gpfsug.org/pipermail/gpfsug-discuss/2014-November/000597.html

BTW: I see there's quite a lot of counters easily available with mmpmon. These
are probably a good start:


"mmpmon io_s"
-------------
mmpmon node 192.168.42.28 name popimap1 io_s OK
timestamp:      1416599060/588460
bytes read:     35495396803892
bytes written:  1132075346666
opens:           390711997
closes:          374730575
reads:          3901906749
writes:          230450025
readdir:          24539957
inode updates:   142537717

"mmpmon fs_io_s"
----------------
mmpmon node 192.168.42.28 name popimap1 fs_io_s OK
cluster:        atmailcluster.ulh
filesystem:     atmailusers
disks:                  11
timestamp:      1416599111/431751
bytes read:      151420466
bytes written:     3914687
opens:              427398
closes:             425296
reads:             1928948
writes:              49214
readdir:               324
inode updates:       17270

mmpmon node 192.168.42.28 name popimap1 fs_io_s OK
cluster:        atmailcluster.ulh
filesystem:     gpfsmailstore
disks:                  12
timestamp:      1416599111/431751
bytes read:     35495469736251
bytes written:  1132076341564
opens:           390287891
closes:          374308462
reads:          3900001260
writes:          230402025
readdir:          24539918
inode updates:   142521531

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416599288.6D2d3.9686
Date: Fri, 21 Nov 2014 13:48:08 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
      

        

            Comment # 3
              on bug 1083
              from 

        
Ok, I've asked on the gpfsug if anybody has any ideas for how the snmp agent
gathers its metrics:

http://gpfsug.org/pipermail/gpfsug-discuss/2014-November/000597.html

BTW: I see there's quite a lot of counters easily available with mmpmon. These
are probably a good start:


"mmpmon io_s"
-------------
mmpmon node 192.168.42.28 name popimap1 io_s OK
timestamp:      1416599060/588460
bytes read:     35495396803892
bytes written:  1132075346666
opens:           390711997
closes:          374730575
reads:          3901906749
writes:          230450025
readdir:          24539957
inode updates:   142537717

"mmpmon fs_io_s"
----------------
mmpmon node 192.168.42.28 name popimap1 fs_io_s OK
cluster:        atmailcluster.ulh
filesystem:     atmailusers
disks:                  11
timestamp:      1416599111/431751
bytes read:      151420466
bytes written:     3914687
opens:              427398
closes:             425296
reads:             1928948
writes:              49214
readdir:               324
inode updates:       17270

mmpmon node 192.168.42.28 name popimap1 fs_io_s OK
cluster:        atmailcluster.ulh
filesystem:     gpfsmailstore
disks:                  12
timestamp:      1416599111/431751
bytes read:     35495469736251
bytes written:  1132076341564
opens:           390287891
closes:          374308462
reads:          3900001260
writes:          230402025
readdir:          24539918
inode updates:   142521531

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416599288.6D2d3.9686--

From kenj@internode.on.net  Fri Nov 21 14:20:29 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 3E0A67F3F
	for ; Fri, 21 Nov 2014 14:20:29 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id BA04DAC005
	for ; Fri, 21 Nov 2014 12:20:28 -0800 (PST)
X-ASG-Debug-ID: 1416601225-04cbb01e5b20aec0001-S8gJnT
Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id YqjZqbHlgPsuU6wt for ; Fri, 21 Nov 2014 12:20:26 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.141
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AoYLAFSdb1R20XOZPGdsb2JhbABcgw6BLoMGhAPLeAQCAoEEFwEBAQEBBgEBAQE4O4QCAQEBBAgCGQQvLwEDAgYDEQQBAQMCIwMCAhkgCgMJCAIEARILBYgwtmeXEQEKAQEBHoEtj2UGgnOBVQWSZGukdoFZNDCCSwEBAQ
Received: from ppp118-209-115-153.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.115.153])
  by ipmail04.adl6.internode.on.net with ESMTP; 22 Nov 2014 06:50:24 +1030
From: "Ken McDonell" 
To: "'Shirshendu Chakrabarti'" ,
	
References: 
In-Reply-To: 
Subject: RE: [pcp] Converting and SQL statement into a PMIE rule for evaluation against pmlogger archives
Date: Sat, 22 Nov 2014 07:20:23 +1100
X-ASG-Orig-Subj: RE: [pcp] Converting and SQL statement into a PMIE rule for evaluation against pmlogger archives
Message-ID: <021601d005c8$961c2790$c25476b0$@internode.on.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Outlook 15.0
Thread-Index: AQEzM2NICEcvH5Znr/HJpXMR+feGaZ2lKWxQ
Content-Language: en-au
X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141]
X-Barracuda-Start-Time: 1416601225
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.01
X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11870
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==

Shirsendu,

There is no generic tool that I'm aware of for translating SQL into pmie =
rules.

And in general this is not going to be possible due to things like =
differences in data models and value comparisons (pmie only works with =
numeric expressions).

But it may be possible in some restricted cases.  We'd probably need =
specific examples of the PCP metrics you're interested and some =
associated SQL queries.

> -----Original Message-----
> From: pcp-bounces@oss.sgi.com [mailto:pcp-bounces@oss.sgi.com] On
> Behalf Of Shirshendu Chakrabarti
> Sent: Wednesday, 19 November 2014 2:50 AM
> To: pcp@oss.sgi.com
> Subject: [pcp] Converting and SQL statement into a PMIE rule for =
evaluation
> against pmlogger archives
>=20
> Hi PCP Team,
>=20
> Is there any way where we can translate a SQL query into a pmie rule =
which
> can be evaluated against a live stream of metrics data or against some
> pmlogger created archive.
>=20
> An example of what I am trying to achieve:
>=20
> SQL statement:
>=20
> select attribute_1, attribute_2 from table where attribute_1 =3D =
'some_val1'
> and attribute_2 =3D 'some_val2' group by 'attribute_1';
>=20
> equivalent metrics, that need to be processed:
>=20
> ..
> .
.
>=20
>=20
> probable pmie rule:
>=20
> result =3D some_inst (
>                               $.
. =3D =
'some_val1' &&
>                               $.
. =3D =
'some_val2'
> ) -> alarm "some message" "%i ";
>=20
> The intended audience for this use-case is business users and analysts =
with
> basic SQL skill set.
>=20
> Thanks,
>=20
> Shirshendu Chakrabarti

From dsmith@redhat.com  Fri Nov 21 16:01:10 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 5CB8F7F3F
	for ; Fri, 21 Nov 2014 16:01:10 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id DB889AC006
	for ; Fri, 21 Nov 2014 14:01:06 -0800 (PST)
X-ASG-Debug-ID: 1416607264-04cb6c0573208ad0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id xLsJGHRazGf13zCW (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 21 Nov 2014 14:01:05 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sALM132Q007418
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Fri, 21 Nov 2014 17:01:04 -0500
Received: from t540p.usersys.redhat.com (vpn-51-68.rdu2.redhat.com [10.10.51.68])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sALM10nF012944
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Fri, 21 Nov 2014 17:01:02 -0500
Message-ID: <546FB61C.2090103@redhat.com>
Date: Fri, 21 Nov 2014 16:01:00 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <545CEE9A.5060007@redhat.com> <615631257.11639679.1415673601327.JavaMail.zimbra@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com> <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416607265
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/19/2014 03:29 AM, Nathan Scott wrote:
> Hi David,
> 
> ----- Original Message -----
>> On 11/17/2014 04:53 PM, Nathan Scott wrote:
>>> Oh, I just had the impression from your earlier mail you weren't
>>> completely satisfied that we'd covered off all the cases ... its
>>> likely I've just misinterpreted that "except for [1]" reference.
>>
>> I believe I've covered all the cases. All list/dictionary references are
>> cached down in cpmda and no longer thrown away after use (a). In the
>> PMDA class, I made sure all lists/dictionaries are cleared, not recreated.
>>
> 
> OK, good stuff.  Do you want to merge the PMDA API changes at this
> stage?
> 
> I tried cherry-picking (there's lots of other commits in dsmith/dev):
> cec13bfd0297ecc755265ba2db69a86daf32a05c
> f2f5a51cde0646fcdf35bc2f60798024c0931c9e
> 1fc0bbf9d517810e8512fb3a7775b68fc6f64572
> ... but there was a fair few test failures (I haven't dug deeper yet -
> they all look like python PMDAs failing to start or exiting early on,
> from a quick glance).

OK, I've tested this. I cherry-picked the following 3 commits from the
pcpfans.git dsmith/dev branch to a local copy of the pcpfans.git dev branch.

====
commit b2e3b5cdd15ddbe57c14b2eaa04cfbfebe2e190b
Author: David Smith 
Date:   Fri Nov 14 15:03:20 2014 -0600

    Improve pcp python support by simplifying refresh metrics callback.

commit 4a265214337a5313466fdd7586f00bdefc417f2c
Author: David Smith 
Date:   Tue Nov 11 16:29:56 2014 -0600

    Removed dead/unused code from src/python/pmda.c

commit 38983a531d83544d3f5d8c73348d0a7c543e90e8
Author: David Smith 
Date:   Wed Oct 29 12:58:40 2014 -0500

    Improve pcp python support by adding refresh metrics callback.
====

I ran all the qa tests before and after the cherry pick, and saw passes
both ways. The python "simple" pmda seems to work fine also.

I'm not sure why your commit ids are different than mine.

> Or, do you want to wait for that JSON PMDA to be closer to complete so
> we have at least one use case for the API?  (either that or some form
> of specialised API test case will be needed - a real PMDA is probably
> easiest, with its tests covering use of the new API).

The latest version of the stap_json PMDA on the dsmith/dev branch, while
not finished does use the new refresh_metrics API.

I'm flexible on how we proceed here, we can do whatever you think makes
the most sense.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From shirshendu@riva.co  Sat Nov 22 06:06:30 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id B0A317F3F
	for ; Sat, 22 Nov 2014 06:06:30 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id 2E7A4AC001
	for ; Sat, 22 Nov 2014 04:06:26 -0800 (PST)
X-ASG-Debug-ID: 1416657983-04cb6c0573251640001-S8gJnT
Received: from mail-qc0-f172.google.com (mail-qc0-f172.google.com [209.85.216.172]) by cuda.sgi.com with ESMTP id KqhIoDMetwRD4Kpb (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Sat, 22 Nov 2014 04:06:23 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.172
Received: by mail-qc0-f172.google.com with SMTP id m20so5006389qcx.31
        for ; Sat, 22 Nov 2014 04:06:23 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=AcHp6LIGeLcOUn8hJ3XTac2nWmI3YSmg4eornTlQrNk=;
        b=FCjop4+Acr7H5qwj3lGBKC5ce4XJS0dgz9Wi/ho3TLuiaVgArwuM4pM9IuogT6XDkH
         eGQl3BMjWEfMAx5updlQqFlc2mHorBXN2Qn+wDoe30K1J4wYHJKMU1QLJYZW5RkMJw7w
         TlDr5Au3MDUCnh3VdUn3846OjxYaxj9uWSZga7QPt2mUQP67nDzqG93ZB61L8JyhFBlE
         e99w3h8I77wXIo64Zq+xuQ9f6U0NOjQHbHzQU39jEE6pXaE1k8P93mf/hpvVw+q5BW/C
         aysTmW2hzhrtZ/zKoStKKNPEXDD7BDl3DYSHr8zX7Wftz9th5XTxPwheyCuXAFnx1uqg
         g8dw==
X-Gm-Message-State: ALoCoQk9MfkRbRpdA+ZK0sICdC8TzGLKc+NWAkCijMI7qAhx05lsfdkjl9iXCseHlnccJMvtbbq/
MIME-Version: 1.0
X-Received: by 10.224.65.4 with SMTP id g4mr14398702qai.20.1416657983333; Sat,
 22 Nov 2014 04:06:23 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Sat, 22 Nov 2014 04:06:23 -0800 (PST)
In-Reply-To: 
References: 
	<546B9C7F.1050706@internode.on.net>
	<432584280.1372959.1416341277776.JavaMail.zimbra@redhat.com>
	
	
Date: Sat, 22 Nov 2014 17:36:23 +0530
Message-ID: 
Subject: Re: [pcp] Non-availability of pmwebd on Amazon Linux
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Non-availability of pmwebd on Amazon Linux
To: Nathan Scott 
Cc: Ken McDonell , pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2b980dd814305087163f9
X-Barracuda-Connect: mail-qc0-f172.google.com[209.85.216.172]
X-Barracuda-Start-Time: 1416657983
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11897
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2b980dd814305087163f9
Content-Type: text/plain; charset=UTF-8

Hi Team,

I have finally been able to get the pmwebd coupled with grafana working and
its amazing.

Based on the pointers received above:

1. I had to make a rpm for libmicrohttpd, since for the Amazon Linux
release we use, the rpm was not available.
2. While making the spec file, there was unusual conflict for
/usr/share/info/dir. Since I am deploying it on server, I did not create
the doc rpms and bypassed the issue.
    I would like to admit that it will probably be considered a hack.
3. After installing the libmicrohttpd rpm, I spun the rpms for pcp from the
git repo, installed pcp-webapi and it all worked after that.

Thanks Nathan and Ken and the PCP team for all the help :)

Thanks,

Shirshendu Chakrabarti

On Wed, Nov 19, 2014 at 9:57 PM, Shirshendu Chakrabarti 
wrote:

> Hi Ken and Nathan,
>
> Please find the pastie at http://pastie.org/private/kj1ow3plpriuuwfx4xpdsg
>
> This pastie is the output of check-vm command.
>
> As was correctly pointed out by both of you, qt-devel and
> libmicrohttpd-devel are missing.
>
> Working to fix them. Will keep the team posted.
>
> Thanks,
>
> Shirshendu Chakrabarti
>
> On Wed, Nov 19, 2014 at 11:45 AM, Shirshendu Chakrabarti <
> shirshendu@riva.co> wrote:
>
>> Hi Team,
>>
>> Thanks for the pointers, I will try both missing packages listed above
>> and report back.
>>
>> Thanks,
>>
>> Shirshendu Chakrabarti
>>
>> On Wed, Nov 19, 2014 at 1:37 AM, Nathan Scott  wrote:
>>
>>>
>>>
>>> ----- Original Message -----
>>> > On 19/11/14 05:17, Shirshendu Chakrabarti wrote:
>>> > > Hi PCP Team,
>>> > >
>>> > > I have bee trying to obtain the pmwebd binary from pcp sources on
>>> Amazon
>>> > > Linux.
>>> > >
>>> > > I build the rpms but I never see the pmwebd daemon installed.
>>> >
>>> > I don't have access to an Amazon Linux system, but it is most likely
>>> > that there are missing RPMS, probably qt-devel.
>>>
>>> It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 required)
>>> package
>>> on the build machine ("libmicrohttpd-dev" package on Debian/Ubuntu).
>>>
>>> cheers.
>>>
>>> --
>>> Nathan
>>>
>>
>>
>

--001a11c2b980dd814305087163f9
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Team,

I have finally been able to ge=
t the pmwebd coupled with grafana working and its amazing.

Based on the pointers received above:

1. =
I had to make a rpm for libmicrohttpd, since for the Amazon Linux release w=
e use, the rpm was not available.
2. While making the spec file, =
there was unusual conflict for /usr/share/info/dir. Since I am deploying it=
 on server, I did not create the doc rpms and bypassed the issue.
=C2=A0 =C2=A0 I would like to admit that it will probably be considered a =
hack.
3. After installing the libmicrohttpd rpm, I spun the rpms =
for pcp from the git repo, installed pcp-webapi and it all worked after tha=
t.

Thanks Nathan and Ken and the PCP team for all =
the help :)

Thanks,

Shirs=
hendu Chakrabarti

On Wed, Nov 19, 2014 at 9:57 PM, Shirshendu Chakrabarti <shir=
shendu@riva.co> wrote:
Hi Ken and Nathan,

Please find the pastie a=
t=C2=A0http://pastie.org/private/kj1ow3plpriuuwfx4xpdsg
=

This pastie is the output of check-vm command.
As was correctly pointed out by both of you, qt-devel and libmi=
crohttpd-devel are missing.

Working to fix them. W=
ill keep the team posted.

Thanks,

Shirshendu Chakrabarti

On Wed, N=
ov 19, 2014 at 11:45 AM, Shirshendu Chakrabarti <shirshendu@riva.co&g=
t; wrote:
Hi Team=
,

Thanks for the pointers, I will try both missing packa=
ges listed above and report back.

Thanks,

Shirshendu Chakrabarti

On Wed, Nov 19, 2014 at 1:37 AM=
, Nathan Scott <nathans@redhat.com> wrote:




----- Original Message -----

> On 19/11/14 05:17, Shirshendu Chakrabarti wrote:

> > Hi PCP Team,

> >

> > I have bee trying to obtain the pmwebd binary from pcp sources on=
 Amazon

> > Linux.

> >

> > I build the rpms but I never see the pmwebd daemon installed.

>

> I don't have access to an Amazon Linux system, but it is most like=
ly

> that there are missing RPMS, probably qt-devel.



It'll be a missing or too-old libmicrohttpd-devel (>0.9.9 req=
uired) package

on the build machine ("libmicrohttpd-dev" package on Debian/Ubunt=
u).



cheers.



--

Nathan










--001a11c2b980dd814305087163f9--

From wwwrun@oss.sgi.com  Sun Nov 23 16:43:54 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 741877F57; Sun, 23 Nov 2014 16:43:54 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1083] gpfs pmda
Date: Sun, 23 Nov 2014 22:43:54 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: 
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416782634.FC1372dc3.14728"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416782634.FC1372dc3.14728
Date: Sun, 23 Nov 2014 16:43:54 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1083

--- Comment #4 from Nathan Scott  ---
(In reply to comment #3)
> BTW: I see there's quite a lot of counters easily available with mmpmon.
> These are probably a good start:

Yep, looks like a good interface to use.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416782634.FC1372dc3.14728
Date: Sun, 23 Nov 2014 16:43:54 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
      

        

            Comment # 4
              on bug 1083
              from 

        
(In reply to comment #3)
> BTW: I see there's quite a lot of counters easily available with mmpmon.
> These are probably a good start:

Yep, looks like a good interface to use.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416782634.FC1372dc3.14728--

From nscott@redhat.com  Sun Nov 23 17:38:34 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id E873E7F50
	for ; Sun, 23 Nov 2014 17:38:34 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id D55748F8039
	for ; Sun, 23 Nov 2014 15:38:34 -0800 (PST)
X-ASG-Debug-ID: 1416785912-04cb6c05712d3270001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id zNgKHeaGgpxrC2xz (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 23 Nov 2014 15:38:33 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sANNcWkh011096;
	Sun, 23 Nov 2014 18:38:32 -0500
Date: Sun, 23 Nov 2014 18:38:32 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: David Smith 
Cc: pcp 
Message-ID: <1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com>
In-Reply-To: <546FB61C.2090103@redhat.com>
References: <54512E80.9090302@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com> <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com> <546FB61C.2090103@redhat.com>
Subject: Re: [pcp] [RFC] pcp python patch
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp python patch
Thread-Index: G4jqUqFvAW3urDeCIjtHACEVV6qAPA==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1416785913
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.11954
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain

Hi David,

----- Original Message -----
> On 11/19/2014 03:29 AM, Nathan Scott wrote:
> > OK, good stuff.  Do you want to merge the PMDA API changes at this
> > stage?
> > 
> > I tried cherry-picking (there's lots of other commits in dsmith/dev):
> > cec13bfd0297ecc755265ba2db69a86daf32a05c
> > f2f5a51cde0646fcdf35bc2f60798024c0931c9e
> > 1fc0bbf9d517810e8512fb3a7775b68fc6f64572
> > ... but there was a fair few test failures (I haven't dug deeper yet -
> > they all look like python PMDAs failing to start or exiting early on,
> > from a quick glance).
> [...]
> I ran all the qa tests before and after the cherry pick, and saw passes
> both ways. The python "simple" pmda seems to work fine also.

Odd.  Only thing I can think of is a python version difference - I'm
using RHEL6 on my primary test machine, which has a relatively old
python version.  I'll have another go and look more deeply into the
failures.

> > Or, do you want to wait for that JSON PMDA to be closer to complete so
> > we have at least one use case for the API?  (either that or some form
> > of specialised API test case will be needed - a real PMDA is probably
> > easiest, with its tests covering use of the new API).
> 
> The latest version of the stap_json PMDA on the dsmith/dev branch, while
> not finished does use the new refresh_metrics API.
> 
> I'm flexible on how we proceed here, we can do whatever you think makes
> the most sense.

We can go either way, just need an appropriate level of testing to go in
with the API addition.  Since there's no rush, and a release in a week -
maybe it'll make most sense to wait for the PMDA that uses it, and piggy-
back on the testing of that, rather than writing a special test case just
for the one API function.

Also, please consider reclaiming the systemtap domain number for the new
PMDA if this is is going to become the preferred stap interface for folks
to use (and, "git rm" on the src/pmdas/systemtap - that's getting old and
moldy at this stage.  Will reports its broken on modern kernels due to it
hooking into kernel vfs functions that haven't existed for many years, so
it seems noone is using it anymore).

cheers.

--
Nathan

From fche@redhat.com  Mon Nov 24 13:32:00 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id DA8397F60
	for ; Mon, 24 Nov 2014 13:32:00 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id BC126304066
	for ; Mon, 24 Nov 2014 11:32:00 -0800 (PST)
X-ASG-Debug-ID: 1416857516-04cbb01e593820e0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 9yKHF723KBKzRLlC (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 24 Nov 2014 11:31:56 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAOJVta4019002
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 24 Nov 2014 14:31:56 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAOJVtup011265;
	Mon, 24 Nov 2014 14:31:55 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 0000B58259; Mon, 24 Nov 2014 14:31:54 -0500 (EST)
To: Nathan Scott 
Cc: David Smith , pcp@oss.sgi.com
Subject: Re: [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <54667179.1060605@redhat.com>
	<370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com>
	<546A44F0.1070001@redhat.com>
	<207038253.507858.1416264783839.JavaMail.zimbra@redhat.com>
	<546A7EDD.9000009@redhat.com>
	<134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com>
	<546FB61C.2090103@redhat.com>
	<1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [RFC] pcp python patch
From: fche@redhat.com (Frank Ch. Eigler)
Date: Mon, 24 Nov 2014 14:31:54 -0500
In-Reply-To: <1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Sun, 23 Nov 2014 18:38:32 -0500 (EST)")
Message-ID: 
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416857516
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1


nathans wrote:

> [...]  Also, please consider reclaiming the systemtap domain number
> for the new PMDA

No objection here, but is there something special about this old perl
pmda that makes this widespread cautionary verbiage moot?

       -d   It is absolutely crucial that the performance metrics domain num-
            ber  specified  here  is  unique and consistent.  That is, domain
            should be different for every PMDA on the one host, and the  same
            domain number should be used for the same PMDA on all hosts.

(In this case, 88 happens to be hard-coded into the pmdasystemtap.pl instead
of being supplied as a command line option.)


> if this is is going to become the preferred stap interface for folks
> to use [...]

(If we do it right, it will be useful far beyond stap.)

- FChE

From dsmith@redhat.com  Mon Nov 24 16:46:12 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 3C4617F4E
	for ; Mon, 24 Nov 2014 16:46:12 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id BFB3AAC004
	for ; Mon, 24 Nov 2014 14:46:11 -0800 (PST)
X-ASG-Debug-ID: 1416869166-04cbb01e593acbc0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id DeET0aoAolmcyc6Y (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 24 Nov 2014 14:46:07 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAOMk62a015216
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 24 Nov 2014 17:46:06 -0500
Received: from t540p.usersys.redhat.com (dhcp-10-15-1-13.hsv.redhat.com [10.15.1.13])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAOMk5Gn024088
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Mon, 24 Nov 2014 17:46:05 -0500
Message-ID: <5473B52D.2030004@redhat.com>
Date: Mon, 24 Nov 2014 16:46:05 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com> <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com> <546FB61C.2090103@redhat.com> <1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416869167
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/23/2014 05:38 PM, Nathan Scott wrote:
> Hi David,
> 
> ----- Original Message -----
>> On 11/19/2014 03:29 AM, Nathan Scott wrote:
>>> OK, good stuff.  Do you want to merge the PMDA API changes at this
>>> stage?
>>>
>>> I tried cherry-picking (there's lots of other commits in dsmith/dev):
>>> cec13bfd0297ecc755265ba2db69a86daf32a05c
>>> f2f5a51cde0646fcdf35bc2f60798024c0931c9e
>>> 1fc0bbf9d517810e8512fb3a7775b68fc6f64572
>>> ... but there was a fair few test failures (I haven't dug deeper yet -
>>> they all look like python PMDAs failing to start or exiting early on,
>>> from a quick glance).
>> [...]
>> I ran all the qa tests before and after the cherry pick, and saw passes
>> both ways. The python "simple" pmda seems to work fine also.
> 
> Odd.  Only thing I can think of is a python version difference - I'm
> using RHEL6 on my primary test machine, which has a relatively old
> python version.  I'll have another go and look more deeply into the
> failures.

Hmm, I'll try running this code on RHEL6 (instead of F20) and see what I
get.

>>> Or, do you want to wait for that JSON PMDA to be closer to complete so
>>> we have at least one use case for the API?  (either that or some form
>>> of specialised API test case will be needed - a real PMDA is probably
>>> easiest, with its tests covering use of the new API).
>>
>> The latest version of the stap_json PMDA on the dsmith/dev branch, while
>> not finished does use the new refresh_metrics API.
>>
>> I'm flexible on how we proceed here, we can do whatever you think makes
>> the most sense.
> 
> We can go either way, just need an appropriate level of testing to go in
> with the API addition.  Since there's no rush, and a release in a week -
> maybe it'll make most sense to wait for the PMDA that uses it, and piggy-
> back on the testing of that, rather than writing a special test case just
> for the one API function.
> 
> Also, please consider reclaiming the systemtap domain number for the new
> PMDA if this is is going to become the preferred stap interface for folks
> to use (and, "git rm" on the src/pmdas/systemtap - that's getting old and
> moldy at this stage.  Will reports its broken on modern kernels due to it
> hooking into kernel vfs functions that haven't existed for many years, so
> it seems noone is using it anymore).

I'm not sure what the plan is (assuming there is one) for the systemtap
pmda.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From wwwrun@oss.sgi.com  Mon Nov 24 17:41:28 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 3CEE17F51; Mon, 24 Nov 2014 17:41:28 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 983] /proc/net/snmp metric fetch shortcomings exposed by recent
 kernel changes
Date: Mon, 24 Nov 2014 23:41:27 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_status resolution
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416872488.4a1ACb1.18191"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416872488.4a1ACb1.18191
Date: Mon, 24 Nov 2014 17:41:28 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=983

Nathan Scott  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #1 from Nathan Scott  ---
Fixed over a year ago.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416872488.4a1ACb1.18191
Date: Mon, 24 Nov 2014 17:41:28 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 983
        

             

          

            What
            Removed
            Added
          


         

           Status
           NEW
           
           RESOLVED
           
         


         

           Resolution
           ---
           
           FIXED
           
         

      

        

            Comment # 1
              on bug 983
              from 

        
Fixed over a year ago.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416872488.4a1ACb1.18191--

From wwwrun@oss.sgi.com  Mon Nov 24 17:53:56 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id A7C747F51; Mon, 24 Nov 2014 17:53:56 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1087] New: Add a pcp-mpstat(1) command
Date: Mon, 24 Nov 2014 23:53:56 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status keywords bug_severity priority component assigned_to
 reporter cc classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416873236.fcFB4C81.19306"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873236.fcFB4C81.19306
Date: Mon, 24 Nov 2014 17:53:56 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1087

            Bug ID: 1087
           Summary: Add a pcp-mpstat(1) command
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Keywords: EasyFix
          Severity: enhancement
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: nathans@debian.org
                CC: pcp@oss.sgi.com
    Classification: Unclassified

A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like mpstat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, as is access to
additional PCP functionality like archive replay and remote hosts.

The python pcp.pmcc module provides a nice way to implement such tools, along
with the pcp(1) frontend command that allows for option compatibility alongside
the new PCP functionality.  See the pcp-free(1) and pcp-numastat(1) man pages
for examples, as well as their sources (src/pcp/{free,numastat} and test cases
(qa/{743,991} in the git tree).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873236.fcFB4C81.19306
Date: Mon, 24 Nov 2014 17:53:56 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1087
          
        


        

          Summary
          Add a pcp-mpstat(1) command
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          All
          
        


        

          Status
          NEW
          
        


        

          Keywords
          EasyFix
          
        


        

          Severity
          enhancement
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          nathans@debian.org
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like mpstat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, as is access to
additional PCP functionality like archive replay and remote hosts.

The python pcp.pmcc module provides a nice way to implement such tools, along
with the pcp(1) frontend command that allows for option compatibility alongside
the new PCP functionality.  See the pcp-free(1) and pcp-numastat(1) man pages
for examples, as well as their sources (src/pcp/{free,numastat} and test cases
(qa/{743,991} in the git tree).

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873236.fcFB4C81.19306--

From wwwrun@oss.sgi.com  Mon Nov 24 17:55:00 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 017BC7F51; Mon, 24 Nov 2014 17:55:00 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1088] New: Add a pcp-pidstat(1) command
Date: Mon, 24 Nov 2014 23:54:59 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status keywords bug_severity priority component assigned_to
 reporter cc classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416873299.3b4FDEF41.19453"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873299.3b4FDEF41.19453
Date: Mon, 24 Nov 2014 17:54:59 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1088

            Bug ID: 1088
           Summary: Add a pcp-pidstat(1) command
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Keywords: EasyFix
          Severity: enhancement
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: nathans@debian.org
                CC: pcp@oss.sgi.com
    Classification: Unclassified

A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like pidstat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, as is access to
additional PCP functionality like archive replay and remote hosts.

The python pcp.pmcc module provides a nice way to implement such tools, along
with the pcp(1) frontend command that allows for option compatibility alongside
the new PCP functionality.  See the pcp-free(1) and pcp-numastat(1) man pages
for examples, as well as their sources (src/pcp/{free,numastat} and test cases
(qa/{743,991} in the git tree).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873299.3b4FDEF41.19453
Date: Mon, 24 Nov 2014 17:54:59 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1088
          
        


        

          Summary
          Add a pcp-pidstat(1) command
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          All
          
        


        

          Status
          NEW
          
        


        

          Keywords
          EasyFix
          
        


        

          Severity
          enhancement
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          nathans@debian.org
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like pidstat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, as is access to
additional PCP functionality like archive replay and remote hosts.

The python pcp.pmcc module provides a nice way to implement such tools, along
with the pcp(1) frontend command that allows for option compatibility alongside
the new PCP functionality.  See the pcp-free(1) and pcp-numastat(1) man pages
for examples, as well as their sources (src/pcp/{free,numastat} and test cases
(qa/{743,991} in the git tree).

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873299.3b4FDEF41.19453--

From wwwrun@oss.sgi.com  Mon Nov 24 17:56:37 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id E5DA57F51; Mon, 24 Nov 2014 17:56:37 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1089] New: Add a pcp-vmstat(1) command
Date: Mon, 24 Nov 2014 23:56:37 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: major
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status keywords bug_severity priority component assigned_to
 reporter cc classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416873397.f1cb1.19593"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873397.f1cb1.19593
Date: Mon, 24 Nov 2014 17:56:37 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1089

            Bug ID: 1089
           Summary: Add a pcp-vmstat(1) command
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Keywords: EasyFix
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: nathans@debian.org
                CC: pcp@oss.sgi.com
    Classification: Unclassified

A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like vmstat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, as is access to
additional PCP functionality like archive replay and remote hosts.

The python pcp.pmcc module provides a nice way to implement such tools, along
with the pcp(1) frontend command that allows for option compatibility alongside
the new PCP functionality.  See the pcp-free(1) and pcp-numastat(1) man pages
for examples, as well as their sources (src/pcp/{free,numastat} and test cases
(qa/{743,991} in the git tree).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873397.f1cb1.19593
Date: Mon, 24 Nov 2014 17:56:37 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1089
          
        


        

          Summary
          Add a pcp-vmstat(1) command
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          All
          
        


        

          Status
          NEW
          
        


        

          Keywords
          EasyFix
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          nathans@debian.org
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like vmstat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, as is access to
additional PCP functionality like archive replay and remote hosts.

The python pcp.pmcc module provides a nice way to implement such tools, along
with the pcp(1) frontend command that allows for option compatibility alongside
the new PCP functionality.  See the pcp-free(1) and pcp-numastat(1) man pages
for examples, as well as their sources (src/pcp/{free,numastat} and test cases
(qa/{743,991} in the git tree).

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873397.f1cb1.19593--

From wwwrun@oss.sgi.com  Mon Nov 24 17:59:51 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 4C0E57F55; Mon, 24 Nov 2014 17:59:51 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 881] Need a pmcd_check script
Date: Mon, 24 Nov 2014 23:59:51 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: major
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: mort@sgi.com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_status cc resolution
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416873591.f5c73.19837"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873591.f5c73.19837
Date: Mon, 24 Nov 2014 17:59:51 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=881

Nathan Scott  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |nathans@debian.org
         Resolution|---                         |WONTFIX

--- Comment #3 from Nathan Scott  ---
No further information on the original reason for pmcd exiting was forthcoming,
and hundreds of fixes have gone in that may have resolved whatever it was in
the meantime - so closing out.

In terms of high-availability of daemons (like pmcd) - PaceMaker is apparently
the generally prefered way of tackling this nowadays (http://clusterlabs.org/)
and we apparently supply everything we need to from PCP (just init scripts are
enough, I'm reliably informed by the PaceMaker developers).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873591.f5c73.19837
Date: Mon, 24 Nov 2014 17:59:51 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 881
        

             

          

            What
            Removed
            Added
          


         

           Status
           NEW
           
           RESOLVED
           
         


         

           CC
           
                
           
           nathans@debian.org
           
         


         

           Resolution
           ---
           
           WONTFIX
           
         

      

        

            Comment # 3
              on bug 881
              from 

        
No further information on the original reason for pmcd exiting was forthcoming,
and hundreds of fixes have gone in that may have resolved whatever it was in
the meantime - so closing out.

In terms of high-availability of daemons (like pmcd) - PaceMaker is apparently
the generally prefered way of tackling this nowadays (http://clusterlabs.org/)
and we apparently supply everything we need to from PCP (just init scripts are
enough, I'm reliably informed by the PaceMaker developers).

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873591.f5c73.19837--

From wwwrun@oss.sgi.com  Mon Nov 24 18:00:37 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 3634E7F54; Mon, 24 Nov 2014 18:00:37 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 891] IB pmda;  ./Makepkgs -clean fails
Date: Tue, 25 Nov 2014 00:00:37 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: trivial
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: mort@sgi.com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_status cc resolution
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416873637.E7CFa38e2.20016"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873637.E7CFa38e2.20016
Date: Mon, 24 Nov 2014 18:00:37 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=891

Nathan Scott  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |nathans@debian.org
         Resolution|---                         |FIXED

--- Comment #1 from Nathan Scott  ---
Fixed for a long time (pcp-pmda-infiniband now in main PCP tree also).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873637.E7CFa38e2.20016
Date: Mon, 24 Nov 2014 18:00:37 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 891
        

             

          

            What
            Removed
            Added
          


         

           Status
           NEW
           
           RESOLVED
           
         


         

           CC
           
                
           
           nathans@debian.org
           
         


         

           Resolution
           ---
           
           FIXED
           
         

      

        

            Comment # 1
              on bug 891
              from 

        
Fixed for a long time (pcp-pmda-infiniband now in main PCP tree also).

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873637.E7CFa38e2.20016--

From wwwrun@oss.sgi.com  Mon Nov 24 18:04:46 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id DEC7B7F54; Mon, 24 Nov 2014 18:04:46 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1090] New: Extend the pmiostat(1) command for increased
 compatibility
Date: Tue, 25 Nov 2014 00:04:46 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: major
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
 op_sys bug_status keywords bug_severity priority component assigned_to
 reporter cc classification
Message-ID: 
Content-Type: multipart/alternative; boundary="1416873886.A1Dd23651.20442"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873886.A1Dd23651.20442
Date: Mon, 24 Nov 2014 18:04:46 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=1090

            Bug ID: 1090
           Summary: Extend the pmiostat(1) command for increased
                    compatibility
           Product: pcp
           Version: unspecified
          Hardware: All
                OS: Linux
            Status: NEW
          Keywords: EasyFix
          Severity: major
          Priority: P5
         Component: pcp
          Assignee: pcp@kenj.com.au
          Reporter: nathans@debian.org
                CC: pcp@oss.sgi.com
    Classification: Unclassified

A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like iostat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, but the existing
pmiostat(1) python script does not offer that.

It would be possible to add a pcp-iostat(1) symlink to pmiostat(1) that uses a
different pmOptions subclass (switch based on argv[0]), to provide the desired
level of compatibility while still leveraging the work done in the pmiostat(1)
command.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873886.A1Dd23651.20442
Date: Mon, 24 Nov 2014 18:04:46 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    

        

          Bug ID
          1090
          
        


        

          Summary
          Extend the pmiostat(1) command for increased compatibility
          
        


        

          Product
          pcp
          
        


        

          Version
          unspecified
          
        


        

          Hardware
          All
          
        


        

          OS
          Linux
          
        


        

          Status
          NEW
          
        


        

          Keywords
          EasyFix
          
        


        

          Severity
          major
          
        


        

          Priority
          P5
          
        


        

          Component
          pcp
          
        


        

          Assignee
          pcp@kenj.com.au
          
        


        

          Reporter
          nathans@debian.org
          
        


        

          CC
          pcp@oss.sgi.com
          
        


        

          Classification
          Unclassified
          
        

      

        

        
A number of people have asked us to provide drop-in replacements for sysstat
reporting tools like iostat(1), so they can more easily transition to using
PCP.  Command line option compatibility is highly desirable, but the existing
pmiostat(1) python script does not offer that.

It would be possible to add a pcp-iostat(1) symlink to pmiostat(1) that uses a
different pmOptions subclass (switch based on argv[0]), to provide the desired
level of compatibility while still leveraging the work done in the pmiostat(1)
command.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873886.A1Dd23651.20442--

From wwwrun@oss.sgi.com  Mon Nov 24 18:06:00 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 0E2D57F54; Mon, 24 Nov 2014 18:06:00 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 869] Add details on ipc facilities to linux PMDA
Date: Tue, 25 Nov 2014 00:05:59 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: EasyFix
X-Bugzilla-Severity: normal
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: mort@sgi.com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: keywords
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416873960.aAf6B71.20591"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416873960.aAf6B71.20591
Date: Mon, 24 Nov 2014 18:06:00 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=869

Nathan Scott  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |EasyFix

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416873960.aAf6B71.20591
Date: Mon, 24 Nov 2014 18:06:00 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 869
        

             

          

            What
            Removed
            Added
          


         

           Keywords
           
                
           
           EasyFix
           
         

      

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416873960.aAf6B71.20591--

From wwwrun@oss.sgi.com  Mon Nov 24 18:10:22 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id F2C007F56; Mon, 24 Nov 2014 18:10:22 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 974] Installation issues when user "pcp" and/or group "pcp"
 already defined
Date: Tue, 25 Nov 2014 00:10:22 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: normal
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@kenj.com.au
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_status resolution
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416874222.0154Be4.20936"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416874222.0154Be4.20936
Date: Mon, 24 Nov 2014 18:10:22 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=974

Nathan Scott  changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #6 from Nathan Scott  ---
We're well committed to a path here - we now: a/ allow configurable PCP_USER
and PCP_GROUP settings via pcp.conf; and b/ cannot realistically do anything
about the default user/group names used during the RPM installation process at
this stage.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416874222.0154Be4.20936
Date: Mon, 24 Nov 2014 18:10:22 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    
 changed
              bug 974
        

             

          

            What
            Removed
            Added
          


         

           Status
           NEW
           
           RESOLVED
           
         


         

           Resolution
           ---
           
           FIXED
           
         

      

        

            Comment # 6
              on bug 974
              from 

        
We're well committed to a path here - we now: a/ allow configurable PCP_USER
and PCP_GROUP settings via pcp.conf; and b/ cannot realistically do anything
about the default user/group names used during the RPM installation process at
this stage.

        

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416874222.0154Be4.20936--

From wwwrun@oss.sgi.com  Mon Nov 24 18:10:23 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
	autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
	id 187EC7F57; Mon, 24 Nov 2014 18:10:23 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 942] Wish List / Backlog / RFEs
Date: Tue, 25 Nov 2014 00:10:23 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: nathans@debian.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: kenj@internode.on.net
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_status resolution
Message-ID: 
In-Reply-To: 
References: 
Content-Type: multipart/alternative; boundary="1416874223.7a4c86.20936"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0


--1416874223.7a4c86.20936
Date: Mon, 24 Nov 2014 18:10:23 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"

http://oss.sgi.com/bugzilla/show_bug.cgi?id=942

Bug 942 depends on bug 974, which changed state.

Bug 974 Summary: Installation issues when user "pcp" and/or group "pcp" already defined
http://oss.sgi.com/bugzilla/show_bug.cgi?id=974

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

--1416874223.7a4c86.20936
Date: Mon, 24 Nov 2014 18:10:23 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"


    
      
    
    Bug 942
              depends on
              bug 974,
              which changed state.
        

             

          

            What
            Removed
            Added
          


         

           Status
           NEW
           
           RESOLVED
           
         


         

           Resolution
           ---
           
           FIXED
           
         

      

      

      

      You are receiving this mail because:
      
      
    
          
  • You are on the CC list for the bug.
    • 
      

    


--1416874223.7a4c86.20936--

From lberk@redhat.com  Mon Nov 24 18:45:38 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id CCCD87F51
	for ; Mon, 24 Nov 2014 18:45:38 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id AD6C1304062
	for ; Mon, 24 Nov 2014 16:45:35 -0800 (PST)
X-ASG-Debug-ID: 1416876331-04bdf0615e3a2630001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id uQuHnQs6CGIrnPVM (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 24 Nov 2014 16:45:31 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAP0jVHD027300
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Mon, 24 Nov 2014 19:45:31 -0500
Received: from toium ([10.15.16.195])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAP0jU6C030407
	(version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO)
	for ; Mon, 24 Nov 2014 19:45:30 -0500
From: Lukas Berk 
To: pcp@oss.sgi.com
Subject: pcp updates: build, scripts, pmdapapi qa
Date: Mon, 24 Nov 2014 19:45:29 -0500
X-ASG-Orig-Subj: pcp updates: build, scripts, pmdapapi qa
Message-ID: <87fvd8xg46.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416876331
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1


Hi,

Changes committed to: lberk/dev on git://sourceware.org/git/pcpfans.git

A bit of everything in these commits, a couple rpm build fixes, another
qa addition for pmdapapi, and I've contributed the script I use to build
weekly pcp snapshots on fedora-rawhide.  It's able to merge in upstream
spec file differences, tar the latest sources, and send to the build
system.

Cheers,

Lukas

 build/rpm/fedora.spec  |    4 -
 build/rpm/spin-rawhide |  113 ++++++++++++++++++++++++++++++++++++++++++++++++-
 qa/967                 |   19 ++++++++
 qa/967.out             |   47 ++++++++++++++++++++
 4 files changed, 180 insertions(+), 3 deletions(-)

commit 8b249e27719dd821e7d94da11d43a55a82770392
Author: Lukas Berk 
Date:   Mon Nov 24 14:41:07 2014 -0500

    Add check for $PCP_WEB_JS variable, only clone/tar if needed
    
    Currently, there are two sources for pcp builds, the core PCP git
    tree, and a seperate tree for the javascript functionality.  If,
    in the future, these two are combined, lets handle that
    gracefully/automatically, as the individual running this script on
    monday morning may completely forget.

commit 2e0590102d788e513636457271e21586f2a3c766
Author: Lukas Berk 
Date:   Mon Nov 24 14:32:16 2014 -0500

    Add spin-rawhide script to build/rpm/ dir
    
    spin-rawhide is a script that allows the user to (given proper permissions)
    automatically build a pcp package on fedora-rawhide.

commit c9f06c941d601b414b6f2e89bc8a985cf9112e58
Author: Lukas Berk 
Date:   Mon Nov 24 13:44:06 2014 -0500

    Add explicit fetch/store qa testcase for papi.control.multiplex
    
    Append another testcase to ensure we can still fetch metric values
    when mutliplexing is disable as well as papi.control.status returns
    the value stored multiplexing variable

commit 8efd4dc9e8c9fe7cdb6d26836883d3bd1cfd2e98
Author: Lukas Berk 
Date:   Mon Nov 24 13:34:04 2014 -0500

    Remove glob for %{_localstatedir}/lib/pcp/config/pmie
    
    The glob was breaking the build on fedora-rawhide

commit 0d3db7e18f01a1daab6c532f7c4200819e550ae7
Author: Lukas Berk 
Date:   Sun Nov 23 20:53:52 2014 -0500

    Add which requirement to fedora.spec
    
    Dependncy on 'which' needs to be added to fedora.spec

From nscott@redhat.com  Mon Nov 24 21:11:11 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 6E1077F50
	for ; Mon, 24 Nov 2014 21:11:11 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 4D8EC8F8035
	for ; Mon, 24 Nov 2014 19:11:08 -0800 (PST)
X-ASG-Debug-ID: 1416885065-04cb6c0571408140001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id tmf77jR3oAed1Hhn (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 24 Nov 2014 19:11:06 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAP3B5ie030216;
	Mon, 24 Nov 2014 22:11:05 -0500
Date: Mon, 24 Nov 2014 22:11:05 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: "Frank Ch. Eigler" 
Cc: David Smith , pcp@oss.sgi.com
Message-ID: <82347673.4643536.1416885065537.JavaMail.zimbra@redhat.com>
In-Reply-To: 
References: <54512E80.9090302@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com> <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com> <546FB61C.2090103@redhat.com> <1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com> 
Subject: Re: [RFC] pcp python patch
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [RFC] pcp python patch
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.11]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp python patch
Thread-Index: BbvF1KLN4KweT3ze4/eqBWO//Oh7bQ==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1416885066
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12008
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
	                           Domain



----- Original Message -----
> > [...]  but is there something special about this old perl
> pmda that makes this widespread cautionary verbiage moot?

Nope.

> (In this case, 88 happens to be hard-coded into the pmdasystemtap.pl instead
> of being supplied as a command line option.)

The script PMDAs don't process command line options.  They could, they
should, but they don't at this stage.

cheers.

--
Nathan

From brolley@redhat.com  Tue Nov 25 13:35:40 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 58E217F3F
	for ; Tue, 25 Nov 2014 13:35:40 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 393BC304048
	for ; Tue, 25 Nov 2014 11:35:37 -0800 (PST)
X-ASG-Debug-ID: 1416944133-04bdf0616079f210001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id CDVNdIrRT68AWu8E (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 25 Nov 2014 11:35:33 -0800 (PST)
X-Barracuda-Envelope-From: brolley@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAPJZWAR007944
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Tue, 25 Nov 2014 14:35:32 -0500
Received: from [10.15.16.126] (dhcp-10-15-16-126.yyz.redhat.com [10.15.16.126])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAPJZW6h004718
	for ; Tue, 25 Nov 2014 14:35:32 -0500
Message-ID: <5474DA9E.9020309@redhat.com>
Date: Tue, 25 Nov 2014 14:38:06 -0500
From: Dave Brolley 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: PCP Mailing List 
Subject: pcp updates: lberk configuration and build
Content-Type: text/plain; charset=utf-8; format=flowed
X-ASG-Orig-Subj: pcp updates: lberk configuration and build
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416944133
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Changes committed to git://git.pcp.io/pcp.git dev

Lukas Berk (5):
       Add which requirement to fedora.spec
       Remove glob for %{_localstatedir}/lib/pcp/config/pmie
       Add explicit fetch/store qa testcase for papi.control.multiplex
       Add spin-rawhide script to build/rpm/ dir
       Add check for $PCP_WEB_JS variable, only clone/tar if needed

  build/rpm/fedora.spec  |    4 -
  build/rpm/spin-rawhide |  113 
++++++++++++++++++++++++++++++++++++++++++++++++-
  qa/967                 |   19 ++++++++
  qa/967.out             |   47 ++++++++++++++++++++
  4 files changed, 180 insertions(+), 3 deletions(-)

Dave

From fche@redhat.com  Tue Nov 25 13:57:25 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id CFA1A7F47
	for ; Tue, 25 Nov 2014 13:57:25 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id BCDCD8F8059
	for ; Tue, 25 Nov 2014 11:57:22 -0800 (PST)
X-ASG-Debug-ID: 1416945437-04cb6c0570637b00001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id zJDIA1vug0Dg54qR (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 25 Nov 2014 11:57:18 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAPJvHoF028921
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Tue, 25 Nov 2014 14:57:17 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAPJvGql018691
	for ; Tue, 25 Nov 2014 14:57:16 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id 5072658259; Tue, 25 Nov 2014 14:57:16 -0500 (EST)
Date: Tue, 25 Nov 2014 14:57:16 -0500
From: "Frank Ch. Eigler" 
To: pcp developers 
Subject: pcp update: papi-pmda dynamic metrictab[]
Message-ID: <20141125195716.GE5088@redhat.com>
X-ASG-Orig-Subj: pcp update: papi-pmda dynamic metrictab[]
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416945438
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

Thanks to kenj's fine work from last week, this patch from pcpfans.git
simplifies the papi pmda to eliminate the malloc'd metrictab[].

I didn't publish this patch earlier because during regression
testing, the linux-proc pmda crashes.  But a brief investigation
showed that this particular failure (a memory corruption during the
fetch / value-stuff loop) also exists in git/dev pcp as per lberk's
fine rawhide-pcp build, so this change was not responsible.


commit ede409944fe58fef2eaba2dd275a0899745bb9bd
Author: Frank Ch. Eigler 
Date:   Sun Nov 23 19:50:26 2014 -0500

    papi pmda: use dynamic metric description
    
    Using kenj's commit #f856e2c171, it becomes possible to avoid
    allocating a metrictab[] in the pmda, when it is so easy to generate
    each pmDesc on demand.  This is done by having the pmdaFetch()
    function in libpcp_pmda fall back to the pmdaInterface->desc callback
    (if set), if a metrictab[] was not specified during pmdaInit().
    
    The papi pmda is converted to this scheme.  (Also, multiplexing is
    turned back on by default, as per the qa/967 test case, and .gitignore
    is extended with help.* stuff.)

From dsmith@redhat.com  Tue Nov 25 14:33:37 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id B6E397F47
	for ; Tue, 25 Nov 2014 14:33:37 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id A5D648F8052
	for ; Tue, 25 Nov 2014 12:33:34 -0800 (PST)
X-ASG-Debug-ID: 1416947609-04cbb01e5b6484c0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id GzWpE9jAdG3DdJFH (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 25 Nov 2014 12:33:30 -0800 (PST)
X-Barracuda-Envelope-From: dsmith@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAPKXScU011592
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Tue, 25 Nov 2014 15:33:29 -0500
Received: from t540p.usersys.redhat.com (vpn-53-15.rdu2.redhat.com [10.10.53.15])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAPKXQpC007303
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
	Tue, 25 Nov 2014 15:33:27 -0500
Message-ID: <5474E795.7010302@redhat.com>
Date: Tue, 25 Nov 2014 14:33:25 -0600
From: David Smith 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Nathan Scott 
CC: pcp 
Subject: Re: [pcp] [RFC] pcp python patch
References: <54512E80.9090302@redhat.com> <54667179.1060605@redhat.com> <370186244.15487866.1416205739744.JavaMail.zimbra@redhat.com> <546A44F0.1070001@redhat.com> <207038253.507858.1416264783839.JavaMail.zimbra@redhat.com> <546A7EDD.9000009@redhat.com> <134603578.1719924.1416389380267.JavaMail.zimbra@redhat.com> <546FB61C.2090103@redhat.com> <1332365833.3983034.1416785912055.JavaMail.zimbra@redhat.com> <5473B52D.2030004@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] [RFC] pcp python patch
In-Reply-To: <5473B52D.2030004@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1416947609
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/24/2014 04:46 PM, David Smith wrote:
> On 11/23/2014 05:38 PM, Nathan Scott wrote:
>> Hi David,
>>
>> ----- Original Message -----
>>> On 11/19/2014 03:29 AM, Nathan Scott wrote:
>>>> OK, good stuff.  Do you want to merge the PMDA API changes at this
>>>> stage?
>>>>
>>>> I tried cherry-picking (there's lots of other commits in dsmith/dev):
>>>> cec13bfd0297ecc755265ba2db69a86daf32a05c
>>>> f2f5a51cde0646fcdf35bc2f60798024c0931c9e
>>>> 1fc0bbf9d517810e8512fb3a7775b68fc6f64572
>>>> ... but there was a fair few test failures (I haven't dug deeper yet -
>>>> they all look like python PMDAs failing to start or exiting early on,
>>>> from a quick glance).
>>> [...]
>>> I ran all the qa tests before and after the cherry pick, and saw passes
>>> both ways. The python "simple" pmda seems to work fine also.
>>
>> Odd.  Only thing I can think of is a python version difference - I'm
>> using RHEL6 on my primary test machine, which has a relatively old
>> python version.  I'll have another go and look more deeply into the
>> failures.
> 
> Hmm, I'll try running this code on RHEL6 (instead of F20) and see what I
> get.

I'm seeing failures on RHEL6 also, digging into it now.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

From shirshendu@riva.co  Wed Nov 26 04:17:22 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 9FE537F4E
	for ; Wed, 26 Nov 2014 04:17:22 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 704D48F8033
	for ; Wed, 26 Nov 2014 02:17:19 -0800 (PST)
X-ASG-Debug-ID: 1416997034-04bdf0615f868d20001-S8gJnT
Received: from mail-qa0-f50.google.com (mail-qa0-f50.google.com [209.85.216.50]) by cuda.sgi.com with ESMTP id CeFHvNNzhKWtQVcK (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Wed, 26 Nov 2014 02:17:15 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.50
Received: by mail-qa0-f50.google.com with SMTP id w8so1650630qac.37
        for ; Wed, 26 Nov 2014 02:17:14 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:date:message-id:subject:from:to
         :content-type;
        bh=6MSwB3IapBbdiv3Atf40a/wiJLID8h5SkwXUS1x0KsE=;
        b=jJLPX47Cz9Pgxz4vq+sLisavgspvkeFfD1OM0zpneL50PRnTDcWF3oSDOkpzSCX4c8
         A9IX6wbHepRs0ZsKDFx/J8Bk/vQ2VMBq04LFVuvBxRLNSb96+/FbKYClM3poQpy4Ez5C
         wpYz/SnuFzpgYLq5loIqoWdOfNBEaoQLiXK0b5w402l7m+llCv1SUREz08rNvyxcJs4l
         wYNAOGiECc3j/Q/xU4DQzPU7L0HEpClpa0tCjc5IYUTQum3jHqbf+iYUsbjyCEIz+8RG
         dyPkQqgKgNgYAaP34YWLdBSjct5FDxBy9NWD5mIsGnhu6rEzSAdVOK972oWyBYWkxb2d
         ufEQ==
X-Gm-Message-State: ALoCoQn68y2oOhatVHoN4p+TZIiau8oqhzsbk9ykN+fGcy1+RolbmTCql7wxZFMYyvGFkQrLoFpD
MIME-Version: 1.0
X-Received: by 10.140.86.175 with SMTP id p44mr7324748qgd.54.1416997034014;
 Wed, 26 Nov 2014 02:17:14 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Wed, 26 Nov 2014 02:17:13 -0800 (PST)
Date: Wed, 26 Nov 2014 15:47:13 +0530
Message-ID: 
Subject: Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c12592dc4eca0508c0546a
X-Barracuda-Connect: mail-qa0-f50.google.com[209.85.216.50]
X-Barracuda-Start-Time: 1416997034
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12068
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c12592dc4eca0508c0546a
Content-Type: text/plain; charset=UTF-8

Hi PCP team,

Could anyone in the team point me to any literature which explains the
difference between a per node metric and percpu metric. The one-liner and
extended description are absent in pernode metric case and terse in percpu
case.

For example:

[root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.user

kernel.percpu.cpu.user
    inst [0 or "cpu0"] value 993240
[root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user

kernel.pernode.cpu.user
    inst [0 or "node0"] value 0

I was under the assumption that, kernel.pernode.* and kernel.all.* metrics
are the same. I am clearly missing something important.

Thanks,

Shirshendu Chakrabarti

--001a11c12592dc4eca0508c0546a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi PCP team,

Could anyone in the team point m=
e to any literature which explains the difference between a per node metric=
 and percpu metric. The one-liner and extended description are absent in pe=
rnode metric case and terse in percpu case.

For ex=
ample:

[root@pcp-test-shir3 pmlogger]# pminfo -f k=
ernel.percpu.cpu.user

kernel.percpu.cpu.user
=

=C2=A0 =C2=A0 inst [0 or "cpu0"] value 993240
[roo=
t@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
kernel.pernode.cpu.user
=C2=A0 =C2=A0 inst [0 or &quo=
t;node0"] value 0
=C2=A0
I was under the ass=
umption that, kernel.pernode.* and kernel.all.* metrics are the same. I am =
clearly missing something important.

Thanks,
=


Shirshendu Chakrabarti


--001a11c12592dc4eca0508c0546a--

From caokl@turblade.com  Wed Nov 26 11:06:56 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_FONT_FACE_BAD,
	HTML_MESSAGE autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 727AD7F4E
	for ; Wed, 26 Nov 2014 11:06:56 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 60EE0304070
	for ; Wed, 26 Nov 2014 09:06:53 -0800 (PST)
X-ASG-Debug-ID: 1417021599-04cb6c057072a6e0001-S8gJnT
Received: from surfront.trublade.com ([61.160.78.18]) by cuda.sgi.com with SMTP id 0FUpsvi0g2H9zkBw for ; Wed, 26 Nov 2014 09:06:42 -0800 (PST)
X-Barracuda-Envelope-From: caokl@turblade.com
X-Barracuda-Apparent-Source-IP: 61.160.78.18
Received: from [58.221.58.13] by surfront.trublade.com with SURFRONT ESMTP id 308493414992852;
	Thu, 27 Nov 2014 01:13:48 +0800 (CST)
Message-ID: <20141127010649444650@turblade.com>
From: =?utf-8?B?6auY6Imz?= 
To: 
Subject: 
Date: Thu, 27 Nov 2014 01:06:37 +0800
X-ASG-Orig-Subj: 
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_0EB5_0157C5D0.16289A60"
X-Priority: 5
X-mailer: Zwnrqxusy 1
X-Barracuda-Connect: UNKNOWN[61.160.78.18]
X-Barracuda-Start-Time: 1417021601
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 2.50
X-Barracuda-Spam-Status: No, SCORE=2.50 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC5_MJ1963, HTML_FONT_FACE_BAD, HTML_MESSAGE, MISSING_SUBJECT, MISSING_SUBJECT_2, RDNS_NONE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12079
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	0.00 HTML_MESSAGE           BODY: HTML included in message
	0.61 HTML_FONT_FACE_BAD     BODY: HTML font face is not a word
	0.01 MISSING_SUBJECT        Missing Subject: header
	0.10 RDNS_NONE              Delivered to trusted network by a host with no rDNS
	1.28 MISSING_SUBJECT_2      Missing Subject: header
	0.50 BSF_SC5_MJ1963         Custom Rule MJ1963

This is a multi-part message in MIME format.

------=_NextPart_000_0EB5_0157C5D0.16289A60
Content-Type: text/plain;
	charset="utf-8"
Content-Transfer-Encoding: base64

5bel44CB5Yac44CB5bu644CB6YKu44CB5Lit44CB5oub5ZWG44CB5rCR55Sf6ZO26KGM5Y2hDQrl
kKvvvJrlgqjok4TljaHjgIHouqvku73or4HjgIHnvZHpk7bnm77jgIHlvIDmiLfmiYvmnLrljaHv
vIzlvIDmiLfkv6Hmga/nurgNCuS4gOaJi+WNoea6kCzpm7bkuqTmmJPorrDlvZXlronlhajlj6/p
naDjgILnlKjpgJTlub/ms5vvvIzmrKLov47lkqjor6LjgIINCiAgIG4uLiAgQSAgcG9pDQoNCuac
iemcgOimgeeahOivt+WKoFHvvJoxMTMwNjMyNDMgIOaIluiHtOeUtTE1MDAyMDk5MDg5ICBzY2Fs
ZSBuLuagh+W6pu+8m+avlOS+i++8m+Wkp+Wwjw0KDQpicWxxanF0cw==

------=_NextPart_000_0EB5_0157C5D0.16289A60
Content-Type: text/html;
	charset="utf-8"
Content-Transfer-Encoding: base64

PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMCBUcmFuc2l0aW9uYWwv
L0VOIj4NCjxIVE1MPjxIRUFEPg0KPE1FVEEgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0
Zi04IiBodHRwLWVxdWl2PUNvbnRlbnQtVHlwZT4NCjxNRVRBIG5hbWU9R0VORVJBVE9SIGNvbnRl
bnQ9Ik1TSFRNTCA4LjAwLjYwMDEuMjM1MDciPjwvSEVBRD4NCjxCT0RZPjxGT05UIGNvbG9yPXJl
ZCBzaXplPTQgZmFjZT3mpbfkvZM+DQo8RElWPg0KPERJViBpZD1tYWlsQ29udGVudENvbnRhaW5l
ciBjbGFzcz0icW1ib3ggcW1fY29uX2JvZHlfY29udGVudCI+PEZPTlQgY29sb3I9cmVkIA0Kc2l6
ZT00IGZhY2U95qW35L2TPg0KPERJVj48Rk9OVCANCmNvbG9yPXJlZD7lt6XjgIHlhpzjgIHlu7rj
gIHpgq7jgIHkuK3jgIHmi5vllYbjgIHmsJHnlJ/pk7booYzljaE8QlI+5ZCr77ya5YKo6JOE5Y2h
44CB6Lqr5Lu96K+B44CB572R6ZO255u+44CB5byA5oi35omL5py65Y2h77yM5byA5oi35L+h5oGv
57q4PEJSPuS4gOaJi+WNoea6kCzpm7bkuqTmmJPorrDlvZXlronlhajlj6/pnaDjgILnlKjpgJTl
ub/ms5vvvIzmrKLov47lkqjor6LjgII8QlI+PC9GT05UPjxGT05UIA0KY29sb3I9d2hpdGU+Jm5i
c3A7Jm5ic3A7IG4uLiAgQSAgcG9pPC9GT05UPjwvRElWPg0KPERJVj4mbmJzcDs8L0RJVj4NCjxE
SVY+5pyJ6ZyA6KaB55qE6K+35YqgUe+8mjxTUEFOIA0Kc3R5bGU9IlotSU5ERVg6IDE7IEJPUkRF
Ui1CT1RUT006ICNjY2MgMXB4IGRhc2hlZDsgUE9TSVRJT046IHN0YXRpYyI+MTEzMDYzMjQzPC9T
UEFOPiZuYnNwOyANCuaIluiHtOeUtTxTUEFOIHN0eWxlPSJaLUlOREVYOiAxOyBCT1JERVItQk9U
VE9NOiAjY2NjIDFweCBkYXNoZWQ7IFBPU0lUSU9OOiBzdGF0aWMiIA0KdD0iNiIgZGF0YT0iMTUw
MDIwOTkwODkiPjE1MDAyMDk5MDg5PC9TUEFOPiZuYnNwOyA8Rk9OVCANCmNvbG9yPXdoaXRlPnNj
YWxlIG4u5qCH5bqm77yb5q+U5L6L77yb5aSn5bCPPC9GT05UPjwvRElWPg0KPERJVj48Rk9OVCBj
b2xvcj0jZmZmZmZmPjwvRk9OVD4mbmJzcDs8L0RJVj4NCjxESVY+PEZPTlQgY29sb3I9I2ZmZmZm
Zj5mcnJwMzwvRk9OVD48L0RJVj48L0ZPTlQ+PC9ESVY+PCEtLSAtLT4NCjxTVFlMRT4jbWFpbENv
bnRlbnRDb250YWluZXIgLnR4dCB7aGVpZ2h0OmF1dG87fTwvU1RZTEU+DQo8L0RJVj48L0ZPTlQ+
PC9CT0RZPjwvSFRNTD4NCg==

------=_NextPart_000_0EB5_0157C5D0.16289A60--


From pevans@redhat.com  Wed Nov 26 13:24:33 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id C45437F5E
	for ; Wed, 26 Nov 2014 13:24:33 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id B28B78F8050
	for ; Wed, 26 Nov 2014 11:24:30 -0800 (PST)
X-ASG-Debug-ID: 1417029866-04cb6c057373f220001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id D7jyizNiN5xbCD2z (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 26 Nov 2014 11:24:26 -0800 (PST)
X-Barracuda-Envelope-From: pevans@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAQJ00ah011132
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 26 Nov 2014 14:00:01 -0500
Received: from [10.36.7.35] (vpn1-7-35.ams2.redhat.com [10.36.7.35])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAQIxxTf012417;
	Wed, 26 Nov 2014 13:59:59 -0500
Message-ID: <54762329.3080201@redhat.com>
Date: Wed, 26 Nov 2014 18:59:53 +0000
From: Paul Evans 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: pcp@oss.sgi.com, nathan Scott 
Subject: CIFS PMDA
Content-Type: text/plain; charset=utf-8; format=flowed
X-ASG-Orig-Subj: CIFS PMDA
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1417029866
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi,

It was mentioned in the last developers meeting that there is CIFS 
kernel code available that currently is not being exported in PCP.

To add this coverage into PCP I have made an initial version of a CIFS 
PMDA for general review/inclusion and suggestions. This PMDA makes use 
of statistics exported from procfs (/proc/fs/cifs/Stats).

Changes committed to git://github.com/pauljevans/pcp-cifs.git dev

  qa/656                           |  58 ++++++
  qa/656.out                       | 302 +++++++++++++++++++++++++++
  qa/GNUmakefile                   |   2 +-
  qa/cifs/GNUmakefile              |  15 ++
  qa/cifs/GNUmakefile.install      |   1 +
  qa/cifs/cifs-root-2014-11-25.tgz | Bin 0 -> 543 bytes
  qa/group                         |   1 +
  src/pmdas/GNUmakefile            |   2 +-
  src/pmdas/cifs/.gitignore        |   4 +
  src/pmdas/cifs/GNUmakefile       |  61 ++++++
  src/pmdas/cifs/Install           |  30 +++
  src/pmdas/cifs/README            |  64 ++++++
  src/pmdas/cifs/Remove            |  24 +++
  src/pmdas/cifs/help              | 136 +++++++++++++
  src/pmdas/cifs/pmda.c            | 429 
+++++++++++++++++++++++++++++++++++++++
  src/pmdas/cifs/pmdacifs.1        |  67 ++++++
  src/pmdas/cifs/pmdacifs.h        |  41 ++++
  src/pmdas/cifs/pmns              |  58 ++++++
  src/pmdas/cifs/root              |   9 +
  src/pmdas/cifs/stats.c           | 195 ++++++++++++++++++
  src/pmdas/cifs/stats.h           |  71 +++++++
  21 files changed, 1568 insertions(+), 2 deletions(-)

commit 5606ce87e5a2d249efc0fd5e312c73c7f79b3c9b
Author: Paul Evans 
Date:   Wed Nov 26 18:39:45 2014 +0000

     pmdacifs: Initial PMDA Code

     Inital CIFS PMDA code using the stats given in procfs (DSO and Daemon),
     using previously reserved domain 121.

     Exports statistics from the /proc/fs/cifs/Stats file. In this initial
     version we make use of the stats given globally for the cifs module
     along with data provided per each mounted cifs share.

     Requires at least the cifs kernel module to be loaded in order to
     give the global cifs metrics and with each mounted cifs share comes
     it's own instance of metrics.

     QA is given with qa/656 which runs the pmda in DSO mode with a fake
     root given (borrowed from xfs pmda getenv()) to provide testing data
     instead of needed mounted cifs shares on the qa host. Also running
     with valgrind during the test.

     Code checked with coverity and has no reported issues.

All of the patches have been tested and have had covscan run on them. As
always please let me know if there are any issues and feedback is 
welcome :).

Cheers

Paul

From fche@redhat.com  Wed Nov 26 20:06:45 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 39DF37F4E
	for ; Wed, 26 Nov 2014 20:06:45 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 28A51304059
	for ; Wed, 26 Nov 2014 18:06:44 -0800 (PST)
X-ASG-Debug-ID: 1417054000-04cb6c05717717a0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 4VrrDUzNC4WmFIdy (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 26 Nov 2014 18:06:40 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAR26d73016158
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Wed, 26 Nov 2014 21:06:39 -0500
Received: from fche.csb (vpn-238-111.phx2.redhat.com [10.3.238.111])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAR26dtf020741
	for ; Wed, 26 Nov 2014 21:06:39 -0500
Received: by fche.csb (Postfix, from userid 2569)
	id C8DED58259; Wed, 26 Nov 2014 21:06:38 -0500 (EST)
Date: Wed, 26 Nov 2014 21:06:38 -0500
From: "Frank Ch. Eigler" 
To: pcp developers 
Subject: linux_proc caches; error handling; segv upon unprivileged proc.* reads
Message-ID: <20141127020638.GH5088@redhat.com>
X-ASG-Orig-Subj: linux_proc caches; error handling; segv upon unprivileged proc.* reads
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.2i
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1417054000
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi -

I'd like to share some notes of analysis of how come some gittish
builds of pcp cause a linux_proc pmda segv during normal operations,
or during the qa/022 test case (when invoked as pcpqa).  The common
element seems to be invocation as root vs. an unprivileged userid.
This occurs on a fedora rawhide VM, i.e.:

Linux vm-rawhide-64 3.18.0-rc2-00053-g9f76628da20f #1 SMP Wed Nov 12 15:12:35 EST 2014 x86_64 x86_64 x86_64 GNU/Linux



% pcp 
[...] Version 3.10.1-1, 8 agents, 2 clients
% rpm -q pcp
pcp-3.10.1-0.759.gf6d6a93.fc22.x86_64
% sudo service pmcd restart
% sudo pminfo -f proc.memory.maps
[... ok ...]
% pminfo -f proc.memory.maps
proc.memory.maps: pmLookupDesc: No PMCD agent for domain of request

indeed, the procpmda process exits with a SEGV thusly:

Program received signal SIGSEGV, Segmentation fault.
0x00007f4cf9bcad8a in strlen () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f4cf9bcad8a in strlen () from /lib64/libc.so.6
#1  0x00007f4cf9f3c66b in __pmStuffValue (avp=avp@entry=0x7fff088f9fb0, vp=vp@entry=0x1eaaf80, type=type@entry=6) at stuffvalue.c:55
#2  0x00007f4cfa181e71 in pmdaFetch (numpmid=numpmid@entry=1, pmidlist=pmidlist@entry=0x1e8a01c, resp=resp@entry=0x7fff088fa1c8, pmda=pmda@entry=0x1e84010)
    at callback.c:603
#3  0x0000000000402dc3 in proc_fetch (numpmid=1, pmidlist=0x1e8a01c, resp=0x7fff088fa1c8, pmda=0x1e84010) at pmda.c:2393
#4  0x00007f4cfa184572 in __pmdaMainPDU (dispatch=dispatch@entry=0x7fff088fa270) at mainloop.c:179
#5  0x00007f4cfa184e08 in pmdaMain (dispatch=dispatch@entry=0x7fff088fa270) at mainloop.c:428
#6  0x00000000004025f7 in main (argc=6, argv=) at pmda.c:2668

This happens because the input pmAtom* is unfilled:

(gdb) frame 1
#1  0x00007f4cf9f3c66b in __pmStuffValue (avp=avp@entry=0x7fff088f9fb0, vp=vp@entry=0x1eaaf80, type=type@entry=6) at stuffvalue.c:55
55                         body = strlen(avp->cp) + 1;
(gdb) p *avp
$2 = {l = 0, ul = 0, ll = 0, ull = 0, f = 0, d = 0, cp = 0x0, vbp = 0x0}

This happens because proc_fetchCallBack returns sts >= 0 but fails to
fill in the output atom.

This happens because fetch_proc_pid_maps returns an incomplete pointer
(a proc_pid_entry_t with a NULL maps_buf) *and* a 0 sts.

$9 = {id = 481, flags = 1, name = 0x1ca3ee0 "000481 avahi-daemon:
  running [vm-rawhide-64.local]", stat_buflen = 0, stat_buf = 0x0,
  statm_buflen = 0, statm_buf = 0x0, maps_buflen = 0, maps_buf = 0x0,
  status_buflen = 0, status_buf = 0x0, status_lines = {uid = 0x0, gid
  = 0x0, sigpnd = 0x0, sigblk = 0x0, sigign = 0x0, sigcgt = 0x0,
  vmsize = 0x0, vmlck = 0x0, vmrss = 0x0, vmdata = 0x0, vmstk = 0x0,
  vmexe = 0x0, vmlib = 0x0, vmswap = 0x0, threads = 0x0, vctxsw = 0x0,
  nvctxsw = 0x0}, schedstat_buflen = 0, schedstat_buf = 0x0, io_buflen
  = 0, io_buf = 0x0, io_lines = {rchar = 0x0, wchar = 0x0, syscr =
  0x0, syscw = 0x0, readb = 0x0, writeb = 0x0, cancel = 0x0},
  wchan_buflen = 0, wchan_buf = 0x0, fd_buflen = 0, fd_count = 0,
  fd_buf = 0x0, cgroup_id = 0, label_id = 0}

This happens because when fetch_proc_pid_maps tries to populate that
data structure on behalf of the client (so already temp-seteuid'd),
proc_open("maps", ep) comes back with an error, but leaves *sts=0 and
skips the rest of the function without ->maps_buf init.  This is
almost certainly a bug (->maps_buf should be set).

The *sts=0 part happens because maperr() hides (only!) EACCES & EINVAL
errors and maps them to *sts=0.  This is mysterious.  This is the most
salient place the pmda could pass knowledge up toward libpcp_pmda &
the client to identify this metric instance as missing (e.g.,
PM_ERR_INST).  Instead it pretends everything's OK, enabling the above
bug.

This logic was changed as a part of commit #85d06e790e, and has seen
at least one full release.  I didn't initially figure out why this is 
affecting only my VM instead of every recent pcp build & machine.  It
turns out that this could be a kernel version difference.  An strace
of the same type of operation on an older 3.14.9 kernel shows:

  open("/proc/31830/maps", O_RDONLY)      = 5
  read(5, 0x7fff8e9833c0, 1024)           = -1 EACCES (Permission denied)

whereas on the new 3.18-rc:

  open("/proc/481/maps", O_RDONLY)        = -1 EACCES (Permission denied)

So it now fails earlier.


On the virtual machine, the qa/022 test passes if run as root, but
fails if run as pcpqa.  How do other people run the testsuite?  It
seems like we need the test extended to run both ways, and better
describe such invocation issues in qa/README.


Reading related code got me thinking.
 
The ACL idea for the linux_proc pmda is simply that a user invoking
proc.* fetches should see (or not see) the exact same amount of data
as she would if looking directly at /proc.  The linux_proc pmda
doesn't quite accomplish that.  One possible problem is that it keeps
a shared hash-table of process-data (pidhash), which could only be
fully populated by root.  If that data happens to be retained (ie.,
escapes a refresh, or ep->flags | PROC_PID_..._FETCHED), then any
subsequent less-privileged pcp client user can copy out the contents!
Any temp-seteuid access control effort is moot:

% hostname
HOST
% service pmcd restart
% sudo pmval -i 1 -f proc.memory.maps
[... prime the cache ...]
then thereafter, from an unprivileged userid:
% pmval -i 1 -f proc.memory.maps

will return the cached information, even though the second
client has no business seeing all that information!

We need to revisit the general issue of these caches in the proc_pmda,
from a security point of view, not just the scaling problems
identified at .


- FChE

From shirshendu@riva.co  Thu Nov 27 00:17:38 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 5EC8C7F4E
	for ; Thu, 27 Nov 2014 00:17:38 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 3E924304039
	for ; Wed, 26 Nov 2014 22:17:35 -0800 (PST)
X-ASG-Debug-ID: 1417069052-04cb6c05727859d0001-S8gJnT
Received: from mail-qa0-f42.google.com (mail-qa0-f42.google.com [209.85.216.42]) by cuda.sgi.com with ESMTP id l34zW69KGkc9QlCH (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Wed, 26 Nov 2014 22:17:33 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.42
Received: by mail-qa0-f42.google.com with SMTP id j7so2967878qaq.29
        for ; Wed, 26 Nov 2014 22:17:32 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:content-type;
        bh=JpKC5Om68RcMCjv9VbetTUVO5FcOW5NPD9iH7sjw3Lk=;
        b=dkZYxtngQn8OUr0fobl7cu++ERGA03sAdjiKSQ/a0GBHKLqRcdu8DIb0HUohB2uStK
         13ZQ3iZimsEUqYdvHVco7o3beoQvMpcPsuciCckt72EUNbr4knivO7Eua1O979U9QeIM
         5idPmweWkLZcbToXI0l9Yr0oqm6pnXVNXAum4E7arXaBGAEQDu3MsGn0XrtKgv5ooY26
         tFu8qriJmBaopo5L7e+C6VUQeh+2zlhv8sRq7k39Zzv0/XBq1yzT8bwfxaEseq7PFi0m
         lCbX0ordRPPUmd6HdR3Vs7V8FcP9iaqBGVzrULjzp3+zpXwc22hMUo8BHlWU2OMPaMLr
         a2ug==
X-Gm-Message-State: ALoCoQk3ZQN17SmPVNqPoWCqmbvjY6xWJ6W9Ex6P/12c6BKbnSncSmYe+TC9X1y1ZRBgr38ZOvdy
MIME-Version: 1.0
X-Received: by 10.140.88.100 with SMTP id s91mr50034488qgd.65.1417069052317;
 Wed, 26 Nov 2014 22:17:32 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Wed, 26 Nov 2014 22:17:32 -0800 (PST)
In-Reply-To: 
References: 
Date: Thu, 27 Nov 2014 11:47:32 +0530
Message-ID: 
Subject: Re: Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c119907c6a9b0508d1190d
X-Barracuda-Connect: mail-qa0-f42.google.com[209.85.216.42]
X-Barracuda-Start-Time: 1417069053
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12106
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c119907c6a9b0508d1190d
Content-Type: text/plain; charset=UTF-8

Hi PCP team,

Could anyone please respond the question above :)

Thanks,

Shirshendu Chakrabarti

On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti 
wrote:

> Hi PCP team,
>
> Could anyone in the team point me to any literature which explains the
> difference between a per node metric and percpu metric. The one-liner and
> extended description are absent in pernode metric case and terse in percpu
> case.
>
> For example:
>
> [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.user
>
> kernel.percpu.cpu.user
>     inst [0 or "cpu0"] value 993240
> [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
>
> kernel.pernode.cpu.user
>     inst [0 or "node0"] value 0
>
> I was under the assumption that, kernel.pernode.* and kernel.all.* metrics
> are the same. I am clearly missing something important.
>
> Thanks,
>
> Shirshendu Chakrabarti
>

--001a11c119907c6a9b0508d1190d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi PCP team,

Could anyone please respon=
d the question above :)

Thanks,

Shirshendu Chakrabarti

On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrab=
arti <shirshendu@riva.co> wrote:
Hi PCP team,

Could anyone in the team=
 point me to any literature which explains the difference between a per nod=
e metric and percpu metric. The one-liner and extended description are abse=
nt in pernode metric case and terse in percpu case.

For example:

[root@pcp-test-shir3 pmlogger]# pmi=
nfo -f kernel.percpu.cpu.user

kernel.percpu.cpu.us=
er
=C2=A0 =C2=A0 inst [0 or "cpu0"] value 993240
<=
div>[root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
=


kernel.pernode.cpu.user
=C2=A0 =C2=A0 inst [0=
 or "node0"] value 0
=C2=A0
I was under=
 the assumption that, kernel.pernode.* and kernel.all.* metrics are the sam=
e. I am clearly missing something important.

Thank=
s,

Shirshendu Chakrabarti




--001a11c119907c6a9b0508d1190d--

From nicholaskevin668@gmail.com  Thu Nov 27 00:48:38 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: *
X-Spam-Status: No, score=1.3 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,
	FREEMAIL_FROM,FREEMAIL_REPLYTO,T_DKIM_INVALID autolearn=no version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 4AAC07F4E
	for ; Thu, 27 Nov 2014 00:48:38 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id CA96CAC002
	for ; Wed, 26 Nov 2014 22:48:37 -0800 (PST)
X-ASG-Debug-ID: 1417070916-04cbb01e5c77d360001-S8gJnT
Received: from mail-ig0-f196.google.com (mail-ig0-f196.google.com [209.85.213.196]) by cuda.sgi.com with ESMTP id B3WcXCSOytOnKzHR (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Wed, 26 Nov 2014 22:48:36 -0800 (PST)
X-Barracuda-Envelope-From: nicholaskevin668@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.213.196
X-Barracuda-IPDD: Level1 [gmail.com/209.85.213.196]
Received: by mail-ig0-f196.google.com with SMTP id h15so585357igd.7
        for ; Wed, 26 Nov 2014 22:48:36 -0800 (PST)
X-Barracuda-IPDD: Level1 [gmail.com/209.85.213.196]
X-Barracuda-IPDD: Level1 [gmail.com/209.85.213.196]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:reply-to:date:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        bh=k09Cnpy+otTZ3i01wRMLkdTl6a0Tq2KXMv3fLdJ/fmo=;
        b=OCLmiOz2yPMcWqJqLGJpaI9269V6aaEcX8FayBXPSZUSjzSlR1n1DRKp57m0WxvuWY
         Be9Ci1gkWE6bBbSPhKgpbyWChZgJMmOeQwhn+5xmUDA8hBkJxft91CESNRqepnKduerh
         K7ax7h/e3Q4Q9XqK6xYVWqR4lDeRuMsbvde2TBASvm/Kdz9Cb1oiELJQAV/sVOBS3dn5
         upGA1zmCVMCz6Wlrx8cDfhxaErdUyLiBOfFYOTUiSPa6/C/2W8uPGJqdvgo1httbel7m
         jRCenfvu/4MGG7YYjO07PufA+C855IGV7NqkfKdPSVosO6eR/4Zw/hqS0BqA089JM9+7
         TG7A==
MIME-Version: 1.0
X-Received: by 10.107.163.142 with SMTP id m136mr31521017ioe.86.1417070916096;
 Wed, 26 Nov 2014 22:48:36 -0800 (PST)
Received: by 10.64.77.98 with HTTP; Wed, 26 Nov 2014 22:48:36 -0800 (PST)
Reply-To: barclayslender668@outlook.com
Date: Wed, 26 Nov 2014 22:48:36 -0800
Message-ID: 
Subject: =?UTF-8?B?Z8O2cmEgbmkgQmVow7Z2ZXIgZHUgbMOlbj8=?=
From: Barclays lender 
X-ASG-Orig-Subj: =?UTF-8?B?Z8O2cmEgbmkgQmVow7Z2ZXIgZHUgbMOlbj8=?=
To: undisclosed-recipients:;
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Barracuda-Connect: mail-ig0-f196.google.com[209.85.213.196]
X-Barracuda-Start-Time: 1417070916
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED, MAILTO_TO_SPAM_ADDR
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12107
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature
	0.00 MAILTO_TO_SPAM_ADDR    URI: Includes a link to a likely spammer email

--=20


--=20
Caro,

Oferta de empr=C3=A9stimo =C3=A0 taxa de juros de 3%, eu sou nicholas uma e=
quipe
de base de unitedleader no Reino Unido. Voc=C3=AA precisa de um empr=C3=A9s=
timo
de qualquer tipo, por favor contacte-nos para ajudar.

Precisa de informa=C3=A7=C3=B5es para processamento:

nome:
pa=C3=ADs:
Sexo:
Finalidade do empr=C3=A9stimo de:
Montante necess=C3=A1rio:
dura=C3=A7=C3=A3o:
E-mail:
Celular:
Renda mensal:

Tudo em informa=C3=A7=C3=A3o tem que enviar para
unitedlender1010@outlook.com.com esperamos para o seu
responder para um melhor suporte.

obrigado
Nicholos

From anne.dai@lklchina.com  Thu Nov 27 01:28:28 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_FONT_FACE_BAD,
	HTML_MESSAGE autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 300DF7F4E
	for ; Thu, 27 Nov 2014 01:28:28 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id E8213304039
	for ; Wed, 26 Nov 2014 23:28:23 -0800 (PST)
X-ASG-Debug-ID: 1417073296-04bdf0615e909ef0001-S8gJnT
Received: from mail.lklchina.com (mail.lklchina.com [58.215.224.165]) by cuda.sgi.com with ESMTP id h1dATshsoi5fxs3H for ; Wed, 26 Nov 2014 23:28:17 -0800 (PST)
X-Barracuda-Envelope-From: anne.dai@lklchina.com
X-Barracuda-Apparent-Source-IP: 58.215.224.165
Received: (qmail 16089 invoked by uid 889); 27 Nov 2014 13:39:45 +0800
Received: by simscan 1.4.0 ppid: 16081, pid: 16086, t: 0.0271s
         scanners: attach: 1.4.0 clamav: 0.95.2/m:51/d:9450
Received: from unknown (HELO kejqnyx) (anne.dai@lklchina.com@58.221.58.13)
  by mail.lklchina.com with ESMTPA; 27 Nov 2014 13:39:45 +0800
Message-ID: <20141127152824015518@lklchina.com>
From: =?utf-8?B?5ZCV56aP5b+g?= 
To: 
Subject: 
Date: Thu, 27 Nov 2014 15:28:13 +0800
X-ASG-Orig-Subj: 
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_0B3C_01ABE04E.14F1BF40"
X-Priority: 5
X-mailer: Uvf 2
X-Barracuda-Connect: mail.lklchina.com[58.215.224.165]
X-Barracuda-Start-Time: 1417073297
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 1.90
X-Barracuda-Spam-Status: No, SCORE=1.90 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, HTML_FONT_FACE_BAD, HTML_MESSAGE, MISSING_SUBJECT, MISSING_SUBJECT_2
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12108
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 BSF_SC0_MISMATCH_TO    Envelope rcpt doesn't match header
	0.00 HTML_MESSAGE           BODY: HTML included in message
	0.61 HTML_FONT_FACE_BAD     BODY: HTML font face is not a word
	0.01 MISSING_SUBJECT        Missing Subject: header
	1.28 MISSING_SUBJECT_2      Missing Subject: header

This is a multi-part message in MIME format.

------=_NextPart_000_0B3C_01ABE04E.14F1BF40
Content-Type: text/plain;
	charset="utf-8"
Content-Transfer-Encoding: base64

5bel44CB5Yac44CB5bu644CB6YKu44CB5Lit44CB5oub5ZWG44CB5rCR55Sf6ZO26KGM5Y2hDQrl
kKvvvJrlgqjok4TljaHjgIHouqvku73or4HjgIHnvZHpk7bnm77jgIHlvIDmiLfmiYvmnLrljaHv
vIzlvIDmiLfkv6Hmga/nurgNCuS4gOaJi+WNoea6kCzpm7bkuqTmmJPorrDlvZXlronlhajlj6/p
naDjgILnlKjpgJTlub/ms5vvvIzmrKLov47lkqjor6LjgIINCiAgIG4ucHVibGljIGtleSBlDQoN
CuaciemcgOimgeeahOivt+WKoFHvvJoxMTMwNjMyNDMgIOaIluiHtOeUtTE1MDAyMDk5MDg5ICB2
YXBvdXJuLuaxve+8jOiSuOawlA0KDQp4d3h2Nmo=

------=_NextPart_000_0B3C_01ABE04E.14F1BF40
Content-Type: text/html;
	charset="utf-8"
Content-Transfer-Encoding: base64

PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMCBUcmFuc2l0aW9uYWwv
L0VOIj4NCjxIVE1MPjxIRUFEPg0KPE1FVEEgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0
Zi04IiBodHRwLWVxdWl2PUNvbnRlbnQtVHlwZT4NCjxNRVRBIG5hbWU9R0VORVJBVE9SIGNvbnRl
bnQ9Ik1TSFRNTCA4LjAwLjYwMDEuMjM1MDciPjwvSEVBRD4NCjxCT0RZPjxGT05UIGNvbG9yPXJl
ZCBzaXplPTQgZmFjZT3mpbfkvZM+DQo8RElWPg0KPERJViBpZD1tYWlsQ29udGVudENvbnRhaW5l
ciBjbGFzcz0icW1ib3ggcW1fY29uX2JvZHlfY29udGVudCI+PEZPTlQgY29sb3I9cmVkIA0Kc2l6
ZT00IGZhY2U95qW35L2TPg0KPERJVj48Rk9OVCANCmNvbG9yPXJlZD7lt6XjgIHlhpzjgIHlu7rj
gIHpgq7jgIHkuK3jgIHmi5vllYbjgIHmsJHnlJ/pk7booYzljaE8QlI+5ZCr77ya5YKo6JOE5Y2h
44CB6Lqr5Lu96K+B44CB572R6ZO255u+44CB5byA5oi35omL5py65Y2h77yM5byA5oi35L+h5oGv
57q4PEJSPuS4gOaJi+WNoea6kCzpm7bkuqTmmJPorrDlvZXlronlhajlj6/pnaDjgILnlKjpgJTl
ub/ms5vvvIzmrKLov47lkqjor6LjgII8QlI+PC9GT05UPjxGT05UIA0KY29sb3I9d2hpdGU+Jm5i
c3A7Jm5ic3A7IG4ucHVibGljIGtleSBlPC9GT05UPjwvRElWPg0KPERJVj4mbmJzcDs8L0RJVj4N
CjxESVY+5pyJ6ZyA6KaB55qE6K+35YqgUe+8mjxTUEFOIA0Kc3R5bGU9IlotSU5ERVg6IDE7IEJP
UkRFUi1CT1RUT006ICNjY2MgMXB4IGRhc2hlZDsgUE9TSVRJT046IHN0YXRpYyI+MTEzMDYzMjQz
PC9TUEFOPiZuYnNwOyANCuaIluiHtOeUtTxTUEFOIHN0eWxlPSJaLUlOREVYOiAxOyBCT1JERVIt
Qk9UVE9NOiAjY2NjIDFweCBkYXNoZWQ7IFBPU0lUSU9OOiBzdGF0aWMiIA0KdD0iNiIgZGF0YT0i
MTUwMDIwOTkwODkiPjE1MDAyMDk5MDg5PC9TUEFOPiZuYnNwOyA8Rk9OVCANCmNvbG9yPXdoaXRl
PnZhcG91cm4u5rG977yM6JK45rCUPC9GT05UPjwvRElWPg0KPERJVj48Rk9OVCBjb2xvcj0jZmZm
ZmZmPjwvRk9OVD4mbmJzcDs8L0RJVj4NCjxESVY+PEZPTlQgY29sb3I9I2ZmZmZmZj50a3hjZ3F0
MDwvRk9OVD48L0RJVj48L0ZPTlQ+PC9ESVY+PCEtLSAtLT4NCjxTVFlMRT4jbWFpbENvbnRlbnRD
b250YWluZXIgLnR4dCB7aGVpZ2h0OmF1dG87fTwvU1RZTEU+DQo8L0RJVj48L0ZPTlQ+PC9CT0RZ
PjwvSFRNTD4NCg==

------=_NextPart_000_0B3C_01ABE04E.14F1BF40--

From mgoodwin@redhat.com  Thu Nov 27 02:02:31 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 865297F4E
	for ; Thu, 27 Nov 2014 02:02:31 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 665388F8054
	for ; Thu, 27 Nov 2014 00:02:28 -0800 (PST)
X-ASG-Debug-ID: 1417075346-04cbb01e5a7841b0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id ziZCMVTgfa3cb3Kb (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 27 Nov 2014 00:02:27 -0800 (PST)
X-Barracuda-Envelope-From: mgoodwin@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAR82Qr3030334
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL);
	Thu, 27 Nov 2014 03:02:26 -0500
Received: from [10.64.48.240] (vpn1-48-240.bne.redhat.com [10.64.48.240])
	by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAR82OSb029596;
	Thu, 27 Nov 2014 03:02:25 -0500
Message-ID: <5476DA8F.2060202@redhat.com>
Date: Thu, 27 Nov 2014 19:02:23 +1100
From: Mark Goodwin 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Shirshendu Chakrabarti , pcp@oss.sgi.com
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
References:  
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1417075347
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/27/2014 05:17 PM, Shirshendu Chakrabarti wrote:
> Hi PCP team,
>
> Could anyone please respond the question above :)

The difference is the instance domains and counter aggregation.

kernel.pernode is aggregated over all CPUs on each numa node.
On my laptop with 4 CPU cores on one (fake) numa node, we should
expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and
the sum of the per-cpu counters to equal the same, e.g. :

# pminfo -f kernel.{all,pernode,percpu}.cpu.user

kernel.all.cpu.user
     value 254548080

kernel.pernode.cpu.user
     inst [0 or "node0"] value 254548060

kernel.percpu.cpu.user
     inst [0 or "cpu0"] value 66377110
     inst [1 or "cpu1"] value 61187360
     inst [2 or "cpu2"] value 64051110
     inst [3 or "cpu3"] value 62932480

The mapping between CPUs and nodes is in hinv.map.cpu_node,
where each CPU (instance) is mapped to a node number, e.g.

# pminfo -f hinv.map.cpu_node

hinv.map.cpu_node
     inst [0 or "cpu0"] value 0
     inst [1 or "cpu1"] value 0
     inst [2 or "cpu2"] value 0
     inst [3 or "cpu3"] value 0

In your case, kernel.pernode.cpu.user is zero, which isn't correct.
What platform and PCP version are you running?

Regards
-- Mark

>
> Thanks,
>
> Shirshendu Chakrabarti
>
> On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti  > wrote:
>
>     Hi PCP team,
>
>     Could anyone in the team point me to any literature which explains the
>     difference between a per node metric and percpu metric. The one-liner and
>     extended description are absent in pernode metric case and terse in percpu case.
>
>     For example:
>
>     [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.user
>
>     kernel.percpu.cpu.user
>          inst [0 or "cpu0"] value 993240
>     [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
>
>     kernel.pernode.cpu.user
>          inst [0 or "node0"] value 0
>     I was under the assumption that, kernel.pernode.* and kernel.all.* metrics
>     are the same. I am clearly missing something important.
>
>     Thanks,
>
>     Shirshendu Chakrabarti
>
>
>
>
> _______________________________________________
> pcp mailing list
> pcp@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/pcp
>

From shirshendu@riva.co  Thu Nov 27 02:34:27 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id B00387F4E
	for ; Thu, 27 Nov 2014 02:34:27 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 5D138304053
	for ; Thu, 27 Nov 2014 00:34:27 -0800 (PST)
X-ASG-Debug-ID: 1417077264-04bdf06160910cd0001-S8gJnT
Received: from mail-qc0-f173.google.com (mail-qc0-f173.google.com [209.85.216.173]) by cuda.sgi.com with ESMTP id DuZDo5czG0QreWEs (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 27 Nov 2014 00:34:25 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.173
Received: by mail-qc0-f173.google.com with SMTP id i17so3262275qcy.32
        for ; Thu, 27 Nov 2014 00:34:24 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=z4ibegrXxw3wXzlLo+K79eFeKxqGcJpTNy9kAAiaSv4=;
        b=DK1mZAhTYmZzDvNVBZAWlOyezY+ZO7kshWHsfcvt5jCXum4eiqWfLsB6CuBLTYk4Cy
         Yv+625bJYPTdgIHtDO0234nGdmVdtILpTSIPlGZkUMZsVqcNO3JOvbJmWZSfDIqN1fNU
         GW+/6JMytzDpvbk6lnNmykEzp7wWVbjBX32DUwWHouJ4LitFR1nhT81KcDzUlZFD0ypj
         TvLLKL7xJjzWHpKtI7lrf6pQGs3s8qzOyBmy4DmH5gi+NNt2GVKoI0mxtWhodElkXJjE
         SaPaAn6mdloNx3t7jdy1diFDLUFi0NZ69p67Qzmp7y+e7YqoV3oSfGRk2N/zm6m/IiI3
         Adcg==
X-Gm-Message-State: ALoCoQmnyy9kyWquu7yhYBPF5M+9saaiHZ87bb6t3b+88ookQdsLYJXCkYqxzrb2JPmybjHAIAzi
MIME-Version: 1.0
X-Received: by 10.224.65.4 with SMTP id g4mr53156305qai.20.1417077264552; Thu,
 27 Nov 2014 00:34:24 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 27 Nov 2014 00:34:24 -0800 (PST)
In-Reply-To: <5476DA8F.2060202@redhat.com>
References: 
	
	<5476DA8F.2060202@redhat.com>
Date: Thu, 27 Nov 2014 14:04:24 +0530
Message-ID: 
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: Mark Goodwin 
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2b980f92d630508d30209
X-Barracuda-Connect: mail-qc0-f173.google.com[209.85.216.173]
X-Barracuda-Start-Time: 1417077265
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12110
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2b980f92d630508d30209
Content-Type: text/plain; charset=UTF-8

Hi Mark,

Thanks  for the explanation.

I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64

I am running, PCP-3-10.0-1.

Please see its a t1.micro instance.

For more details:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts_micro_instances.html

I have a few other questions:

1. Does PCP have any different requirements for running on the cloud.
2. Do the metrics need to be interpreted differently when running on cloud.

Thanks,

Shirshendu Chakrabarti

On Thu, Nov 27, 2014 at 1:32 PM, Mark Goodwin  wrote:

> On 11/27/2014 05:17 PM, Shirshendu Chakrabarti wrote:
>
>> Hi PCP team,
>>
>> Could anyone please respond the question above :)
>>
>
> The difference is the instance domains and counter aggregation.
>
> kernel.pernode is aggregated over all CPUs on each numa node.
> On my laptop with 4 CPU cores on one (fake) numa node, we should
> expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and
> the sum of the per-cpu counters to equal the same, e.g. :
>
> # pminfo -f kernel.{all,pernode,percpu}.cpu.user
>
> kernel.all.cpu.user
>     value 254548080
>
> kernel.pernode.cpu.user
>     inst [0 or "node0"] value 254548060
>
> kernel.percpu.cpu.user
>     inst [0 or "cpu0"] value 66377110
>     inst [1 or "cpu1"] value 61187360
>     inst [2 or "cpu2"] value 64051110
>     inst [3 or "cpu3"] value 62932480
>
> The mapping between CPUs and nodes is in hinv.map.cpu_node,
> where each CPU (instance) is mapped to a node number, e.g.
>
> # pminfo -f hinv.map.cpu_node
>
> hinv.map.cpu_node
>     inst [0 or "cpu0"] value 0
>     inst [1 or "cpu1"] value 0
>     inst [2 or "cpu2"] value 0
>     inst [3 or "cpu3"] value 0
>
> In your case, kernel.pernode.cpu.user is zero, which isn't correct.
> What platform and PCP version are you running?
>
> Regards
> -- Mark
>
>
>> Thanks,
>>
>> Shirshendu Chakrabarti
>>
>> On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti <
>> shirshendu@riva.co
>> > wrote:
>>
>>     Hi PCP team,
>>
>>     Could anyone in the team point me to any literature which explains the
>>     difference between a per node metric and percpu metric. The one-liner
>> and
>>     extended description are absent in pernode metric case and terse in
>> percpu case.
>>
>>     For example:
>>
>>     [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.user
>>
>>     kernel.percpu.cpu.user
>>          inst [0 or "cpu0"] value 993240
>>     [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
>>
>>     kernel.pernode.cpu.user
>>          inst [0 or "node0"] value 0
>>     I was under the assumption that, kernel.pernode.* and kernel.all.*
>> metrics
>>     are the same. I am clearly missing something important.
>>
>>     Thanks,
>>
>>     Shirshendu Chakrabarti
>>
>>
>>
>>
>> _______________________________________________
>> pcp mailing list
>> pcp@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/pcp
>>
>>
>

--001a11c2b980f92d630508d30209
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Mark,

Thanks =C2=A0for the explanati=
on.

I am running on Amazon Linux - 3.4.71-63.amzn1=
.x86_64

I am running, PCP-3-10.0-1.

=


<=
div>I have a few other questions:

1. Does PCP have=
 any different requirements for running on the cloud.=C2=A0
2. Do=
 the metrics need to be interpreted differently when running on cloud.

Thanks,

Shirshendu Chakrabart=
i

On T=
hu, Nov 27, 2014 at 1:32 PM, Mark Goodwin <mgoodwin@redhat.com> wrote:
On 11/27/20=
14 05:17 PM, Shirshendu Chakrabarti wrote:



Hi PCP team,



Could anyone please respond the question above :)





The difference is the instance domains and counter aggregation.



kernel.pernode is aggregated over all CPUs on each numa node.

On my laptop with 4 CPU cores on one (fake) numa node, we should

expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and

the sum of the per-cpu counters to equal the same, e.g. :



# pminfo -f kernel.{all,pernode,percpu}.cpu.user



kernel.all.cpu.user

=C2=A0 =C2=A0 value 254548080



kernel.pernode.cpu.user

=C2=A0 =C2=A0 inst [0 or "node0"] value 254548060



kernel.percpu.cpu.user

=C2=A0 =C2=A0 inst [0 or "cpu0"] value 66377110

=C2=A0 =C2=A0 inst [1 or "cpu1"] value 61187360

=C2=A0 =C2=A0 inst [2 or "cpu2"] value 64051110

=C2=A0 =C2=A0 inst [3 or "cpu3"] value 62932480



The mapping between CPUs and nodes is in hinv.map.cpu_node,

where each CPU (instance) is mapped to a node number, e.g.



# pminfo -f hinv.map.cpu_node



hinv.map.cpu_node

=C2=A0 =C2=A0 inst [0 or "cpu0"] value 0

=C2=A0 =C2=A0 inst [1 or "cpu1"] value 0

=C2=A0 =C2=A0 inst [2 or "cpu2"] value 0

=C2=A0 =C2=A0 inst [3 or "cpu3"] value 0



In your case, kernel.pernode.cpu.user is zero, which isn't correct.

What platform and PCP version are you running?



Regards

-- Mark







Thanks,



Shirshendu Chakrabarti



On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti <shirshendu@riva.co
<=
span class=3D"">
<mailto:shirshen=
du@riva.co>> wrote:



=C2=A0 =C2=A0 Hi PCP team,



=C2=A0 =C2=A0 Could anyone in the team point me to any literature which exp=
lains the

=C2=A0 =C2=A0 difference between a per node metric and percpu metric. The o=
ne-liner and

=C2=A0 =C2=A0 extended description are absent in pernode metric case and te=
rse in percpu case.



=C2=A0 =C2=A0 For example:



=C2=A0 =C2=A0 [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.u=
ser



=C2=A0 =C2=A0 kernel.percpu.cpu.user

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inst [0 or "cpu0"] value 993240=


=C2=A0 =C2=A0 [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.=
user



=C2=A0 =C2=A0 kernel.pernode.cpu.user

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inst [0 or "node0"] value 0

=C2=A0 =C2=A0 I was under the assumption that, kernel.pernode.* and kernel.=
all.* metrics

=C2=A0 =C2=A0 are the same. I am clearly missing something important.



=C2=A0 =C2=A0 Thanks,



=C2=A0 =C2=A0 Shirshendu Chakrabarti









_______________________________________________

pcp mailing list

pcp@oss.sgi.com
http:=
//oss.sgi.com/mailman/listinfo/pcp











--001a11c2b980f92d630508d30209--

From shirshendu@riva.co  Thu Nov 27 02:50:45 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 894D37F4E
	for ; Thu, 27 Nov 2014 02:50:45 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id EC886AC002
	for ; Thu, 27 Nov 2014 00:50:41 -0800 (PST)
X-ASG-Debug-ID: 1417078239-04cbb01e5b788d30001-S8gJnT
Received: from mail-qa0-f49.google.com (mail-qa0-f49.google.com [209.85.216.49]) by cuda.sgi.com with ESMTP id iuJfAcDdyLlRdIPE (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 27 Nov 2014 00:50:39 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.49
Received: by mail-qa0-f49.google.com with SMTP id s7so3014299qap.36
        for ; Thu, 27 Nov 2014 00:50:39 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=xBXcZfLqZzi/2ovPyyMDahej6B626+SXbFaezfjUIo8=;
        b=G/hjuoh9DdTOtlxrSycDDuZX7P1DmYlv81f69HpG6Wt4L6aSAT5wNgug/DZh/ORz9L
         h+Tud5O7j5N9i71DyD668Jy6zX28ojneQD9Kqs26VKlVj+QwMXvQ00U8vVYWsnc2eIEM
         sdi3TLoh25Xzk1UXJpZOVjxT3FYvKkTNGGGsMnuHKnxAfmkSA27Ou8KUQtGbQgbcq0VW
         3zi+AoXUTvOL3ry7hMmBoJXIj8cQT+Q7NPX+ZE8PYSWUAHMkKcEwOvFXxSWKv5WZj3Lz
         9yDQVRoGbrLE+o+BbPQ5oJZMrU0AfHJcqT1T2b8TIApf1Pl4Jx1qg+PHDAlB8gzlEOuV
         7Qvw==
X-Gm-Message-State: ALoCoQk0Wu8lmM5s0EqoEeZKdoQg2utc+bkUHIrpoHg7A2cXtBWNrFOucyGuqmq1s4NymUm5ie6u
MIME-Version: 1.0
X-Received: by 10.224.88.193 with SMTP id b1mr52890899qam.30.1417078239138;
 Thu, 27 Nov 2014 00:50:39 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 27 Nov 2014 00:50:39 -0800 (PST)
In-Reply-To: 
References: 
	
	<5476DA8F.2060202@redhat.com>
	
Date: Thu, 27 Nov 2014 14:20:39 +0530
Message-ID: 
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: Mark Goodwin 
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2d758102c6d0508d33dbe
X-Barracuda-Connect: mail-qa0-f49.google.com[209.85.216.49]
X-Barracuda-Start-Time: 1417078239
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12110
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2d758102c6d0508d33dbe
Content-Type: text/plain; charset=UTF-8

Hi Mark,

Please find the statistics gathered below:

[root@pcp-test-shir3 ~]# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0
node 0 size: 622 MB
node 0 free: 418 MB
node distances:
node   0
  0:  10

[root@pcp-test-shir3 ~]# numastat -v

Per-node numastat info (in MBs):
                          Node 0           Total
                 --------------- ---------------
Numa_Hit                46805.92        46805.92
Numa_Miss                   0.00            0.00
Numa_Foreign                0.00            0.00
Interleave_Hit             31.29           31.29
Local_Node              46805.92        46805.92
Other_Node                  0.00            0.00

Thanks,

Shirshendu Chakrabarti



On Thu, Nov 27, 2014 at 2:04 PM, Shirshendu Chakrabarti 
wrote:

> Hi Mark,
>
> Thanks  for the explanation.
>
> I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64
>
> I am running, PCP-3-10.0-1.
>
> Please see its a t1.micro instance.
>
> For more details:
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts_micro_instances.html
>
> I have a few other questions:
>
> 1. Does PCP have any different requirements for running on the cloud.
> 2. Do the metrics need to be interpreted differently when running on cloud.
>
> Thanks,
>
> Shirshendu Chakrabarti
>
> On Thu, Nov 27, 2014 at 1:32 PM, Mark Goodwin  wrote:
>
>> On 11/27/2014 05:17 PM, Shirshendu Chakrabarti wrote:
>>
>>> Hi PCP team,
>>>
>>> Could anyone please respond the question above :)
>>>
>>
>> The difference is the instance domains and counter aggregation.
>>
>> kernel.pernode is aggregated over all CPUs on each numa node.
>> On my laptop with 4 CPU cores on one (fake) numa node, we should
>> expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and
>> the sum of the per-cpu counters to equal the same, e.g. :
>>
>> # pminfo -f kernel.{all,pernode,percpu}.cpu.user
>>
>> kernel.all.cpu.user
>>     value 254548080
>>
>> kernel.pernode.cpu.user
>>     inst [0 or "node0"] value 254548060
>>
>> kernel.percpu.cpu.user
>>     inst [0 or "cpu0"] value 66377110
>>     inst [1 or "cpu1"] value 61187360
>>     inst [2 or "cpu2"] value 64051110
>>     inst [3 or "cpu3"] value 62932480
>>
>> The mapping between CPUs and nodes is in hinv.map.cpu_node,
>> where each CPU (instance) is mapped to a node number, e.g.
>>
>> # pminfo -f hinv.map.cpu_node
>>
>> hinv.map.cpu_node
>>     inst [0 or "cpu0"] value 0
>>     inst [1 or "cpu1"] value 0
>>     inst [2 or "cpu2"] value 0
>>     inst [3 or "cpu3"] value 0
>>
>> In your case, kernel.pernode.cpu.user is zero, which isn't correct.
>> What platform and PCP version are you running?
>>
>> Regards
>> -- Mark
>>
>>
>>> Thanks,
>>>
>>> Shirshendu Chakrabarti
>>>
>>> On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti <
>>> shirshendu@riva.co
>>> > wrote:
>>>
>>>     Hi PCP team,
>>>
>>>     Could anyone in the team point me to any literature which explains
>>> the
>>>     difference between a per node metric and percpu metric. The
>>> one-liner and
>>>     extended description are absent in pernode metric case and terse in
>>> percpu case.
>>>
>>>     For example:
>>>
>>>     [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.user
>>>
>>>     kernel.percpu.cpu.user
>>>          inst [0 or "cpu0"] value 993240
>>>     [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
>>>
>>>     kernel.pernode.cpu.user
>>>          inst [0 or "node0"] value 0
>>>     I was under the assumption that, kernel.pernode.* and kernel.all.*
>>> metrics
>>>     are the same. I am clearly missing something important.
>>>
>>>     Thanks,
>>>
>>>     Shirshendu Chakrabarti
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> pcp mailing list
>>> pcp@oss.sgi.com
>>> http://oss.sgi.com/mailman/listinfo/pcp
>>>
>>>
>>
>

--001a11c2d758102c6d0508d33dbe
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Mark,

Please find the statistics gat=
hered below:

[root@pcp-test-shir3 ~]# numactl=
 --hardware
available: 1 nodes (0)
node 0 cpus: 0
=

node 0 size: 622 MB
node 0 free: 418 MB
node dista=
nces:
node =C2=A0 0=C2=A0
=C2=A0 0: =C2=A010=C2=A0

[root@pcp-test-shir3 ~]# numastat -v
<=
div>
Per-node numastat info (in MBs):
=C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 Node 0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Total
=C2=A0 =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0--------------- ----------=
-----
Numa_Hit =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A046805.92 =C2=A0 =C2=A0 =C2=A0 =C2=A046805.92
Numa_Miss =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0.00 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.00
Numa_Foreign =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.00 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A00.00
Interleave_Hit =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 31.29 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 31.29
Local_N=
ode =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A046805.92 =C2=A0 =C2=A0 =
=C2=A0 =C2=A046805.92
Other_Node =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.00 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A00.00

Thanks,

Shi=
rshendu Chakrabarti



On Thu, Nov 27, 2014 at 2:04 PM=
, Shirshendu Chakrabarti <shirshendu@riva.co> wrote:
Hi Mark,

Th=
anks =C2=A0for the explanation.

I am running on Am=
azon Linux - 3.4.71-63.amzn1.x86_64

I am running, =
PCP-3-10.0-1.

Please see its a t1.micro insta=
nce.


I have a few other questi=
ons:

1. Does PCP have any different requirements f=
or running on the cloud.=C2=A0
2. Do the metrics need to be inter=
preted differently when running on cloud.

Thanks,<=
/div>

Shirshendu Chakrabarti

On Thu, Nov 27, 2014 at 1:32 PM, Mark Goodwin <=
mgoodwin@redhat.co=
m> wrote:
On 11/27/20=
14 05:17 PM, Shirshendu Chakrabarti wrote:



Hi PCP team,



Could anyone please respond the question above :)





The difference is the instance domains and counter aggregation.



kernel.pernode is aggregated over all CPUs on each numa node.

On my laptop with 4 CPU cores on one (fake) numa node, we should

expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and

the sum of the per-cpu counters to equal the same, e.g. :



# pminfo -f kernel.{all,pernode,percpu}.cpu.user



kernel.all.cpu.user

=C2=A0 =C2=A0 value 254548080



kernel.pernode.cpu.user

=C2=A0 =C2=A0 inst [0 or "node0"] value 254548060



kernel.percpu.cpu.user

=C2=A0 =C2=A0 inst [0 or "cpu0"] value 66377110

=C2=A0 =C2=A0 inst [1 or "cpu1"] value 61187360

=C2=A0 =C2=A0 inst [2 or "cpu2"] value 64051110

=C2=A0 =C2=A0 inst [3 or "cpu3"] value 62932480



The mapping between CPUs and nodes is in hinv.map.cpu_node,

where each CPU (instance) is mapped to a node number, e.g.



# pminfo -f hinv.map.cpu_node



hinv.map.cpu_node

=C2=A0 =C2=A0 inst [0 or "cpu0"] value 0

=C2=A0 =C2=A0 inst [1 or "cpu1"] value 0

=C2=A0 =C2=A0 inst [2 or "cpu2"] value 0

=C2=A0 =C2=A0 inst [3 or "cpu3"] value 0



In your case, kernel.pernode.cpu.user is zero, which isn't correct.

What platform and PCP version are you running?



Regards

-- Mark







Thanks,



Shirshendu Chakrabarti



On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti <shirshendu@riva.co
<=
span>
<mailto:shirshen=
du@riva.co>> wrote:



=C2=A0 =C2=A0 Hi PCP team,



=C2=A0 =C2=A0 Could anyone in the team point me to any literature which exp=
lains the

=C2=A0 =C2=A0 difference between a per node metric and percpu metric. The o=
ne-liner and

=C2=A0 =C2=A0 extended description are absent in pernode metric case and te=
rse in percpu case.



=C2=A0 =C2=A0 For example:



=C2=A0 =C2=A0 [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.u=
ser



=C2=A0 =C2=A0 kernel.percpu.cpu.user

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inst [0 or "cpu0"] value 993240=


=C2=A0 =C2=A0 [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.=
user



=C2=A0 =C2=A0 kernel.pernode.cpu.user

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inst [0 or "node0"] value 0

=C2=A0 =C2=A0 I was under the assumption that, kernel.pernode.* and kernel.=
all.* metrics

=C2=A0 =C2=A0 are the same. I am clearly missing something important.



=C2=A0 =C2=A0 Thanks,



=C2=A0 =C2=A0 Shirshendu Chakrabarti









_______________________________________________

pcp mailing list

pcp@oss.sgi.com
http:=
//oss.sgi.com/mailman/listinfo/pcp














--001a11c2d758102c6d0508d33dbe--

From janfrode@tanso.net  Thu Nov 27 04:41:04 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 8E9E67F4E
	for ; Thu, 27 Nov 2014 04:41:04 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id D6EEFAC002
	for ; Thu, 27 Nov 2014 02:41:00 -0800 (PST)
X-ASG-Debug-ID: 1417084850-04cb6c05737c97e0001-S8gJnT
Received: from mail-lb0-f176.google.com (mail-lb0-f176.google.com [209.85.217.176]) by cuda.sgi.com with ESMTP id NTuZpWNPefVbDRHm (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 27 Nov 2014 02:40:51 -0800 (PST)
X-Barracuda-Envelope-From: janfrode@tanso.net
X-Barracuda-Apparent-Source-IP: 209.85.217.176
Received: by mail-lb0-f176.google.com with SMTP id p9so4006093lbv.35
        for ; Thu, 27 Nov 2014 02:40:50 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:date:from:to:subject:message-id:mime-version
         :content-type:content-disposition:user-agent;
        bh=TEaIVP41A9mrvn5Pdki9kBCClh3Lzhzs9I9Mk4EQai8=;
        b=RrhZ1ckDLtwNJ8RQU8wB/dFU5SC3qDOdqTSk4cmYwmrH6SaH8hjg9QiHCGTstEmR2h
         Ayb66/TJo+pfBKhPWi9wvz0aOpOxaa6/dKV9GOnwsgz0OG2kEUU517s4WPXERrYHYyTk
         fw/XEyNyxL62wDdiiPSuS5q3jwrF0AGpWKrDRPKBgKZBGGFjfdPxjxaYmKudOFEk1zdi
         3ZRbdCiqnDtOXmEOVzdHTvVcuSo4RTbvz6dZ+oJAYt5YnmXGxJbqg4+X3vMeGDt/7e1C
         wFF6ErKmC5KbtYtQtDPSnQ13FJgR1PjcyTA7L5K6sTRvMX7o+hQr2nmBg+kWkfE+Al0/
         Se9g==
X-Gm-Message-State: ALoCoQnDzPJH8SRdjI/SdrMIYsDrEPRQbTFSgd5gk97sFYFTJMvyOH/z3hgDgH7+wIRJ+cum5quz
X-Received: by 10.112.150.136 with SMTP id ui8mr38463804lbb.60.1417084850052;
        Thu, 27 Nov 2014 02:40:50 -0800 (PST)
Received: from localhost (120.81-167-109.customer.lyse.net. [81.167.109.120])
        by mx.google.com with ESMTPSA id y3sm293313lad.7.2014.11.27.02.40.47
        for 
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 27 Nov 2014 02:40:48 -0800 (PST)
Date: Thu, 27 Nov 2014 11:40:47 +0100
From: Jan-Frode Myklebust 
To: pcp@oss.sgi.com
Subject: [PATCH] Add PMDA for the Unbound DNS resolver.
Message-ID: <20141127104047.GA7900@mushkin.tanso.net>
X-ASG-Orig-Subj: [PATCH] Add PMDA for the Unbound DNS resolver.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
X-Barracuda-Connect: mail-lb0-f176.google.com[209.85.217.176]
X-Barracuda-Start-Time: 1417084851
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12113
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------

This adds a PMDA for the Unbound DNS resolver. Currently exposing 140
metrics gathered from Unbounds "unbound-control stats" command. It's 
missing the per thread metrics, but I plan on adding these at a later
point.

Signed-off-by: Jan-Frode Myklebust 
---
 src/pmdas/unbound/Install            |   28 +
 src/pmdas/unbound/Remove             |   25 +
 src/pmdas/unbound/pmdaunbound.python | 1344 ++++++++++++++++++++++++++++++++++
 3 files changed, 1397 insertions(+)
 create mode 100755 src/pmdas/unbound/Install
 create mode 100755 src/pmdas/unbound/Remove
 create mode 100644 src/pmdas/unbound/pmdaunbound.python

diff --git a/src/pmdas/unbound/Install b/src/pmdas/unbound/Install
new file mode 100755
index 0000000..a375fbe
--- /dev/null
+++ b/src/pmdas/unbound/Install
@@ -0,0 +1,28 @@
+#! /bin/sh
+#
+# Copyright (c) 2014 Red Hat.
+# 
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+# 
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+# or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+# 
+# Install the Unbound PMDA
+#
+
+. $PCP_DIR/etc/pcp.env
+. $PCP_SHARE_DIR/lib/pmdaproc.sh
+
+iam=unbound
+python_opt=true
+daemon_opt=false
+forced_restart=true
+
+pmdaSetup
+pmdaInstall
+exit 0
diff --git a/src/pmdas/unbound/Remove b/src/pmdas/unbound/Remove
new file mode 100755
index 0000000..7ff291f
--- /dev/null
+++ b/src/pmdas/unbound/Remove
@@ -0,0 +1,25 @@
+#! /bin/sh
+#
+# Copyright (c) 2014 Red Hat.
+# 
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+# 
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+# or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+# 
+# Remove the Linux Unbound PMDA
+#
+
+. $PCP_DIR/etc/pcp.env
+. $PCP_SHARE_DIR/lib/pmdaproc.sh
+
+iam=unbound
+
+pmdaSetup
+pmdaRemove
+exit 0
diff --git a/src/pmdas/unbound/pmdaunbound.python b/src/pmdas/unbound/pmdaunbound.python
new file mode 100644
index 0000000..6575bb2
--- /dev/null
+++ b/src/pmdas/unbound/pmdaunbound.python
@@ -0,0 +1,1344 @@
+'''
+Performance Metrics Domain Agent exporting Unbound statistics.
+'''
+
+import cpmapi as c_api
+from pcp.pmapi import pmUnits
+from pcp.pmda import PMDA, pmdaMetric, pmdaIndom
+from subprocess import Popen, PIPE
+import string
+
+class UnboundPMDA(PMDA):
+    '''
+    Performance Metrics Domain Agent exporting Unbound metrics.
+    Install it and make basic use of it if you use Unbound, as follows:
+
+    # $PCP_PMDAS_DIR/unbound/Install
+    $ pminfo -fmdtT unbound
+    '''
+
+    def unbound_fetch(self):
+        '''
+        Called once per PCP "fetch" PDU from pmcd(1)
+        Iterates over the unbound statistics,
+        '''
+	p = Popen(['/usr/sbin/unbound-control', 'stats_noreset'], stdin=PIPE, stdout=PIPE, stderr=PIPE)
+        p.wait()
+        stdout, stderr = p.communicate()
+        stdoutlines = string.split(stdout.strip(), '\n')
+        for line in stdoutlines:
+                keyval = string.split(line, '=')
+		if keyval[0].startswith( 'histogram' ):
+		    # Replace "." with "_" to avoid splitting metric name in multiple sub sections
+                    keyval[0] = "histogram." + "_".join(str(v) for v in keyval[0].split(".")[1:])
+		# only try to populate known metrics:
+		if keyval[0] in self.values:
+                    if (
+                        keyval[0] == "time.now" or
+		        keyval[0] == "time.up" or 
+		        keyval[0] == "time.elapsed" or 
+		        keyval[0] == "total.requestlist.avg" or
+		        keyval[0] == "total.recursion.time.avg" or
+		        keyval[0] == "total.recursion.time.median"
+                       ):
+		          self.values[keyval[0]] = float(keyval[1])
+		    else:
+                          self.values[keyval[0]] = long(keyval[1])
+
+    def unbound_fetch_callback(self, cluster, item, inst):
+        '''
+        Main fetch callback - looks up value associated with requested PMID
+        (from unbound_fetch), converts units if needed, then returns a list of
+        value,status (single pair) for the requested pmid
+        '''
+        # self.log("fetch callback for %d.%d[%d]" % (cluster, item, inst))
+        if inst != c_api.PM_IN_NULL:
+            return [c_api.PM_ERR_INST, 0]
+        elif cluster != 0:
+            return [c_api.PM_ERR_PMID, 0]
+        if item >= 0 and item < self.nmetrics and self.patherrors == 1:
+            return [c_api.PM_ERR_AGAIN, 0]
+
+        if item == 0:
+            return [self.values['total.num.queries'], 1]
+        elif item == 1:
+            return [self.values['total.num.cachehits'], 1]
+        elif item == 2:
+            return [self.values['total.num.cachemiss'], 1]
+        elif item == 3:
+            return [self.values['total.num.prefetch'], 1]
+        elif item == 4:
+            return [self.values['total.num.recursivereplies'], 1]
+        elif item == 5:
+            return [self.values['total.requestlist.avg'], 1]
+        elif item == 6:
+            return [self.values['total.requestlist.max'], 1]
+        elif item == 7:
+            return [self.values['total.requestlist.overwritten'], 1]
+        elif item == 8:
+            return [self.values['total.requestlist.exceeded'], 1]
+        elif item == 9:
+            return [self.values['total.requestlist.current.all'], 1]
+        elif item == 10:
+            return [self.values['total.requestlist.current.user'], 1]
+        elif item == 11:
+            return [self.values['total.recursion.time.avg'], 1]
+        elif item == 12:
+            return [self.values['total.recursion.time.median'], 1]
+        elif item == 13:
+            return [self.values['unwanted.queries'], 1]
+        elif item == 14:
+            return [self.values['unwanted.replies'], 1]
+        elif item == 15:
+            return [self.values['num.answer.rcode.NOERROR'], 1]
+        elif item == 16:
+            return [self.values['num.answer.rcode.FORMERR'], 1]
+        elif item == 17:
+            return [self.values['num.answer.rcode.SERVFAIL'], 1]
+        elif item == 18:
+            return [self.values['num.answer.rcode.NXDOMAIN'], 1]
+        elif item == 19:
+            return [self.values['num.answer.rcode.REFUSED'], 1]
+        elif item == 20:
+            return [self.values['num.answer.rcode.nodata'], 1]
+        elif item == 21:
+            return [self.values['num.answer.secure'], 1]
+        elif item == 22:
+            return [self.values['num.answer.bogus'], 1]
+        elif item == 23:
+            return [self.values['num.rrset.bogus'], 1]
+        elif item == 24:
+            return [self.values['num.query.type.TYPE0'], 1]
+        elif item == 25:
+            return [self.values['num.query.type.A'], 1]
+        elif item == 26:
+            return [self.values['num.query.type.NS'], 1]
+        elif item == 27:
+            return [self.values['num.query.type.MD'], 1]
+        elif item == 28:
+            return [self.values['num.query.type.MF'], 1]
+        elif item == 29:
+            return [self.values['num.query.type.CNAME'], 1]
+        elif item == 30:
+            return [self.values['num.query.type.SOA'], 1]
+        elif item == 31:
+            return [self.values['num.query.type.MB'], 1]
+        elif item == 32:
+            return [self.values['num.query.type.MG'], 1]
+        elif item == 33:
+            return [self.values['num.query.type.MR'], 1]
+        elif item == 34:
+            return [self.values['num.query.type.NULL'], 1]
+        elif item == 35:
+            return [self.values['num.query.type.WKS'], 1]
+        elif item == 36:
+            return [self.values['num.query.type.PTR'], 1]
+        elif item == 37:
+            return [self.values['num.query.type.HINFO'], 1]
+        elif item == 38:
+            return [self.values['num.query.type.MINFO'], 1]
+        elif item == 39:
+            return [self.values['num.query.type.MX'], 1]
+        elif item == 40:
+            return [self.values['num.query.type.TXT'], 1]
+        elif item == 41:
+            return [self.values['num.query.type.RP'], 1]
+        elif item == 42:
+            return [self.values['num.query.type.AFSDB'], 1]
+        elif item == 43:
+            return [self.values['num.query.type.X25'], 1]
+        elif item == 44:
+            return [self.values['num.query.type.ISDN'], 1]
+        elif item == 45:
+            return [self.values['num.query.type.RT'], 1]
+        elif item == 46:
+            return [self.values['num.query.type.AAAA'], 1]
+        elif item == 47:
+            return [self.values['num.query.type.SRV'], 1]
+        elif item == 48:
+            return [self.values['num.query.type.ATMA'], 1]
+        elif item == 49:
+            return [self.values['num.query.type.NAPTR'], 1]
+        elif item == 50:
+            return [self.values['num.query.type.A6'], 1]
+        elif item == 51:
+            return [self.values['num.query.type.DS'], 1]
+        elif item == 52:
+            return [self.values['num.query.type.SSHFP'], 1]
+        elif item == 53:
+            return [self.values['num.query.type.RRSIG'], 1]
+        elif item == 54:
+            return [self.values['num.query.type.NSEC'], 1]
+        elif item == 55:
+            return [self.values['num.query.type.DNSKEY'], 1]
+        elif item == 56:
+            return [self.values['num.query.type.TLSA'], 1]
+        elif item == 57:
+            return [self.values['num.query.type.TYPE65'], 1]
+        elif item == 58:
+            return [self.values['num.query.type.TYPE67'], 1]
+        elif item == 59:
+            return [self.values['num.query.type.TYPE76'], 1]
+        elif item == 60:
+            return [self.values['num.query.type.TYPE97'], 1]
+        elif item == 61:
+            return [self.values['num.query.type.SPF'], 1]
+        elif item == 62:
+            return [self.values['num.query.type.TYPE103'], 1]
+        elif item == 63:
+            return [self.values['num.query.type.EUI64'], 1]
+        elif item == 64:
+            return [self.values['num.query.type.TYPE110'], 1]
+        elif item == 65:
+            return [self.values['num.query.type.TYPE117'], 1]
+        elif item == 66:
+            return [self.values['num.query.type.TYPE119'], 1]
+        elif item == 67:
+            return [self.values['num.query.type.TYPE123'], 1]
+        elif item == 68:
+            return [self.values['num.query.type.TKEY'], 1]
+        elif item == 69:
+            return [self.values['num.query.type.IXFR'], 1]
+        elif item == 70:
+            return [self.values['num.query.type.AXFR'], 1]
+        elif item == 71:
+            return [self.values['num.query.type.ANY'], 1]
+        elif item == 72:
+            return [self.values['num.query.type.other'], 1]
+        elif item == 73:
+            return [self.values['num.query.class.CLASS0'], 1]
+        elif item == 74:
+            return [self.values['num.query.class.IN'], 1]
+        elif item == 75:
+            return [self.values['num.query.class.CH'], 1]
+        elif item == 76:
+            return [self.values['num.query.class.CLASS6'], 1]
+        elif item == 77:
+            return [self.values['num.query.class.CLASS65'], 1]
+        elif item == 78:
+            return [self.values['num.query.class.CLASS115'], 1]
+        elif item == 79:
+            return [self.values['num.query.class.CLASS240'], 1]
+        elif item == 80:
+            return [self.values['num.query.class.ANY'], 1]
+        elif item == 81:
+            return [self.values['num.query.class.other'], 1]
+        elif item == 82:
+            return [self.values['num.query.opcode.QUERY'], 1]
+        elif item == 83:
+            return [self.values['num.query.tcp'], 1]
+        elif item == 84:
+            return [self.values['num.query.ipv6'], 1]
+        elif item == 85:
+            return [self.values['num.query.flags.QR'], 1]
+        elif item == 86:
+            return [self.values['num.query.flags.AA'], 1]
+        elif item == 87:
+            return [self.values['num.query.flags.TC'], 1]
+        elif item == 88:
+            return [self.values['num.query.flags.RD'], 1]
+        elif item == 89:
+            return [self.values['num.query.flags.RA'], 1]
+        elif item == 90:
+            return [self.values['num.query.flags.Z'], 1]
+        elif item == 91:
+            return [self.values['num.query.flags.AD'], 1]
+        elif item == 92:
+            return [self.values['num.query.flags.CD'], 1]
+        elif item == 93:
+            return [self.values['num.query.edns.present'], 1]
+        elif item == 94:
+            return [self.values['num.query.edns.DO'], 1]
+        elif item == 95:
+            return [self.values['mem.total.sbrk'], 1]
+        elif item == 96:
+            return [self.values['mem.cache.rrset'], 1]
+        elif item == 97:
+            return [self.values['mem.cache.message'], 1]
+        elif item == 98:
+            return [self.values['mem.mod.iterator'], 1]
+        elif item == 99:
+            return [self.values['mem.mod.validator'], 1]
+        elif item == 100:
+            return [self.values['histogram.000000_000000_to_000000_000001'], 1]
+        elif item == 101:
+            return [self.values['histogram.000000_000001_to_000000_000002'], 1]
+        elif item == 102:
+            return [self.values['histogram.000000_000002_to_000000_000004'], 1]
+        elif item == 103:
+            return [self.values['histogram.000000_000004_to_000000_000008'], 1]
+        elif item == 104:
+            return [self.values['histogram.000000_000008_to_000000_000016'], 1]
+        elif item == 105:
+            return [self.values['histogram.000000_000016_to_000000_000032'], 1]
+        elif item == 106:
+            return [self.values['histogram.000000_000032_to_000000_000064'], 1]
+        elif item == 107:
+            return [self.values['histogram.000000_000064_to_000000_000128'], 1]
+        elif item == 108:
+            return [self.values['histogram.000000_000128_to_000000_000256'], 1]
+        elif item == 109:
+            return [self.values['histogram.000000_000256_to_000000_000512'], 1]
+        elif item == 110:
+            return [self.values['histogram.000000_000512_to_000000_001024'], 1]
+        elif item == 111:
+            return [self.values['histogram.000000_001024_to_000000_002048'], 1]
+        elif item == 112:
+            return [self.values['histogram.000000_002048_to_000000_004096'], 1]
+        elif item == 113:
+            return [self.values['histogram.000000_004096_to_000000_008192'], 1]
+        elif item == 114:
+            return [self.values['histogram.000000_008192_to_000000_016384'], 1]
+        elif item == 115:
+            return [self.values['histogram.000000_016384_to_000000_032768'], 1]
+        elif item == 116:
+            return [self.values['histogram.000000_032768_to_000000_065536'], 1]
+        elif item == 117:
+            return [self.values['histogram.000000_065536_to_000000_131072'], 1]
+        elif item == 118:
+            return [self.values['histogram.000000_131072_to_000000_262144'], 1]
+        elif item == 119:
+            return [self.values['histogram.000000_262144_to_000000_524288'], 1]
+        elif item == 120:
+            return [self.values['histogram.000000_524288_to_000001_000000'], 1]
+        elif item == 121:
+            return [self.values['histogram.000001_000000_to_000002_000000'], 1]
+        elif item == 122:
+            return [self.values['histogram.000002_000000_to_000004_000000'], 1]
+        elif item == 123:
+            return [self.values['histogram.000004_000000_to_000008_000000'], 1]
+        elif item == 124:
+            return [self.values['histogram.000008_000000_to_000016_000000'], 1]
+        elif item == 125:
+            return [self.values['histogram.000016_000000_to_000032_000000'], 1]
+        elif item == 126:
+            return [self.values['histogram.000032_000000_to_000064_000000'], 1]
+        elif item == 127:
+            return [self.values['histogram.000064_000000_to_000128_000000'], 1]
+        elif item == 128:
+            return [self.values['histogram.000128_000000_to_000256_000000'], 1]
+        elif item == 129:
+            return [self.values['histogram.000256_000000_to_000512_000000'], 1]
+        elif item == 130:
+            return [self.values['histogram.000512_000000_to_001024_000000'], 1]
+        elif item == 131:
+            return [self.values['histogram.001024_000000_to_002048_000000'], 1]
+        elif item == 132:
+            return [self.values['histogram.002048_000000_to_004096_000000'], 1]
+        elif item == 133:
+            return [self.values['histogram.004096_000000_to_008192_000000'], 1]
+        elif item == 134:
+            return [self.values['histogram.008192_000000_to_016384_000000'], 1]
+        elif item == 135:
+            return [self.values['histogram.016384_000000_to_032768_000000'], 1]
+        elif item == 136:
+            return [self.values['histogram.032768_000000_to_065536_000000'], 1]
+        elif item == 137:
+            return [self.values['histogram.065536_000000_to_131072_000000'], 1]
+        elif item == 138:
+            return [self.values['histogram.131072_000000_to_262144_000000'], 1]
+        elif item == 139:
+            return [self.values['histogram.262144_000000_to_524288_000000'], 1]
+        return [c_api.PM_ERR_PMID, 0]
+
+    def setup_unbound_metrics(self, name):
+        '''
+        Setup the metric table - ensure a values hash entry is setup for
+        each metric; that way we don't need to worry about KeyErrors in
+        the fetch callback routine (e.g. if the kernel interface changes).
+        '''
+        self.values['total.num.queries'] = 0
+        self.add_metric(name + '.total.num.queries', pmdaMetric(
+                self.pmid(0, 0),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Total number of queries received from all threads.')
+
+        self.values['total.num.cachehits'] = 0
+        self.add_metric(name + '.total.num.cachehits', pmdaMetric(
+                self.pmid(0, 1),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Total number of queries that were successfully answered using a cache lookup.')
+
+        self.values['total.num.cachemiss'] = 0
+        self.add_metric(name + '.total.num.cachemiss', pmdaMetric(
+                self.pmid(0, 2),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Total number of queries that needed recursive processing.')
+
+        self.values['total.num.prefetch'] = 0
+        self.add_metric(name + '.total.num.prefetch', pmdaMetric(
+                self.pmid(0, 3),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Total number cache prefetches performed.')
+
+        self.values['total.num.recursivereplies'] = 0
+        self.add_metric(name + '.total.num.recursivereplies', pmdaMetric(
+                self.pmid(0, 4),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Total number of replies sent to queries that needed recursive processing.')
+
+        self.values['total.requestlist.avg'] = 0
+        self.add_metric(name + '.total.requestlist.avg', pmdaMetric(
+                self.pmid(0, 5),
+                c_api.PM_TYPE_FLOAT, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Average number of requests in the internal recursive processing request list on insert of a new incoming recursive processing query.')
+
+        self.values['total.requestlist.max'] = 0
+        self.add_metric(name + '.total.requestlist.max', pmdaMetric(
+                self.pmid(0, 6),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Maximum size attained by the internal recursive processing request list.')
+
+        self.values['total.requestlist.overwritten'] = 0
+        self.add_metric(name + '.total.requestlist.overwritten', pmdaMetric(
+                self.pmid(0, 7),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of requests in the request list that were overwritten by newer entries.')
+
+        self.values['total.requestlist.exceeded'] = 0
+        self.add_metric(name + '.total.requestlist.exceeded', pmdaMetric(
+                self.pmid(0, 8),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Queries that were dropped because the request list was full.')
+
+        self.values['total.requestlist.current.all'] = 0
+        self.add_metric(name + '.total.requestlist.current.all', pmdaMetric(
+                self.pmid(0, 9),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Current size of the request list, includes internally generated queries.')
+
+        self.values['total.requestlist.current.user'] = 0
+        self.add_metric(name + '.total.requestlist.current.user', pmdaMetric(
+                self.pmid(0, 10),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Current size of the request list, only the requests from client queries.')
+
+        self.values['total.recursion.time.avg'] = 0
+        self.add_metric(name + '.total.recursion.time.avg', pmdaMetric(
+                self.pmid(0, 11),
+                c_api.PM_TYPE_FLOAT, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(0, 1, 0, 0, c_api.PM_TIME_SEC, 0)),
+        'Average time it took to answer queries that needed recursive processing.')
+
+        self.values['total.recursion.time.median'] = 0
+        self.add_metric(name + '.total.recursion.time.median', pmdaMetric(
+                self.pmid(0, 12),
+                c_api.PM_TYPE_FLOAT, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(0, 1, 0, 0, c_api.PM_TIME_SEC, 0)),
+        'Median time it took to answer queries that needed recursive processing.')
+
+        self.values['unwanted.queries'] = 0
+        self.add_metric(name + '.unwanted.queries', pmdaMetric(
+                self.pmid(0, 13),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of queries that were refused or dropped because they failed the access control settings.')
+
+        self.values['unwanted.replies'] = 0
+        self.add_metric(name + '.unwanted.replies', pmdaMetric(
+                self.pmid(0, 14),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Replies that were unwanted or unsolicited.')
+
+        self.values['num.answer.rcode.NOERROR'] = 0
+        self.add_metric(name + '.num.answer.rcode.NOERROR', pmdaMetric(
+                self.pmid(0, 15),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers that were without faults.')
+
+        self.values['num.answer.rcode.FORMERR'] = 0
+        self.add_metric(name + '.num.answer.rcode.FORMERR', pmdaMetric(
+                self.pmid(0, 16),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers with format error.')
+
+        self.values['num.answer.rcode.SERVFAIL'] = 0
+        self.add_metric(name + '.num.answer.rcode.SERVFAIL', pmdaMetric(
+                self.pmid(0, 17),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers with SERVFAIL.')
+
+        self.values['num.answer.rcode.NXDOMAIN'] = 0
+        self.add_metric(name + '.num.answer.rcode.NXDOMAIN', pmdaMetric(
+                self.pmid(0, 18),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers with NXDOMAIN.')
+
+        self.values['num.answer.rcode.REFUSED'] = 0
+        self.add_metric(name + '.num.answer.rcode.REFUSED', pmdaMetric(
+                self.pmid(0, 19),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers status REFUSED.')
+
+        self.values['num.answer.rcode.nodata'] = 0
+        self.add_metric(name + '.num.answer.rcode.nodata', pmdaMetric(
+                self.pmid(0, 20),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'The number of answers to queries that had the pseudo return code nodata.')
+
+        self.values['num.answer.secure'] = 0
+        self.add_metric(name + '.num.answer.secure', pmdaMetric(
+                self.pmid(0, 21),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers that were secure.  The answer validated correctly.')
+
+        self.values['num.answer.bogus'] = 0
+        self.add_metric(name + '.num.answer.bogus', pmdaMetric(
+                self.pmid(0, 22),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Number of answers that were bogus. Answer failed validation.')
+
+        self.values['num.rrset.bogus'] = 0
+        self.add_metric(name + '.num.rrset.bogus', pmdaMetric(
+                self.pmid(0, 23),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'The number of rrsets marked bogus by the validator.')
+
+        self.values['num.query.type.TYPE0'] = 0
+        self.add_metric(name + '.num.query.type.TYPE0', pmdaMetric(
+                self.pmid(0, 24),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE0')
+
+        self.values['num.query.type.A'] = 0
+        self.add_metric(name + '.num.query.type.A', pmdaMetric(
+                self.pmid(0, 25),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.A')
+
+        self.values['num.query.type.NS'] = 0
+        self.add_metric(name + '.num.query.type.NS', pmdaMetric(
+                self.pmid(0, 26),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.NS')
+
+        self.values['num.query.type.MD'] = 0
+        self.add_metric(name + '.num.query.type.MD', pmdaMetric(
+                self.pmid(0, 27),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MD')
+
+        self.values['num.query.type.MF'] = 0
+        self.add_metric(name + '.num.query.type.MF', pmdaMetric(
+                self.pmid(0, 28),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MF')
+
+        self.values['num.query.type.CNAME'] = 0
+        self.add_metric(name + '.num.query.type.CNAME', pmdaMetric(
+                self.pmid(0, 29),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.CNAME')
+
+        self.values['num.query.type.SOA'] = 0
+        self.add_metric(name + '.num.query.type.SOA', pmdaMetric(
+                self.pmid(0, 30),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.SOA')
+
+        self.values['num.query.type.MB'] = 0
+        self.add_metric(name + '.num.query.type.MB', pmdaMetric(
+                self.pmid(0, 31),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MB')
+
+        self.values['num.query.type.MG'] = 0
+        self.add_metric(name + '.num.query.type.MG', pmdaMetric(
+                self.pmid(0, 32),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MG')
+
+        self.values['num.query.type.MR'] = 0
+        self.add_metric(name + '.num.query.type.MR', pmdaMetric(
+                self.pmid(0, 33),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MR')
+
+        self.values['num.query.type.NULL'] = 0
+        self.add_metric(name + '.num.query.type.NULL', pmdaMetric(
+                self.pmid(0, 34),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.NULL')
+
+        self.values['num.query.type.WKS'] = 0
+        self.add_metric(name + '.num.query.type.WKS', pmdaMetric(
+                self.pmid(0, 35),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.WKS')
+
+        self.values['num.query.type.PTR'] = 0
+        self.add_metric(name + '.num.query.type.PTR', pmdaMetric(
+                self.pmid(0, 36),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.PTR')
+
+        self.values['num.query.type.HINFO'] = 0
+        self.add_metric(name + '.num.query.type.HINFO', pmdaMetric(
+                self.pmid(0, 37),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.HINFO')
+
+        self.values['num.query.type.MINFO'] = 0
+        self.add_metric(name + '.num.query.type.MINFO', pmdaMetric(
+                self.pmid(0, 38),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MINFO')
+
+        self.values['num.query.type.MX'] = 0
+        self.add_metric(name + '.num.query.type.MX', pmdaMetric(
+                self.pmid(0, 39),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.MX')
+
+        self.values['num.query.type.TXT'] = 0
+        self.add_metric(name + '.num.query.type.TXT', pmdaMetric(
+                self.pmid(0, 40),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TXT')
+
+        self.values['num.query.type.RP'] = 0
+        self.add_metric(name + '.num.query.type.RP', pmdaMetric(
+                self.pmid(0, 41),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.RP')
+
+        self.values['num.query.type.AFSDB'] = 0
+        self.add_metric(name + '.num.query.type.AFSDB', pmdaMetric(
+                self.pmid(0, 42),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.AFSDB')
+
+        self.values['num.query.type.X25'] = 0
+        self.add_metric(name + '.num.query.type.X25', pmdaMetric(
+                self.pmid(0, 43),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.X25')
+
+        self.values['num.query.type.ISDN'] = 0
+        self.add_metric(name + '.num.query.type.ISDN', pmdaMetric(
+                self.pmid(0, 44),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.ISDN')
+
+        self.values['num.query.type.RT'] = 0
+        self.add_metric(name + '.num.query.type.RT', pmdaMetric(
+                self.pmid(0, 45),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.RT')
+
+        self.values['num.query.type.AAAA'] = 0
+        self.add_metric(name + '.num.query.type.AAAA', pmdaMetric(
+                self.pmid(0, 46),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.AAAA')
+
+        self.values['num.query.type.SRV'] = 0
+        self.add_metric(name + '.num.query.type.SRV', pmdaMetric(
+                self.pmid(0, 47),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.SRV')
+
+        self.values['num.query.type.ATMA'] = 0
+        self.add_metric(name + '.num.query.type.ATMA', pmdaMetric(
+                self.pmid(0, 48),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.ATMA')
+
+        self.values['num.query.type.NAPTR'] = 0
+        self.add_metric(name + '.num.query.type.NAPTR', pmdaMetric(
+                self.pmid(0, 49),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.NAPTR')
+
+        self.values['num.query.type.A6'] = 0
+        self.add_metric(name + '.num.query.type.A6', pmdaMetric(
+                self.pmid(0, 50),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.A6')
+
+        self.values['num.query.type.DS'] = 0
+        self.add_metric(name + '.num.query.type.DS', pmdaMetric(
+                self.pmid(0, 51),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.DS')
+
+        self.values['num.query.type.SSHFP'] = 0
+        self.add_metric(name + '.num.query.type.SSHFP', pmdaMetric(
+                self.pmid(0, 52),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.SSHFP')
+
+        self.values['num.query.type.RRSIG'] = 0
+        self.add_metric(name + '.num.query.type.RRSIG', pmdaMetric(
+                self.pmid(0, 53),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.RRSIG')
+
+        self.values['num.query.type.NSEC'] = 0
+        self.add_metric(name + '.num.query.type.NSEC', pmdaMetric(
+                self.pmid(0, 54),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.NSEC')
+
+        self.values['num.query.type.DNSKEY'] = 0
+        self.add_metric(name + '.num.query.type.DNSKEY', pmdaMetric(
+                self.pmid(0, 55),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.DNSKEY')
+
+        self.values['num.query.type.TLSA'] = 0
+        self.add_metric(name + '.num.query.type.TLSA', pmdaMetric(
+                self.pmid(0, 56),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TLSA')
+
+        self.values['num.query.type.TYPE65'] = 0
+        self.add_metric(name + '.num.query.type.TYPE65', pmdaMetric(
+                self.pmid(0, 57),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE65')
+
+        self.values['num.query.type.TYPE67'] = 0
+        self.add_metric(name + '.num.query.type.TYPE67', pmdaMetric(
+                self.pmid(0, 58),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE67')
+
+        self.values['num.query.type.TYPE76'] = 0
+        self.add_metric(name + '.num.query.type.TYPE76', pmdaMetric(
+                self.pmid(0, 59),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE76')
+
+        self.values['num.query.type.TYPE97'] = 0
+        self.add_metric(name + '.num.query.type.TYPE97', pmdaMetric(
+                self.pmid(0, 60),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE97')
+
+        self.values['num.query.type.SPF'] = 0
+        self.add_metric(name + '.num.query.type.SPF', pmdaMetric(
+                self.pmid(0, 61),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.SPF')
+
+        self.values['num.query.type.TYPE103'] = 0
+        self.add_metric(name + '.num.query.type.TYPE103', pmdaMetric(
+                self.pmid(0, 62),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE103')
+
+        self.values['num.query.type.EUI64'] = 0
+        self.add_metric(name + '.num.query.type.EUI64', pmdaMetric(
+                self.pmid(0, 63),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.EUI64')
+
+        self.values['num.query.type.TYPE110'] = 0
+        self.add_metric(name + '.num.query.type.TYPE110', pmdaMetric(
+                self.pmid(0, 64),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE110')
+
+        self.values['num.query.type.TYPE117'] = 0
+        self.add_metric(name + '.num.query.type.TYPE117', pmdaMetric(
+                self.pmid(0, 65),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE117')
+
+        self.values['num.query.type.TYPE119'] = 0
+        self.add_metric(name + '.num.query.type.TYPE119', pmdaMetric(
+                self.pmid(0, 66),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE119')
+
+        self.values['num.query.type.TYPE123'] = 0
+        self.add_metric(name + '.num.query.type.TYPE123', pmdaMetric(
+                self.pmid(0, 67),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TYPE123')
+
+        self.values['num.query.type.TKEY'] = 0
+        self.add_metric(name + '.num.query.type.TKEY', pmdaMetric(
+                self.pmid(0, 68),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.TKEY')
+
+        self.values['num.query.type.IXFR'] = 0
+        self.add_metric(name + '.num.query.type.IXFR', pmdaMetric(
+                self.pmid(0, 69),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.IXFR')
+
+        self.values['num.query.type.AXFR'] = 0
+        self.add_metric(name + '.num.query.type.AXFR', pmdaMetric(
+                self.pmid(0, 70),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.AXFR')
+
+        self.values['num.query.type.ANY'] = 0
+        self.add_metric(name + '.num.query.type.ANY', pmdaMetric(
+                self.pmid(0, 71),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.ANY')
+
+        self.values['num.query.type.other'] = 0
+        self.add_metric(name + '.num.query.type.other', pmdaMetric(
+                self.pmid(0, 72),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.type.other')
+
+        self.values['num.query.class.CLASS0'] = 0
+        self.add_metric(name + '.num.query.class.CLASS0', pmdaMetric(
+                self.pmid(0, 73),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.CLASS0')
+
+        self.values['num.query.class.IN'] = 0
+        self.add_metric(name + '.num.query.class.IN', pmdaMetric(
+                self.pmid(0, 74),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.IN')
+
+        self.values['num.query.class.CH'] = 0
+        self.add_metric(name + '.num.query.class.CH', pmdaMetric(
+                self.pmid(0, 75),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.CH')
+
+        self.values['num.query.class.CLASS6'] = 0
+        self.add_metric(name + '.num.query.class.CLASS6', pmdaMetric(
+                self.pmid(0, 76),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.CLASS6')
+
+        self.values['num.query.class.CLASS65'] = 0
+        self.add_metric(name + '.num.query.class.CLASS65', pmdaMetric(
+                self.pmid(0, 77),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.CLASS65')
+
+        self.values['num.query.class.CLASS115'] = 0
+        self.add_metric(name + '.num.query.class.CLASS115', pmdaMetric(
+                self.pmid(0, 78),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.CLASS115')
+
+        self.values['num.query.class.CLASS240'] = 0
+        self.add_metric(name + '.num.query.class.CLASS240', pmdaMetric(
+                self.pmid(0, 79),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.CLASS240')
+
+        self.values['num.query.class.ANY'] = 0
+        self.add_metric(name + '.num.query.class.ANY', pmdaMetric(
+                self.pmid(0, 80),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.ANY')
+
+        self.values['num.query.class.other'] = 0
+        self.add_metric(name + '.num.query.class.other', pmdaMetric(
+                self.pmid(0, 81),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.class.other')
+
+        self.values['num.query.opcode.QUERY'] = 0
+        self.add_metric(name + '.num.query.opcode.QUERY', pmdaMetric(
+                self.pmid(0, 82),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.opcode.QUERY')
+
+        self.values['num.query.tcp'] = 0
+        self.add_metric(name + '.num.query.tcp', pmdaMetric(
+                self.pmid(0, 83),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.tcp')
+
+        self.values['num.query.ipv6'] = 0
+        self.add_metric(name + '.num.query.ipv6', pmdaMetric(
+                self.pmid(0, 84),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.ipv6')
+
+        self.values['num.query.flags.QR'] = 0
+        self.add_metric(name + '.num.query.flags.QR', pmdaMetric(
+                self.pmid(0, 85),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.QR')
+
+        self.values['num.query.flags.AA'] = 0
+        self.add_metric(name + '.num.query.flags.AA', pmdaMetric(
+                self.pmid(0, 86),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.AA')
+
+        self.values['num.query.flags.TC'] = 0
+        self.add_metric(name + '.num.query.flags.TC', pmdaMetric(
+                self.pmid(0, 87),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.TC')
+
+        self.values['num.query.flags.RD'] = 0
+        self.add_metric(name + '.num.query.flags.RD', pmdaMetric(
+                self.pmid(0, 88),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.RD')
+
+        self.values['num.query.flags.RA'] = 0
+        self.add_metric(name + '.num.query.flags.RA', pmdaMetric(
+                self.pmid(0, 89),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.RA')
+
+        self.values['num.query.flags.Z'] = 0
+        self.add_metric(name + '.num.query.flags.Z', pmdaMetric(
+                self.pmid(0, 90),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.Z')
+
+        self.values['num.query.flags.AD'] = 0
+        self.add_metric(name + '.num.query.flags.AD', pmdaMetric(
+                self.pmid(0, 91),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.AD')
+
+        self.values['num.query.flags.CD'] = 0
+        self.add_metric(name + '.num.query.flags.CD', pmdaMetric(
+                self.pmid(0, 92),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.flags.CD')
+
+        self.values['num.query.edns.present'] = 0
+        self.add_metric(name + '.num.query.edns.present', pmdaMetric(
+                self.pmid(0, 93),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.edns.present')
+
+        self.values['num.query.edns.DO'] = 0
+        self.add_metric(name + '.num.query.edns.DO', pmdaMetric(
+                self.pmid(0, 94),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'num.query.edns.DO')
+
+        self.values['mem.total.sbrk'] = 0
+        self.add_metric(name + '.mem.total.sbrk', pmdaMetric(
+                self.pmid(0, 95),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(1, 0, 0, c_api.PM_SPACE_BYTE, 0, 0)),
+        'Estimate of the heap size of the program in number of  bytes.')
+
+        self.values['mem.cache.rrset'] = 0
+        self.add_metric(name + '.mem.cache.rrset', pmdaMetric(
+                self.pmid(0, 96),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(1, 0, 0, c_api.PM_SPACE_BYTE, 0, 0)),
+        'Memory in bytes in use by the RRset cache.')
+
+        self.values['mem.cache.message'] = 0
+        self.add_metric(name + '.mem.cache.message', pmdaMetric(
+                self.pmid(0, 97),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(1, 0, 0, c_api.PM_SPACE_BYTE, 0, 0)),
+        'Memory in bytes in use by the message cache.')
+
+        self.values['mem.mod.iterator'] = 0
+        self.add_metric(name + '.mem.mod.iterator', pmdaMetric(
+                self.pmid(0, 98),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(1, 0, 0, c_api.PM_SPACE_BYTE, 0, 0)),
+        'Memory in bytes in use by the iterator module.')
+
+        self.values['mem.mod.validator'] = 0
+        self.add_metric(name + '.mem.mod.validator', pmdaMetric(
+                self.pmid(0, 99),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_INSTANT,
+                pmUnits(1, 0, 0, c_api.PM_SPACE_BYTE, 0, 0)),
+        'Memory in bytes in use by the validator module.')
+
+        self.values['histogram.000000_000000_to_000000_000001'] = 0
+        self.add_metric(name + '.histogram.000000_000000_to_000000_000001', pmdaMetric(
+                self.pmid(0, 100),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000001_to_000000_000002'] = 0
+        self.add_metric(name + '.histogram.000000_000001_to_000000_000002', pmdaMetric(
+                self.pmid(0, 101),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000002_to_000000_000004'] = 0
+        self.add_metric(name + '.histogram.000000_000002_to_000000_000004', pmdaMetric(
+                self.pmid(0, 102),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000004_to_000000_000008'] = 0
+        self.add_metric(name + '.histogram.000000_000004_to_000000_000008', pmdaMetric(
+                self.pmid(0, 103),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000008_to_000000_000016'] = 0
+        self.add_metric(name + '.histogram.000000_000008_to_000000_000016', pmdaMetric(
+                self.pmid(0, 104),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000016_to_000000_000032'] = 0
+        self.add_metric(name + '.histogram.000000_000016_to_000000_000032', pmdaMetric(
+                self.pmid(0, 105),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000032_to_000000_000064'] = 0
+        self.add_metric(name + '.histogram.000000_000032_to_000000_000064', pmdaMetric(
+                self.pmid(0, 106),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000064_to_000000_000128'] = 0
+        self.add_metric(name + '.histogram.000000_000064_to_000000_000128', pmdaMetric(
+                self.pmid(0, 107),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000128_to_000000_000256'] = 0
+        self.add_metric(name + '.histogram.000000_000128_to_000000_000256', pmdaMetric(
+                self.pmid(0, 108),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000256_to_000000_000512'] = 0
+        self.add_metric(name + '.histogram.000000_000256_to_000000_000512', pmdaMetric(
+                self.pmid(0, 109),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_000512_to_000000_001024'] = 0
+        self.add_metric(name + '.histogram.000000_000512_to_000000_001024', pmdaMetric(
+                self.pmid(0, 110),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_001024_to_000000_002048'] = 0
+        self.add_metric(name + '.histogram.000000_001024_to_000000_002048', pmdaMetric(
+                self.pmid(0, 111),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_002048_to_000000_004096'] = 0
+        self.add_metric(name + '.histogram.000000_002048_to_000000_004096', pmdaMetric(
+                self.pmid(0, 112),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_004096_to_000000_008192'] = 0
+        self.add_metric(name + '.histogram.000000_004096_to_000000_008192', pmdaMetric(
+                self.pmid(0, 113),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_008192_to_000000_016384'] = 0
+        self.add_metric(name + '.histogram.000000_008192_to_000000_016384', pmdaMetric(
+                self.pmid(0, 114),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_016384_to_000000_032768'] = 0
+        self.add_metric(name + '.histogram.000000_016384_to_000000_032768', pmdaMetric(
+                self.pmid(0, 115),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_032768_to_000000_065536'] = 0
+        self.add_metric(name + '.histogram.000000_032768_to_000000_065536', pmdaMetric(
+                self.pmid(0, 116),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_065536_to_000000_131072'] = 0
+        self.add_metric(name + '.histogram.000000_065536_to_000000_131072', pmdaMetric(
+                self.pmid(0, 117),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_131072_to_000000_262144'] = 0
+        self.add_metric(name + '.histogram.000000_131072_to_000000_262144', pmdaMetric(
+                self.pmid(0, 118),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_262144_to_000000_524288'] = 0
+        self.add_metric(name + '.histogram.000000_262144_to_000000_524288', pmdaMetric(
+                self.pmid(0, 119),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000000_524288_to_000001_000000'] = 0
+        self.add_metric(name + '.histogram.000000_524288_to_000001_000000', pmdaMetric(
+                self.pmid(0, 120),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000001_000000_to_000002_000000'] = 0
+        self.add_metric(name + '.histogram.000001_000000_to_000002_000000', pmdaMetric(
+                self.pmid(0, 121),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000002_000000_to_000004_000000'] = 0
+        self.add_metric(name + '.histogram.000002_000000_to_000004_000000', pmdaMetric(
+                self.pmid(0, 122),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000004_000000_to_000008_000000'] = 0
+        self.add_metric(name + '.histogram.000004_000000_to_000008_000000', pmdaMetric(
+                self.pmid(0, 123),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000008_000000_to_000016_000000'] = 0
+        self.add_metric(name + '.histogram.000008_000000_to_000016_000000', pmdaMetric(
+                self.pmid(0, 124),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000016_000000_to_000032_000000'] = 0
+        self.add_metric(name + '.histogram.000016_000000_to_000032_000000', pmdaMetric(
+                self.pmid(0, 125),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000032_000000_to_000064_000000'] = 0
+        self.add_metric(name + '.histogram.000032_000000_to_000064_000000', pmdaMetric(
+                self.pmid(0, 126),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000064_000000_to_000128_000000'] = 0
+        self.add_metric(name + '.histogram.000064_000000_to_000128_000000', pmdaMetric(
+                self.pmid(0, 127),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000128_000000_to_000256_000000'] = 0
+        self.add_metric(name + '.histogram.000128_000000_to_000256_000000', pmdaMetric(
+                self.pmid(0, 128),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000256_000000_to_000512_000000'] = 0
+        self.add_metric(name + '.histogram.000256_000000_to_000512_000000', pmdaMetric(
+                self.pmid(0, 129),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.000512_000000_to_001024_000000'] = 0
+        self.add_metric(name + '.histogram.000512_000000_to_001024_000000', pmdaMetric(
+                self.pmid(0, 130),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.001024_000000_to_002048_000000'] = 0
+        self.add_metric(name + '.histogram.001024_000000_to_002048_000000', pmdaMetric(
+                self.pmid(0, 131),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.002048_000000_to_004096_000000'] = 0
+        self.add_metric(name + '.histogram.002048_000000_to_004096_000000', pmdaMetric(
+                self.pmid(0, 132),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.004096_000000_to_008192_000000'] = 0
+        self.add_metric(name + '.histogram.004096_000000_to_008192_000000', pmdaMetric(
+                self.pmid(0, 133),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.008192_000000_to_016384_000000'] = 0
+        self.add_metric(name + '.histogram.008192_000000_to_016384_000000', pmdaMetric(
+                self.pmid(0, 134),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.016384_000000_to_032768_000000'] = 0
+        self.add_metric(name + '.histogram.016384_000000_to_032768_000000', pmdaMetric(
+                self.pmid(0, 135),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.032768_000000_to_065536_000000'] = 0
+        self.add_metric(name + '.histogram.032768_000000_to_065536_000000', pmdaMetric(
+                self.pmid(0, 136),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.065536_000000_to_131072_000000'] = 0
+        self.add_metric(name + '.histogram.065536_000000_to_131072_000000', pmdaMetric(
+                self.pmid(0, 137),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.131072_000000_to_262144_000000'] = 0
+        self.add_metric(name + '.histogram.131072_000000_to_262144_000000', pmdaMetric(
+                self.pmid(0, 138),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+        self.values['histogram.262144_000000_to_524288_000000'] = 0
+        self.add_metric(name + '.histogram.262144_000000_to_524288_000000', pmdaMetric(
+                self.pmid(0, 139),
+                c_api.PM_TYPE_U64, c_api.PM_INDOM_NULL, c_api.PM_SEM_COUNTER,
+                pmUnits(0, 0, 1, 0, 0, c_api.PM_COUNT_ONE)),
+        'Recursive queries whose reply time fit between the lower and upper bound.')
+
+    def __init__(self, name, domain):
+        PMDA.__init__(self, name, domain)
+        self.patherrors = 0
+        self.fileerrors = 0
+
+        self.values = {}
+        self.setup_unbound_metrics(name)
+        self.nmetrics = len(self.values)
+
+        self.set_fetch(self.unbound_fetch)
+        self.set_fetch_callback(self.unbound_fetch_callback)
+
+
+if __name__ == '__main__':
+    UnboundPMDA('unbound', 132).run()
-- 
1.8.3.1

From mgoodwin@redhat.com  Thu Nov 27 04:47:54 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id E9F4A7F4E
	for ; Thu, 27 Nov 2014 04:47:54 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id D5DBA304053
	for ; Thu, 27 Nov 2014 02:47:54 -0800 (PST)
X-ASG-Debug-ID: 1417085273-04cbb01e5a7bd560001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 8IjIz3JdGpFlLdgL (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 27 Nov 2014 02:47:53 -0800 (PST)
X-Barracuda-Envelope-From: mgoodwin@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sARAlqA7021824
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL);
	Thu, 27 Nov 2014 05:47:52 -0500
Received: from [10.64.48.240] (vpn1-48-240.bne.redhat.com [10.64.48.240])
	by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sARAlo6r031203;
	Thu, 27 Nov 2014 05:47:51 -0500
Message-ID: <54770155.9000101@redhat.com>
Date: Thu, 27 Nov 2014 21:47:49 +1100
From: Mark Goodwin 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Shirshendu Chakrabarti 
CC: pcp@oss.sgi.com
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
References: 		<5476DA8F.2060202@redhat.com> 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
In-Reply-To: 
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1417085273
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/27/2014 07:34 PM, Shirshendu Chakrabarti wrote:
[...]
> I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64

is that an SMP kernel?

>
> I am running, PCP-3-10.0-1.
>
> Please see its a t1.micro instance.

are there any 'cpu*' symlinks below /sys/devices/system/node/node0 ?
and does /sys/devices/system/node/node0/cpumap exist?  This is where
the PCP "linux" PMDA gets it's node-cpu map from.

>
> For more details:
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts_micro_instances.html
>
> I have a few other questions:
>
> 1. Does PCP have any different requirements for running on the cloud.
> 2. Do the metrics need to be interpreted differently when running on cloud.

I don't know enough about the Amazon environment to answer that. Is it
possible for me to login and have a poke around? Or can we download an
instance and run it to investigate?

Regards
-- Mark

>
> Thanks,
>
> Shirshendu Chakrabarti
>
> On Thu, Nov 27, 2014 at 1:32 PM, Mark Goodwin  > wrote:
>
>     On 11/27/2014 05:17 PM, Shirshendu Chakrabarti wrote:
>
>         Hi PCP team,
>
>         Could anyone please respond the question above :)
>
>
>     The difference is the instance domains and counter aggregation.
>
>     kernel.pernode is aggregated over all CPUs on each numa node.
>     On my laptop with 4 CPU cores on one (fake) numa node, we should
>     expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and
>     the sum of the per-cpu counters to equal the same, e.g. :
>
>     # pminfo -f kernel.{all,pernode,percpu}.__cpu.user
>
>     kernel.all.cpu.user
>          value 254548080
>
>     kernel.pernode.cpu.user
>          inst [0 or "node0"] value 254548060
>
>     kernel.percpu.cpu.user
>          inst [0 or "cpu0"] value 66377110
>          inst [1 or "cpu1"] value 61187360
>          inst [2 or "cpu2"] value 64051110
>          inst [3 or "cpu3"] value 62932480
>
>     The mapping between CPUs and nodes is in hinv.map.cpu_node,
>     where each CPU (instance) is mapped to a node number, e.g.
>
>     # pminfo -f hinv.map.cpu_node
>
>     hinv.map.cpu_node
>          inst [0 or "cpu0"] value 0
>          inst [1 or "cpu1"] value 0
>          inst [2 or "cpu2"] value 0
>          inst [3 or "cpu3"] value 0
>
>     In your case, kernel.pernode.cpu.user is zero, which isn't correct.
>     What platform and PCP version are you running?
>
>     Regards
>     -- Mark
>
>
>         Thanks,
>
>         Shirshendu Chakrabarti
>
>         On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti
>         
>         >> wrote:
>
>              Hi PCP team,
>
>              Could anyone in the team point me to any literature which explains the
>              difference between a per node metric and percpu metric. The
>         one-liner and
>              extended description are absent in pernode metric case and terse in
>         percpu case.
>
>              For example:
>
>              [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.percpu.cpu.user
>
>              kernel.percpu.cpu.user
>                   inst [0 or "cpu0"] value 993240
>              [root@pcp-test-shir3 pmlogger]# pminfo -f kernel.pernode.cpu.user
>
>              kernel.pernode.cpu.user
>                   inst [0 or "node0"] value 0
>              I was under the assumption that, kernel.pernode.* and kernel.all.*
>         metrics
>              are the same. I am clearly missing something important.
>
>              Thanks,
>
>              Shirshendu Chakrabarti
>
>
>
>
>         _________________________________________________
>         pcp mailing list
>         pcp@oss.sgi.com 
>         http://oss.sgi.com/mailman/__listinfo/pcp
>         
>
>
>

From shirshendu@riva.co  Thu Nov 27 05:53:14 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id 13C5C7F4E
	for ; Thu, 27 Nov 2014 05:53:14 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 0230E8F8054
	for ; Thu, 27 Nov 2014 03:53:10 -0800 (PST)
X-ASG-Debug-ID: 1417089188-04cb6c0572808ff0001-S8gJnT
Received: from mail-qg0-f54.google.com (mail-qg0-f54.google.com [209.85.192.54]) by cuda.sgi.com with ESMTP id nJVz9r7D4GaFsINw (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 27 Nov 2014 03:53:08 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.192.54
Received: by mail-qg0-f54.google.com with SMTP id q107so3320768qgd.41
        for ; Thu, 27 Nov 2014 03:53:07 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=L/CG3KarSAp0DnstlLgNpoIDm0PE8x9Kte3ZieG+E3U=;
        b=AZPrtjBrODW+tU1ZalA/ccvWBsSErjnsXlV7REJFI8Ho7q2uqpJ+2JcC2j2e0ROLE6
         nbIwwkZttwgFfMSGzrkt81x7TufHs6k512rUByDvxS//yJxnNxQg0/SKSSf1YpBeDeys
         aeWZ3dIMb8OywUGUUJ977yoC072/FZdLoflJtPe16ti4vtPISSsKO40eG40SkRjMG+HD
         72NBrOa2Z5kgv9Lu1HCn5KoEypzbkmpcYN8qHCxbA4ZzFEcYIH7aSrYcrNY3mB/ykDNe
         6p6tJaUxhtmlK2Y+/rWdbeMWeX9QeVRuDtknIPmKsi71V1bBJSHgSDKAeYXPSARYp228
         h6UA==
X-Gm-Message-State: ALoCoQmOuOBd2CEBAcUpgIjjBzRrjIhoNj27fzHSVVLErRukhpVE25P9x8SKRZz22sZlc1zqx67a
MIME-Version: 1.0
X-Received: by 10.224.88.193 with SMTP id b1mr54006265qam.30.1417089187525;
 Thu, 27 Nov 2014 03:53:07 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 27 Nov 2014 03:53:07 -0800 (PST)
In-Reply-To: <54770155.9000101@redhat.com>
References: 
	
	<5476DA8F.2060202@redhat.com>
	
	<54770155.9000101@redhat.com>
Date: Thu, 27 Nov 2014 17:23:07 +0530
Message-ID: 
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: Mark Goodwin 
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2d758a351bb0508d5c949
X-Barracuda-Connect: mail-qg0-f54.google.com[209.85.192.54]
X-Barracuda-Start-Time: 1417089188
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12115
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2d758a351bb0508d5c949
Content-Type: text/plain; charset=UTF-8

Hi Mark,

Please see the responses below:


Please notice the cpu0 in the string above (bold)

Yes, I see the symlinks below for



On Thu, Nov 27, 2014 at 4:17 PM, Mark Goodwin  wrote:

> On 11/27/2014 07:34 PM, Shirshendu Chakrabarti wrote:
> [...]
>
>> I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64
>>
>
> is that an SMP kernel?
>
>
Yes it is a SMP kernel:

[root@pcp-test-shir3 pmlogger]# uname -a
Linux pcp-test-shir3.ops.use1b.aws.talk.to 3.4.71-63.98.amzn1.x86_64 #1
*SMP* Tue Dec 3 01:59:29 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

please see the SMP in the string above (bold).

Also, top command, after pressing 1
top - 11:22:33 up 10 days, 16:54,  1 user,  load average: 0.00, 0.01, 0.05
Tasks:  65 total,   1 running,  64 sleeping,   0 stopped,   0 zombie
*Cpu0*  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,
 0.0%st
Mem:    608468k total,   192808k used,   415660k free,    99576k buffers
Swap:        0k total,        0k used,        0k free,    34620k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

    1 root      20   0 19356 1352 1040 S  0.0  0.2   0:00.83 init

    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd

    3 root      20   0     0    0    0 S  0.0  0.0   0:00.07 ksoftirqd/0

    4 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0

    5 root      20   0     0    0    0 S  0.0  0.0   0:00.01 kworker/u:0

    6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0

    7 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 cpuset

    8 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 khelper

    9 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kdevtmpfs

   10 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 netns

   11 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/u:1

   15 root      20   0     0    0    0 S  0.0  0.0   0:00.51 xenwatch

   16 root      20   0     0    0    0 S  0.0  0.0   0:00.00 xenbus

   83 root      20   0     0    0    0 S  0.0  0.0   0:02.49 sync_supers

   85 root      20   0     0    0    0 S  0.0  0.0   0:00.08 bdi-default

   86 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kintegrityd

   88 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kblockd

   97 root      20   0     0    0    0 S  0.0  0.0   0:36.86 kworker/0:1

  102 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 md

  200 root      20   0     0    0    0 S  0.0  0.0   0:00.24 khungtaskd

  205 root      20   0     0    0    0 S  0.0  0.0   0:00.21 kswapd0

  206 root      25   5     0    0    0 S  0.0  0.0   0:00.00 ksmd

  276 root      20   0     0    0    0 S  0.0  0.0   0:00.00 fsnotify_mark

  281 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 crypto

  288 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kthrotld




>
>> I am running, PCP-3-10.0-1.
>>
>> Please see its a t1.micro instance.
>>
>
> are there any 'cpu*' symlinks below /sys/devices/system/node/node0 ?
> and does /sys/devices/system/node/node0/cpumap exist?  This is where
> the PCP "linux" PMDA gets it's node-cpu map from.
>
>
[root@pcp-test-shir3 node0]# pwd
/sys/devices/system/node/node0
[root@pcp-test-shir3 node0]# ls -l
total 0
--w------- 1 root root 4096 Nov 26 07:29 compact
lrwxrwxrwx 1 root root    0 Nov 26 07:29 cpu0 -> ../../cpu/cpu0
-r--r--r-- 1 root root 4096 Nov 26 07:29 cpulist
*-r--r--r-- 1 root root 4096 Nov 16 18:28 cpumap*
-r--r--r-- 1 root root 4096 Nov 26 07:29 distance
drwxr-xr-x 3 root root    0 Nov 26 07:29 hugepages
-r--r--r-- 1 root root 4096 Nov 16 18:28 meminfo
lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory0 -> ../../memory/memory0
lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory1 -> ../../memory/memory1
lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory2 -> ../../memory/memory2
lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory3 -> ../../memory/memory3
lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory4 -> ../../memory/memory4
-r--r--r-- 1 root root 4096 Nov 16 18:32 numastat
drwxr-xr-x 2 root root    0 Nov 26 07:29 power
-rw-r--r-- 1 root root 4096 Nov 26 07:29 scan_unevictable_pages
lrwxrwxrwx 1 root root    0 Nov 16 18:28 subsystem -> ../../../../bus/node
-rw-r--r-- 1 root root 4096 Nov 16 18:28 uevent
-r--r--r-- 1 root root 4096 Nov 26 07:29 vmstat

I see a cpumap file, as can be seen above.

[root@pcp-test-shir3 node0]# less cpumap
00000001
cpumap (END)


>> For more details:
>> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
>> concepts_micro_instances.html
>>
>> I have a few other questions:
>>
>> 1. Does PCP have any different requirements for running on the cloud.
>> 2. Do the metrics need to be interpreted differently when running on
>> cloud.
>>
>
> I don't know enough about the Amazon environment to answer that. Is it
> possible for me to login and have a poke around? Or can we download an
> instance and run it to investigate?
>
>
I am not sure if I can get you access to one of our machines. I will have
to consult with our manager.

Thanks,

Shirshendu Chakrabarti


> Regards
> -- Mark
>
>
>> Thanks,
>>
>> Shirshendu Chakrabarti
>>
>> On Thu, Nov 27, 2014 at 1:32 PM, Mark Goodwin > > wrote:
>>
>>     On 11/27/2014 05:17 PM, Shirshendu Chakrabarti wrote:
>>
>>         Hi PCP team,
>>
>>         Could anyone please respond the question above :)
>>
>>
>>     The difference is the instance domains and counter aggregation.
>>
>>     kernel.pernode is aggregated over all CPUs on each numa node.
>>     On my laptop with 4 CPU cores on one (fake) numa node, we should
>>     expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and
>>     the sum of the per-cpu counters to equal the same, e.g. :
>>
>>     # pminfo -f kernel.{all,pernode,percpu}.__cpu.user
>>
>>
>>     kernel.all.cpu.user
>>          value 254548080
>>
>>     kernel.pernode.cpu.user
>>          inst [0 or "node0"] value 254548060
>>
>>     kernel.percpu.cpu.user
>>          inst [0 or "cpu0"] value 66377110
>>          inst [1 or "cpu1"] value 61187360
>>          inst [2 or "cpu2"] value 64051110
>>          inst [3 or "cpu3"] value 62932480
>>
>>     The mapping between CPUs and nodes is in hinv.map.cpu_node,
>>     where each CPU (instance) is mapped to a node number, e.g.
>>
>>     # pminfo -f hinv.map.cpu_node
>>
>>     hinv.map.cpu_node
>>          inst [0 or "cpu0"] value 0
>>          inst [1 or "cpu1"] value 0
>>          inst [2 or "cpu2"] value 0
>>          inst [3 or "cpu3"] value 0
>>
>>     In your case, kernel.pernode.cpu.user is zero, which isn't correct.
>>     What platform and PCP version are you running?
>>
>>     Regards
>>     -- Mark
>>
>>
>>         Thanks,
>>
>>         Shirshendu Chakrabarti
>>
>>         On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti
>>         
>>         >> wrote:
>>
>>              Hi PCP team,
>>
>>              Could anyone in the team point me to any literature which
>> explains the
>>              difference between a per node metric and percpu metric. The
>>         one-liner and
>>              extended description are absent in pernode metric case and
>> terse in
>>         percpu case.
>>
>>              For example:
>>
>>              [root@pcp-test-shir3 pmlogger]# pminfo -f
>> kernel.percpu.cpu.user
>>
>>              kernel.percpu.cpu.user
>>                   inst [0 or "cpu0"] value 993240
>>              [root@pcp-test-shir3 pmlogger]# pminfo -f
>> kernel.pernode.cpu.user
>>
>>              kernel.pernode.cpu.user
>>                   inst [0 or "node0"] value 0
>>              I was under the assumption that, kernel.pernode.* and
>> kernel.all.*
>>         metrics
>>              are the same. I am clearly missing something important.
>>
>>              Thanks,
>>
>>              Shirshendu Chakrabarti
>>
>>
>>
>>
>>         _________________________________________________
>>         pcp mailing list
>>         pcp@oss.sgi.com 
>>         http://oss.sgi.com/mailman/__listinfo/pcp
>>         
>>
>>
>>
>>
>

--001a11c2d758a351bb0508d5c949
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: base64

PGRpdiBkaXI9Imx0ciI+SGkgTWFyayw8ZGl2Pjxicj48L2Rpdj48ZGl2PlBsZWFzZSBzZWUgdGhl
IHJlc3BvbnNlcyBiZWxvdzo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2
PlBsZWFzZSBub3RpY2UgdGhlIGNwdTAgaW4gdGhlIHN0cmluZyBhYm92ZSAoYm9sZCk8L2Rpdj48
ZGl2Pjxicj48L2Rpdj48ZGl2PlllcywgSSBzZWUgdGhlIHN5bWxpbmtzIGJlbG93IGZvcsKgPC9k
aXY+PGRpdj48YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdiBjbGFzcz0iZ21haWxfZXh0cmEi
Pjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gVGh1LCBOb3YgMjcsIDIwMTQgYXQgNDox
NyBQTSwgTWFyayBHb29kd2luIDxzcGFuIGRpcj0ibHRyIj4mbHQ7PGEgaHJlZj0ibWFpbHRvOm1n
b29kd2luQHJlZGhhdC5jb20iIHRhcmdldD0iX2JsYW5rIj5tZ29vZHdpbkByZWRoYXQuY29tPC9h
PiZndDs8L3NwYW4+IHdyb3RlOjxicj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0
eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQtd2lkdGg6MXB4O2JvcmRl
ci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVyLWxlZnQtc3R5bGU6c29saWQ7cGFk
ZGluZy1sZWZ0OjFleCI+T24gMTEvMjcvMjAxNCAwNzozNCBQTSwgU2hpcnNoZW5kdSBDaGFrcmFi
YXJ0aSB3cm90ZTo8YnI+DQpbLi4uXTxzcGFuIGNsYXNzPSIiPjxicj4NCjxibG9ja3F1b3RlIGNs
YXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXIt
bGVmdC13aWR0aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXIt
bGVmdC1zdHlsZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCkkgYW0gcnVubmluZyBvbiBBbWF6
b24gTGludXggLSAzLjQuNzEtNjMuYW16bjEueDg2XzY0PGJyPg0KPC9ibG9ja3F1b3RlPg0KPGJy
Pjwvc3Bhbj4NCmlzIHRoYXQgYW4gU01QIGtlcm5lbD88c3BhbiBjbGFzcz0iIj48YnI+DQo8YnI+
PC9zcGFuPjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PjxkaXY+WWVzIGl0IGlzIGEg
U01QIGtlcm5lbDo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PjxkaXY+W3Jvb3RAcGNwLXRlc3Qt
c2hpcjMgcG1sb2dnZXJdIyB1bmFtZSAtYTwvZGl2PjxkaXY+TGludXggPGEgaHJlZj0iaHR0cDov
L3BjcC10ZXN0LXNoaXIzLm9wcy51c2UxYi5hd3MudGFsay50byI+cGNwLXRlc3Qtc2hpcjMub3Bz
LnVzZTFiLmF3cy50YWxrLnRvPC9hPiAzLjQuNzEtNjMuOTguYW16bjEueDg2XzY0ICMxwqA8Yj5T
TVA8L2I+wqBUdWUgRGVjIDMgMDE6NTk6MjkgVVRDIDIwMTMgeDg2XzY0IHg4Nl82NCB4ODZfNjQg
R05VL0xpbnV4PC9kaXY+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5wbGVhc2Ugc2VlIHRoZSBT
TVAgaW4gdGhlIHN0cmluZyBhYm92ZSAoYm9sZCkuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5B
bHNvLCB0b3AgY29tbWFuZCwgYWZ0ZXIgcHJlc3NpbmcgMTwvZGl2PjxkaXY+PGRpdj50b3AgLSAx
MToyMjozMyB1cCAxMCBkYXlzLCAxNjo1NCwgwqAxIHVzZXIsIMKgbG9hZCBhdmVyYWdlOiAwLjAw
LCAwLjAxLCAwLjA1PC9kaXY+PGRpdj5UYXNrczogwqA2NSB0b3RhbCwgwqAgMSBydW5uaW5nLCDC
oDY0IHNsZWVwaW5nLCDCoCAwIHN0b3BwZWQsIMKgIDAgem9tYmllPC9kaXY+PGRpdj48Yj5DcHUw
PC9iPsKgwqA6IMKgMC4zJXVzLCDCoDAuMCVzeSwgwqAwLjAlbmksIDk5LjclaWQsIMKgMC4wJXdh
LCDCoDAuMCVoaSwgwqAwLjAlc2ksIMKgMC4wJXN0PC9kaXY+PGRpdj5NZW06IMKgIMKgNjA4NDY4
ayB0b3RhbCwgwqAgMTkyODA4ayB1c2VkLCDCoCA0MTU2NjBrIGZyZWUsIMKgIMKgOTk1NzZrIGJ1
ZmZlcnM8L2Rpdj48ZGl2PlN3YXA6IMKgIMKgIMKgIMKgMGsgdG90YWwsIMKgIMKgIMKgIMKgMGsg
dXNlZCwgwqAgwqAgwqAgwqAwayBmcmVlLCDCoCDCoDM0NjIwayBjYWNoZWQ8L2Rpdj48ZGl2Pjxi
cj48L2Rpdj48ZGl2PsKgIFBJRCBVU0VSIMKgIMKgIMKgUFIgwqBOSSDCoFZJUlQgwqBSRVMgwqBT
SFIgUyAlQ1BVICVNRU0gwqAgwqBUSU1FKyDCoENPTU1BTkQgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqAgMSByb290IMKgIMKgIMKgMjAg
wqAgMCAxOTM1NiAxMzUyIDEwNDAgUyDCoDAuMCDCoDAuMiDCoCAwOjAwLjgzIGluaXQgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA8L2Rpdj48ZGl2PsKgIMKg
IDIgcm9vdCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAw
LjAgwqAgMDowMC4wMCBrdGhyZWFkZCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoDwvZGl2PjxkaXY+wqAgwqAgMyByb290IMKgIMKgIMKgMjAgwqAgMCDCoCDCoCAwIMKg
IMKgMCDCoCDCoDAgUyDCoDAuMCDCoDAuMCDCoCAwOjAwLjA3IGtzb2Z0aXJxZC8wIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIMKgIDQgcm9vdCDCoCDC
oCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4w
MCBrd29ya2VyLzA6MCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+
PGRpdj7CoCDCoCA1IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBT
IMKgMC4wIMKgMC4wIMKgIDA6MDAuMDEga3dvcmtlci91OjAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqAgNiByb290IMKgIMKgIMKgUlQgwqAgMCDC
oCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDCoDAuMCDCoCAwOjAwLjAwIG1pZ3JhdGlvbi8w
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIMKgIDcg
cm9vdCDCoCDCoCDCoCAwIC0yMCDCoCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDCoDAuMCDC
oCAwOjAwLjAwIGNwdXNldCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoDwvZGl2PjxkaXY+wqAgwqAgOCByb290IMKgIMKgIMKgIDAgLTIwIMKgIMKgIDAgwqAgwqAw
IMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAga2hlbHBlciDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7CoCDCoCA5IHJvb3QgwqAgwqAg
wqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAg
a2RldnRtcGZzIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48
ZGl2PsKgIMKgMTAgcm9vdCDCoCDCoCDCoCAwIC0yMCDCoCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDC
oDAuMCDCoDAuMCDCoCAwOjAwLjAwIG5ldG5zIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIMKgMTEgcm9vdCDCoCDCoCDCoDIwIMKgIDAg
wqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBrd29ya2VyL3U6
MSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7CoCDCoDE1
IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4w
IMKgIDA6MDAuNTEgeGVud2F0Y2ggwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqA8L2Rpdj48ZGl2PsKgIMKgMTYgcm9vdCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDC
oDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCB4ZW5idXMgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA8L2Rpdj48ZGl2PsKgIMKgODMgcm9vdCDCoCDC
oCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMi40
OSBzeW5jX3N1cGVycyDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+
PGRpdj7CoCDCoDg1IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBT
IMKgMC4wIMKgMC4wIMKgIDA6MDAuMDggYmRpLWRlZmF1bHQgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqA4NiByb290IMKgIMKgIMKgIDAgLTIwIMKg
IMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAga2ludGVncml0eWQg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqA4OCBy
b290IMKgIMKgIMKgIDAgLTIwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKg
IDA6MDAuMDAga2Jsb2NrZCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oMKgPC9kaXY+PGRpdj7CoCDCoDk3IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAw
IMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MzYuODYga3dvcmtlci8wOjEgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgMTAyIHJvb3QgwqAgwqAgwqAg
MCAtMjAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBtZCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDwvZGl2Pjxk
aXY+wqAgMjAwIHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKg
MC4wIMKgMC4wIMKgIDA6MDAuMjQga2h1bmd0YXNrZCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoDwvZGl2PjxkaXY+wqAgMjA1IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKg
IDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMjEga3N3YXBkMCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7CoCAyMDYgcm9v
dCDCoCDCoCDCoDI1IMKgIDUgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAg
MDowMC4wMCBrc21kIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgPC9kaXY+PGRpdj7CoCAyNzYgcm9vdCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAg
wqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBmc25vdGlmeV9tYXJrIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIDI4MSByb290IMKgIMKgIMKgIDAg
LTIwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAgY3J5cHRv
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgPC9kaXY+PGRpdj7C
oCAyODggcm9vdCDCoCDCoCDCoCAwIC0yMCDCoCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDC
oDAuMCDCoCAwOjAwLjAwIGt0aHJvdGxkIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgPC9kaXY+PC9kaXY+PC9kaXY+PGRpdj48
YnI+PC9kaXY+PGRpdj7CoDwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5
bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0aDoxcHg7Ym9yZGVy
LWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHlsZTpzb2xpZDtwYWRk
aW5nLWxlZnQ6MWV4Ij48c3BhbiBjbGFzcz0iIj4NCjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9x
dW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0aDox
cHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHlsZTpz
b2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxicj4NCkkgYW0gcnVubmluZywgUENQLTMtMTAuMC0x
Ljxicj4NCjxicj4NClBsZWFzZSBzZWUgaXRzIGEgdDEubWljcm8gaW5zdGFuY2UuPGJyPg0KPC9i
bG9ja3F1b3RlPg0KPGJyPjwvc3Bhbj4NCmFyZSB0aGVyZSBhbnkgJiMzOTtjcHUqJiMzOTsgc3lt
bGlua3MgYmVsb3cgL3N5cy9kZXZpY2VzL3N5c3RlbS9ub2RlL25vZGUwID88YnI+DQphbmQgZG9l
cyAvc3lzL2RldmljZXMvc3lzdGVtL25vZGUvPHU+PC91Pm5vZGUwL2NwdW1hcCBleGlzdD/CoCBU
aGlzIGlzIHdoZXJlPGJyPg0KdGhlIFBDUCAmcXVvdDtsaW51eCZxdW90OyBQTURBIGdldHMgaXQm
IzM5O3Mgbm9kZS1jcHUgbWFwIGZyb20uPHNwYW4gY2xhc3M9IiI+PGJyPg0KPGJyPjwvc3Bhbj48
L2Jsb2NrcXVvdGU+PGRpdj48YnI+PC9kaXY+PGRpdj48ZGl2Pltyb290QHBjcC10ZXN0LXNoaXIz
IG5vZGUwXSMgcHdkPC9kaXY+PGRpdj4vc3lzL2RldmljZXMvc3lzdGVtL25vZGUvbm9kZTA8L2Rp
dj48ZGl2Pltyb290QHBjcC10ZXN0LXNoaXIzIG5vZGUwXSMgbHMgLWw8L2Rpdj48ZGl2PnRvdGFs
IDA8L2Rpdj48ZGl2Pi0tdy0tLS0tLS0gMSByb290IHJvb3QgNDA5NiBOb3YgMjYgMDc6MjkgY29t
cGFjdDwvZGl2PjxkaXY+bHJ3eHJ3eHJ3eCAxIHJvb3Qgcm9vdCDCoCDCoDAgTm92IDI2IDA3OjI5
IGNwdTAgLSZndDsgLi4vLi4vY3B1L2NwdTA8L2Rpdj48ZGl2Pi1yLS1yLS1yLS0gMSByb290IHJv
b3QgNDA5NiBOb3YgMjYgMDc6MjkgY3B1bGlzdDwvZGl2PjxkaXY+PGI+LXItLXItLXItLSAxIHJv
b3Qgcm9vdCA0MDk2IE5vdiAxNiAxODoyOCBjcHVtYXA8L2I+PC9kaXY+PGRpdj4tci0tci0tci0t
IDEgcm9vdCByb290IDQwOTYgTm92IDI2IDA3OjI5IGRpc3RhbmNlPC9kaXY+PGRpdj5kcnd4ci14
ci14IDMgcm9vdCByb290IMKgIMKgMCBOb3YgMjYgMDc6MjkgaHVnZXBhZ2VzPC9kaXY+PGRpdj4t
ci0tci0tci0tIDEgcm9vdCByb290IDQwOTYgTm92IDE2IDE4OjI4IG1lbWluZm88L2Rpdj48ZGl2
Pmxyd3hyd3hyd3ggMSByb290IHJvb3QgwqAgwqAwIE5vdiAyNiAwNzoyOSBtZW1vcnkwIC0mZ3Q7
IC4uLy4uL21lbW9yeS9tZW1vcnkwPC9kaXY+PGRpdj5scnd4cnd4cnd4IDEgcm9vdCByb290IMKg
IMKgMCBOb3YgMjYgMDc6MjkgbWVtb3J5MSAtJmd0OyAuLi8uLi9tZW1vcnkvbWVtb3J5MTwvZGl2
PjxkaXY+bHJ3eHJ3eHJ3eCAxIHJvb3Qgcm9vdCDCoCDCoDAgTm92IDI2IDA3OjI5IG1lbW9yeTIg
LSZndDsgLi4vLi4vbWVtb3J5L21lbW9yeTI8L2Rpdj48ZGl2Pmxyd3hyd3hyd3ggMSByb290IHJv
b3QgwqAgwqAwIE5vdiAyNiAwNzoyOSBtZW1vcnkzIC0mZ3Q7IC4uLy4uL21lbW9yeS9tZW1vcnkz
PC9kaXY+PGRpdj5scnd4cnd4cnd4IDEgcm9vdCByb290IMKgIMKgMCBOb3YgMjYgMDc6MjkgbWVt
b3J5NCAtJmd0OyAuLi8uLi9tZW1vcnkvbWVtb3J5NDwvZGl2PjxkaXY+LXItLXItLXItLSAxIHJv
b3Qgcm9vdCA0MDk2IE5vdiAxNiAxODozMiBudW1hc3RhdDwvZGl2PjxkaXY+ZHJ3eHIteHIteCAy
IHJvb3Qgcm9vdCDCoCDCoDAgTm92IDI2IDA3OjI5IHBvd2VyPC9kaXY+PGRpdj4tcnctci0tci0t
IDEgcm9vdCByb290IDQwOTYgTm92IDI2IDA3OjI5IHNjYW5fdW5ldmljdGFibGVfcGFnZXM8L2Rp
dj48ZGl2Pmxyd3hyd3hyd3ggMSByb290IHJvb3QgwqAgwqAwIE5vdiAxNiAxODoyOCBzdWJzeXN0
ZW0gLSZndDsgLi4vLi4vLi4vLi4vYnVzL25vZGU8L2Rpdj48ZGl2Pi1ydy1yLS1yLS0gMSByb290
IHJvb3QgNDA5NiBOb3YgMTYgMTg6MjggdWV2ZW50PC9kaXY+PGRpdj4tci0tci0tci0tIDEgcm9v
dCByb290IDQwOTYgTm92IDI2IDA3OjI5IHZtc3RhdDwvZGl2PjwvZGl2PjxkaXY+PGJyPjwvZGl2
PjxkaXY+SSBzZWUgYSBjcHVtYXAgZmlsZSwgYXMgY2FuIGJlIHNlZW4gYWJvdmUuPC9kaXY+PGRp
dj48YnI+PC9kaXY+PGRpdj5bcm9vdEBwY3AtdGVzdC1zaGlyMyBub2RlMF0jIGxlc3MgY3B1bWFw
PC9kaXY+PGRpdj48ZGl2PjAwMDAwMDAxPC9kaXY+PGRpdj5jcHVtYXAgKEVORCnCoDwvZGl2Pjwv
ZGl2PjxkaXY+PGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9
Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0aDoxcHg7Ym9yZGVyLWxl
ZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHlsZTpzb2xpZDtwYWRkaW5n
LWxlZnQ6MWV4Ij48c3BhbiBjbGFzcz0iIj4NCjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90
ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0aDoxcHg7
Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHlsZTpzb2xp
ZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxicj4NCkZvciBtb3JlIGRldGFpbHM6PGJyPg0KPGEgaHJl
Zj0iaHR0cDovL2RvY3MuYXdzLmFtYXpvbi5jb20vQVdTRUMyL2xhdGVzdC9Vc2VyR3VpZGUvY29u
Y2VwdHNfbWljcm9faW5zdGFuY2VzLmh0bWwiIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vZG9jcy5h
d3MuYW1hem9uLmNvbS88dT48L3U+QVdTRUMyL2xhdGVzdC9Vc2VyR3VpZGUvPHU+PC91PmNvbmNl
cHRzX21pY3JvX2luc3RhbmNlcy5odG1sPC9hPjxicj4NCjxicj4NCkkgaGF2ZSBhIGZldyBvdGhl
ciBxdWVzdGlvbnM6PGJyPg0KPGJyPg0KMS4gRG9lcyBQQ1AgaGF2ZSBhbnkgZGlmZmVyZW50IHJl
cXVpcmVtZW50cyBmb3IgcnVubmluZyBvbiB0aGUgY2xvdWQuPGJyPg0KMi4gRG8gdGhlIG1ldHJp
Y3MgbmVlZCB0byBiZSBpbnRlcnByZXRlZCBkaWZmZXJlbnRseSB3aGVuIHJ1bm5pbmcgb24gY2xv
dWQuPGJyPg0KPC9ibG9ja3F1b3RlPg0KPGJyPjwvc3Bhbj4NCkkgZG9uJiMzOTt0IGtub3cgZW5v
dWdoIGFib3V0IHRoZSBBbWF6b24gZW52aXJvbm1lbnQgdG8gYW5zd2VyIHRoYXQuIElzIGl0PGJy
Pg0KcG9zc2libGUgZm9yIG1lIHRvIGxvZ2luIGFuZCBoYXZlIGEgcG9rZSBhcm91bmQ/IE9yIGNh
biB3ZSBkb3dubG9hZCBhbjxicj4NCmluc3RhbmNlIGFuZCBydW4gaXQgdG8gaW52ZXN0aWdhdGU/
PGJyPg0KPGJyPjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PkkgYW0gbm90IHN1cmUg
aWYgSSBjYW4gZ2V0IHlvdSBhY2Nlc3MgdG8gb25lIG9mIG91ciBtYWNoaW5lcy4gSSB3aWxsIGhh
dmUgdG8gY29uc3VsdCB3aXRoIG91ciBtYW5hZ2VyLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+
VGhhbmtzLDwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+U2hpcnNoZW5kdSBDaGFrcmFiYXJ0aTwv
ZGl2PjxkaXY+wqA8L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJt
YXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQtd2lkdGg6MXB4O2JvcmRlci1sZWZ0
LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVyLWxlZnQtc3R5bGU6c29saWQ7cGFkZGluZy1s
ZWZ0OjFleCI+DQpSZWdhcmRzPGJyPg0KLS0gTWFyazxicj4NCjxicj4NCjxibG9ja3F1b3RlIGNs
YXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXIt
bGVmdC13aWR0aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXIt
bGVmdC1zdHlsZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxicj4NClRoYW5rcyw8YnI+DQo8
YnI+DQpTaGlyc2hlbmR1IENoYWtyYWJhcnRpPGJyPg0KPGJyPjxzcGFuIGNsYXNzPSIiPg0KT24g
VGh1LCBOb3YgMjcsIDIwMTQgYXQgMTozMiBQTSwgTWFyayBHb29kd2luICZsdDs8YSBocmVmPSJt
YWlsdG86bWdvb2R3aW5AcmVkaGF0LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPm1nb29kd2luQHJlZGhh
dC5jb208L2E+PGJyPjwvc3Bhbj48c3BhbiBjbGFzcz0iIj4NCiZsdDttYWlsdG86PGEgaHJlZj0i
bWFpbHRvOm1nb29kd2luQHJlZGhhdC5jb20iIHRhcmdldD0iX2JsYW5rIj5tZ29vZHdpbkByZWRo
YXQuY29tPC9hPiZndDsmZ3Q7IHdyb3RlOjxicj4NCjxicj4NCsKgIMKgIE9uIDExLzI3LzIwMTQg
MDU6MTcgUE0sIFNoaXJzaGVuZHUgQ2hha3JhYmFydGkgd3JvdGU6PGJyPg0KPGJyPg0KwqAgwqAg
wqAgwqAgSGkgUENQIHRlYW0sPGJyPg0KPGJyPg0KwqAgwqAgwqAgwqAgQ291bGQgYW55b25lIHBs
ZWFzZSByZXNwb25kIHRoZSBxdWVzdGlvbiBhYm92ZSA6KTxicj4NCjxicj4NCjxicj4NCsKgIMKg
IFRoZSBkaWZmZXJlbmNlIGlzIHRoZSBpbnN0YW5jZSBkb21haW5zIGFuZCBjb3VudGVyIGFnZ3Jl
Z2F0aW9uLjxicj4NCjxicj4NCsKgIMKgIGtlcm5lbC5wZXJub2RlIGlzIGFnZ3JlZ2F0ZWQgb3Zl
ciBhbGwgQ1BVcyBvbiBlYWNoIG51bWEgbm9kZS48YnI+DQrCoCDCoCBPbiBteSBsYXB0b3Agd2l0
aCA0IENQVSBjb3JlcyBvbiBvbmUgKGZha2UpIG51bWEgbm9kZSwgd2Ugc2hvdWxkPGJyPg0KwqAg
wqAgZXhwZWN0IGtlcm5lbC5hbGwuY3B1LnVzZXIgdG8gZXF1YWwga2VybmVsLnBlcm5vZGUuY3B1
LnVzZXIsIGFuZDxicj4NCsKgIMKgIHRoZSBzdW0gb2YgdGhlIHBlci1jcHUgY291bnRlcnMgdG8g
ZXF1YWwgdGhlIHNhbWUsIGUuZy4gOjxicj4NCjxicj48L3NwYW4+DQrCoCDCoCAjIHBtaW5mbyAt
ZiBrZXJuZWwue2FsbCxwZXJub2RlLHBlcmNwdX0uX188dT48L3U+Y3B1LnVzZXI8ZGl2PjxkaXYg
Y2xhc3M9Img1Ij48YnI+DQo8YnI+DQrCoCDCoCBrZXJuZWwuYWxsLmNwdS51c2VyPGJyPg0KwqAg
wqAgwqAgwqAgwqB2YWx1ZSAyNTQ1NDgwODA8YnI+DQo8YnI+DQrCoCDCoCBrZXJuZWwucGVybm9k
ZS5jcHUudXNlcjxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMCBvciAmcXVvdDtub2RlMCZxdW90
O10gdmFsdWUgMjU0NTQ4MDYwPGJyPg0KPGJyPg0KwqAgwqAga2VybmVsLnBlcmNwdS5jcHUudXNl
cjxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMCBvciAmcXVvdDtjcHUwJnF1b3Q7XSB2YWx1ZSA2
NjM3NzExMDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMSBvciAmcXVvdDtjcHUxJnF1b3Q7XSB2
YWx1ZSA2MTE4NzM2MDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMiBvciAmcXVvdDtjcHUyJnF1
b3Q7XSB2YWx1ZSA2NDA1MTExMDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMyBvciAmcXVvdDtj
cHUzJnF1b3Q7XSB2YWx1ZSA2MjkzMjQ4MDxicj4NCjxicj4NCsKgIMKgIFRoZSBtYXBwaW5nIGJl
dHdlZW4gQ1BVcyBhbmQgbm9kZXMgaXMgaW4gaGludi5tYXAuY3B1X25vZGUsPGJyPg0KwqAgwqAg
d2hlcmUgZWFjaCBDUFUgKGluc3RhbmNlKSBpcyBtYXBwZWQgdG8gYSBub2RlIG51bWJlciwgZS5n
Ljxicj4NCjxicj4NCsKgIMKgICMgcG1pbmZvIC1mIGhpbnYubWFwLmNwdV9ub2RlPGJyPg0KPGJy
Pg0KwqAgwqAgaGludi5tYXAuY3B1X25vZGU8YnI+DQrCoCDCoCDCoCDCoCDCoGluc3QgWzAgb3Ig
JnF1b3Q7Y3B1MCZxdW90O10gdmFsdWUgMDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMSBvciAm
cXVvdDtjcHUxJnF1b3Q7XSB2YWx1ZSAwPGJyPg0KwqAgwqAgwqAgwqAgwqBpbnN0IFsyIG9yICZx
dW90O2NwdTImcXVvdDtdIHZhbHVlIDA8YnI+DQrCoCDCoCDCoCDCoCDCoGluc3QgWzMgb3IgJnF1
b3Q7Y3B1MyZxdW90O10gdmFsdWUgMDxicj4NCjxicj4NCsKgIMKgIEluIHlvdXIgY2FzZSwga2Vy
bmVsLnBlcm5vZGUuY3B1LnVzZXIgaXMgemVybywgd2hpY2ggaXNuJiMzOTt0IGNvcnJlY3QuPGJy
Pg0KwqAgwqAgV2hhdCBwbGF0Zm9ybSBhbmQgUENQIHZlcnNpb24gYXJlIHlvdSBydW5uaW5nPzxi
cj4NCjxicj4NCsKgIMKgIFJlZ2FyZHM8YnI+DQrCoCDCoCAtLSBNYXJrPGJyPg0KPGJyPg0KPGJy
Pg0KwqAgwqAgwqAgwqAgVGhhbmtzLDxicj4NCjxicj4NCsKgIMKgIMKgIMKgIFNoaXJzaGVuZHUg
Q2hha3JhYmFydGk8YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCBPbiBXZWQsIE5vdiAyNiwgMjAxNCBh
dCAzOjQ3IFBNLCBTaGlyc2hlbmR1IENoYWtyYWJhcnRpPGJyPg0KwqAgwqAgwqAgwqAgJmx0Ozxh
IGhyZWY9Im1haWx0bzpzaGlyc2hlbmR1QHJpdmEuY28iIHRhcmdldD0iX2JsYW5rIj5zaGlyc2hl
bmR1QHJpdmEuY288L2E+ICZsdDttYWlsdG86PGEgaHJlZj0ibWFpbHRvOnNoaXJzaGVuZHVAcml2
YS5jbyIgdGFyZ2V0PSJfYmxhbmsiPnNoaXJzaGVuZHVAcml2YS5jbzwvYT4mZ3Q7PGJyPjwvZGl2
PjwvZGl2PjxzcGFuIGNsYXNzPSIiPg0KwqAgwqAgwqAgwqAgJmx0O21haWx0bzo8YSBocmVmPSJt
YWlsdG86c2hpcnNoZW5kdUByaXZhLmNvIiB0YXJnZXQ9Il9ibGFuayI+c2hpcnNoZW5kdUByaXZh
LmNvPC9hPiAmbHQ7bWFpbHRvOjxhIGhyZWY9Im1haWx0bzpzaGlyc2hlbmR1QHJpdmEuY28iIHRh
cmdldD0iX2JsYW5rIj5zaGlyc2hlbmR1QHJpdmEuY288L2E+Jmd0OyZndDsmZ3Q7IHdyb3RlOjxi
cj4NCjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgSGkgUENQIHRlYW0sPGJyPg0KPGJyPg0KwqAg
wqAgwqAgwqAgwqAgwqAgwqBDb3VsZCBhbnlvbmUgaW4gdGhlIHRlYW0gcG9pbnQgbWUgdG8gYW55
IGxpdGVyYXR1cmUgd2hpY2ggZXhwbGFpbnMgdGhlPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBk
aWZmZXJlbmNlIGJldHdlZW4gYSBwZXIgbm9kZSBtZXRyaWMgYW5kIHBlcmNwdSBtZXRyaWMuIFRo
ZTxicj4NCsKgIMKgIMKgIMKgIG9uZS1saW5lciBhbmQ8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDC
oGV4dGVuZGVkIGRlc2NyaXB0aW9uIGFyZSBhYnNlbnQgaW4gcGVybm9kZSBtZXRyaWMgY2FzZSBh
bmQgdGVyc2UgaW48YnI+DQrCoCDCoCDCoCDCoCBwZXJjcHUgY2FzZS48YnI+DQo8YnI+DQrCoCDC
oCDCoCDCoCDCoCDCoCDCoEZvciBleGFtcGxlOjxicj4NCjxicj4NCsKgIMKgIMKgIMKgIMKgIMKg
IMKgW3Jvb3RAcGNwLXRlc3Qtc2hpcjMgcG1sb2dnZXJdIyBwbWluZm8gLWYga2VybmVsLnBlcmNw
dS5jcHUudXNlcjxicj4NCjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKga2VybmVsLnBlcmNwdS5j
cHUudXNlcjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGluc3QgWzAgb3IgJnF1b3Q7
Y3B1MCZxdW90O10gdmFsdWUgOTkzMjQwPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBbcm9vdEBw
Y3AtdGVzdC1zaGlyMyBwbWxvZ2dlcl0jIHBtaW5mbyAtZiBrZXJuZWwucGVybm9kZS5jcHUudXNl
cjxicj4NCjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKga2VybmVsLnBlcm5vZGUuY3B1LnVzZXI8
YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpbnN0IFswIG9yICZxdW90O25vZGUwJnF1
b3Q7XSB2YWx1ZSAwPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBJIHdhcyB1bmRlciB0aGUgYXNz
dW1wdGlvbiB0aGF0LCBrZXJuZWwucGVybm9kZS4qIGFuZCBrZXJuZWwuYWxsLio8YnI+DQrCoCDC
oCDCoCDCoCBtZXRyaWNzPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBhcmUgdGhlIHNhbWUuIEkg
YW0gY2xlYXJseSBtaXNzaW5nIHNvbWV0aGluZyBpbXBvcnRhbnQuPGJyPg0KPGJyPg0KwqAgwqAg
wqAgwqAgwqAgwqAgwqBUaGFua3MsPGJyPg0KPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBTaGly
c2hlbmR1IENoYWtyYWJhcnRpPGJyPg0KPGJyPg0KPGJyPg0KPGJyPg0KPGJyPjwvc3Bhbj4NCsKg
IMKgIMKgIMKgIF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzx1PjwvdT5fX19fX19fX19f
X19fX19fX19fPGJyPg0KwqAgwqAgwqAgwqAgcGNwIG1haWxpbmcgbGlzdDxicj4NCsKgIMKgIMKg
IMKgIDxhIGhyZWY9Im1haWx0bzpwY3BAb3NzLnNnaS5jb20iIHRhcmdldD0iX2JsYW5rIj5wY3BA
b3NzLnNnaS5jb208L2E+ICZsdDttYWlsdG86PGEgaHJlZj0ibWFpbHRvOnBjcEBvc3Muc2dpLmNv
bSIgdGFyZ2V0PSJfYmxhbmsiPnBjcEBvc3Muc2dpLmNvbTwvYT4mZ3Q7PGJyPg0KwqAgwqAgwqAg
wqAgPGEgaHJlZj0iaHR0cDovL29zcy5zZ2kuY29tL21haWxtYW4vX19saXN0aW5mby9wY3AiIHRh
cmdldD0iX2JsYW5rIj5odHRwOi8vb3NzLnNnaS5jb20vbWFpbG1hbi9fXzx1PjwvdT5saXN0aW5m
by9wY3A8L2E+PGJyPg0KwqAgwqAgwqAgwqAgJmx0OzxhIGhyZWY9Imh0dHA6Ly9vc3Muc2dpLmNv
bS9tYWlsbWFuL2xpc3RpbmZvL3BjcCIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9vc3Muc2dpLmNv
bS9tYWlsbWFuLzx1PjwvdT5saXN0aW5mby9wY3A8L2E+Jmd0Ozxicj4NCjxicj4NCjxicj4NCjxi
cj4NCjwvYmxvY2txdW90ZT4NCjxicj4NCjwvYmxvY2txdW90ZT48L2Rpdj48YnI+PC9kaXY+PC9k
aXY+DQo=
--001a11c2d758a351bb0508d5c949--

From shirshendu@riva.co  Thu Nov 27 06:37:17 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 8454E7F4E
	for ; Thu, 27 Nov 2014 06:37:17 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay3.corp.sgi.com (Postfix) with ESMTP id DC7D1AC003
	for ; Thu, 27 Nov 2014 04:37:13 -0800 (PST)
X-ASG-Debug-ID: 1417091827-04cbb01e5c7fbcf0001-S8gJnT
Received: from mail-qc0-f178.google.com (mail-qc0-f178.google.com [209.85.216.178]) by cuda.sgi.com with ESMTP id JuW2uNGaHohfIAK9 (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 27 Nov 2014 04:37:07 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.178
Received: by mail-qc0-f178.google.com with SMTP id b13so3806405qcw.23
        for ; Thu, 27 Nov 2014 04:37:07 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=7wjR5tDfL6t2g2EsatFSRQYgf8kuIcGDnW8J3aTKKXk=;
        b=R62HlMkFHcmRe7HeWlhvxeIldnRc8F+Rlwvu4nfx+7eLD7Alj8Y4ExpVIugaCB36Nx
         mQpKZDxq4dbl5sHwVyujb6+ibTUe5/uVXq2UWdO1fUOO/YCUYiDMm1gaCaQ8LhYoMBXF
         OF2hCDnc+j3D21OPagA2zyHEvbkrMQVJEgndeRcZM15KezQbetTMwYxjuG7GEXOYkRQP
         saNjHMoyTuArarXb0hK3o971YJkcRrgXeA9IbrlXRrZMtA6AAE69A1tgarWlDX4uOl1i
         E9or4eqxAAInBZwd0iO7O6qk0/u1BGACbG5BdzbjD3B8EBWS0d9qaRFXH/jkKaeoLw/a
         ZHsw==
X-Gm-Message-State: ALoCoQlI6FVylQ0oJueZCi5DTwiPy+/wjLlr6HSlyRW0dFyXHhlkV2juVHBx1BrGWEKqUO23LA2I
MIME-Version: 1.0
X-Received: by 10.224.65.4 with SMTP id g4mr54659702qai.20.1417091827255; Thu,
 27 Nov 2014 04:37:07 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 27 Nov 2014 04:37:07 -0800 (PST)
In-Reply-To: 
References: 
	
	<5476DA8F.2060202@redhat.com>
	
	<54770155.9000101@redhat.com>
	
Date: Thu, 27 Nov 2014 18:07:07 +0530
Message-ID: 
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: Mark Goodwin 
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11c2b980fa661d0508d66674
X-Barracuda-Connect: mail-qc0-f178.google.com[209.85.216.178]
X-Barracuda-Start-Time: 1417091827
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12117
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--001a11c2b980fa661d0508d66674
Content-Type: text/plain; charset=UTF-8

Hi Mark,

I had a talk with my manager and he has agreed to give you access to one of
our machines :)

I will setup a t1.micro machine which will replicate our environment.

Please let me know if this sounds good.

Thanks,

Shirshendu Chakrabarti

On Thu, Nov 27, 2014 at 5:23 PM, Shirshendu Chakrabarti 
wrote:

> Hi Mark,
>
> Please see the responses below:
>
>
> Please notice the cpu0 in the string above (bold)
>
> Yes, I see the symlinks below for
>
>
>
> On Thu, Nov 27, 2014 at 4:17 PM, Mark Goodwin  wrote:
>
>> On 11/27/2014 07:34 PM, Shirshendu Chakrabarti wrote:
>> [...]
>>
>>> I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64
>>>
>>
>> is that an SMP kernel?
>>
>>
> Yes it is a SMP kernel:
>
> [root@pcp-test-shir3 pmlogger]# uname -a
> Linux pcp-test-shir3.ops.use1b.aws.talk.to 3.4.71-63.98.amzn1.x86_64 #1
> *SMP* Tue Dec 3 01:59:29 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> please see the SMP in the string above (bold).
>
> Also, top command, after pressing 1
> top - 11:22:33 up 10 days, 16:54,  1 user,  load average: 0.00, 0.01, 0.05
> Tasks:  65 total,   1 running,  64 sleeping,   0 stopped,   0 zombie
> *Cpu0*  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,
>  0.0%st
> Mem:    608468k total,   192808k used,   415660k free,    99576k buffers
> Swap:        0k total,        0k used,        0k free,    34620k cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>
>     1 root      20   0 19356 1352 1040 S  0.0  0.2   0:00.83 init
>
>     2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
>
>     3 root      20   0     0    0    0 S  0.0  0.0   0:00.07 ksoftirqd/0
>
>     4 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0
>
>     5 root      20   0     0    0    0 S  0.0  0.0   0:00.01 kworker/u:0
>
>     6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
>
>     7 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 cpuset
>
>     8 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 khelper
>
>     9 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kdevtmpfs
>
>    10 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 netns
>
>    11 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/u:1
>
>    15 root      20   0     0    0    0 S  0.0  0.0   0:00.51 xenwatch
>
>    16 root      20   0     0    0    0 S  0.0  0.0   0:00.00 xenbus
>
>    83 root      20   0     0    0    0 S  0.0  0.0   0:02.49 sync_supers
>
>    85 root      20   0     0    0    0 S  0.0  0.0   0:00.08 bdi-default
>
>    86 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kintegrityd
>
>    88 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kblockd
>
>    97 root      20   0     0    0    0 S  0.0  0.0   0:36.86 kworker/0:1
>
>   102 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 md
>
>   200 root      20   0     0    0    0 S  0.0  0.0   0:00.24 khungtaskd
>
>   205 root      20   0     0    0    0 S  0.0  0.0   0:00.21 kswapd0
>
>   206 root      25   5     0    0    0 S  0.0  0.0   0:00.00 ksmd
>
>   276 root      20   0     0    0    0 S  0.0  0.0   0:00.00 fsnotify_mark
>
>   281 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 crypto
>
>   288 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kthrotld
>
>
>
>
>>
>>> I am running, PCP-3-10.0-1.
>>>
>>> Please see its a t1.micro instance.
>>>
>>
>> are there any 'cpu*' symlinks below /sys/devices/system/node/node0 ?
>> and does /sys/devices/system/node/node0/cpumap exist?  This is where
>> the PCP "linux" PMDA gets it's node-cpu map from.
>>
>>
> [root@pcp-test-shir3 node0]# pwd
> /sys/devices/system/node/node0
> [root@pcp-test-shir3 node0]# ls -l
> total 0
> --w------- 1 root root 4096 Nov 26 07:29 compact
> lrwxrwxrwx 1 root root    0 Nov 26 07:29 cpu0 -> ../../cpu/cpu0
> -r--r--r-- 1 root root 4096 Nov 26 07:29 cpulist
> *-r--r--r-- 1 root root 4096 Nov 16 18:28 cpumap*
> -r--r--r-- 1 root root 4096 Nov 26 07:29 distance
> drwxr-xr-x 3 root root    0 Nov 26 07:29 hugepages
> -r--r--r-- 1 root root 4096 Nov 16 18:28 meminfo
> lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory0 -> ../../memory/memory0
> lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory1 -> ../../memory/memory1
> lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory2 -> ../../memory/memory2
> lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory3 -> ../../memory/memory3
> lrwxrwxrwx 1 root root    0 Nov 26 07:29 memory4 -> ../../memory/memory4
> -r--r--r-- 1 root root 4096 Nov 16 18:32 numastat
> drwxr-xr-x 2 root root    0 Nov 26 07:29 power
> -rw-r--r-- 1 root root 4096 Nov 26 07:29 scan_unevictable_pages
> lrwxrwxrwx 1 root root    0 Nov 16 18:28 subsystem -> ../../../../bus/node
> -rw-r--r-- 1 root root 4096 Nov 16 18:28 uevent
> -r--r--r-- 1 root root 4096 Nov 26 07:29 vmstat
>
> I see a cpumap file, as can be seen above.
>
> [root@pcp-test-shir3 node0]# less cpumap
> 00000001
> cpumap (END)
>
>
>>> For more details:
>>> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
>>> concepts_micro_instances.html
>>>
>>> I have a few other questions:
>>>
>>> 1. Does PCP have any different requirements for running on the cloud.
>>> 2. Do the metrics need to be interpreted differently when running on
>>> cloud.
>>>
>>
>> I don't know enough about the Amazon environment to answer that. Is it
>> possible for me to login and have a poke around? Or can we download an
>> instance and run it to investigate?
>>
>>
> I am not sure if I can get you access to one of our machines. I will have
> to consult with our manager.
>
> Thanks,
>
> Shirshendu Chakrabarti
>
>
>> Regards
>> -- Mark
>>
>>
>>> Thanks,
>>>
>>> Shirshendu Chakrabarti
>>>
>>> On Thu, Nov 27, 2014 at 1:32 PM, Mark Goodwin >> > wrote:
>>>
>>>     On 11/27/2014 05:17 PM, Shirshendu Chakrabarti wrote:
>>>
>>>         Hi PCP team,
>>>
>>>         Could anyone please respond the question above :)
>>>
>>>
>>>     The difference is the instance domains and counter aggregation.
>>>
>>>     kernel.pernode is aggregated over all CPUs on each numa node.
>>>     On my laptop with 4 CPU cores on one (fake) numa node, we should
>>>     expect kernel.all.cpu.user to equal kernel.pernode.cpu.user, and
>>>     the sum of the per-cpu counters to equal the same, e.g. :
>>>
>>>     # pminfo -f kernel.{all,pernode,percpu}.__cpu.user
>>>
>>>
>>>     kernel.all.cpu.user
>>>          value 254548080
>>>
>>>     kernel.pernode.cpu.user
>>>          inst [0 or "node0"] value 254548060
>>>
>>>     kernel.percpu.cpu.user
>>>          inst [0 or "cpu0"] value 66377110
>>>          inst [1 or "cpu1"] value 61187360
>>>          inst [2 or "cpu2"] value 64051110
>>>          inst [3 or "cpu3"] value 62932480
>>>
>>>     The mapping between CPUs and nodes is in hinv.map.cpu_node,
>>>     where each CPU (instance) is mapped to a node number, e.g.
>>>
>>>     # pminfo -f hinv.map.cpu_node
>>>
>>>     hinv.map.cpu_node
>>>          inst [0 or "cpu0"] value 0
>>>          inst [1 or "cpu1"] value 0
>>>          inst [2 or "cpu2"] value 0
>>>          inst [3 or "cpu3"] value 0
>>>
>>>     In your case, kernel.pernode.cpu.user is zero, which isn't correct.
>>>     What platform and PCP version are you running?
>>>
>>>     Regards
>>>     -- Mark
>>>
>>>
>>>         Thanks,
>>>
>>>         Shirshendu Chakrabarti
>>>
>>>         On Wed, Nov 26, 2014 at 3:47 PM, Shirshendu Chakrabarti
>>>         
>>>         >> wrote:
>>>
>>>              Hi PCP team,
>>>
>>>              Could anyone in the team point me to any literature which
>>> explains the
>>>              difference between a per node metric and percpu metric. The
>>>         one-liner and
>>>              extended description are absent in pernode metric case and
>>> terse in
>>>         percpu case.
>>>
>>>              For example:
>>>
>>>              [root@pcp-test-shir3 pmlogger]# pminfo -f
>>> kernel.percpu.cpu.user
>>>
>>>              kernel.percpu.cpu.user
>>>                   inst [0 or "cpu0"] value 993240
>>>              [root@pcp-test-shir3 pmlogger]# pminfo -f
>>> kernel.pernode.cpu.user
>>>
>>>              kernel.pernode.cpu.user
>>>                   inst [0 or "node0"] value 0
>>>              I was under the assumption that, kernel.pernode.* and
>>> kernel.all.*
>>>         metrics
>>>              are the same. I am clearly missing something important.
>>>
>>>              Thanks,
>>>
>>>              Shirshendu Chakrabarti
>>>
>>>
>>>
>>>
>>>         _________________________________________________
>>>         pcp mailing list
>>>         pcp@oss.sgi.com 
>>>         http://oss.sgi.com/mailman/__listinfo/pcp
>>>         
>>>
>>>
>>>
>>>
>>
>

--001a11c2b980fa661d0508d66674
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: base64

PGRpdiBkaXI9Imx0ciI+SGkgTWFyayw8ZGl2Pjxicj48L2Rpdj48ZGl2PkkgaGFkIGEgdGFsayB3
aXRoIG15IG1hbmFnZXIgYW5kIGhlIGhhcyBhZ3JlZWQgdG8gZ2l2ZSB5b3UgYWNjZXNzIHRvIG9u
ZSBvZiBvdXIgbWFjaGluZXMgOik8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pkkgd2lsbCBzZXR1
cCBhIHQxLm1pY3JvIG1hY2hpbmUgd2hpY2ggd2lsbCByZXBsaWNhdGUgb3VyIGVudmlyb25tZW50
LjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+UGxlYXNlIGxldCBtZSBrbm93IGlmIHRoaXMgc291
bmRzIGdvb2QuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5UaGFua3MsPC9kaXY+PGRpdj48YnI+
PC9kaXY+PGRpdj5TaGlyc2hlbmR1IENoYWtyYWJhcnRpPC9kaXY+PC9kaXY+PGRpdiBjbGFzcz0i
Z21haWxfZXh0cmEiPjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gVGh1LCBOb3YgMjcs
IDIwMTQgYXQgNToyMyBQTSwgU2hpcnNoZW5kdSBDaGFrcmFiYXJ0aSA8c3BhbiBkaXI9Imx0ciI+
Jmx0OzxhIGhyZWY9Im1haWx0bzpzaGlyc2hlbmR1QHJpdmEuY28iIHRhcmdldD0iX2JsYW5rIj5z
aGlyc2hlbmR1QHJpdmEuY288L2E+Jmd0Ozwvc3Bhbj4gd3JvdGU6PGJyPjxibG9ja3F1b3RlIGNs
YXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMCAuOGV4O2JvcmRlci1sZWZ0OjFw
eCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPjxkaXYgZGlyPSJsdHIiPkhpIE1hcmssPGRp
dj48YnI+PC9kaXY+PGRpdj5QbGVhc2Ugc2VlIHRoZSByZXNwb25zZXMgYmVsb3c6PC9kaXY+PGRp
dj48YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5QbGVhc2Ugbm90aWNlIHRoZSBjcHUwIGlu
IHRoZSBzdHJpbmcgYWJvdmUgKGJvbGQpPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5ZZXMsIEkg
c2VlIHRoZSBzeW1saW5rcyBiZWxvdyBmb3LCoDwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PGJy
PjwvZGl2PjxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj48YnI+PGRpdiBjbGFzcz0iZ21haWxfcXVv
dGUiPjxzcGFuIGNsYXNzPSIiPk9uIFRodSwgTm92IDI3LCAyMDE0IGF0IDQ6MTcgUE0sIE1hcmsg
R29vZHdpbiA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhIGhyZWY9Im1haWx0bzptZ29vZHdpbkByZWRo
YXQuY29tIiB0YXJnZXQ9Il9ibGFuayI+bWdvb2R3aW5AcmVkaGF0LmNvbTwvYT4mZ3Q7PC9zcGFu
PiB3cm90ZTo8YnI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2lu
OjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0LXdpZHRoOjFweDtib3JkZXItbGVmdC1jb2xv
cjpyZ2IoMjA0LDIwNCwyMDQpO2JvcmRlci1sZWZ0LXN0eWxlOnNvbGlkO3BhZGRpbmctbGVmdDox
ZXgiPk9uIDExLzI3LzIwMTQgMDc6MzQgUE0sIFNoaXJzaGVuZHUgQ2hha3JhYmFydGkgd3JvdGU6
PGJyPg0KWy4uLl08c3Bhbj48YnI+DQo8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0
eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQtd2lkdGg6MXB4O2JvcmRl
ci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVyLWxlZnQtc3R5bGU6c29saWQ7cGFk
ZGluZy1sZWZ0OjFleCI+DQpJIGFtIHJ1bm5pbmcgb24gQW1hem9uIExpbnV4IC0gMy40LjcxLTYz
LmFtem4xLng4Nl82NDxicj4NCjwvYmxvY2txdW90ZT4NCjxicj48L3NwYW4+DQppcyB0aGF0IGFu
IFNNUCBrZXJuZWw/PHNwYW4+PGJyPg0KPGJyPjwvc3Bhbj48L2Jsb2NrcXVvdGU+PGRpdj48YnI+
PC9kaXY+PC9zcGFuPjxkaXY+PGRpdj5ZZXMgaXQgaXMgYSBTTVAga2VybmVsOjwvZGl2PjxkaXY+
PGJyPjwvZGl2PjxkaXY+PGRpdj5bcm9vdEBwY3AtdGVzdC1zaGlyMyBwbWxvZ2dlcl0jIHVuYW1l
IC1hPC9kaXY+PGRpdj5MaW51eCA8YSBocmVmPSJodHRwOi8vcGNwLXRlc3Qtc2hpcjMub3BzLnVz
ZTFiLmF3cy50YWxrLnRvIiB0YXJnZXQ9Il9ibGFuayI+cGNwLXRlc3Qtc2hpcjMub3BzLnVzZTFi
LmF3cy50YWxrLnRvPC9hPiAzLjQuNzEtNjMuOTguYW16bjEueDg2XzY0ICMxwqA8Yj5TTVA8L2I+
wqBUdWUgRGVjIDMgMDE6NTk6MjkgVVRDIDIwMTMgeDg2XzY0IHg4Nl82NCB4ODZfNjQgR05VL0xp
bnV4PC9kaXY+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5wbGVhc2Ugc2VlIHRoZSBTTVAgaW4g
dGhlIHN0cmluZyBhYm92ZSAoYm9sZCkuPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5BbHNvLCB0
b3AgY29tbWFuZCwgYWZ0ZXIgcHJlc3NpbmcgMTwvZGl2PjxkaXY+PGRpdj50b3AgLSAxMToyMjoz
MyB1cCAxMCBkYXlzLCAxNjo1NCwgwqAxIHVzZXIsIMKgbG9hZCBhdmVyYWdlOiAwLjAwLCAwLjAx
LCAwLjA1PC9kaXY+PGRpdj5UYXNrczogwqA2NSB0b3RhbCwgwqAgMSBydW5uaW5nLCDCoDY0IHNs
ZWVwaW5nLCDCoCAwIHN0b3BwZWQsIMKgIDAgem9tYmllPC9kaXY+PGRpdj48Yj5DcHUwPC9iPsKg
wqA6IMKgMC4zJXVzLCDCoDAuMCVzeSwgwqAwLjAlbmksIDk5LjclaWQsIMKgMC4wJXdhLCDCoDAu
MCVoaSwgwqAwLjAlc2ksIMKgMC4wJXN0PC9kaXY+PGRpdj5NZW06IMKgIMKgNjA4NDY4ayB0b3Rh
bCwgwqAgMTkyODA4ayB1c2VkLCDCoCA0MTU2NjBrIGZyZWUsIMKgIMKgOTk1NzZrIGJ1ZmZlcnM8
L2Rpdj48ZGl2PlN3YXA6IMKgIMKgIMKgIMKgMGsgdG90YWwsIMKgIMKgIMKgIMKgMGsgdXNlZCwg
wqAgwqAgwqAgwqAwayBmcmVlLCDCoCDCoDM0NjIwayBjYWNoZWQ8L2Rpdj48ZGl2Pjxicj48L2Rp
dj48ZGl2PsKgIFBJRCBVU0VSIMKgIMKgIMKgUFIgwqBOSSDCoFZJUlQgwqBSRVMgwqBTSFIgUyAl
Q1BVICVNRU0gwqAgwqBUSU1FKyDCoENPTU1BTkQgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqAgMSByb290IMKgIMKgIMKgMjAgwqAgMCAx
OTM1NiAxMzUyIDEwNDAgUyDCoDAuMCDCoDAuMiDCoCAwOjAwLjgzIGluaXQgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA8L2Rpdj48ZGl2PsKgIMKgIDIgcm9v
dCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAg
MDowMC4wMCBrdGhyZWFkZCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oDwvZGl2PjxkaXY+wqAgwqAgMyByb290IMKgIMKgIMKgMjAgwqAgMCDCoCDCoCAwIMKgIMKgMCDC
oCDCoDAgUyDCoDAuMCDCoDAuMCDCoCAwOjAwLjA3IGtzb2Z0aXJxZC8wIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIMKgIDQgcm9vdCDCoCDCoCDCoDIw
IMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBrd29y
a2VyLzA6MCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7C
oCDCoCA1IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4w
IMKgMC4wIMKgIDA6MDAuMDEga3dvcmtlci91OjAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqAgNiByb290IMKgIMKgIMKgUlQgwqAgMCDCoCDCoCAw
IMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDCoDAuMCDCoCAwOjAwLjAwIG1pZ3JhdGlvbi8wIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIMKgIDcgcm9vdCDC
oCDCoCDCoCAwIC0yMCDCoCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDCoDAuMCDCoCAwOjAw
LjAwIGNwdXNldCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDwv
ZGl2PjxkaXY+wqAgwqAgOCByb290IMKgIMKgIMKgIDAgLTIwIMKgIMKgIDAgwqAgwqAwIMKgIMKg
MCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAga2hlbHBlciDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7CoCDCoCA5IHJvb3QgwqAgwqAgwqAyMCDC
oCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAga2RldnRt
cGZzIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKg
IMKgMTAgcm9vdCDCoCDCoCDCoCAwIC0yMCDCoCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDC
oDAuMCDCoCAwOjAwLjAwIG5ldG5zIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIMKgMTEgcm9vdCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAg
MCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBrd29ya2VyL3U6MSDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7CoCDCoDE1IHJvb3Qg
wqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6
MDAuNTEgeGVud2F0Y2ggwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA8
L2Rpdj48ZGl2PsKgIMKgMTYgcm9vdCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAg
wqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCB4ZW5idXMgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA8L2Rpdj48ZGl2PsKgIMKgODMgcm9vdCDCoCDCoCDCoDIw
IMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMi40OSBzeW5j
X3N1cGVycyDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7C
oCDCoDg1IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4w
IMKgMC4wIMKgIDA6MDAuMDggYmRpLWRlZmF1bHQgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqA4NiByb290IMKgIMKgIMKgIDAgLTIwIMKgIMKgIDAg
wqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAga2ludGVncml0eWQgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgwqA4OCByb290IMKg
IMKgIMKgIDAgLTIwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAu
MDAga2Jsb2NrZCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9k
aXY+PGRpdj7CoCDCoDk3IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKg
MCBTIMKgMC4wIMKgMC4wIMKgIDA6MzYuODYga3dvcmtlci8wOjEgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqAgwqAgwqAgwqDCoDwvZGl2PjxkaXY+wqAgMTAyIHJvb3QgwqAgwqAgwqAgMCAtMjAg
wqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBtZCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDwvZGl2PjxkaXY+wqAg
MjAwIHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKg
MC4wIMKgIDA6MDAuMjQga2h1bmd0YXNrZCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoDwvZGl2PjxkaXY+wqAgMjA1IHJvb3QgwqAgwqAgwqAyMCDCoCAwIMKgIMKgIDAgwqAg
wqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMjEga3N3YXBkMCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoMKgPC9kaXY+PGRpdj7CoCAyMDYgcm9vdCDCoCDC
oCDCoDI1IMKgIDUgwqAgwqAgMCDCoCDCoDAgwqAgwqAwIFMgwqAwLjAgwqAwLjAgwqAgMDowMC4w
MCBrc21kIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgPC9k
aXY+PGRpdj7CoCAyNzYgcm9vdCDCoCDCoCDCoDIwIMKgIDAgwqAgwqAgMCDCoCDCoDAgwqAgwqAw
IFMgwqAwLjAgwqAwLjAgwqAgMDowMC4wMCBmc25vdGlmeV9tYXJrIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgwqA8L2Rpdj48ZGl2PsKgIDI4MSByb290IMKgIMKgIMKgIDAgLTIwIMKg
IMKgIDAgwqAgwqAwIMKgIMKgMCBTIMKgMC4wIMKgMC4wIMKgIDA6MDAuMDAgY3J5cHRvIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgPC9kaXY+PGRpdj7CoCAyODgg
cm9vdCDCoCDCoCDCoCAwIC0yMCDCoCDCoCAwIMKgIMKgMCDCoCDCoDAgUyDCoDAuMCDCoDAuMCDC
oCAwOjAwLjAwIGt0aHJvdGxkIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgPC9kaXY+PC9kaXY+PC9kaXY+PHNwYW4gY2xhc3M9
IiI+PGRpdj48YnI+PC9kaXY+PGRpdj7CoDwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9x
dW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0aDox
cHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHlsZTpz
b2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48c3Bhbj4NCjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9x
dW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdC13aWR0aDox
cHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTtib3JkZXItbGVmdC1zdHlsZTpz
b2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NCjxicj4NCkkgYW0gcnVubmluZywgUENQLTMtMTAuMC0x
Ljxicj4NCjxicj4NClBsZWFzZSBzZWUgaXRzIGEgdDEubWljcm8gaW5zdGFuY2UuPGJyPg0KPC9i
bG9ja3F1b3RlPg0KPGJyPjwvc3Bhbj4NCmFyZSB0aGVyZSBhbnkgJiMzOTtjcHUqJiMzOTsgc3lt
bGlua3MgYmVsb3cgL3N5cy9kZXZpY2VzL3N5c3RlbS9ub2RlL25vZGUwID88YnI+DQphbmQgZG9l
cyAvc3lzL2RldmljZXMvc3lzdGVtL25vZGUvPHU+PC91Pm5vZGUwL2NwdW1hcCBleGlzdD/CoCBU
aGlzIGlzIHdoZXJlPGJyPg0KdGhlIFBDUCAmcXVvdDtsaW51eCZxdW90OyBQTURBIGdldHMgaXQm
IzM5O3Mgbm9kZS1jcHUgbWFwIGZyb20uPHNwYW4+PGJyPg0KPGJyPjwvc3Bhbj48L2Jsb2NrcXVv
dGU+PGRpdj48YnI+PC9kaXY+PC9zcGFuPjxkaXY+PGRpdj5bcm9vdEBwY3AtdGVzdC1zaGlyMyBu
b2RlMF0jIHB3ZDwvZGl2PjxkaXY+L3N5cy9kZXZpY2VzL3N5c3RlbS9ub2RlL25vZGUwPC9kaXY+
PGRpdj5bcm9vdEBwY3AtdGVzdC1zaGlyMyBub2RlMF0jIGxzIC1sPC9kaXY+PGRpdj50b3RhbCAw
PC9kaXY+PGRpdj4tLXctLS0tLS0tIDEgcm9vdCByb290IDQwOTYgTm92IDI2IDA3OjI5IGNvbXBh
Y3Q8L2Rpdj48ZGl2Pmxyd3hyd3hyd3ggMSByb290IHJvb3QgwqAgwqAwIE5vdiAyNiAwNzoyOSBj
cHUwIC0mZ3Q7IC4uLy4uL2NwdS9jcHUwPC9kaXY+PGRpdj4tci0tci0tci0tIDEgcm9vdCByb290
IDQwOTYgTm92IDI2IDA3OjI5IGNwdWxpc3Q8L2Rpdj48ZGl2PjxiPi1yLS1yLS1yLS0gMSByb290
IHJvb3QgNDA5NiBOb3YgMTYgMTg6MjggY3B1bWFwPC9iPjwvZGl2PjxkaXY+LXItLXItLXItLSAx
IHJvb3Qgcm9vdCA0MDk2IE5vdiAyNiAwNzoyOSBkaXN0YW5jZTwvZGl2PjxkaXY+ZHJ3eHIteHIt
eCAzIHJvb3Qgcm9vdCDCoCDCoDAgTm92IDI2IDA3OjI5IGh1Z2VwYWdlczwvZGl2PjxkaXY+LXIt
LXItLXItLSAxIHJvb3Qgcm9vdCA0MDk2IE5vdiAxNiAxODoyOCBtZW1pbmZvPC9kaXY+PGRpdj5s
cnd4cnd4cnd4IDEgcm9vdCByb290IMKgIMKgMCBOb3YgMjYgMDc6MjkgbWVtb3J5MCAtJmd0OyAu
Li8uLi9tZW1vcnkvbWVtb3J5MDwvZGl2PjxkaXY+bHJ3eHJ3eHJ3eCAxIHJvb3Qgcm9vdCDCoCDC
oDAgTm92IDI2IDA3OjI5IG1lbW9yeTEgLSZndDsgLi4vLi4vbWVtb3J5L21lbW9yeTE8L2Rpdj48
ZGl2Pmxyd3hyd3hyd3ggMSByb290IHJvb3QgwqAgwqAwIE5vdiAyNiAwNzoyOSBtZW1vcnkyIC0m
Z3Q7IC4uLy4uL21lbW9yeS9tZW1vcnkyPC9kaXY+PGRpdj5scnd4cnd4cnd4IDEgcm9vdCByb290
IMKgIMKgMCBOb3YgMjYgMDc6MjkgbWVtb3J5MyAtJmd0OyAuLi8uLi9tZW1vcnkvbWVtb3J5Mzwv
ZGl2PjxkaXY+bHJ3eHJ3eHJ3eCAxIHJvb3Qgcm9vdCDCoCDCoDAgTm92IDI2IDA3OjI5IG1lbW9y
eTQgLSZndDsgLi4vLi4vbWVtb3J5L21lbW9yeTQ8L2Rpdj48ZGl2Pi1yLS1yLS1yLS0gMSByb290
IHJvb3QgNDA5NiBOb3YgMTYgMTg6MzIgbnVtYXN0YXQ8L2Rpdj48ZGl2PmRyd3hyLXhyLXggMiBy
b290IHJvb3QgwqAgwqAwIE5vdiAyNiAwNzoyOSBwb3dlcjwvZGl2PjxkaXY+LXJ3LXItLXItLSAx
IHJvb3Qgcm9vdCA0MDk2IE5vdiAyNiAwNzoyOSBzY2FuX3VuZXZpY3RhYmxlX3BhZ2VzPC9kaXY+
PGRpdj5scnd4cnd4cnd4IDEgcm9vdCByb290IMKgIMKgMCBOb3YgMTYgMTg6Mjggc3Vic3lzdGVt
IC0mZ3Q7IC4uLy4uLy4uLy4uL2J1cy9ub2RlPC9kaXY+PGRpdj4tcnctci0tci0tIDEgcm9vdCBy
b290IDQwOTYgTm92IDE2IDE4OjI4IHVldmVudDwvZGl2PjxkaXY+LXItLXItLXItLSAxIHJvb3Qg
cm9vdCA0MDk2IE5vdiAyNiAwNzoyOSB2bXN0YXQ8L2Rpdj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48
ZGl2Pkkgc2VlIGEgY3B1bWFwIGZpbGUsIGFzIGNhbiBiZSBzZWVuIGFib3ZlLjwvZGl2PjxkaXY+
PGJyPjwvZGl2PjxkaXY+W3Jvb3RAcGNwLXRlc3Qtc2hpcjMgbm9kZTBdIyBsZXNzIGNwdW1hcDwv
ZGl2PjxkaXY+PGRpdj4wMDAwMDAwMTwvZGl2PjxkaXY+Y3B1bWFwIChFTkQpwqA8L2Rpdj48L2Rp
dj48c3BhbiBjbGFzcz0iIj48ZGl2Pjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxf
cXVvdGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQtd2lkdGg6
MXB4O2JvcmRlci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVyLWxlZnQtc3R5bGU6
c29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PHNwYW4+DQo8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxf
cXVvdGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQtd2lkdGg6
MXB4O2JvcmRlci1sZWZ0LWNvbG9yOnJnYigyMDQsMjA0LDIwNCk7Ym9yZGVyLWxlZnQtc3R5bGU6
c29saWQ7cGFkZGluZy1sZWZ0OjFleCI+DQo8YnI+DQpGb3IgbW9yZSBkZXRhaWxzOjxicj4NCjxh
IGhyZWY9Imh0dHA6Ly9kb2NzLmF3cy5hbWF6b24uY29tL0FXU0VDMi9sYXRlc3QvVXNlckd1aWRl
L2NvbmNlcHRzX21pY3JvX2luc3RhbmNlcy5odG1sIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL2Rv
Y3MuYXdzLmFtYXpvbi5jb20vPHU+PC91PkFXU0VDMi9sYXRlc3QvVXNlckd1aWRlLzx1PjwvdT5j
b25jZXB0c19taWNyb19pbnN0YW5jZXMuaHRtbDwvYT48YnI+DQo8YnI+DQpJIGhhdmUgYSBmZXcg
b3RoZXIgcXVlc3Rpb25zOjxicj4NCjxicj4NCjEuIERvZXMgUENQIGhhdmUgYW55IGRpZmZlcmVu
dCByZXF1aXJlbWVudHMgZm9yIHJ1bm5pbmcgb24gdGhlIGNsb3VkLjxicj4NCjIuIERvIHRoZSBt
ZXRyaWNzIG5lZWQgdG8gYmUgaW50ZXJwcmV0ZWQgZGlmZmVyZW50bHkgd2hlbiBydW5uaW5nIG9u
IGNsb3VkLjxicj4NCjwvYmxvY2txdW90ZT4NCjxicj48L3NwYW4+DQpJIGRvbiYjMzk7dCBrbm93
IGVub3VnaCBhYm91dCB0aGUgQW1hem9uIGVudmlyb25tZW50IHRvIGFuc3dlciB0aGF0LiBJcyBp
dDxicj4NCnBvc3NpYmxlIGZvciBtZSB0byBsb2dpbiBhbmQgaGF2ZSBhIHBva2UgYXJvdW5kPyBP
ciBjYW4gd2UgZG93bmxvYWQgYW48YnI+DQppbnN0YW5jZSBhbmQgcnVuIGl0IHRvIGludmVzdGln
YXRlPzxicj4NCjxicj48L2Jsb2NrcXVvdGU+PGRpdj48YnI+PC9kaXY+PC9zcGFuPjxkaXY+SSBh
bSBub3Qgc3VyZSBpZiBJIGNhbiBnZXQgeW91IGFjY2VzcyB0byBvbmUgb2Ygb3VyIG1hY2hpbmVz
LiBJIHdpbGwgaGF2ZSB0byBjb25zdWx0IHdpdGggb3VyIG1hbmFnZXIuPC9kaXY+PGRpdj48YnI+
PC9kaXY+PGRpdj5UaGFua3MsPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5TaGlyc2hlbmR1IENo
YWtyYWJhcnRpPC9kaXY+PGRpdj48ZGl2IGNsYXNzPSJoNSI+PGRpdj7CoDwvZGl2PjxibG9ja3F1
b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDti
b3JkZXItbGVmdC13aWR0aDoxcHg7Ym9yZGVyLWxlZnQtY29sb3I6cmdiKDIwNCwyMDQsMjA0KTti
b3JkZXItbGVmdC1zdHlsZTpzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4NClJlZ2FyZHM8YnI+DQot
LSBNYXJrPGJyPg0KPGJyPg0KPGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0i
bWFyZ2luOjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0LXdpZHRoOjFweDtib3JkZXItbGVm
dC1jb2xvcjpyZ2IoMjA0LDIwNCwyMDQpO2JvcmRlci1sZWZ0LXN0eWxlOnNvbGlkO3BhZGRpbmct
bGVmdDoxZXgiPg0KPGJyPg0KVGhhbmtzLDxicj4NCjxicj4NClNoaXJzaGVuZHUgQ2hha3JhYmFy
dGk8YnI+DQo8YnI+PHNwYW4+DQpPbiBUaHUsIE5vdiAyNywgMjAxNCBhdCAxOjMyIFBNLCBNYXJr
IEdvb2R3aW4gJmx0OzxhIGhyZWY9Im1haWx0bzptZ29vZHdpbkByZWRoYXQuY29tIiB0YXJnZXQ9
Il9ibGFuayI+bWdvb2R3aW5AcmVkaGF0LmNvbTwvYT48YnI+PC9zcGFuPjxzcGFuPg0KJmx0O21h
aWx0bzo8YSBocmVmPSJtYWlsdG86bWdvb2R3aW5AcmVkaGF0LmNvbSIgdGFyZ2V0PSJfYmxhbmsi
Pm1nb29kd2luQHJlZGhhdC5jb208L2E+Jmd0OyZndDsgd3JvdGU6PGJyPg0KPGJyPg0KwqAgwqAg
T24gMTEvMjcvMjAxNCAwNToxNyBQTSwgU2hpcnNoZW5kdSBDaGFrcmFiYXJ0aSB3cm90ZTo8YnI+
DQo8YnI+DQrCoCDCoCDCoCDCoCBIaSBQQ1AgdGVhbSw8YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCBD
b3VsZCBhbnlvbmUgcGxlYXNlIHJlc3BvbmQgdGhlIHF1ZXN0aW9uIGFib3ZlIDopPGJyPg0KPGJy
Pg0KPGJyPg0KwqAgwqAgVGhlIGRpZmZlcmVuY2UgaXMgdGhlIGluc3RhbmNlIGRvbWFpbnMgYW5k
IGNvdW50ZXIgYWdncmVnYXRpb24uPGJyPg0KPGJyPg0KwqAgwqAga2VybmVsLnBlcm5vZGUgaXMg
YWdncmVnYXRlZCBvdmVyIGFsbCBDUFVzIG9uIGVhY2ggbnVtYSBub2RlLjxicj4NCsKgIMKgIE9u
IG15IGxhcHRvcCB3aXRoIDQgQ1BVIGNvcmVzIG9uIG9uZSAoZmFrZSkgbnVtYSBub2RlLCB3ZSBz
aG91bGQ8YnI+DQrCoCDCoCBleHBlY3Qga2VybmVsLmFsbC5jcHUudXNlciB0byBlcXVhbCBrZXJu
ZWwucGVybm9kZS5jcHUudXNlciwgYW5kPGJyPg0KwqAgwqAgdGhlIHN1bSBvZiB0aGUgcGVyLWNw
dSBjb3VudGVycyB0byBlcXVhbCB0aGUgc2FtZSwgZS5nLiA6PGJyPg0KPGJyPjwvc3Bhbj4NCsKg
IMKgICMgcG1pbmZvIC1mIGtlcm5lbC57YWxsLHBlcm5vZGUscGVyY3B1fS5fXzx1PjwvdT5jcHUu
dXNlcjxkaXY+PGRpdj48YnI+DQo8YnI+DQrCoCDCoCBrZXJuZWwuYWxsLmNwdS51c2VyPGJyPg0K
wqAgwqAgwqAgwqAgwqB2YWx1ZSAyNTQ1NDgwODA8YnI+DQo8YnI+DQrCoCDCoCBrZXJuZWwucGVy
bm9kZS5jcHUudXNlcjxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMCBvciAmcXVvdDtub2RlMCZx
dW90O10gdmFsdWUgMjU0NTQ4MDYwPGJyPg0KPGJyPg0KwqAgwqAga2VybmVsLnBlcmNwdS5jcHUu
dXNlcjxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMCBvciAmcXVvdDtjcHUwJnF1b3Q7XSB2YWx1
ZSA2NjM3NzExMDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMSBvciAmcXVvdDtjcHUxJnF1b3Q7
XSB2YWx1ZSA2MTE4NzM2MDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMiBvciAmcXVvdDtjcHUy
JnF1b3Q7XSB2YWx1ZSA2NDA1MTExMDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMyBvciAmcXVv
dDtjcHUzJnF1b3Q7XSB2YWx1ZSA2MjkzMjQ4MDxicj4NCjxicj4NCsKgIMKgIFRoZSBtYXBwaW5n
IGJldHdlZW4gQ1BVcyBhbmQgbm9kZXMgaXMgaW4gaGludi5tYXAuY3B1X25vZGUsPGJyPg0KwqAg
wqAgd2hlcmUgZWFjaCBDUFUgKGluc3RhbmNlKSBpcyBtYXBwZWQgdG8gYSBub2RlIG51bWJlciwg
ZS5nLjxicj4NCjxicj4NCsKgIMKgICMgcG1pbmZvIC1mIGhpbnYubWFwLmNwdV9ub2RlPGJyPg0K
PGJyPg0KwqAgwqAgaGludi5tYXAuY3B1X25vZGU8YnI+DQrCoCDCoCDCoCDCoCDCoGluc3QgWzAg
b3IgJnF1b3Q7Y3B1MCZxdW90O10gdmFsdWUgMDxicj4NCsKgIMKgIMKgIMKgIMKgaW5zdCBbMSBv
ciAmcXVvdDtjcHUxJnF1b3Q7XSB2YWx1ZSAwPGJyPg0KwqAgwqAgwqAgwqAgwqBpbnN0IFsyIG9y
ICZxdW90O2NwdTImcXVvdDtdIHZhbHVlIDA8YnI+DQrCoCDCoCDCoCDCoCDCoGluc3QgWzMgb3Ig
JnF1b3Q7Y3B1MyZxdW90O10gdmFsdWUgMDxicj4NCjxicj4NCsKgIMKgIEluIHlvdXIgY2FzZSwg
a2VybmVsLnBlcm5vZGUuY3B1LnVzZXIgaXMgemVybywgd2hpY2ggaXNuJiMzOTt0IGNvcnJlY3Qu
PGJyPg0KwqAgwqAgV2hhdCBwbGF0Zm9ybSBhbmQgUENQIHZlcnNpb24gYXJlIHlvdSBydW5uaW5n
Pzxicj4NCjxicj4NCsKgIMKgIFJlZ2FyZHM8YnI+DQrCoCDCoCAtLSBNYXJrPGJyPg0KPGJyPg0K
PGJyPg0KwqAgwqAgwqAgwqAgVGhhbmtzLDxicj4NCjxicj4NCsKgIMKgIMKgIMKgIFNoaXJzaGVu
ZHUgQ2hha3JhYmFydGk8YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCBPbiBXZWQsIE5vdiAyNiwgMjAx
NCBhdCAzOjQ3IFBNLCBTaGlyc2hlbmR1IENoYWtyYWJhcnRpPGJyPg0KwqAgwqAgwqAgwqAgJmx0
OzxhIGhyZWY9Im1haWx0bzpzaGlyc2hlbmR1QHJpdmEuY28iIHRhcmdldD0iX2JsYW5rIj5zaGly
c2hlbmR1QHJpdmEuY288L2E+ICZsdDttYWlsdG86PGEgaHJlZj0ibWFpbHRvOnNoaXJzaGVuZHVA
cml2YS5jbyIgdGFyZ2V0PSJfYmxhbmsiPnNoaXJzaGVuZHVAcml2YS5jbzwvYT4mZ3Q7PGJyPjwv
ZGl2PjwvZGl2PjxzcGFuPg0KwqAgwqAgwqAgwqAgJmx0O21haWx0bzo8YSBocmVmPSJtYWlsdG86
c2hpcnNoZW5kdUByaXZhLmNvIiB0YXJnZXQ9Il9ibGFuayI+c2hpcnNoZW5kdUByaXZhLmNvPC9h
PiAmbHQ7bWFpbHRvOjxhIGhyZWY9Im1haWx0bzpzaGlyc2hlbmR1QHJpdmEuY28iIHRhcmdldD0i
X2JsYW5rIj5zaGlyc2hlbmR1QHJpdmEuY288L2E+Jmd0OyZndDsmZ3Q7IHdyb3RlOjxicj4NCjxi
cj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgSGkgUENQIHRlYW0sPGJyPg0KPGJyPg0KwqAgwqAgwqAg
wqAgwqAgwqAgwqBDb3VsZCBhbnlvbmUgaW4gdGhlIHRlYW0gcG9pbnQgbWUgdG8gYW55IGxpdGVy
YXR1cmUgd2hpY2ggZXhwbGFpbnMgdGhlPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBkaWZmZXJl
bmNlIGJldHdlZW4gYSBwZXIgbm9kZSBtZXRyaWMgYW5kIHBlcmNwdSBtZXRyaWMuIFRoZTxicj4N
CsKgIMKgIMKgIMKgIG9uZS1saW5lciBhbmQ8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoGV4dGVu
ZGVkIGRlc2NyaXB0aW9uIGFyZSBhYnNlbnQgaW4gcGVybm9kZSBtZXRyaWMgY2FzZSBhbmQgdGVy
c2UgaW48YnI+DQrCoCDCoCDCoCDCoCBwZXJjcHUgY2FzZS48YnI+DQo8YnI+DQrCoCDCoCDCoCDC
oCDCoCDCoCDCoEZvciBleGFtcGxlOjxicj4NCjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgW3Jv
b3RAcGNwLXRlc3Qtc2hpcjMgcG1sb2dnZXJdIyBwbWluZm8gLWYga2VybmVsLnBlcmNwdS5jcHUu
dXNlcjxicj4NCjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKga2VybmVsLnBlcmNwdS5jcHUudXNl
cjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGluc3QgWzAgb3IgJnF1b3Q7Y3B1MCZx
dW90O10gdmFsdWUgOTkzMjQwPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBbcm9vdEBwY3AtdGVz
dC1zaGlyMyBwbWxvZ2dlcl0jIHBtaW5mbyAtZiBrZXJuZWwucGVybm9kZS5jcHUudXNlcjxicj4N
Cjxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKga2VybmVsLnBlcm5vZGUuY3B1LnVzZXI8YnI+DQrC
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpbnN0IFswIG9yICZxdW90O25vZGUwJnF1b3Q7XSB2
YWx1ZSAwPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBJIHdhcyB1bmRlciB0aGUgYXNzdW1wdGlv
biB0aGF0LCBrZXJuZWwucGVybm9kZS4qIGFuZCBrZXJuZWwuYWxsLio8YnI+DQrCoCDCoCDCoCDC
oCBtZXRyaWNzPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBhcmUgdGhlIHNhbWUuIEkgYW0gY2xl
YXJseSBtaXNzaW5nIHNvbWV0aGluZyBpbXBvcnRhbnQuPGJyPg0KPGJyPg0KwqAgwqAgwqAgwqAg
wqAgwqAgwqBUaGFua3MsPGJyPg0KPGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqBTaGlyc2hlbmR1
IENoYWtyYWJhcnRpPGJyPg0KPGJyPg0KPGJyPg0KPGJyPg0KPGJyPjwvc3Bhbj4NCsKgIMKgIMKg
IMKgIF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzx1PjwvdT5fX19fX19fX19fX19fX19f
X19fPGJyPg0KwqAgwqAgwqAgwqAgcGNwIG1haWxpbmcgbGlzdDxicj4NCsKgIMKgIMKgIMKgIDxh
IGhyZWY9Im1haWx0bzpwY3BAb3NzLnNnaS5jb20iIHRhcmdldD0iX2JsYW5rIj5wY3BAb3NzLnNn
aS5jb208L2E+ICZsdDttYWlsdG86PGEgaHJlZj0ibWFpbHRvOnBjcEBvc3Muc2dpLmNvbSIgdGFy
Z2V0PSJfYmxhbmsiPnBjcEBvc3Muc2dpLmNvbTwvYT4mZ3Q7PGJyPg0KwqAgwqAgwqAgwqAgPGEg
aHJlZj0iaHR0cDovL29zcy5zZ2kuY29tL21haWxtYW4vX19saXN0aW5mby9wY3AiIHRhcmdldD0i
X2JsYW5rIj5odHRwOi8vb3NzLnNnaS5jb20vbWFpbG1hbi9fXzx1PjwvdT5saXN0aW5mby9wY3A8
L2E+PGJyPg0KwqAgwqAgwqAgwqAgJmx0OzxhIGhyZWY9Imh0dHA6Ly9vc3Muc2dpLmNvbS9tYWls
bWFuL2xpc3RpbmZvL3BjcCIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9vc3Muc2dpLmNvbS9tYWls
bWFuLzx1PjwvdT5saXN0aW5mby9wY3A8L2E+Jmd0Ozxicj4NCjxicj4NCjxicj4NCjxicj4NCjwv
YmxvY2txdW90ZT4NCjxicj4NCjwvYmxvY2txdW90ZT48L2Rpdj48L2Rpdj48L2Rpdj48YnI+PC9k
aXY+PC9kaXY+DQo8L2Jsb2NrcXVvdGU+PC9kaXY+PGJyPjwvZGl2Pg0K
--001a11c2b980fa661d0508d66674--

From michele@acksyn.org  Thu Nov 27 06:38:02 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id C5A357F4E
	for ; Thu, 27 Nov 2014 06:38:02 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
	by relay2.corp.sgi.com (Postfix) with ESMTP id B3A3D304039
	for ; Thu, 27 Nov 2014 04:38:02 -0800 (PST)
X-ASG-Debug-ID: 1417091880-04cbb01e5a7fc1b0001-S8gJnT
Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id 6rs1QWEAvUbrjtcG for ; Thu, 27 Nov 2014 04:38:01 -0800 (PST)
X-Barracuda-Envelope-From: michele@acksyn.org
X-Barracuda-Apparent-Source-IP: 5.9.7.26
Received: from localhost (localhost [127.0.0.1])
	by palahniuk.acksyn.org (Postfix) with ESMTP id 763A528FD3;
	Thu, 27 Nov 2014 07:38:00 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h=
	user-agent:in-reply-to:content-disposition:content-type
	:content-type:mime-version:references:message-id:subject:subject
	:from:from:date:date:received:received; s=2010; t=1417091879;
	 bh=w8ThEKeSUqkEoLquQGfHshAwrKKIVD2pZyt7UfPWgJQ=; b=V9VPC/g+AKhR
	i23z/AN2iVvTwbuHh3gxqn3byS75xWh/p/VzuwDQIbawsKjiBc4BUD77c930/xKb
	1KhzxMVvaCIWGyeeC/b7xtzgXZnTU9YPo7dag2TtRylFGUOOVGllP+ITlwnj+0aK
	npUuKSmKAiOFZYpAvYVgyNFH6jrFc1k=
Received: from palahniuk.acksyn.org ([127.0.0.1])
	by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026)
	with ESMTP id 40nHfCZbUdOF; Thu, 27 Nov 2014 07:37:59 -0500 (EST)
Received: from localhost (host54-190-dynamic.24-79-r.retail.telecomitalia.it [79.24.190.54])
	by palahniuk.acksyn.org (Postfix) with ESMTPSA id 417E72657A;
	Thu, 27 Nov 2014 07:37:59 -0500 (EST)
Date: Thu, 27 Nov 2014 13:37:58 +0100
From: Michele Baldessari 
To: Mark Goodwin 
Cc: Shirshendu Chakrabarti , pcp@oss.sgi.com
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and
 kernel.pernode.cpu.user
Message-ID: <20141127123758.GA19904@marquez.int.rhx>
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and
 kernel.pernode.cpu.user
References: 
 
 <5476DA8F.2060202@redhat.com>
 
 <54770155.9000101@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <54770155.9000101@redhat.com>
User-Agent: Mutt/1.5.21 (2012-12-30)
X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26]
X-Barracuda-Start-Time: 1417091881
X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12117
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	-0.00 DKIM_VERIFIED          Domain Keys Identified Mail: signature passes
	                           verification
	0.00 DKIM_SIGNED            Domain Keys Identified Mail: message has a signature

On Thu, Nov 27, 2014 at 09:47:49PM +1100, Mark Goodwin wrote:
> On 11/27/2014 07:34 PM, Shirshendu Chakrabarti wrote:
> [...]
> >I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64
> 
> is that an SMP kernel?
[...]

> I don't know enough about the Amazon environment to answer that. Is it
> possible for me to login and have a poke around? Or can we download an
> instance and run it to investigate?

I'd guess this is related to how XEN exposes this stuff? AFAIK amazon
runs on XEN

cheers,
Michele
-- 
Michele Baldessari            
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D

From shirshendu@riva.co  Thu Nov 27 06:44:42 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
	by oss.sgi.com (Postfix) with ESMTP id 2F7287F4E
	for ; Thu, 27 Nov 2014 06:44:42 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay3.corp.sgi.com (Postfix) with ESMTP id A3C32AC007
	for ; Thu, 27 Nov 2014 04:44:41 -0800 (PST)
X-ASG-Debug-ID: 1417092276-04cb6c057082b390001-S8gJnT
Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by cuda.sgi.com with ESMTP id n4Nva2JvPvwZ5AfH (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Thu, 27 Nov 2014 04:44:36 -0800 (PST)
X-Barracuda-Envelope-From: shirshendu@riva.co
X-Barracuda-Apparent-Source-IP: 209.85.216.182
Received: by mail-qc0-f182.google.com with SMTP id r5so3520947qcx.13
        for ; Thu, 27 Nov 2014 04:44:36 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type;
        bh=965gcGtMUNjNqhH1g0HfYMplPsKOT1yDFkkD1n7DMdk=;
        b=AkdvQ2H8xh6hYOCXBVBrpd4ylCiv+1vd7mSfHgRPpW/khrzG8aMYaq1sjbGSLh+v6H
         MmGWGaHIBguXxsIC91/3EtwBdx9tJStbL7XdpE4Ot0MtYZe76LRPFJFx2SLzrFp9ddPj
         py50WheBHXFOY8ka+4qTkEUbXTr7nvCYHN99CT8HWrHEg4LQUmWkCRr9/ZDMHOo/6+mI
         TAAEhOrtoNpBVSu88/Nuobpov33Zp97fS4+Nx+oTPaxMZPm3OxOw3Dqu6y2dtjyK8rjh
         fLsQe22/QfvIYfR3jZDVG01KiIzQfmVxJT4M5evdW9tjInC2gkQcDikpb1JFDyxueF4Z
         CyjA==
X-Gm-Message-State: ALoCoQmegQOwU2i0wingskJrMc7J9pJ2Ff559gt1fxdgGUVoH8Fxrgd4GQks14lH/GYdiWO9Espt
MIME-Version: 1.0
X-Received: by 10.224.93.18 with SMTP id t18mr54390243qam.102.1417092276382;
 Thu, 27 Nov 2014 04:44:36 -0800 (PST)
Received: by 10.140.22.201 with HTTP; Thu, 27 Nov 2014 04:44:36 -0800 (PST)
In-Reply-To: <20141127123758.GA19904@marquez.int.rhx>
References: 
	
	<5476DA8F.2060202@redhat.com>
	
	<54770155.9000101@redhat.com>
	<20141127123758.GA19904@marquez.int.rhx>
Date: Thu, 27 Nov 2014 18:14:36 +0530
Message-ID: 
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
From: Shirshendu Chakrabarti 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
To: Michele Baldessari 
Cc: Mark Goodwin , pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=089e0149c9b4bf839b0508d68183
X-Barracuda-Connect: mail-qc0-f182.google.com[209.85.216.182]
X-Barracuda-Start-Time: 1417092276
X-Barracuda-Encrypted: RC4-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12117
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.00 HTML_MESSAGE           BODY: HTML included in message

--089e0149c9b4bf839b0508d68183
Content-Type: text/plain; charset=UTF-8

Hi Michele,

Yes, AWS runs on Xen. Though new AWS EC2 machines use HVM.

Please note, we use older generation machines, i.e., Xen based.

Thanks,

Shirshendu Chakrabarti

On Thu, Nov 27, 2014 at 6:07 PM, Michele Baldessari 
wrote:

> On Thu, Nov 27, 2014 at 09:47:49PM +1100, Mark Goodwin wrote:
> > On 11/27/2014 07:34 PM, Shirshendu Chakrabarti wrote:
> > [...]
> > >I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64
> >
> > is that an SMP kernel?
> [...]
>
> > I don't know enough about the Amazon environment to answer that. Is it
> > possible for me to login and have a poke around? Or can we download an
> > instance and run it to investigate?
>
> I'd guess this is related to how XEN exposes this stuff? AFAIK amazon
> runs on XEN
>
> cheers,
> Michele
> --
> Michele Baldessari            
> C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D
>

--089e0149c9b4bf839b0508d68183
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Michele,

Yes, AWS runs on Xen. Thoug=
h new AWS EC2 machines use HVM.

Please note, we us=
e older generation machines, i.e., Xen based.

Than=
ks,

Shirshendu Chakrabarti

On Thu, Nov 27, 2014 at 6:0=
7 PM, Michele Baldessari <michele@acksyn.org> wrote:
On Thu, Nov 27, 2014 at 09:47:=
49PM +1100, Mark Goodwin wrote:

> On 11/27/2014 07:34 PM, Shirshendu Chakrabarti wrote:

> [...]

> >I am running on Amazon Linux - 3.4.71-63.amzn1.x86_64

>

> is that an SMP kernel?

[...]



> I don't know enough about the Amazon environment to answer that. I=
s it

> possible for me to login and have a poke around? Or can we download an=


> instance and run it to investigate?



I'd guess this is related to how XEN exposes this stuff? AFAIK a=
mazon

runs on XEN



cheers,

Michele

--

Michele Baldessari=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <michele@acksyn.org>

C2A5 9DA3 9961 4FFB E01B=C2=A0 D0BC DDD4 DCCB 7515 5C6D





--089e0149c9b4bf839b0508d68183--

From mgoodwin@redhat.com  Thu Nov 27 17:33:04 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 85ABE7F4E
	for ; Thu, 27 Nov 2014 17:33:04 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 662DD304048
	for ; Thu, 27 Nov 2014 15:33:04 -0800 (PST)
X-ASG-Debug-ID: 1417131179-04bdf0615fa04cb0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id wW0usEsPQ7c2sxW1 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 27 Nov 2014 15:33:00 -0800 (PST)
X-Barracuda-Envelope-From: mgoodwin@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sARNWxuc015563
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL);
	Thu, 27 Nov 2014 18:32:59 -0500
Received: from [10.64.48.241] (vpn1-48-241.bne.redhat.com [10.64.48.241])
	by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sARNWvXM022772;
	Thu, 27 Nov 2014 18:32:58 -0500
Message-ID: <5477B4A9.8040801@redhat.com>
Date: Fri, 28 Nov 2014 10:32:57 +1100
From: Mark Goodwin 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
MIME-Version: 1.0
To: Shirshendu Chakrabarti 
CC: pcp@oss.sgi.com
Subject: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
References: 		<5476DA8F.2060202@redhat.com>		<54770155.9000101@redhat.com>	 
X-ASG-Orig-Subj: Re: [pcp] Difference between: kernel.percpu.cpu.user and kernel.pernode.cpu.user
In-Reply-To: 
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1417131180
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

On 11/27/2014 11:37 PM, Shirshendu Chakrabarti wrote:
> Hi Mark,
>
> I had a talk with my manager and he has agreed to give you access to one of our
> machines :)
>
> I will setup a t1.micro machine which will replicate our environment.
>
> Please let me know if this sounds good.

yes that sounds good - it would be even better if we could set up one
of these systems in our QA lab.

>         are there any 'cpu*' symlinks below /sys/devices/system/node/node0 ?
>         and does /sys/devices/system/node/__node0/cpumap exist?  This is where
>         the PCP "linux" PMDA gets it's node-cpu map from.
>
>
[...]
>
>     I see a cpumap file, as can be seen above.
>
>     [root@pcp-test-shir3 node0]# less cpumap
>     00000001
>     cpumap (END)

that cpumap is in a different format to my system (fedora fc19 / x86_64).
Can you run the follwoing please to check on the node-cpu mapping on your
system :

# pminfo -DAPPL2 -L -f kernel.{percpu,pernode}.cpu.user

Thanks
-- Mark

From nscott@redhat.com  Thu Nov 27 21:30:52 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 14DFF7F4E
	for ; Thu, 27 Nov 2014 21:30:52 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 04120304048
	for ; Thu, 27 Nov 2014 19:30:51 -0800 (PST)
X-ASG-Debug-ID: 1417145446-04cb6c05738c4270001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id XQ7DV9QjDPUNqCuG (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 27 Nov 2014 19:30:47 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAS3UgXM001590;
	Thu, 27 Nov 2014 22:30:42 -0500
Date: Thu, 27 Nov 2014 22:30:42 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: Ken McDonell 
Cc: pcp 
Message-ID: <8149026.6994945.1417145442845.JavaMail.zimbra@redhat.com>
In-Reply-To: <1224656795.6945862.1417136806064.JavaMail.zimbra@redhat.com>
Subject: Regression in qa/628
MIME-Version: 1.0
X-ASG-Orig-Subj: Regression in qa/628
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: Regression in qa/628
Thread-Index: HiiQSGXbemOjUFKO70iV7fh8KfFM3w==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1417145447
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12143
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Hi Ken,

I'm seeing the following failure on several hosts:

$ ./check -q -l 628
628 27s ... - output mismatch (see 628.out.bad)
4c4
<     value 3
---
> No value(s) available!
16c16
<     value 4
---
> No value(s) available!

I've git bisected it to the recent e_ext_t changes, I think (its
behaviour seems to have some dependence on uninitialised memory,
so its not 100% certainly that commit).  I've never seen it fail
without that commit in play though, and since I've never seen it
before either, it has to be something very recent and something
that affects pmcd/pmda_simple.so - that one's the only candidate
change I can see.

This test is exercising 2 DSO PMDAs and basic value fetches.  They
both use PMDA_INTERFACE_2, which looks like a factor - the change
to fetch callback return code being zero/non-zero was after v2.  A
'No values' being returned from pminfo may indicate PMDA version
confusion at the pmcd (libpcp_pmda) end of the connection?

Beyond that, its proving really difficult to diagnose.  Could I
get you to take a closer look when you get some time?  I think it
is best to revert this change for this release, and reinstate it
in dev for 3.10.2 while investigations continue.  Let me know if
its something that is immediately obvious, or if not reproducible
I'll continue the hunt next week.

thanks!

--
Nathan

From nscott@redhat.com  Fri Nov 28 00:35:44 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
	by oss.sgi.com (Postfix) with ESMTP id A99B97F4E
	for ; Fri, 28 Nov 2014 00:35:44 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay1.corp.sgi.com (Postfix) with ESMTP id 992A68F8054
	for ; Thu, 27 Nov 2014 22:35:41 -0800 (PST)
X-ASG-Debug-ID: 1417156539-04cb6c05738e4d30001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id etucYqgAhtiAoRg9 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 27 Nov 2014 22:35:39 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id sAS6ZcbK028504
	for ; Fri, 28 Nov 2014 01:35:38 -0500
Date: Fri, 28 Nov 2014 01:35:38 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <955932433.7025171.1417156538806.JavaMail.zimbra@redhat.com>
In-Reply-To: <1184932149.7025036.1417156451937.JavaMail.zimbra@redhat.com>
Subject: pcp updates: minor pending-release tweaks
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: minor pending-release tweaks
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: minor pending-release tweaks
Thread-Index: TQFZvcBYsrEeeTY6G1VDqPGOkT+rYA==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1417156539
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12147
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Nathan Scott (3):
      Revert "libpcp_pmda: change e_ext_t to expose pmdaInterface"
      pmda papi: log errors that occur with no client context
      docs: update changelog for pending release

 CHANGELOG                      |   16 +++++++++++++
 src/libpcp_pmda/src/callback.c |   14 ++++++------
 src/libpcp_pmda/src/libdefs.h  |    8 +++---
 src/libpcp_pmda/src/open.c     |    2 -
 src/pmdas/papi/papi.c          |   47 ++++++++++++++++++++++-------------------
 5 files changed, 53 insertions(+), 34 deletions(-)

From lberk@redhat.com  Fri Nov 28 16:46:50 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id 1ED2A7F3F
	for ; Fri, 28 Nov 2014 16:46:50 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 0E784304059
	for ; Fri, 28 Nov 2014 14:46:46 -0800 (PST)
X-ASG-Debug-ID: 1417214805-04cb6c0573962050001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id ucWTicNt9mKnwke9 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 28 Nov 2014 14:46:46 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
	by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sASMkibB030792
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
	for ; Fri, 28 Nov 2014 17:46:45 -0500
Received: from toium ([10.15.16.195])
	by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sASMkh3v000546
	(version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO)
	for ; Fri, 28 Nov 2014 17:46:44 -0500
From: Lukas Berk 
To: pcp@oss.sgi.com
Subject: pcp updates: qa improvements and spin-rawhide tweak
Date: Fri, 28 Nov 2014 17:46:43 -0500
X-ASG-Orig-Subj: pcp updates: qa improvements and spin-rawhide tweak
Message-ID: <871tonymcs.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1417214806
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1

Hi,

Changes commited to lberk/dev on git://sourceware.org/git/pcpfans.git 

Initial results of the qa runs on my fedora-rawhide spins showed some
failing tests that should not have been run.  For example, the pmdapapi
tests were failing because there were no metrics present (same for the
perfevent pmda).  These changes create basic awareness tests and _notrun
accordingly.  I've also tweaked the spin-rawhide script with updated git
urls (pointed to git.pcp.io now), and dropped the dependency on a
specific SourceX: keyword.

build/rpm/spin-rawhide |   17 ++++++++---------
qa/661                 |    2 ++
qa/757                 |    4 +++-
qa/813                 |   14 ++++++++++++++
qa/903                 |    1 +
qa/914                 |    7 +++++++
qa/967                 |   14 ++++++++++++++
7 files changed, 49 insertions(+), 10 deletions(-)

commit 6dbc5cab917e12aa13a252d15f204c6f2bed143a
Author: Lukas Berk 
Date:   Fri Nov 28 17:39:41 2014 -0500

    More qa tweaks based on availablity of papi metrics on current hardware
    
    qa/903 - test for PAPI availability
    qa/914 - test for PAPI availability

commit 94ab15ed9c4a1b96d399e24cf35eb5b8b2dea2d4
Author: Lukas Berk 
Date:   Fri Nov 28 17:17:56 2014 -0500

    Tweak spin-rawhide
    
    Attempt to make spin-rawhide even more robust by making pcp-webjs
    no longer dependent on a specific SourceX: keyword.  Also, change
    repository urls to pcp.io based.

commit d6455eeb2d6f93bff232a3b967ac349776f5df09
Author: Lukas Berk 
Date:   Fri Nov 28 16:55:10 2014 -0500

    qa fixes for situations where tests shouldn't be run
    
    qa/661 - add _notrun test for simplejson.tool
    qa/757 - add _notrun test for when no perfevents are found
    qa/813 - add _notrun test for when no pmu metrics are found
    qa/867 - add _notrun test for when no pmu metrics are found

From nscott@redhat.com  Sun Nov 30 20:17:40 2014
Return-Path: 
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
	version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
	by oss.sgi.com (Postfix) with ESMTP id A482D7F59
	for ; Sun, 30 Nov 2014 20:17:40 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
	by relay2.corp.sgi.com (Postfix) with ESMTP id 81259304032
	for ; Sun, 30 Nov 2014 18:17:37 -0800 (PST)
X-ASG-Debug-ID: 1417400252-04cb6c0570be9d10001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id I5J0zFNVirM47BDf (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 30 Nov 2014 18:17:32 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
	by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sB12HVPJ014183
	for ; Sun, 30 Nov 2014 21:17:31 -0500
Date: Sun, 30 Nov 2014 21:17:31 -0500 (EST)
From: Nathan Scott 
Reply-To: Nathan Scott 
To: pcp 
Message-ID: <1824686059.7643583.1417400251602.JavaMail.zimbra@redhat.com>
In-Reply-To: <546072657.7643463.1417400145203.JavaMail.zimbra@redhat.com>
Subject: pcp updates: qa, pmdaproc, builds
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: qa, pmdaproc, builds
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.5.82.6]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: qa, pmdaproc, builds
Thread-Index: +JieYS5zQx2oOtb5A5OjpLHb67NxAA==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1417400252
X-Barracuda-Encrypted: AES256-SHA
X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.12266
	Rule breakdown below
	 pts rule name              description
	---- ---------------------- --------------------------------------------------
	0.01 THREAD_INDEX           thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
	0.01 THREAD_TOPIC           Thread-Topic: ...(Japanese Subject)...

Changes committed to git://git.pcp.io/pcp.git dev

Lukas Berk (3):
      qa fixes for situations where tests shouldn't be run
      Tweak spin-rawhide
      More qa tweaks based on availablity of papi metrics on current hardware

Nathan Scott (3):
      docs: update changelog for pending release
      pmda proc: add defensive coding measures for recent kernels
      build: update packaging for pending release

 CHANGELOG                       |   16 +++++++++++++++-
 build/rpm/fedora.spec           |    8 ++++----
 build/rpm/spin-rawhide          |   17 ++++++++---------
 debian/changelog                |    2 +-
 qa/661                          |    2 ++
 qa/757                          |    4 +++-
 qa/813                          |   14 ++++++++++++++
 qa/903                          |    1 +
 qa/914                          |    7 +++++++
 qa/967                          |   14 ++++++++++++++
 src/pmdas/linux_proc/pmda.c     |    2 +-
 src/pmdas/linux_proc/proc_pid.c |   14 ++++++++++++++
 12 files changed, 84 insertions(+), 17 deletions(-)