From nscott@redhat.com Tue Apr 1 01:50:10 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 99C9A7F37 for ; Tue, 1 Apr 2014 01:50:10 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 3A303AC006 for ; Mon, 31 Mar 2014 23:50:07 -0700 (PDT) X-ASG-Debug-ID: 1396335001-04bdf05daabbaaf0001-S8gJnT Received: from mx3-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id Bi2ROmI2n1PySTsy for ; Mon, 31 Mar 2014 23:50:02 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s316o1rr024274 for ; Tue, 1 Apr 2014 02:50:01 -0400 Date: Tue, 1 Apr 2014 02:50:01 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <217346373.5127001.1396335001736.JavaMail.zimbra@redhat.com> In-Reply-To: <603402953.5126803.1396334966784.JavaMail.zimbra@redhat.com> Subject: pcp updates: python vs getopts (wip) MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: python vs getopts (wip) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.6] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: python vs getopts (wip) Thread-Index: 1M3+9PMrud3pVCO/twSBykJiQW3FhQ== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1396335002 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4479 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/nathans/pcp.git dev qa/src/GNUlocaldefs | 1 qa/src/test_pcp_getopts.python | 110 +++++ qa/src/test_pcp_options.python | 93 ++++ src/pcp/uptime/GNUmakefile | 28 + src/pcp/uptime/pcp-uptime.py | 121 ++++++ src/python/pcp/pmapi.py | 260 ++++++++++++- src/python/pmapi.c | 809 ++++++++++++++++++++++++++++++++++++++++- 7 files changed, 1403 insertions(+), 19 deletions(-) commit 1b4dedbabcc9eab396e28b34ca5dfeab8f1a4277 Author: Nathan Scott Date: Tue Apr 1 17:47:26 2014 +1100 Initial Python script support for auto-command-line-parsing Support for long options and automatic PCP option parsing on the command lines of python monitor tools. A new pmOptions class is added to the pmapi python module, and a classmethod on the pmContext class allows it to be injected into the new context creation process. A sample program pcp-uptime.py is added here, which will go on to become one of the pcp(1) sub-commands talked about so long ago now. It's output-compatible with the Linux uptime command, but supports historical queries like "what was the machine uptime/loadavg/nusers at time X on day Y". Two test scripts are also added to exercise the python APIs. Further work remains to drive these scripts via qa/check... will follow soon, next commit. From wwwrun@oss.sgi.com Tue Apr 1 13:22:55 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: by oss.sgi.com (Postfix, from userid 30) id 0F93C7F59; Tue, 1 Apr 2014 13:22:55 -0500 (CDT) From: bugzilla-daemon@oss.sgi.com To: pcp@oss.sgi.com Subject: [Bug 1035] PMCD Should Not Fail to Start if NSS Fails to Initialize Date: Tue, 01 Apr 2014 18:22:54 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Classification: Unclassified X-Bugzilla-Product: pcp X-Bugzilla-Component: pcp X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: brolley@redhat.com X-Bugzilla-Status: RESOLVED X-Bugzilla-Priority: P5 X-Bugzilla-Assigned-To: brolley@redhat.com X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: multipart/alternative; boundary="1396376574.Cf1a5D0.25093"; charset="us-ascii" X-Bugzilla-URL: http://oss.sgi.com/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 --1396376574.Cf1a5D0.25093 Date: Tue, 1 Apr 2014 13:22:54 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" http://oss.sgi.com/bugzilla/show_bug.cgi?id=1035 brolley@redhat.com changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution|--- |FIXED --- Comment #2 from brolley@redhat.com --- Some discussion about possibly changing __pmSecureServerSetup() to return void since, with this change, it always returns zero. It was decided to leave it as-is in case a non-zero return code becomes necessary in the future. -- You are receiving this mail because: You are on the CC list for the bug. --1396376574.Cf1a5D0.25093 Date: Tue, 1 Apr 2014 13:22:54 -0500 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" changed bug 1035
What Removed Added
Status ASSIGNED RESOLVED
Resolution --- FIXED

Comment # 2 on bug 1035 from 
Some discussion about possibly changing __pmSecureServerSetup() to return void
since, with this change, it always returns zero. It was decided to leave it
as-is in case a non-zero return code becomes necessary in the future.

You are receiving this mail because:
  • You are on the CC list for the bug.
--1396376574.Cf1a5D0.25093-- From nscott@redhat.com Tue Apr 1 17:50:19 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 5DA997F5F for ; Tue, 1 Apr 2014 17:50:19 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 1E4858F8064 for ; Tue, 1 Apr 2014 15:50:15 -0700 (PDT) X-ASG-Debug-ID: 1396392584-04bdf0743a05b90001-S8gJnT Received: from mx3-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id ioG4SNu3Hs7niNer for ; Tue, 01 Apr 2014 15:49:45 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s31MnijD014510 for ; Tue, 1 Apr 2014 18:49:44 -0400 Date: Tue, 1 Apr 2014 18:49:44 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <139916861.5673703.1396392584584.JavaMail.zimbra@redhat.com> In-Reply-To: <1128463178.5657276.1396389913870.JavaMail.zimbra@redhat.com> Subject: 2 weeks out from pcp-3.9.2 MIME-Version: 1.0 X-ASG-Orig-Subj: 2 weeks out from pcp-3.9.2 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.6] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: 2 weeks out from pcp-3.9.2 Thread-Index: CtdmstNn051MDfBGUiNHfp9Orn8rQw== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1396392584 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4500 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi all, This is just a general heads-up that we're two weeks out from the next PCP point release, as discussed at the last dev meet. If you have big work-in-progress items, please think about when you'd like to land those, bearing in mind several days leading up to the release will see increased merge/QA effort all-round. The following release is always an option, too, and will arrive with the usual boring regularity - same time next month. Please take some time to review outstanding trees around the place, so it doesn't all get left to that final week. It goes without saying that reviewed and tested code makes for an easy merge ... so do help out. Anyone can review and test anyone elses changes, of course - just dive right in! IIRC, both Frank and Stan have outstanding review requests ATM and there's some juicy python tidbits in my personal tree too, if that takes your fancy. I believe work is ongoing with Toms python fix below - Tom and Stan are working on that currently, but more eyes would surely be welcomed there too. http://oss.sgi.com/pipermail/pcp/2014-March/004637.html git://sourceware.org/git/pcpfans.git fche/dev git://sourceware.org/git/pcpfans.git scox/dev git://oss.sgi.com/git/nathans/pcp.git dev cheers. -- Nathan From nscott@redhat.com Wed Apr 2 01:32:54 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id E2EE07F6A for ; Wed, 2 Apr 2014 01:32:53 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 58505AC005 for ; Tue, 1 Apr 2014 23:32:53 -0700 (PDT) X-ASG-Debug-ID: 1396420367-04cb6c5675be39e0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id 1rJcmHH904jeFnq9 for ; Tue, 01 Apr 2014 23:32:48 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s326Wl1F012023 for ; Wed, 2 Apr 2014 02:32:47 -0400 Date: Wed, 2 Apr 2014 02:32:47 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <1208175917.5830503.1396420367454.JavaMail.zimbra@redhat.com> In-Reply-To: <1030455667.5826994.1396419881620.JavaMail.zimbra@redhat.com> Subject: pcp updates: python getopts code, exception handling fixes MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: python getopts code, exception handling fixes Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.6] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: python getopts code, exception handling fixes Thread-Index: K3MUHItI5xcHUHVFDp/5P3dfwHLRRg== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1396420367 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4509 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev qa/739 | 57 ++ qa/739.out | 115 +++++ qa/741 | 42 ++ qa/741.out | 23 + qa/group | 2 qa/src/GNUlocaldefs | 1 qa/src/test_pcp_getopts.python | 113 +++++ qa/src/test_pcp_options.python | 94 ++++ src/libpcp/src/getopt.c | 9 src/python/pcp/pmapi.py | 295 +++++++++++++- src/python/pmapi.c | 818 ++++++++++++++++++++++++++++++++++++++++- 11 files changed, 1529 insertions(+), 40 deletions(-) commit 15be6ce0f478aefe5dcdd9ce138f3801fc373fd7 Author: Nathan Scott Date: Wed Apr 2 17:22:15 2014 +1100 Improvements to the python pmErr exception class The interaction between the pmLookupName PMAPI wrapper and the exception class was not quite right. Firstly, the test-for-failure in the lookup wrapper came after the test-for-not-the-same-number-of-PMIDs-passed-in, rendering the error case unreachable. Secondly, the value passed into the exception handler (which *must* be an error code) was the positive return value, for the case where some, but not all, names resolve. This meant the error-symbol dictionary lookup would fail and neither an error message nor the list of names which didn't resolve would be generated. Finally, when a pmContext constructor stalls (eg for faraway connections) and is interrupted (eg killed), the destructor is called with a partially-initialised object. It then attempts to call pmDestroyContext(3) on an uninitialised context, and badness results. These bugs were found while testing some new python monitor tools, automated regression testing will be added shortly using those new tools as the vehicle. commit 59b5cd0a1c48ad071e405dccd496a31142a5281a Author: Nathan Scott Date: Wed Apr 2 17:05:34 2014 +1100 Small improvements to the core getopt handling in libpcp Do not pass EOF into option override() methods, its not an actual option, hence no need to ever override - busy work. Make the -V,--version output match usual output convention. Switch over to using the thread-safe pmGetContextHostName_r variant. Allow pmUsageMessage() to be used to flush the version output too, instead of making callers deal with that case specially. Used in the python APIs, in particular, to simplify things. commit 10d67282c58093b7cffdbfda3b76df9cbd0b5f47 Author: Nathan Scott Date: Tue Apr 1 17:47:26 2014 +1100 Initial Python script support for auto-command-line-parsing Support for long options and automatic PCP option parsing on the command lines of python monitor tools. A new pmOptions class is added to the pmapi python module, and a classmethod on the pmContext class allows it to be injected into the new context creation process. Two test scripts are also added to exercise the python APIs, exercised via new tests qa/739 and qa/741. The former looks at specifics of the pmOptions class, the latter ensures that this new class interfaces correctly with the PMAPI pmContext creation mechanism. From apache@vps.com Wed Apr 2 08:05:27 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.3 required=5.0 tests=HTML_MESSAGE,MIME_HTML_ONLY, SUBJECT_NEEDS_ENCODING,SUBJ_ILLEGAL_CHARS,T_FRT_CLICK,T_FRT_CONTACT autolearn=no version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 072FB29DF7 for ; Wed, 2 Apr 2014 08:05:27 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id E9063304032 for ; Wed, 2 Apr 2014 06:05:23 -0700 (PDT) X-ASG-Debug-ID: 1396443916-04cb6c5677bfd440001-S8gJnT Received: from mail.vps.com ([168.61.8.253]) by cuda.sgi.com with ESMTP id TpgaVvgB4HNJtfkS for ; Wed, 02 Apr 2014 06:05:17 -0700 (PDT) X-Barracuda-Envelope-From: apache@vps.com X-Barracuda-Apparent-Source-IP: 168.61.8.253 Received: by mail.vps.com (Postfix, from userid 48) id 43EF02BA97; Wed, 2 Apr 2014 13:02:50 +0000 (UTC) To: pcp@oss.sgi.com Subject: Benachrichtigung von Sparkasse‏ X-PHP-Originating-Script: 0:mailer.php X-ASG-Orig-Subj: Benachrichtigung von Sparkasse‏ From: © SPARKASSE 2014 Reply-To: noreply@sparkasse.de MIME-Version: 1.0 Content-Type: text/html Content-Transfer-Encoding: 8bit Message-Id: <20140402130250.43EF02BA97@mail.vps.com> Date: Wed, 2 Apr 2014 13:02:50 +0000 (UTC) X-Barracuda-Connect: UNKNOWN[168.61.8.253] X-Barracuda-Start-Time: 1396443917 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.88 X-Barracuda-Spam-Status: No, SCORE=1.88 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC5_MJ1963, HTML_MESSAGE, MIME_HTML_ONLY, RDNS_NONE, SUBJECT_NEEDS_ENCODING X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4515 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.00 HTML_MESSAGE BODY: HTML included in message 1.28 SUBJECT_NEEDS_ENCODING SUBJECT_NEEDS_ENCODING 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 This is HTML source of message you composed. Do not modify here. To modify this message press HTML Messages Editor button.
Sehr geehrter Kunde,
 
wir möchten Sie darauf hinweisen, dass der Zugang zu Ihrem Online-Konto in Kurze abläuft.
Um dieses weiterhin nützen zu können, bitten wir Sie Ihre Daten bei folgendem Link zu bestätigen:
 
 
Anschließend wir Ihr Online-Konto automatisch wiederhergestellt und Sie werden von einem unserer Mitarbeiter kontaktiert.
 
Beim Online-Banking haben Sie per Klick alles im Griff.
 
Mit dem komfortablen Online-Banking haben Sie schnellen und problemlosen Zugang zu Ihrem Girokonto.
Bequem können Sie Überweisungen und Daueraufträge per Mausklick erledigen.
 
Das Online-Banking bietet aber noch viel mehr:
 
DIE VORTEILE AUF EINEM BLICK:
 
- Kontozugang rund um die Uhr
- Schneller Zugriff aufs Girokonto
- Online-Banking bequem vom Handy oder PC aus
- Flexibel in jedem Winkel der Welt
- Übersichtliche Kontoführung
- Hohe Sicherheitsstandards
- Kombinierbar mit Telefon-Banking
 
Wir freuen uns sehr Sie weiterhin als unseren Online Konto Kunden begrüßen zu dürfen!
 
Mit freundlichen Grüßen,
 
Sparkasse Kundenservice
 From kenj@internode.on.net Thu Apr 3 16:39:58 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 94AAF29E06 for ; Thu, 3 Apr 2014 16:39:58 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 19D06AC003 for ; Thu, 3 Apr 2014 14:39:54 -0700 (PDT) X-ASG-Debug-ID: 1396561188-04cbb054b8a72350001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id fGXTvYreCqSQ9CAI for ; Thu, 03 Apr 2014 14:39:48 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArkVAA7UPVN20fAIPGdsb2JhbAANS4NBg2GFXLxMAwEBAQE4gwQERwosERYLAgsDAgECATEaDQYCAQEVs0t2om4XjHOEdIFJBJBfiTCUTg Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail05.adl6.internode.on.net with ESMTP; 04 Apr 2014 08:09:46 +1030 Message-ID: <533DD572.7070505@internode.on.net> Date: Fri, 04 Apr 2014 08:41:06 +1100 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: PCP Mailing List Subject: Perl PMDA slow start Content-Type: multipart/mixed; boundary="------------020300000604000303090001" X-ASG-Orig-Subj: Perl PMDA slow start X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396561188 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4553 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- This is a multi-part message in MIME format. --------------020300000604000303090001 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I am pretty sure I have the first part of this working. It involves a new $pmda->connect_pmcd() method. I went to document connect_pmcd and could not find any man page documentation for the PCP::PMDA package. Do I need to be looking someplace other than PMDA.pm? I've hacked together additional text for PMDA.pm (assuming the answer to my own question is "no"), and attached the generated man page. Could those that grok Perl (and I am definitely not one of you) please check this out and let me know if it is useful before I spend any more time filling in the TODO parts. --------------020300000604000303090001 Content-Type: text/plain; charset=UTF-8; name="PCP::PMDA.3pm" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="PCP::PMDA.3pm" .\" Automatically generated by Pod::Man 2.25 (Pod::Simple 3.16) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .ie \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .el \{\ . de IX .. .\} .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "PMDA 3pm" .TH PMDA 3pm "2014-04-04" "perl v5.14.2" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" PCP::PMDA \- Perl extension for Performance Metrics Domain Agents .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 1 \& use PCP::PMDA; \& \& $pmda = PCP::PMDA\->new(\*(Aqmyname\*(Aq, $MYDOMAIN); \& \& $pmda\->connect_pmcd; \& \& $pmda\->add_metric($pmid, $type, $indom, $sem, $units, \*(Aqname\*(Aq, \*(Aq\*(Aq, \*(Aq\*(Aq); \& $pmda\->add_indom($indom, [0 => \*(Aqwhite\*(Aq, 1 => \*(Aqblack\*(Aq, ...], \*(Aq\*(Aq, \*(Aq\*(Aq); \& \& $pmda\->set_fetch(\e&fetch_method); \& $pmda\->set_refresh(\e&refresh_method); \& $pmda\->set_instance(\e&instance_method); \& $pmda\->set_fetch_callback(\e&fetch_callback_method); \& $pmda\->set_store_callback(\e&store_callback_method); \& \& $pmda\->set_user(\*(Aqpcp\*(Aq); \& \& $pmda\->run; .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" The \s-1PCP::PMDA\s0 Perl module contains the language bindings for building Performance Metric Domain Agents (PMDAs) using Perl. Each \s-1PMDA\s0 exports performance data for one specific domain, for example the operating system kernel, Cisco routers, a database, an application, etc. .SH "METHODS" .IX Header "METHODS" .IP "\s-1PCP::PMDA\-\s0>new(name, domain)" 4 .IX Item "PCP::PMDA->new(name, domain)" \&\s-1PCP::PMDA\s0 class constructor. \fIname\fR is a string that becomes the name of the \s-1PMDA\s0 for messages and default prefix for the names of external files used by the \s-1PMDA\s0. \fIdomain\fR is an integer domain number for the \s-1PMDA\s0, usually from the register of domain numbers found in \fB\f(CB$PCP_VAR_DIR\fB/pmns/stdpmid\fR. .ie n .IP "$pmda\->\fIconnect_pmcd()\fR" 4 .el .IP "\f(CW$pmda\fR\->\fIconnect_pmcd()\fR" 4 .IX Item "$pmda->connect_pmcd()" Allows the \s-1PMDA\s0 to set up the \s-1IPC\s0 channel to \fBpmcd\fR(1) and complete the credentials handshake with \fBpmcd\fR(1). If \fIconnect_pmcd\fR is not explicitly called the setup and handshake will be done when the \&\fIrun\fR method is called. .Sp The advantage of explicitly calling \fIconnect_pmcd\fR early in the life of the \s-1PMDA\s0 is that this reduces the risk of a fatal timeout during the credentials handshake, which may be an issue if the \s-1PMDA\s0 has considerable work to do, e.g. determining which metrics and instance domains are available, before calling \fIrun\fR. .ie n .IP "$pmda\->add_indom(indom, insts, help, longhelp)" 4 .el .IP "\f(CW$pmda\fR\->add_indom(indom, insts, help, longhelp)" 4 .IX Item "$pmda->add_indom(indom, insts, help, longhelp)" \&\s-1TODO\s0 .ie n .IP "$pmda\->add_metric(pmid, type, indom, sem, units, name, help, longhelp)" 4 .el .IP "\f(CW$pmda\fR\->add_metric(pmid, type, indom, sem, units, name, help, longhelp)" 4 .IX Item "$pmda->add_metric(pmid, type, indom, sem, units, name, help, longhelp)" \&\s-1TODO\s0 .ie n .IP "$pmda\->add_pipe(command, callback, data)" 4 .el .IP "\f(CW$pmda\fR\->add_pipe(command, callback, data)" 4 .IX Item "$pmda->add_pipe(command, callback, data)" \&\s-1TODO\s0 .ie n .IP "$pmda\->add_sock(hostname, port, callback, data)" 4 .el .IP "\f(CW$pmda\fR\->add_sock(hostname, port, callback, data)" 4 .IX Item "$pmda->add_sock(hostname, port, callback, data)" \&\s-1TODO\s0 .ie n .IP "$pmda\->add_tail(filename, callback, data)" 4 .el .IP "\f(CW$pmda\fR\->add_tail(filename, callback, data)" 4 .IX Item "$pmda->add_tail(filename, callback, data)" \&\s-1TODO\s0 .ie n .IP "$pmda\->add_timer(timeout, callback, data)" 4 .el .IP "\f(CW$pmda\fR\->add_timer(timeout, callback, data)" 4 .IX Item "$pmda->add_timer(timeout, callback, data)" \&\s-1TODO\s0 .ie n .IP "$pmda\->err(message)" 4 .el .IP "\f(CW$pmda\fR\->err(message)" 4 .IX Item "$pmda->err(message)" \&\s-1TODO\s0 .ie n .IP "$pmda\->error(message)" 4 .el .IP "\f(CW$pmda\fR\->error(message)" 4 .IX Item "$pmda->error(message)" \&\s-1TODO\s0 .ie n .IP "$pmda\->log(message)" 4 .el .IP "\f(CW$pmda\fR\->log(message)" 4 .IX Item "$pmda->log(message)" \&\s-1TODO\s0 .ie n .IP "$pmda\->put_sock(id, output)" 4 .el .IP "\f(CW$pmda\fR\->put_sock(id, output)" 4 .IX Item "$pmda->put_sock(id, output)" \&\s-1TODO\s0 .ie n .IP "$pmda\->replace_indom(index, insts)" 4 .el .IP "\f(CW$pmda\fR\->replace_indom(index, insts)" 4 .IX Item "$pmda->replace_indom(index, insts)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_fetch_callback(cb_function)" 4 .el .IP "\f(CW$pmda\fR\->set_fetch_callback(cb_function)" 4 .IX Item "$pmda->set_fetch_callback(cb_function)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_fetch(function)" 4 .el .IP "\f(CW$pmda\fR\->set_fetch(function)" 4 .IX Item "$pmda->set_fetch(function)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_inet_socket(port)" 4 .el .IP "\f(CW$pmda\fR\->set_inet_socket(port)" 4 .IX Item "$pmda->set_inet_socket(port)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_instance(function)" 4 .el .IP "\f(CW$pmda\fR\->set_instance(function)" 4 .IX Item "$pmda->set_instance(function)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_ipv6_socket(port)" 4 .el .IP "\f(CW$pmda\fR\->set_ipv6_socket(port)" 4 .IX Item "$pmda->set_ipv6_socket(port)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_refresh(function)" 4 .el .IP "\f(CW$pmda\fR\->set_refresh(function)" 4 .IX Item "$pmda->set_refresh(function)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_store_callback(cb_function)" 4 .el .IP "\f(CW$pmda\fR\->set_store_callback(cb_function)" 4 .IX Item "$pmda->set_store_callback(cb_function)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_unix_socket(socket_name)" 4 .el .IP "\f(CW$pmda\fR\->set_unix_socket(socket_name)" 4 .IX Item "$pmda->set_unix_socket(socket_name)" \&\s-1TODO\s0 .ie n .IP "$pmda\->set_user(username)" 4 .el .IP "\f(CW$pmda\fR\->set_user(username)" 4 .IX Item "$pmda->set_user(username)" \&\s-1TODO\s0 .SH "HELPER METHODS" .IX Header "HELPER METHODS" .IP "pmda_pmid(cluster, item)" 4 .IX Item "pmda_pmid(cluster, item)" Construct a Performance Metric Identifier (\s-1PMID\s0) from the domain number (passed as an argument to the \fInew\fR constructor), the \&\fIcluster\fR (an integer in the range 0 to 2^12\-1) and the \&\fIitem\fR (an integer in the range 0 to 2^10\-1). .Sp Every performance metric exported from a \s-1PMDA\s0 must have a unique \&\s-1PMID\s0. .IP "pmda_pmid_name(cluster, item)" 4 .IX Item "pmda_pmid_name(cluster, item)" \&\s-1TODO\s0 .IP "pmda_pmid_text(cluster, item)" 4 .IX Item "pmda_pmid_text(cluster, item)" \&\s-1TODO\s0 .IP "pmda_inst_name(index, instance)" 4 .IX Item "pmda_inst_name(index, instance)" \&\s-1TODO\s0 .IP "pmda_inst_lookup(index, instance)" 4 .IX Item "pmda_inst_lookup(index, instance)" \&\s-1TODO\s0 .IP "pmda_units(dim_space, dim_time, dim_count, scale_space, scale_time, scale_count)" 4 .IX Item "pmda_units(dim_space, dim_time, dim_count, scale_space, scale_time, scale_count)" \&\s-1TODO\s0 .IP "pmda_config(name)" 4 .IX Item "pmda_config(name)" \&\s-1TODO\s0 .IP "pmda_uptime(now)" 4 .IX Item "pmda_uptime(now)" \&\s-1TODO\s0 .IP "\fIpmda_long()\fR" 4 .IX Item "pmda_long()" \&\s-1TODO\s0 .IP "\fIpmda_ulong()\fR" 4 .IX Item "pmda_ulong()" .SH "MACROS" .IX Header "MACROS" Most of the PM_* macros from the \s-1PCP\s0 C headers are available. .PP For example the \fItype\fR of a metric's value may be directly specified as one of \&\fB\s-1PM_TYPE_32\s0\fR, \fB\s-1PM_TYPE_U32\s0\fR, \fB\s-1PM_TYPE_64\s0\fR, \fB\s-1PM_TYPE_U64\s0\fR, \&\fB\s-1PM_TYPE_FLOAT\s0\fR, \fB\s-1PM_TYPE_DOUBLE\s0\fR, \fB\s-1PM_TYPE_STRING\s0\fR or \&\fB\s-1PM_TYPE_NOSUPPORT\s0\fR. .SH "SEE ALSO" .IX Header "SEE ALSO" \&\fIperl\fR\|(1) and \fIpcpintro\fR\|(3). .PP The \s-1PCP\s0 mailing list pcp@oss.sgi.com can be used for questions about this module. .PP Further details can be found at http://oss.sgi.com/projects/pcp .SH "AUTHOR" .IX Header "AUTHOR" Nathan Scott, .PP Copyright (C) 2008\-2010 by Aconex. Copyright (C) 2004 by Silicon Graphics, Inc. .PP This library is free software; you can redistribute it and/or modify it under the terms of the \s-1GNU\s0 General Public License, version 2 (see the \*(L"\s-1COPYING\s0\*(R" file in the \s-1PCP\s0 source tree for further details). --------------020300000604000303090001-- From nscott@redhat.com Thu Apr 3 17:55:58 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id E694D7FBD for ; Thu, 3 Apr 2014 17:55:57 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7E82AAC002 for ; Thu, 3 Apr 2014 15:55:54 -0700 (PDT) X-ASG-Debug-ID: 1396565749-04cb6c5675c63840001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id EJFeJkpGs8NnYy2S for ; Thu, 03 Apr 2014 15:55:49 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s33MtXc2009721; Thu, 3 Apr 2014 18:55:33 -0400 Date: Thu, 3 Apr 2014 18:55:33 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: PCP Mailing List Message-ID: <1956804155.7267263.1396565733668.JavaMail.zimbra@redhat.com> In-Reply-To: <533DD572.7070505@internode.on.net> References: <533DD572.7070505@internode.on.net> Subject: Re: [pcp] Perl PMDA slow start MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] Perl PMDA slow start Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: Perl PMDA slow start Thread-Index: I3jV0CgP7QhhOh2J0TpXM7hJc/b4Ag== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1396565749 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4557 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... ----- Original Message ----- > I am pretty sure I have the first part of this working. It involves a > new $pmda->connect_pmcd() method. Good stuff! > I went to document connect_pmcd and could not find any man page > documentation for the PCP::PMDA package. > > Do I need to be looking someplace other than PMDA.pm? > > I've hacked together additional text for PMDA.pm (assuming the answer to > my own question is "no"), and attached the generated man page. There is a small man page; perl uses its POD (plain old documentation) concept to inject docs inline within the code. src/perl/PMDA/PMDA.pm is the place to look to start adding in new text (not much there atm). "man PCP::PMDA" should find something in section 3 for installed pcp. > Could those that grok Perl (and I am definitely not one of you) please > check this out and let me know if it is useful before I spend any more > time filling in the TODO parts. Shall do & will get back to you. cheers. -- Nathan From kenj@internode.on.net Thu Apr 3 18:14:57 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 4DD037FD7 for ; Thu, 3 Apr 2014 18:14:57 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id E3BA7AC003 for ; Thu, 3 Apr 2014 16:14:53 -0700 (PDT) X-ASG-Debug-ID: 1396566891-04cb6c5676c63d40001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id Z0TCE7kIMxezbJ1t for ; Thu, 03 Apr 2014 16:14:51 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYVAFPqPVN20fAIPGdsb2JhbAANSotbuSuEQQMBAQEBOIMZQD0WGAMCAQIBMRoNCAEBs2yjXxeTMASuXQ Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail05.adl6.internode.on.net with ESMTP; 04 Apr 2014 09:44:49 +1030 Message-ID: <533DEBB9.7010608@internode.on.net> Date: Fri, 04 Apr 2014 10:16:09 +1100 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: PCP Mailing List Subject: something not right in pmmgr land? Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: something not right in pmmgr land? Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396566891 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4557 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- After a full QA run on xubuntu 13.10 I see ... kenj@bozo:~/src/pcp/qa$ pcp Performance Co-Pilot configuration on bozo: platform: Linux bozo 3.11.0-19-generic #33-Ubuntu SMP Tue Mar 11 18:48:34 UTC 2014 x86_64 hardware: 6 cpus, 3 disks, 1 node, 7985MB RAM timezone: EST-11 pmcd: Version 3.9.2-1, 9 agents, 14 clients pmda: pmcd proc xfs sample sampledso linux mmv jbd2 simple pmlogger: bozo: /var/log/pcp/pmmgr/bozo/archive-20140403.224137 primary logger: bozo/20140404.09.41 pmie: bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log kenj@bozo:~/src/pcp/qa$ ps -ef | grep '[p]mie.*pmmgr' | wc -l 13 It is not a fluke ... I've seen it on another Ubuntu system. From fche@redhat.com Thu Apr 3 20:55:36 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id DB4D47FC7 for ; Thu, 3 Apr 2014 20:55:36 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id C7F9A8F80A3 for ; Thu, 3 Apr 2014 18:55:36 -0700 (PDT) X-ASG-Debug-ID: 1396576531-04cbb054b6a77e00001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id kPVUVIsGMJzPOFZY for ; Thu, 03 Apr 2014 18:55:32 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s341tSOW006283 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 3 Apr 2014 21:55:28 -0400 Received: from fche.csb (vpn-59-222.rdu2.redhat.com [10.10.59.222]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s341tRtB019703; Thu, 3 Apr 2014 21:55:27 -0400 Received: by fche.csb (Postfix, from userid 2569) id A016358506; Thu, 3 Apr 2014 21:55:26 -0400 (EDT) To: Ken McDonell Cc: PCP Mailing List Subject: Re: something not right in pmmgr land? References: <533DEBB9.7010608@internode.on.net> X-ASG-Orig-Subj: Re: something not right in pmmgr land? From: fche@redhat.com (Frank Ch. Eigler) Date: Thu, 03 Apr 2014 21:55:26 -0400 In-Reply-To: <533DEBB9.7010608@internode.on.net> (Ken McDonell's message of "Fri, 04 Apr 2014 10:16:09 +1100") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1396576532 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 kenj wrote: > [...] > pmlogger: bozo: /var/log/pcp/pmmgr/bozo/archive-20140403.224137 > primary logger: bozo/20140404.09.41 > pmie: bozo: /var/log/pcp/pmmgr/bozo/pmie.log > bozo: /var/log/pcp/pmmgr/bozo/pmie.log > bozo: /var/log/pcp/pmmgr/bozo/pmie.log > [...and a bunch more...] > kenj@bozo:~/src/pcp/qa$ ps -ef | grep '[p]mie.*pmmgr' | wc -l > 13 > It is not a fluke ... I've seen it on another Ubuntu system. Please arrange to turn on pmmgr verbosity (pmmgr -v) and see if those pmie processes were sent a SIGTERM (pmmgr_daemon dtor) like they should've been. Anything interesting in the pmie.log or pmmgr.log file(s)? - FChE From nscott@redhat.com Fri Apr 4 00:34:21 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 43FB47FC6 for ; Fri, 4 Apr 2014 00:34:21 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id C21FAAC004 for ; Thu, 3 Apr 2014 22:34:20 -0700 (PDT) X-ASG-Debug-ID: 1396589658-04cb6c5678c6c640001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id wsYKbY3QHpFZHAgq for ; Thu, 03 Apr 2014 22:34:18 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s345YFD2017359; Fri, 4 Apr 2014 01:34:15 -0400 Date: Fri, 4 Apr 2014 01:34:15 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: PCP Mailing List Message-ID: <998043446.7393891.1396589655417.JavaMail.zimbra@redhat.com> In-Reply-To: <1956804155.7267263.1396565733668.JavaMail.zimbra@redhat.com> References: <533DD572.7070505@internode.on.net> <1956804155.7267263.1396565733668.JavaMail.zimbra@redhat.com> Subject: Re: [pcp] Perl PMDA slow start MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] Perl PMDA slow start Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.7] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: Perl PMDA slow start Thread-Index: I3jV0CgP7QhhOh2J0TpXM7hJc/b4AooCoeNj X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1396589658 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4561 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Ken, ----- Original Message ----- > ----- Original Message ----- > > Could those that grok Perl (and I am definitely not one of you) please > > check this out and let me know if it is useful before I spend any more > > time filling in the TODO parts. > Looks good (pmcd_connect), and thanks so much for starting to properly document that API - 'tis fairly embarrassing on my part as to how the API has evolved to date with so little documentation. The run() method could mention the environment variables that are used to generate the namespace and domain.h files (see the C code getenv(3) calls over in PMDA.xs on PCP_PERL_PMNS and PCP_PERL_DOMAIN). If you run out of steam documenting things here, please toss it back over the fence to me and I'll humbly finish it off. Thank you!!! cheers. -- Nathan From nscott@redhat.com Fri Apr 4 02:11:09 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 0A4877FD3 for ; Fri, 4 Apr 2014 02:11:09 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id E4407304062 for ; Fri, 4 Apr 2014 00:11:08 -0700 (PDT) X-ASG-Debug-ID: 1396595462-04cbb054b7a7ef50001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id FFZro12e6xZevhf9 for ; Fri, 04 Apr 2014 00:11:02 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s347B2Q5004851 for ; Fri, 4 Apr 2014 03:11:02 -0400 Date: Fri, 4 Apr 2014 03:11:01 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp@oss.sgi.com Message-ID: <940301097.7457294.1396595461773.JavaMail.zimbra@redhat.com> In-Reply-To: <1916844157.7456876.1396595411674.JavaMail.zimbra@redhat.com> Subject: pcp updates: python free & uptime tools MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: python free & uptime tools Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.7] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: python free & uptime tools Thread-Index: +gPdjGYSC4DujJDePK8uSEOiT/VCow== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1396595462 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.62 X-Barracuda-Spam-Status: No, SCORE=0.62 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=MARKETING_SUBJECT, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4561 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.60 MARKETING_SUBJECT Subject contains popular marketing words Changes committed to git://oss.sgi.com/pcp/pcp.git dev qa/366.out | 55 ++++++++ qa/739 | 4 qa/742 | 28 ++++ qa/742.out | 7 + qa/991 | 47 +++++++ qa/991.out | 57 ++++++++ qa/group | 2 qa/src/GNUlocaldefs | 2 qa/src/pcp-free.0 |binary qa/src/pcp-free.index |binary qa/src/pcp-free.meta |binary qa/src/pcp-uptime.0 |binary qa/src/pcp-uptime.index |binary qa/src/pcp-uptime.meta |binary src/pcp/GNUmakefile | 16 +- src/pcp/free/.gitignore | 1 src/pcp/free/GNUmakefile | 34 +++++ src/pcp/free/pcp-free.1 | 125 +++++++++++++++++++ src/pcp/free/pcp-free.py | 224 +++++++++++++++++++++++++++++++++++ src/pcp/uptime/.gitignore | 1 src/pcp/uptime/GNUmakefile | 34 +++++ src/pcp/uptime/pcp-uptime.1 | 107 ++++++++++++++++ src/pcp/uptime/pcp-uptime.py | 133 ++++++++++++++++++++ src/pmlogconf/tools/atop | 3 src/pmlogconf/tools/atop-summary | 6 src/pmlogconf/tools/collectl-summary | 5 src/pmlogconf/tools/free | 13 ++ src/pmlogconf/tools/free-summary | 4 src/pmlogconf/tools/localdefs | 2 src/pmlogconf/tools/sar-summary | 2 src/pmlogconf/tools/uptime | 6 src/pmlogconf/tools/vmstat | 2 src/pmlogconf/tools/vmstat-summary | 5 src/python/pcp/pmapi.py | 86 ++++++++++--- src/python/pmapi.c | 145 ++++++++++++++++++---- 35 files changed, 1089 insertions(+), 67 deletions(-) commit 19b320a9955693a2d07596776a14a8966806308b Author: Nathan Scott Date: Fri Apr 4 17:50:17 2014 +1100 PMAPI python implementations of uptime(1) and free(1) To exercise some of the python PMAPI additions for automated getopt handling in realistic settings, I've been using some hand-rolled implementations of uptime(1) & free(1). These are actually useful, allowing one to generate well-known output formats of these tools retrospectively (--archive, --origin). So, add these into the installed mix. Some very small example archives are added to exercise these tools, along with tests qa/742 and qa/991. Man pages also added describing all of the regular tool options, plus the PCP additions. A number of API changes are made as a result of building these tools and getting them properly functional (esp archive mode): - further improve error handling in the python exception class, such that any pmErr exception can extend the report using the same model pmLookupName does - pass attempted connection string into pmNewContext exceptions for improved reporting detail - automatically setup the archive pmSetMode configuration for scripts based on arguments passed in (pmOptions class grows mode and delta fields) - the python options callback does not require a return code, this was accidentally, erroneously being mandated - fix the handling of --origin for sub-second time windows In adding these tools, and their associated pmlogconf files, I came across several cases where other tools log configurations were not splitting the log-once metrics from the continually sampled metrics - I've fixed these up too here (atop, collectl, sar configs affected). From kenj@internode.on.net Sun Apr 6 17:48:48 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id A7A257F55 for ; Sun, 6 Apr 2014 17:48:47 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id EEB3AAC003 for ; Sun, 6 Apr 2014 15:48:42 -0700 (PDT) X-ASG-Debug-ID: 1396824510-04cb6c5675d28240001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id YlyYRczJZh0F3KIE for ; Sun, 06 Apr 2014 15:48:31 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjUiAPzYQVN20fAIPGdsb2JhbAANTIcihDm5R4MOgS4DAQEBATiCWgEBAQQaAQhVARAJAhgJFgsCAgkDAgECATEUBg0BBwEBlR6bInaiGheOcQeCb4FJAQOQX4E1nEg Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail05.adl6.internode.on.net with ESMTP; 07 Apr 2014 08:18:27 +0930 Message-ID: <5341DA0F.6000202@internode.on.net> Date: Mon, 07 Apr 2014 08:49:51 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: PCP Mailing List Subject: Re: something not right in pmmgr land? References: <533DEBB9.7010608@internode.on.net> X-ASG-Orig-Subj: Re: something not right in pmmgr land? In-Reply-To: Content-Type: multipart/mixed; boundary="------------000602030702050708010401" X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396824511 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4636 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header This is a multi-part message in MIME format. --------------000602030702050708010401 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 04/04/14 12:55, Frank Ch. Eigler wrote: > > kenj wrote: > >> [...] >> pmlogger: bozo: /var/log/pcp/pmmgr/bozo/archive-20140403.224137 >> primary logger: bozo/20140404.09.41 >> pmie: bozo: /var/log/pcp/pmmgr/bozo/pmie.log >> bozo: /var/log/pcp/pmmgr/bozo/pmie.log >> bozo: /var/log/pcp/pmmgr/bozo/pmie.log >> [...and a bunch more...] >> kenj@bozo:~/src/pcp/qa$ ps -ef | grep '[p]mie.*pmmgr' | wc -l >> 13 >> It is not a fluke ... I've seen it on another Ubuntu system. > > Please arrange to turn on pmmgr verbosity (pmmgr -v) and see if those > pmie processes were sent a SIGTERM (pmmgr_daemon dtor) like they > should've been. Anything interesting in the pmie.log or pmmgr.log > file(s)? > > - FChE > Attached is the pmmgr.log with -v enabled. Here is the evidence of the additional pmie processes ... kenj@bozo:~/src/pcp/qa$ pcp Performance Co-Pilot configuration on bozo: platform: Linux bozo 3.11.0-19-generic #33-Ubuntu SMP Tue Mar 11 18:48:34 UTC 2014 x86_64 hardware: 6 cpus, 3 disks, 1 node, 7985MB RAM timezone: EST-10 pmcd: Version 3.9.2-1, 9 agents, 6 clients pmda: pmcd proc xfs sample sampledso linux mmv jbd2 simple pmlogger: bozo: /var/log/pcp/pmmgr/bozo/archive-20140406.223755 primary logger: bozo/20140407.08.36 pmie: bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log bozo: /var/log/pcp/pmmgr/bozo/pmie.log kenj@bozo:~/src/pcp/qa$ ps -ef | grep '[p]mie.*pmmgr' pcp 7698 29916 0 08:21 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log pcp 14241 14776 0 08:37 pts/1 00:00:00 sh -c /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log pcp 14243 14241 0 08:37 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log pcp 14794 29916 0 07:46 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log pcp 32065 29916 0 08:20 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log The word "sig" (ignoring case) does not appear to be in the pmmgr.log file. pid 29916 is "init". the others seem to be the real deal ... kenj@bozo:~/src/pcp/qa$ pstree 14776 pmmgr─┬─sh───pmie └─sh───pmlogger Ledt me know if you need more, or there are other experiments I can run. This is very easy to reproduce. --------------000602030702050708010401 Content-Type: text/x-log; name="pmmgr.log" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="pmmgr.log" [Mon Apr 7 07:46:56] pmmgr(14776/14776): Log started [Mon Apr 7 07:46:56] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.004113 [Mon Apr 7 07:46:56] pmmgr(14776/14776): /etc/pcp/pmmgr: new hostid bozo at local: [Mon Apr 7 07:46:56] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/lib/pcp/bin/pmlogconf -c -r -h local: /var/log/pcp/pmmgr/bozo/config.pmlogger >/dev/null [Mon Apr 7 07:46:56] pmmgr(14776/14781): /etc/pcp/pmmgr: running /usr/bin/pmieconf -F -c -f /var/log/pcp/pmmgr/bozo/config.pmie [Mon Apr 7 07:46:56] pmmgr(14776/14781): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 07:46:56] pmmgr(14776/14781): /etc/pcp/pmmgr: daemon pid 14792 started: /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140323.000012 >/dev/null [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140323.000012 [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140324.000014 >/dev/null [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140324.000014 [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000009 >/dev/null [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000009 [Mon Apr 7 07:47:02] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000116 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000116 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000114 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000114 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000115 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000115 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140329.000012 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140329.000012 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140330.000008 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140330.000008 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140331.000012 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140331.000012 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140401.000011 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140401.000011 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140402.000009 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140402.000009 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140404.000007 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140404.000007 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140405.000007 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140405.000007 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000008 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000115 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.000115 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.105020 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.105020 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113650 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113650 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113801 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113801 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113914 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113914 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114221 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114221 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114429 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114429 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115133 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115133 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115938 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115938 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120212 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120212 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120918 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120918 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121042 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121042 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121159 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121159 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121516 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121516 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121801 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121801 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121915 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121915 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122242 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122242 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122425 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122425 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123043 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123043 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123501 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123501 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123635 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123635 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123748 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123748 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123907 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123907 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124018 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124018 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124130 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124130 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124254 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124254 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124422 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124422 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124827 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124827 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125002 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125002 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125312 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125312 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125426 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125426 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125546 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125546 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201300 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201300 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201540 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201540 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202301 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202301 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202435 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202435 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202619 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202619 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202750 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202750 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203010 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203010 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203146 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203146 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203653 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203653 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203916 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203916 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204047 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204047 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204215 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204215 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204539 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204539 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204739 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204739 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204906 >/dev/null [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204906 [Mon Apr 7 07:47:03] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205112 >/dev/null [Mon Apr 7 07:47:04] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205112 [Mon Apr 7 07:47:04] pmmgr(14776/14780): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205354 >/dev/null [Mon Apr 7 07:47:04] pmmgr(14776/14780): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205354 [Mon Apr 7 07:47:04] pmmgr(14776/14780): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.214704 [Mon Apr 7 07:47:04] pmmgr(14776/14780): /etc/pcp/pmmgr: daemon pid 17919 started: /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.214704 [Mon Apr 7 07:48:04] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000706 [Mon Apr 7 07:49:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 2.59784 [Mon Apr 7 07:50:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000727 [Mon Apr 7 07:51:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.00067 [Mon Apr 7 07:52:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.00067 [Mon Apr 7 07:53:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.00071 [Mon Apr 7 07:54:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000679 [Mon Apr 7 07:55:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000669 [Mon Apr 7 07:56:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000718 [Mon Apr 7 07:57:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000649 [Mon Apr 7 07:58:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000671 [Mon Apr 7 07:59:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000694 [Mon Apr 7 08:00:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000671 [Mon Apr 7 08:01:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000663 [Mon Apr 7 08:02:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000677 [Mon Apr 7 08:03:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000667 [Mon Apr 7 08:04:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000692 [Mon Apr 7 08:05:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000697 [Mon Apr 7 08:06:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000654 [Mon Apr 7 08:07:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000667 [Mon Apr 7 08:08:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000673 [Mon Apr 7 08:09:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000673 [Mon Apr 7 08:10:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000645 [Mon Apr 7 08:11:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000662 [Mon Apr 7 08:12:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000695 [Mon Apr 7 08:13:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000663 [Mon Apr 7 08:14:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000659 [Mon Apr 7 08:15:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000712 [Mon Apr 7 08:16:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000701 [Mon Apr 7 08:17:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000667 [Mon Apr 7 08:18:06] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000698 [Mon Apr 7 08:19:04] pmmgr(14776/14776): /etc/pcp/pmmgr: dead hostid bozo [Mon Apr 7 08:19:04] pmmgr(14776/14776): /etc/pcp/pmmgr: daemon pid 17919 killed [Mon Apr 7 08:19:04] pmmgr(14776/14776): /etc/pcp/pmmgr: daemon pid 14792 killed [Mon Apr 7 08:20:04] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000741 [Mon Apr 7 08:20:04] pmmgr(14776/14776): /etc/pcp/pmmgr: new hostid bozo at local: [Mon Apr 7 08:20:04] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/lib/pcp/bin/pmlogconf -c -r -h local: /var/log/pcp/pmmgr/bozo/config.pmlogger >/dev/null [Mon Apr 7 08:20:04] pmmgr(14776/32051): /etc/pcp/pmmgr: running /usr/bin/pmieconf -F -c -f /var/log/pcp/pmmgr/bozo/config.pmie [Mon Apr 7 08:20:04] pmmgr(14776/32051): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 08:20:04] pmmgr(14776/32051): /etc/pcp/pmmgr: daemon pid 32063 started: /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140323.000012 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140323.000012 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140324.000014 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140324.000014 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000009 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000009 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000116 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000116 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000114 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000114 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000115 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000115 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140329.000012 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140329.000012 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140330.000008 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140330.000008 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140331.000012 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140331.000012 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140401.000011 >/dev/null [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140401.000011 [Mon Apr 7 08:20:11] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140402.000009 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140402.000009 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140404.000007 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140404.000007 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140405.000007 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140405.000007 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000008 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000115 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.000115 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.105020 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.105020 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113650 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113650 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113801 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113801 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113914 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113914 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114221 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114221 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114429 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114429 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115133 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115133 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115938 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115938 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120212 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120212 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120918 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120918 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121042 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121042 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121159 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121159 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121516 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121516 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121801 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121801 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121915 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121915 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122242 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122242 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122425 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122425 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123043 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123043 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123501 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123501 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123635 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123635 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123748 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123748 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123907 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123907 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124018 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124018 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124130 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124130 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124254 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124254 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124422 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124422 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124827 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124827 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125002 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125002 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125312 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125312 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125426 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125426 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125546 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125546 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201300 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201300 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201540 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201540 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202301 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202301 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202435 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202435 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202619 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202619 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202750 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202750 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203010 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203010 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203146 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203146 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203653 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203653 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203916 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203916 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204047 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204047 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204215 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204215 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204539 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204539 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204739 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204739 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204906 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204906 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205112 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205112 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205354 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205354 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.214704 >/dev/null [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.214704 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.222012 [Mon Apr 7 08:20:12] pmmgr(14776/32050): /etc/pcp/pmmgr: daemon pid 2841 started: /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.222012 [Mon Apr 7 08:20:27] pmmgr(14776/14776): /etc/pcp/pmmgr: dead hostid bozo [Mon Apr 7 08:20:27] pmmgr(14776/14776): /etc/pcp/pmmgr: daemon pid 2841 killed [Mon Apr 7 08:20:27] pmmgr(14776/14776): /etc/pcp/pmmgr: daemon pid 32063 killed [Mon Apr 7 08:21:27] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000743 [Mon Apr 7 08:21:27] pmmgr(14776/14776): /etc/pcp/pmmgr: new hostid bozo at local: [Mon Apr 7 08:21:27] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/lib/pcp/bin/pmlogconf -c -r -h local: /var/log/pcp/pmmgr/bozo/config.pmlogger >/dev/null [Mon Apr 7 08:21:27] pmmgr(14776/7685): /etc/pcp/pmmgr: running /usr/bin/pmieconf -F -c -f /var/log/pcp/pmmgr/bozo/config.pmie [Mon Apr 7 08:21:27] pmmgr(14776/7685): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 08:21:27] pmmgr(14776/7685): /etc/pcp/pmmgr: daemon pid 7696 started: /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140323.000012 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140323.000012 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140324.000014 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140324.000014 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000009 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000009 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000116 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000116 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000114 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000114 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000115 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000115 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140329.000012 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140329.000012 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140330.000008 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140330.000008 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140331.000012 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140331.000012 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140401.000011 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140401.000011 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140402.000009 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140402.000009 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140404.000007 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140404.000007 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140405.000007 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140405.000007 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000008 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000115 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.000115 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.105020 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.105020 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113650 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113650 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113801 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113801 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113914 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113914 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114221 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114221 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114429 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114429 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115133 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115133 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115938 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115938 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120212 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120212 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120918 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120918 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121042 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121042 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121159 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121159 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121516 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121516 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121801 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121801 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121915 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121915 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122242 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122242 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122425 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122425 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123043 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123043 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123501 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123501 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123635 >/dev/null [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123635 [Mon Apr 7 08:21:34] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123748 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123748 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123907 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123907 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124018 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124018 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124130 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124130 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124254 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124254 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124422 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124422 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124827 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124827 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125002 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125002 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125312 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125312 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125426 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125426 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125546 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125546 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201300 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201300 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201540 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201540 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202301 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202301 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202435 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202435 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202619 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202619 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202750 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202750 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203010 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203010 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203146 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203146 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203653 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203653 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203916 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203916 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204047 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204047 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204215 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204215 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204539 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204539 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204739 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204739 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204906 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204906 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205112 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205112 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205354 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205354 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.214704 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.214704 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.222012 >/dev/null [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.222012 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.222135 [Mon Apr 7 08:21:35] pmmgr(14776/7684): /etc/pcp/pmmgr: daemon pid 10828 started: /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.222135 [Mon Apr 7 08:22:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000707 [Mon Apr 7 08:23:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000663 [Mon Apr 7 08:24:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000738 [Mon Apr 7 08:25:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000682 [Mon Apr 7 08:26:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000683 [Mon Apr 7 08:27:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000674 [Mon Apr 7 08:28:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.0007 [Mon Apr 7 08:29:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.00066 [Mon Apr 7 08:30:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000691 [Mon Apr 7 08:31:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000719 [Mon Apr 7 08:32:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000665 [Mon Apr 7 08:33:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000678 [Mon Apr 7 08:34:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000817 [Mon Apr 7 08:35:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000684 [Mon Apr 7 08:36:35] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000733 [Mon Apr 7 08:36:47] pmmgr(14776/14776): /etc/pcp/pmmgr: dead hostid bozo [Mon Apr 7 08:36:47] pmmgr(14776/14776): /etc/pcp/pmmgr: daemon pid 10828 killed [Mon Apr 7 08:36:47] pmmgr(14776/14776): /etc/pcp/pmmgr: daemon pid 7696 killed [Mon Apr 7 08:37:47] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000721 [Mon Apr 7 08:37:47] pmmgr(14776/14776): /etc/pcp/pmmgr: new hostid bozo at local: [Mon Apr 7 08:37:47] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/lib/pcp/bin/pmlogconf -c -r -h local: /var/log/pcp/pmmgr/bozo/config.pmlogger >/dev/null [Mon Apr 7 08:37:47] pmmgr(14776/14230): /etc/pcp/pmmgr: running /usr/bin/pmieconf -F -c -f /var/log/pcp/pmmgr/bozo/config.pmie [Mon Apr 7 08:37:47] pmmgr(14776/14230): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 08:37:47] pmmgr(14776/14230): /etc/pcp/pmmgr: daemon pid 14241 started: /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log [Mon Apr 7 08:37:53] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140323.000012 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140323.000012 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140324.000014 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140324.000014 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000009 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000009 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140326.000116 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140326.000116 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000114 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000114 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140328.000115 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140328.000115 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140329.000012 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140329.000012 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140330.000008 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140330.000008 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140331.000012 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140331.000012 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140401.000011 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140401.000011 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140402.000009 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140402.000009 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140404.000007 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140404.000007 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140405.000007 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-old archive /var/log/pcp/pmmgr/bozo/archive-20140405.000007 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000008 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.000115 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.000115 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.105020 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.105020 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113650 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113650 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113801 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113801 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.113914 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.113914 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114221 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114221 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.114429 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.114429 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115133 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115133 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.115938 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.115938 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120212 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120212 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.120918 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.120918 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121042 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121042 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121159 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121159 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121516 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121516 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121801 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121801 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.121915 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.121915 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122242 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122242 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.122425 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.122425 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123043 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123043 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123501 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123501 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123635 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123635 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123748 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123748 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.123907 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.123907 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124018 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124018 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124130 >/dev/null [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124130 [Mon Apr 7 08:37:54] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124254 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124254 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124422 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124422 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.124827 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.124827 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125002 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125002 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125312 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125312 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125426 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125426 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.125546 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.125546 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201300 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201300 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.201540 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.201540 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202301 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202301 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202435 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202435 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202619 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202619 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.202750 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.202750 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203010 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203010 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203146 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203146 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203653 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203653 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.203916 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.203916 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204047 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204047 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204215 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204215 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204539 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204539 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204739 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204739 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.204906 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.204906 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205112 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205112 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.205354 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.205354 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.214704 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.214704 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.222012 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.222012 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: running /usr/bin/pmlogcheck /var/log/pcp/pmmgr/bozo/archive-20140406.222135 >/dev/null [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: skipping merge of too-new archive /var/log/pcp/pmmgr/bozo/archive-20140406.222135 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: fork/exec sh -c /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.223755 [Mon Apr 7 08:37:55] pmmgr(14776/14229): /etc/pcp/pmmgr: daemon pid 17377 started: /usr/bin/pmlogger -c /var/log/pcp/pmmgr/bozo/config.pmlogger -h local: -r -l /var/log/pcp/pmmgr/bozo/pmlogger.log -y -T \ \@Mon\ Apr\ \ 7\ 10:00:00\ 2014 /var/log/pcp/pmmgr/bozo/archive-20140406.223755 [Mon Apr 7 08:38:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000733 [Mon Apr 7 08:39:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000669 [Mon Apr 7 08:40:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000682 [Mon Apr 7 08:41:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.00067 [Mon Apr 7 08:42:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.00068 [Mon Apr 7 08:43:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000731 [Mon Apr 7 08:44:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000692 [Mon Apr 7 08:45:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000683 [Mon Apr 7 08:46:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000715 [Mon Apr 7 08:47:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.0007 [Mon Apr 7 08:48:55] pmmgr(14776/14776): /etc/pcp/pmmgr: hostid bozo via local: time 0.000689 --------------000602030702050708010401-- From kenj@internode.on.net Sun Apr 6 18:16:25 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id B36DA7F55 for ; Sun, 6 Apr 2014 18:16:25 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 9D6CF8F8035 for ; Sun, 6 Apr 2014 16:16:22 -0700 (PDT) X-ASG-Debug-ID: 1396826175-04cbb054b9b3b3c0001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id FiwaIqOJoPOKTnG6 for ; Sun, 06 Apr 2014 16:16:15 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Au0aABLfQVN20fAIPGdsb2JhbAANTINBiBq8VYEuAwEBAQE4gwhRAS8NFhgDAgECATEnAQUCAQGwRaJxF4lGhSuEPwSMfo0RhGaBOI4v Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail05.adl6.internode.on.net with ESMTP; 07 Apr 2014 08:46:13 +0930 Message-ID: <5341E094.5090304@internode.on.net> Date: Mon, 07 Apr 2014 09:17:40 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com CC: Martins Innus , "Eigler, Frank" Subject: pcp updates - slow start PMDAs Content-Type: text/plain; charset=ISO-8859-1 X-ASG-Orig-Subj: pcp updates - slow start PMDAs Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396826175 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4636 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- This commit will allow all PMDAs (C, Perl and Python) to get early access to pmdaConnect(). This should avoid most of the slow start issues where a PMDA is launched but takes some time to discover what metrics and/or instance domains are available ... if this process takes too long, pmcd will assume the initial credentials handshake has timed out and the PMDA will not be a happy camper thereafter. By giving the PMDA developer the option to connect to pmcd sooner, this problem is largely avoided. What remains is the slow PMDA issue where a subsequent PDU request from a PMAPI client is received from pmcd while the PMDA is "busy" (including slow start) ... this requires much more serious surgery to fix. If you've been seeing PMDAs not starting (perhaps sometimes, not all times), this change should help ... feedback and confirmation of this hypothesis would be most valuable. Changes committed to git://oss.sgi.com/kenj/pcp.git dev man/man3/pmdaconnect.3 | 40 ++++- qa/274.out.2 | 6 qa/553 | 1 qa/763 | 84 +++++++++- qa/763.out | 49 ++++++ qa/779 | 63 ++++++++ qa/779.out | 26 +++ qa/841 | 82 +++++++++- qa/841.out | 71 +++++++-- qa/972 | 1 qa/admin/pcp-daily | 54 ++++-- qa/group | 5 qa/pmdas/GNUmakefile | 6 qa/pmdas/GNUmakefile.install | 6 qa/pmdas/slow-python/GNUmakefile | 76 ++++----- qa/pmdas/slow-python/GNUmakefile.install | 14 - qa/pmdas/slow-python/Install | 76 ++++----- qa/pmdas/slow-python/Remove | 50 +++--- qa/pmdas/slow-python/pmdaslow.python | 168 ++++++++++----------- qa/pmdas/slow/GNUmakefile | 83 +++++++--- qa/pmdas/slow/GNUmakefile.install | 13 + qa/pmdas/slow/Install | 40 ++++- qa/pmdas/slow/Remove | 25 +++ qa/pmdas/slow/pmdaslow.pl | 75 +++++++++ qa/pmdas/slow_python/GNUmakefile | 40 +++++ qa/pmdas/slow_python/GNUmakefile.install | 5 qa/pmdas/slow_python/Install | 38 ++++ qa/pmdas/slow_python/Remove | 25 +++ qa/pmdas/slow_python/pmdaslow_python.python | 84 ++++++++++ qa/src/.gitignore | 1 qa/src/GNUlocaldefs | 5 qa/src/badpmda.c | 126 ++++++++++++++++ src/include/builddefs.in | 2 src/include/pcp/pmda.h | 7 src/libpcp_pmda/src/open.c | 31 +++ src/perl/PMDA/PMDA.pm | 219 ++++++++++++++++++++++++++++ src/perl/PMDA/PMDA.xs | 34 ++++ src/pmcd/pmdaproc.sh | 80 +++++++--- src/pmdas/kvm/pmdakvm.pl | 1 src/pmdas/postfix/pmdapostfix.pl | 1 src/pmdas/rsyslog/pmdarsyslog.pl | 2 src/pmdas/simple/pmdasimple.perl | 1 src/pmdas/simple/pmdasimple.python | 2 src/pmdas/simple/simple.c | 2 src/pmns/stdpmid.pcp | 6 src/python/pcp/pmda.py | 3 src/python/pmda.c | 60 ++++++- 47 files changed, 1562 insertions(+), 327 deletions(-) commit 6ebf92133c77d0e74965ff33dfebd8ae6f792cb2 Author: Ken McDonell Date: Mon Apr 7 09:09:13 2014 +1000 src/perl/PMDA/PMDA.pm - add connect_pmcd() and some basic documentation The functional change is the Perl support for connect_pmcd(). The cosmetic change is start documentation for the Perl PMDA API ... this is a WIP. commit 2ae3d2080695bac68517fe3bf9b5734102776e09 Author: Ken McDonell Date: Mon Apr 7 08:37:34 2014 +1000 stdpmid.pcp - allocate unique domain number for SLOW_PYTHON Just causes too many qa issues to have SLOW and SLOW_PYTHON using the same domain number. commit ce4c821f37d2c86185dab1fbf184d260cb52c323 Author: Ken McDonell Date: Mon Apr 7 06:38:27 2014 +1000 qa/slow PMDA (Perl version) - cleanup Changes to track changes made elsewhere in pmdaproc.sh and qa/763. commit 0e92676a7d84eec38eee717407d2688f1f5ea891 Author: Ken McDonell Date: Mon Apr 7 06:37:29 2014 +1000 qa/763 - install Perl version of slow PMDA from qa/pmdas/slow commit 24e5bf2b0d5caf31ee6117ecac096c9157d29e17 Author: Ken McDonell Date: Mon Apr 7 06:31:00 2014 +1000 src/pmcd/pmdaproc.sh - allow Install/Remove from any directory When the Perl and Python PMDA support was added, it introduced a restriction that these PMDAs can only be installed from a directory below $PCP_VAR_DIR/pmdas ... this is quite unnecessary. Remove that restriction. Also promote $pmda_dir (that was being used locally in pmdaSetup to the global scope so that an Install/Remove script can call source pmdaproc.sh and then change $pmda_dir to be some other absolute pathname before calling pmdaSetup. This would only be useful if your PMDA files for some bizarre reason live in a different directory to the one where the Install/Remove scripts are being run ... but does allow reversion to the previous behaviour on the off chance that some PMDA was depending upon it. commit 47cda57a87c0cf4106c6bc4e98464ff2d46a5741 Author: Ken McDonell Date: Mon Apr 7 06:29:00 2014 +1000 qa/slow PMDA (Python variant) - changes, part 2 Remove the old qa/pmdas/slow-python files from the git tree. commit 42f1ab0167a048f02709221b3fdcc282825fcc14 Author: Ken McDonell Date: Mon Apr 7 06:23:40 2014 +1000 qa/slow PMDA (Python variant) - changes To make this work with the Perl version of the PMDA both in the tree and in the "testsuite" package environmment, needed to make a number of changes. 1. change "iam" from "slow" (duplicating the Perl version) to "slow_python" 2. need stdpmid reference for SLOW_PYTHON (use the same domain number as SLOW) 3. rename slow-python to slow_python in the git tree (needed for PMDA Insall/Remove scripts to work) Much of this originally triggered by moving the qa use of the Install and Remove scripts to be run in qa/pmdas/slow_python instead of $PCP_VAR_DIR/pmdas/slow which is much tidier. commit ec05a573ce8362989dab492bf9bdb6cd11950371 Author: Ken McDonell Date: Sun Apr 6 17:53:19 2014 +1000 qa/553 & qa/972 - remove bad $pmda_dir use Both scripts used to cd "$pmda_dir" but pmda_dir was never set. Dropped the cd, and both still pass. commit d2b670eb565c2f50a6cb60f70754b62b78972095 Author: Ken McDonell Date: Sun Apr 6 06:38:57 2014 +1000 src/include/builddefs.in - remove diagnostic fluff Not sure when the appending to /tmp/eek.raw was added, but this is totally useless ... and fails if /tmp/eek.raw exists and is owned by someone else (root in my failing case). Given the name of the file, I am sure I added the diagnostic without review, so I'll remove it without review. commit e40149046b5e80ac958d6284ea70cd87afc7b73f Author: Ken McDonell Date: Sat Apr 5 18:27:26 2014 +1100 qa/763 - update Better diags and cleanup ... tracking changes in the dual qa/841 test. commit 394949d471b907c0fe57ab8bfdc9825559a58b72 Author: Ken McDonell Date: Sat Apr 5 18:22:00 2014 +1100 qa/841 (new) & slow PMDA (Python version) - slow start PMDA The Python version of the slow PMDA shows how self.connect_pmcd() may be used to better handle slow startup for Python PMDAs ... the use of self.connect_pmcd() is optional to allow the passing and failing cases to be explored. The PMDA will also do slow fetches, but the library support for this is not yet done. qa/841 exercises the Python version of the slow PMDA in the passing and failing cases. This is the dual of qa/763 that exercise the Perl version of the slow PMDA. commit bce4758e26df0090825af111b660d79e2841fafd Author: Ken McDonell Date: Sat Apr 5 18:19:43 2014 +1100 slow PMDA (Perl version) - makefile rework commit 2b57dacef947d94cd5600210f377f43eca14f613 Author: Ken McDonell Date: Sat Apr 5 18:14:03 2014 +1100 slow PMDA (Perl version) - update copyright commit ddc6e7ce8c495753843d2a8291c3203ad9aeb4ea Author: Ken McDonell Date: Sat Apr 5 07:23:36 2014 +1100 slow start PMDA changes - part 3 (PMDAs written in Python) Add the new self.connect_pmcd() method that allows Python PMDAs to make a pmdaConnect() call anytime after the PMDA.__init__() constructor. This will help address slow start issues for Python PMDAs. Change the Python version of the simple PMDA connect_pmcd. commit 41716278a0f1855e712ed7b6805025c8d9e1c647 Author: Ken McDonell Date: Sat Apr 5 07:19:14 2014 +1100 qa/763 & slow PMDA - slow starting PMDA The slow PMDA shows how connect_pmcd() may be used to better handle slow startup for Perl PMDAs ... the use of connect_pmcd() is optional to allow tha passing an failing cases to be explored. The PMDA will also do slow fetches, but the library support for this is not yet done. qa/763 exercises the slow PMDA in the passing and failing cases. commit 3cb41fcd7d425dbb4f5ab7497839e062c2982b26 Author: Ken McDonell Date: Sat Apr 5 07:13:06 2014 +1100 pmdaproc.sh - add $perl_args and $python_args Hooks to allow Perl and Python PMDAs to receive command line arguments. These can be setup in the Install script and then they are saved in pmcd.conf. commit e709ec43c3f313443d83afb602c5c46efb4f8b69 Author: Ken McDonell Date: Fri Apr 4 07:21:01 2014 +1100 slow start PMDA changes - part 2 (PMDAs written in Perl) Add the new pmda->connect_pmcd() method that allows Perl PMDAs to make a pmdaConnect() call anytime after the new() constructor. This will address slow start issues for many PMDAs. Change the following (randomly selected) PMDAs to use connect_pmcd as a proof of concept: kvm, postfix, rsyslog and simple. commit acef3014542b0cc78f69608ff56e91612242c421 Author: Ken McDonell Date: Fri Apr 4 06:42:51 2014 +1100 qa/src/badpmda.c - fix compilation warning commit 5bac3133fe63fa44aa07c604ddf75b83b9e8fec8 Author: Ken McDonell Date: Thu Apr 3 16:38:22 2014 +1100 libpcp_pmda - slow start PMDA changes - part 1 (PMDAs written in C) Allow PMDAs to call pmdaConnect() earlier. This reduces the risk of a slow starting PMDA being killed with a timeout in the credentials handshake that pmcd starts as soon as a daemon PMDA is launched. There is little code change here, just additional internal state and checking to make sure the pmdaInterface structure has been properly setup when pmdaConnect is called. The rest is QA (qa/799 is new) and more concise wording for the pmdaConnect(3) man page. The C version of the simple PMDA has been changed to show how pmdaConnect() can be called earlier in the PMDA's initialization. commit c45073faf55275fde086c636f3aa66cb92a71d61 Author: Ken McDonell Date: Thu Apr 3 06:29:24 2014 +1100 qa/admin/pcp-daily - track recent build changes Use --clean to Makepkgs (to be clear). Need to look elsewhere for packages for many builds. From fche@redhat.com Sun Apr 6 19:04:25 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 154227F59 for ; Sun, 6 Apr 2014 19:04:25 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 02C6F8F8033 for ; Sun, 6 Apr 2014 17:04:21 -0700 (PDT) X-ASG-Debug-ID: 1396829060-04cbb054b7b3dac0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id VjYWjCcCDuhm8nV7 for ; Sun, 06 Apr 2014 17:04:21 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3704HwN023051 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 6 Apr 2014 20:04:17 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3704GU5031831; Sun, 6 Apr 2014 20:04:17 -0400 Received: by fche.csb (Postfix, from userid 2569) id 0C9B3585D8; Sun, 6 Apr 2014 20:04:15 -0400 (EDT) Date: Sun, 6 Apr 2014 20:04:15 -0400 From: "Frank Ch. Eigler" To: Ken McDonell Cc: PCP Mailing List Subject: Re: something not right in pmmgr land? Message-ID: <20140407000415.GA13940@redhat.com> X-ASG-Orig-Subj: Re: something not right in pmmgr land? References: <533DEBB9.7010608@internode.on.net> <5341DA0F.6000202@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5341DA0F.6000202@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1396829060 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - Thanks a lot for the trace. > kenj@bozo:~/src/pcp/qa$ ps -ef | grep '[p]mie.*pmmgr' > pcp 7698 29916 0 08:21 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log > pcp 14241 14776 0 08:37 pts/1 00:00:00 sh -c /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log > pcp 14243 14241 0 08:37 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log > pcp 14794 29916 0 07:46 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log > pcp 32065 29916 0 08:20 pts/1 00:00:00 /usr/bin/pmie -c /var/log/pcp/pmmgr/bozo/config.pmie -h local: -f -l /var/log/pcp/pmmgr/bozo/pmie.log Note that the pmmgr.log file comes from pmmgr pid 14776, which has killed its child pmies/etc. multiple times. Just one (pid 14241/14243 from your list) is still alive, which is correct. The question is where the others are from: who is pid 29916 and why is she still running? Oh, you later say it's "init"? Interesting (why not pid 1?); if so, those old pmies must have come from a previous pmmgr process that has gone away and didn't send out a SIGTERM memo. Was that perhaps a pmmgr run during pcpqa? If so, please consider shutting down the system pmmgr during test time, so we get a clean pmmgr.log etc. (It's unfortunate that system processes conflict with pcpqa.) > The word "sig" (ignoring case) does not appear to be in the pmmgr.log file. (Sorry, "killed" is in there.) - FChE From nscott@redhat.com Sun Apr 6 20:59:59 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 60E867F59 for ; Sun, 6 Apr 2014 20:59:59 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 4657730404E for ; Sun, 6 Apr 2014 18:59:56 -0700 (PDT) X-ASG-Debug-ID: 1396835990-04cb6c5678d32fa0001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id LoeqyiRMPkaMBE4A for ; Sun, 06 Apr 2014 18:59:51 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s371xoTW009793 for ; Sun, 6 Apr 2014 21:59:50 -0400 Date: Sun, 6 Apr 2014 21:59:50 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp@oss.sgi.com Message-ID: <923762078.354909.1396835990488.JavaMail.zimbra@redhat.com> In-Reply-To: <735244779.354215.1396835652289.JavaMail.zimbra@redhat.com> Subject: pcp updates: pmdarpm, python api, makepkgs MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: pmdarpm, python api, makepkgs Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.7] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: pmdarpm, python api, makepkgs Thread-Index: h9uQDHU9HBrMUkcpvjCSTBHLTjt9nQ== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1396835990 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4640 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev HACKING | 66 ++++++------- Makepkgs | 31 ++++-- qa/728.out |binary qa/750 | 5 - qa/common.rpm | 47 +++++++++ qa/src/test_pcp.python | 14 ++ src/pmdas/rpm/rpm.c | 230 +++++++++++++++++++++++++++++------------------- src/python/pcp/pmapi.py | 4 8 files changed, 256 insertions(+), 141 deletions(-) commit 5e080559cb1f2032414451657c37153b14a64f39 Merge: df34bee 9b8c9b1 Author: Nathan Scott Date: Mon Apr 7 11:30:43 2014 +1000 Merge branch 'fche/rpm' of ../pcpfans into dev commit df34bee42082be8f9d73bca5f1118a9dab58aa26 Author: Nathan Scott Date: Mon Apr 7 11:29:41 2014 +1000 Add a Makepkgs mode allowing for absence of GNU tar If we have no GNU tar, we can still stumble on with building the packages via Makepkgs, thanks to git-archive. Then downside of this mode is that it does not pick up not-yet-committed changes. So its added as a fallback mode, and is used only when a GNU tar binary cannot be found. commit 0f0563ad023e45feb86db2eb4de57fc96db470e4 Author: Nathan Scott Date: Mon Apr 7 11:24:53 2014 +1000 Provide a python pmLookupName API mode allowing partial failure The python wrapper for pmLookupName takes a hard-line approach to metric name resolution - if any are missing, an exception will be thrown, and any names which did resolve are tossed in the bin. The underlying C API is designed to cater for missing metrics and tools use this facility surprisingly often. It allows for cross- platform quirks to be elegantly handled, and other fallback modes (eg. where two similar metrics exist, and a prefered one might be unavailable). This API extension is tackled in a backwards-compatible way, by using an optional second argument to allow the "relaxed" handling of missing metric names. Tests qa/702 and qa/707 now exercise this little addition, via an extension to the qa/src/test_pcp.python script which they share. commit 9b8c9b1a5dc9c9fbf6266fc8884a576ee4c11f00 Author: Nathan Scott Date: Mon Apr 7 11:19:11 2014 +1000 Resolve missed change-does-not-match-surrounding-code in pmdarpm A recent threading fix to pmdarpm introduced an exciting new coding style into this file, contrary to HACKING Conventions. On review, some were removed, but others overlooked. This fixes the remaining ones so that this code does not start to look like Frankenstein, in the long term (too late, you may argue ;) ... but at least we try!) commit 862b2032bd618850258db3ca7b092bc1e30b342c Author: Nathan Scott Date: Mon Apr 7 10:53:35 2014 +1000 Update test 750 to more robustly detect an installed rpm Test qa/750 and common.rpm are updated to remove the increasing sleep-waiting-for-rpmdb-refresh interval, and replace it with a check on the presence/absence (depending on whether we are doing addition/removal) of a named package in the rpm instance domain. As an added bonus for folks running QA often, the test now runs quite a bit more quickly in the case where the rpm refresh takes only a short time (which appears to be the normal case, locally at least). commit 1e1fee65e733698c2b2daf6da4cc9348d8947d38 Author: Nathan Scott Date: Mon Apr 7 09:35:21 2014 +1000 Add missing output from test 728 commit dae50a1c31256e2cf1369ee53740257037e3ee4b Author: Nathan Scott Date: Mon Apr 7 09:08:59 2014 +1000 Refine HACKING by removing unnecessary/ambiguous words commit bd4ca2a97d41a33f509428a3aa98ef7f5ddfeb76 Author: Frank Ch. Eigler Date: Thu Apr 3 16:52:30 2014 -0400 RHBZ1044297: whitespace cleanup Because any time is whitespace fixing time. commit 2b819ff78995cf243413afe67996827a04dae551 Author: Frank Ch. Eigler Date: Thu Apr 3 11:53:28 2014 -0400 RHBZ1044297: improve rpm-pmda concurrency complications A review with rhbz1044297 in mind indicated a few possible problems with the way rpm-pmda was using threads. The main problem is that the inotify_thread could run concurrently with the one-shot indom_thread, which could lead to all kinds of mayhem, because the librpm api functions in rpm_update_cache() are called without concurrency protection. We eliminate the indom-thread entirely, instead triggering an initial round of rpm-cache population at the head of inotify_thread. Part of the trigger appeared to be rpmReadConfigFiles(), which is highly stateful inside librpm. We also now preclude multiple calls to that function with a simple static flag. Smaller problems included lack of error protection in the inotify setup, and unnecessary processing of the received inotify events. (If we get any events, it means we must refresh.) The qa/750 test case's pmsleep fudge-factor was raised yet higher to account for machines with a large rpm database. There is an inherent race here between system rpm-database updates and the rpmpmda listener. We might be able to solve more robustly polling the rpm.refresh.count value, but this is left for the future. From kenj@internode.on.net Mon Apr 7 01:47:55 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D8C957F56 for ; Mon, 7 Apr 2014 01:47:55 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 5E906AC004 for ; Sun, 6 Apr 2014 23:47:55 -0700 (PDT) X-ASG-Debug-ID: 1396853267-04cbb054b6b551c0001-S8gJnT Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by cuda.sgi.com with ESMTP id GGpeyHIsKyxpQjKP for ; Sun, 06 Apr 2014 23:47:48 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.145 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiYeAL9IQlN20fAIPGdsb2JhbAANTINBiBq+DAMBAQEBOIMZQDANFhgDAgECATEnBgIBAbBCowEXjg2BAYQiBK5cgVc Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail06.adl6.internode.on.net with ESMTP; 07 Apr 2014 16:17:46 +0930 Message-ID: <53424A69.7070100@internode.on.net> Date: Mon, 07 Apr 2014 16:49:13 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - oops Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - oops Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail06.adl6.internode.on.net[150.101.137.145] X-Barracuda-Start-Time: 1396853268 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4644 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/pmdas/slow_python/pmdaslow_python.python | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit f3ea03e533805babdedb2cfa075171f5a255f5cb Author: Ken McDonell Date: Mon Apr 7 16:48:15 2014 +1000 slow_python PMDA - fix domain number in one last place From anastasiou@teiser.gr Mon Apr 7 03:07:01 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 62BD27F56 for ; Mon, 7 Apr 2014 03:07:01 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 50542304039 for ; Mon, 7 Apr 2014 01:06:57 -0700 (PDT) X-ASG-Debug-ID: 1396858012-04cbb054b7b59b20001-S8gJnT Received: from teiser.gr (mail.teiser.gr [195.130.67.2]) by cuda.sgi.com with ESMTP id AlfgYXOisBigW1c5 (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 07 Apr 2014 01:06:54 -0700 (PDT) X-Barracuda-Envelope-From: anastasiou@teiser.gr X-Barracuda-Apparent-Source-IP: 195.130.67.2 Received: from mail.teiser.gr ([195.130.67.2]) by teiser.gr with esmtpa (Exim 4.72 (FreeBSD)) (envelope-from ) id 1WX4WH-000HSy-7s; Mon, 07 Apr 2014 11:03:05 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 07 Apr 2014 10:03:04 +0200 From: Karin LOCK To: undisclosed-recipients:; Subject: =?UTF-8?Q?=CF=80=CF=81=CE=BF=CF=83=CF=86=CE=BF=CF=81=CE=AC=20?= =?UTF-8?Q?=CE=B4=CE=B1=CE=BD=CE=B5=CE=AF=CE=BF=CF=85=20=CF=83=CE=B5=20=32?= =?UTF-8?Q?=25?= Reply-To: lockkarin@gmail.com X-ASG-Orig-Subj: =?UTF-8?Q?=CF=80=CF=81=CE=BF=CF=83=CF=86=CE=BF=CF=81=CE=AC=20?= =?UTF-8?Q?=CE=B4=CE=B1=CE=BD=CE=B5=CE=AF=CE=BF=CF=85=20=CF=83=CE=B5=20=32?= =?UTF-8?Q?=25?= Mail-Reply-To: lockkarin@gmail.com Message-ID: <771f507543fe7299dd52275770ce1838@teiser.gr> X-Sender: anastasiou@teiser.gr User-Agent: RoundCube Webmail/0.2 X-Barracuda-Connect: mail.teiser.gr[195.130.67.2] X-Barracuda-Start-Time: 1396858013 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.70 X-Barracuda-Spam-Status: No, SCORE=0.70 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA620a, BSF_SC7_SA298e X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4646 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_SC0_SA620a Custom Rule SA620a 0.20 BSF_SC7_SA298e Custom Rule SA298e -- Γεια σας, Σας ενδιαφέρει σε ένα οικονομικό δάνειο στο 2%; επικοινωνήστε μαζί μου για λεπτομέρειες και προϋποθέσεις. ταχυδρομείο μου: lockkarin@gmail.com σας ευχαριστώ From fche@redhat.com Mon Apr 7 09:10:03 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B11E97F4E for ; Mon, 7 Apr 2014 09:10:03 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 40B1FAC00B for ; Mon, 7 Apr 2014 07:10:00 -0700 (PDT) X-ASG-Debug-ID: 1396879795-04cb6c5675d679b0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id T2yFg1iQALb6Mhxk for ; Mon, 07 Apr 2014 07:09:56 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s37E9orM007396 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 7 Apr 2014 10:09:50 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s37E9ort023145; Mon, 7 Apr 2014 10:09:50 -0400 Received: by fche.csb (Postfix, from userid 2569) id BEDF9585D8; Mon, 7 Apr 2014 10:09:49 -0400 (EDT) Date: Mon, 7 Apr 2014 10:09:49 -0400 From: "Frank Ch. Eigler" To: Ken McDonell Cc: pcp developers Subject: Re: something not right in pmmgr land? Message-ID: <20140407140949.GB14052@redhat.com> X-ASG-Orig-Subj: Re: something not right in pmmgr land? References: <533DEBB9.7010608@internode.on.net> <5341DA0F.6000202@internode.on.net> <20140407000415.GA13940@redhat.com> <003501cf5252$a3c0ef40$eb42cdc0$@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <003501cf5252$a3c0ef40$eb42cdc0$@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1396879796 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - Ken's keen eye pointed at a difference in behavior between my workstation (Fedora* etc.) and his: the presence of "sh -c FOO" processes. It seems to have come down to a matter of bash configuration, of all things: >From git://git.savannah.gnu.org/bash.git config-top.h: 35 /* Define ONESHOT if you want sh -c 'command' to avoid forking to execute 36 `command' whenever possible. This is a big efficiency improvement. */ 37 #define ONESHOT 38 A quickie pmmgr patch (prefixing sh -c "exec FOO") should force this behavior on non-bash shells too. - FChE From kenj@internode.on.net Mon Apr 7 17:24:56 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 31FFC7FF9 for ; Mon, 7 Apr 2014 17:24:56 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 0EEF08F8052 for ; Mon, 7 Apr 2014 15:24:52 -0700 (PDT) X-ASG-Debug-ID: 1396909486-04bdf05dabdc1c80001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id sWSGXQItoCGcp6rE for ; Mon, 07 Apr 2014 15:24:47 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjcbABYlQ1N20fAIPGdsb2JhbAANTINBiBq+JgMBAQEBOIMZQDANFhgDAgECATEnBgIBAbAaoyUXjw6EIgSaD5RN Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail04.adl6.internode.on.net with ESMTP; 08 Apr 2014 07:54:46 +0930 Message-ID: <53432605.9040701@internode.on.net> Date: Tue, 08 Apr 2014 08:26:13 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - misc Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - misc Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1396909487 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4668 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/pmdas/slow_python/pmdaslow_python.python | 2 +- src/perl/PMDA/PMDA.pm | 6 +++++- src/pmdas/rsyslog/pmdarsyslog.pl | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-) commit ea28d8fed628792af2b9fe229603d951ac41caea Author: Ken McDonell Date: Tue Apr 8 08:25:11 2014 +1000 src/perl/PMDA/PMDA.pm - add a bit more to the PCP::PMDA man page commit 384973b679ed16d7a538864803111156b5861df9 Author: Ken McDonell Date: Tue Apr 8 08:00:24 2014 +1000 rsyslog PMDA - fix botch at last change I mangled the connect_pmcd() change .. oops. commit f3ea03e533805babdedb2cfa075171f5a255f5cb Author: Ken McDonell Date: Mon Apr 7 16:48:15 2014 +1000 slow_python PMDA - fix domain number in one last place From kenj@internode.on.net Mon Apr 7 17:40:04 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C0E2029E00 for ; Mon, 7 Apr 2014 17:40:04 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id AA69C8F8054 for ; Mon, 7 Apr 2014 15:40:04 -0700 (PDT) X-ASG-Debug-ID: 1396910402-04cbb054b7ba4750001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id 6iXyQu8rsKMXgQ26 for ; Mon, 07 Apr 2014 15:40:02 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ag05ANQoQ1N20fAIPGdsb2JhbAANTINBiBq8XwGBRwMBAQEBOINZMA0WGAMCAQIBMScGAgEBsB6jJRePDoQiBJoPlE0 Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail04.adl6.internode.on.net with ESMTP; 08 Apr 2014 08:10:01 +0930 Message-ID: <53432999.1030706@internode.on.net> Date: Tue, 08 Apr 2014 08:41:29 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - Frank's pmmgr exec fix Content-Type: text/plain; charset=ISO-8859-1 X-ASG-Orig-Subj: pcp updates - Frank's pmmgr exec fix Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1396910402 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4668 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Applied locally and tested. All's well. Thanks Frank. Changes committed to git://oss.sgi.com/kenj/pcp.git dev src/pmmgr/pmmgr.cxx | 5 +++++ 1 file changed, 5 insertions(+) commit 71e081e914dda7e172c45be84c872c1dc2331036 Merge: ea28d8f fcc4c3e Author: Ken McDonell Date: Tue Apr 8 08:31:26 2014 +1000 Merge branch 'fche/for-merge' of git://sourceware.org/git/pcpfans into dev commit fcc4c3ee6f7c131e5327f234cde77adcb0721857 Author: Frank Ch. Eigler Date: Mon Apr 7 13:00:10 2014 -0400 pmmgr: use /bin/sh -c "exec FOO" instead of "FOO" for launching daemons It turns out that bash behaves differently from other shells when being invoked as "/bin/sh -c COMMAND". For pmmgr's daemon-killing signals to work, the shell's pid must belong to the COMMAND (or at least, signals to $pid would need to be relayed). The easiest fix seems to be having pmmgr prefix "exec COMMAND" to force any shell's behavior into the bash default. From nscott@redhat.com Mon Apr 7 21:09:15 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 54DEE8032 for ; Mon, 7 Apr 2014 21:09:15 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id AC13BAC012 for ; Mon, 7 Apr 2014 19:09:11 -0700 (PDT) X-ASG-Debug-ID: 1396922945-04cbb054b9bb2200001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id FkMqlMRUQzSPDXpL for ; Mon, 07 Apr 2014 19:09:05 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s38295iW025811 for ; Mon, 7 Apr 2014 22:09:05 -0400 Date: Mon, 7 Apr 2014 22:09:05 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <2013647381.1294320.1396922945121.JavaMail.zimbra@redhat.com> In-Reply-To: <2086742713.1293791.1396922645854.JavaMail.zimbra@redhat.com> Subject: pcp updates: kenj + fche merge, docs, qa galore MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: kenj + fche merge, docs, qa galore Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.7] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: kenj + fche merge, docs, qa galore Thread-Index: t6sTIBIQGxws1DF4CryVJ+EmsCqajg== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1396922945 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4674 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev man/man3/pmdaconnect.3 | 40 ++ qa/274.out.2 | 6 qa/524 | 2 qa/553 | 1 qa/763 | 84 ++++- qa/763.out | 49 +++ qa/779 | 71 ++++ qa/779.out | 28 + qa/841 | 82 ++++- qa/841.out | 71 ++++ qa/972 | 1 qa/admin/pcp-daily | 54 ++- qa/group | 5 qa/pmdas/GNUmakefile | 6 qa/pmdas/GNUmakefile.install | 6 qa/pmdas/slow-python/GNUmakefile | 76 ++--- qa/pmdas/slow-python/GNUmakefile.install | 14 qa/pmdas/slow-python/Install | 76 ++--- qa/pmdas/slow-python/Remove | 50 +-- qa/pmdas/slow-python/pmdaslow.python | 168 +++++------ qa/pmdas/slow/GNUmakefile | 89 ++++- qa/pmdas/slow/GNUmakefile.install | 13 qa/pmdas/slow/Install | 40 ++ qa/pmdas/slow/Remove | 27 + qa/pmdas/slow/pmdaslow.pl | 78 +++++ qa/pmdas/slow_python/GNUmakefile | 40 ++ qa/pmdas/slow_python/GNUmakefile.install | 5 qa/pmdas/slow_python/Install | 38 ++ qa/pmdas/slow_python/Remove | 25 + qa/pmdas/slow_python/pmdaslow_python.python | 86 +++++ qa/src/.gitignore | 1 qa/src/GNUlocaldefs | 5 qa/src/badpmda.c | 126 ++++++++ src/include/builddefs.in | 2 src/include/pcp/pmda.h | 7 src/libpcp_pmda/src/open.c | 31 +- src/perl/PMDA/PMDA.pm | 417 +++++++++++++++++++++++----- src/perl/PMDA/PMDA.xs | 34 ++ src/pmcd/pmdaproc.sh | 80 +++-- src/pmdas/kvm/pmdakvm.pl | 1 src/pmdas/postfix/pmdapostfix.pl | 1 src/pmdas/rsyslog/pmdarsyslog.pl | 4 src/pmdas/simple/pmdasimple.perl | 1 src/pmdas/simple/pmdasimple.python | 2 src/pmdas/simple/simple.c | 2 src/pmmgr/pmmgr.cxx | 5 src/pmns/stdpmid.pcp | 6 src/python/pcp/pmda.py | 3 src/python/pmda.c | 60 +++- 49 files changed, 1714 insertions(+), 405 deletions(-) commit c7b1bb5ec75f3f2974cd5f261e85b63a32c74ae8 Author: Nathan Scott Date: Tue Apr 8 12:01:55 2014 +1000 Use _get_port in qa/779, fix missed comments Make use of the _get_port function to find a suitable port, rather than hard-coding one with "#hack" comments :) Several spots where cut&paste of comments was NQR resolved. commit 589868263e8e9335b9bcf4fbd7346af87312a470 Author: Nathan Scott Date: Tue Apr 8 11:26:39 2014 +1000 Document all of the remaining PCP::PMDA perl module interfaces Visit every TODO and add explanatory text, as well as refining the words for some existing descriptions. commit 80f1ccc4862b61937512bfb45402d4fb3f104da8 Merge: ac69185 71e081e Author: Nathan Scott Date: Tue Apr 8 09:58:10 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit 71e081e914dda7e172c45be84c872c1dc2331036 Merge: ea28d8f fcc4c3e Author: Ken McDonell Date: Tue Apr 8 08:31:26 2014 +1000 Merge branch 'fche/for-merge' of git://sourceware.org/git/pcpfans into dev commit ea28d8fed628792af2b9fe229603d951ac41caea Author: Ken McDonell Date: Tue Apr 8 08:25:11 2014 +1000 src/perl/PMDA/PMDA.pm - add a bit more to the PCP::PMDA man page commit 384973b679ed16d7a538864803111156b5861df9 Author: Ken McDonell Date: Tue Apr 8 08:00:24 2014 +1000 rsyslog PMDA - fix botch at last change I mangled the connect_pmcd() change .. oops. commit fcc4c3ee6f7c131e5327f234cde77adcb0721857 Author: Frank Ch. Eigler Date: Mon Apr 7 13:00:10 2014 -0400 pmmgr: use /bin/sh -c "exec FOO" instead of "FOO" for launching daemons It turns out that bash behaves differently from other shells when being invoked as "/bin/sh -c COMMAND". For pmmgr's daemon-killing signals to work, the shell's pid must belong to the COMMAND (or at least, signals to $pid would need to be relayed). The easiest fix seems to be having pmmgr prefix "exec COMMAND" to force any shell's behavior into the bash default. commit f3ea03e533805babdedb2cfa075171f5a255f5cb Author: Ken McDonell Date: Mon Apr 7 16:48:15 2014 +1000 slow_python PMDA - fix domain number in one last place commit ac69185e53c2f17395e9039de35b600bd7152fbe Merge: 5e08055 6ebf921 Author: Nathan Scott Date: Mon Apr 7 16:42:42 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit 7cd7cfa1989fe1ae13fbe009737175990e88b5f2 Merge: e2bd861 5e08055 Author: Ken McDonell Date: Mon Apr 7 15:54:16 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/pcp/pcp into dev commit e2bd861b06e208061020dc4f0ca7225ff6b3656d Merge: 6ebf921 4c61b2a Author: Ken McDonell Date: Mon Apr 7 09:18:27 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/pcp/pcp into dev commit 6ebf92133c77d0e74965ff33dfebd8ae6f792cb2 Author: Ken McDonell Date: Mon Apr 7 09:09:13 2014 +1000 src/perl/PMDA/PMDA.pm - add connect_pmcd() and some basic documentation The functional change is the Perl support for connect_pmcd(). The cosmetic change is start documentation for the Perl PMDA API ... this is a WIP. commit 2ae3d2080695bac68517fe3bf9b5734102776e09 Author: Ken McDonell Date: Mon Apr 7 08:37:34 2014 +1000 stdpmid.pcp - allocate unique domain number for SLOW_PYTHON Just causes too many qa issues to have SLOW and SLOW_PYTHON using the same domain number. commit ce4c821f37d2c86185dab1fbf184d260cb52c323 Author: Ken McDonell Date: Mon Apr 7 06:38:27 2014 +1000 qa/slow PMDA (Perl version) - cleanup Changes to track changes made elsewhere in pmdaproc.sh and qa/763. commit 0e92676a7d84eec38eee717407d2688f1f5ea891 Author: Ken McDonell Date: Mon Apr 7 06:37:29 2014 +1000 qa/763 - install Perl version of slow PMDA from qa/pmdas/slow commit 24e5bf2b0d5caf31ee6117ecac096c9157d29e17 Author: Ken McDonell Date: Mon Apr 7 06:31:00 2014 +1000 src/pmcd/pmdaproc.sh - allow Install/Remove from any directory When the Perl and Python PMDA support was added, it introduced a restriction that these PMDAs can only be installed from a directory below $PCP_VAR_DIR/pmdas ... this is quite unnecessary. Remove that restriction. Also promote $pmda_dir (that was being used locally in pmdaSetup to the global scope so that an Install/Remove script can call source pmdaproc.sh and then change $pmda_dir to be some other absolute pathname before calling pmdaSetup. This would only be useful if your PMDA files for some bizarre reason live in a different directory to the one where the Install/Remove scripts are being run ... but does allow reversion to the previous behaviour on the off chance that some PMDA was depending upon it. commit 47cda57a87c0cf4106c6bc4e98464ff2d46a5741 Author: Ken McDonell Date: Mon Apr 7 06:29:00 2014 +1000 qa/slow PMDA (Python variant) - changes, part 2 Remove the old qa/pmdas/slow-python files from the git tree. commit 42f1ab0167a048f02709221b3fdcc282825fcc14 Author: Ken McDonell Date: Mon Apr 7 06:23:40 2014 +1000 qa/slow PMDA (Python variant) - changes To make this work with the Perl version of the PMDA both in the tree and in the "testsuite" package environmment, needed to make a number of changes. 1. change "iam" from "slow" (duplicating the Perl version) to "slow_python" 2. need stdpmid reference for SLOW_PYTHON (use the same domain number as SLOW) 3. rename slow-python to slow_python in the git tree (needed for PMDA Insall/Remove scripts to work) Much of this originally triggered by moving the qa use of the Install and Remove scripts to be run in qa/pmdas/slow_python instead of $PCP_VAR_DIR/pmdas/slow which is much tidier. commit ec05a573ce8362989dab492bf9bdb6cd11950371 Author: Ken McDonell Date: Sun Apr 6 17:53:19 2014 +1000 qa/553 & qa/972 - remove bad $pmda_dir use Both scripts used to cd "$pmda_dir" but pmda_dir was never set. Dropped the cd, and both still pass. commit d2b670eb565c2f50a6cb60f70754b62b78972095 Author: Ken McDonell Date: Sun Apr 6 06:38:57 2014 +1000 src/include/builddefs.in - remove diagnostic fluff Not sure when the appending to /tmp/eek.raw was added, but this is totally useless ... and fails if /tmp/eek.raw exists and is owned by someone else (root in my failing case). Given the name of the file, I am sure I added the diagnostic without review, so I'll remove it without review. commit e40149046b5e80ac958d6284ea70cd87afc7b73f Author: Ken McDonell Date: Sat Apr 5 18:27:26 2014 +1100 qa/763 - update Better diags and cleanup ... tracking changes in the dual qa/841 test. commit 394949d471b907c0fe57ab8bfdc9825559a58b72 Author: Ken McDonell Date: Sat Apr 5 18:22:00 2014 +1100 qa/841 (new) & slow PMDA (Python version) - slow start PMDA The Python version of the slow PMDA shows how self.connect_pmcd() may be used to better handle slow startup for Python PMDAs ... the use of self.connect_pmcd() is optional to allow the passing and failing cases to be explored. The PMDA will also do slow fetches, but the library support for this is not yet done. qa/841 exercises the Python version of the slow PMDA in the passing and failing cases. This is the dual of qa/763 that exercise the Perl version of the slow PMDA. commit bce4758e26df0090825af111b660d79e2841fafd Author: Ken McDonell Date: Sat Apr 5 18:19:43 2014 +1100 slow PMDA (Perl version) - makefile rework commit 2b57dacef947d94cd5600210f377f43eca14f613 Author: Ken McDonell Date: Sat Apr 5 18:14:03 2014 +1100 slow PMDA (Perl version) - update copyright commit ddc6e7ce8c495753843d2a8291c3203ad9aeb4ea Author: Ken McDonell Date: Sat Apr 5 07:23:36 2014 +1100 slow start PMDA changes - part 3 (PMDAs written in Python) Add the new self.connect_pmcd() method that allows Python PMDAs to make a pmdaConnect() call anytime after the PMDA.__init__() constructor. This will help address slow start issues for Python PMDAs. Change the Python version of the simple PMDA connect_pmcd. commit 41716278a0f1855e712ed7b6805025c8d9e1c647 Author: Ken McDonell Date: Sat Apr 5 07:19:14 2014 +1100 qa/763 & slow PMDA - slow starting PMDA The slow PMDA shows how connect_pmcd() may be used to better handle slow startup for Perl PMDAs ... the use of connect_pmcd() is optional to allow tha passing an failing cases to be explored. The PMDA will also do slow fetches, but the library support for this is not yet done. qa/763 exercises the slow PMDA in the passing and failing cases. commit 3cb41fcd7d425dbb4f5ab7497839e062c2982b26 Author: Ken McDonell Date: Sat Apr 5 07:13:06 2014 +1100 pmdaproc.sh - add $perl_args and $python_args Hooks to allow Perl and Python PMDAs to receive command line arguments. These can be setup in the Install script and then they are saved in pmcd.conf. commit e709ec43c3f313443d83afb602c5c46efb4f8b69 Author: Ken McDonell Date: Fri Apr 4 07:21:01 2014 +1100 slow start PMDA changes - part 2 (PMDAs written in Perl) Add the new pmda->connect_pmcd() method that allows Perl PMDAs to make a pmdaConnect() call anytime after the new() constructor. This will address slow start issues for many PMDAs. Change the following (randomly selected) PMDAs to use connect_pmcd as a proof of concept: kvm, postfix, rsyslog and simple. commit acef3014542b0cc78f69608ff56e91612242c421 Author: Ken McDonell Date: Fri Apr 4 06:42:51 2014 +1100 qa/src/badpmda.c - fix compilation warning commit 5bac3133fe63fa44aa07c604ddf75b83b9e8fec8 Author: Ken McDonell Date: Thu Apr 3 16:38:22 2014 +1100 libpcp_pmda - slow start PMDA changes - part 1 (PMDAs written in C) Allow PMDAs to call pmdaConnect() earlier. This reduces the risk of a slow starting PMDA being killed with a timeout in the credentials handshake that pmcd starts as soon as a daemon PMDA is launched. There is little code change here, just additional internal state and checking to make sure the pmdaInterface structure has been properly setup when pmdaConnect is called. The rest is QA (qa/799 is new) and more concise wording for the pmdaConnect(3) man page. The C version of the simple PMDA has been changed to show how pmdaConnect() can be called earlier in the PMDA's initialization. commit c45073faf55275fde086c636f3aa66cb92a71d61 Author: Ken McDonell Date: Thu Apr 3 06:29:24 2014 +1100 qa/admin/pcp-daily - track recent build changes Use --clean to Makepkgs (to be clear). Need to look elsewhere for packages for many builds. From nscott@redhat.com Mon Apr 7 21:13:34 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 0240A8032 for ; Mon, 7 Apr 2014 21:13:34 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id C8B758F8035 for ; Mon, 7 Apr 2014 19:13:30 -0700 (PDT) X-ASG-Debug-ID: 1396923208-04cbb054b6bb2760001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id TcBJlZHE0FmBCwDA for ; Mon, 07 Apr 2014 19:13:29 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s382DNfR009347; Mon, 7 Apr 2014 22:13:23 -0400 Date: Mon, 7 Apr 2014 22:13:23 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell , "Frank Ch. Eigler" Cc: pcp@oss.sgi.com Message-ID: <1960971043.1295159.1396923203345.JavaMail.zimbra@redhat.com> In-Reply-To: <53432999.1030706@internode.on.net> References: <53432999.1030706@internode.on.net> Subject: Re: [pcp] pcp updates - Frank's pmmgr exec fix MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pcp updates - Frank's pmmgr exec fix Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.7] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates - Frank's pmmgr exec fix Thread-Index: lGmlFBKE7vVzvt7Z3VBtSUdImuAoMg== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1396923209 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4674 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... ----- Original Message ----- > Applied locally and tested. All's well. > > Thanks Frank. > Thanks guys. Ken I've also completed the initial cut at all of the remaining PCP::PMDA doc TODOs, if you could review that and one/two other small tidy-ups to your changes, that'd be great. cheers. -- Nathan From kenj@internode.on.net Mon Apr 7 22:34:21 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id D357D29E12 for ; Mon, 7 Apr 2014 22:34:21 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id C10898F8052 for ; Mon, 7 Apr 2014 20:34:18 -0700 (PDT) X-ASG-Debug-ID: 1396928053-04cb6c5678da6210001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id fDZWCWha5FbuuEVo for ; Mon, 07 Apr 2014 20:34:14 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgscAAxtQ1N20fAIPGdsb2JhbAANTINBiBq+GwMBAQEBOIMZQDANFhgDAgECATEnBgIBAa92oyEXjweEIgSuXg Received: from ppp118-209-240-8.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.8]) by ipmail06.adl2.internode.on.net with ESMTP; 08 Apr 2014 13:04:12 +0930 Message-ID: <53436E8C.1040801@internode.on.net> Date: Tue, 08 Apr 2014 13:35:40 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - minor Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - minor Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1396928053 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4675 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev src/libpcp/src/p_auth.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit bb9f98e7a0864c8b5d2cb49593f1de47c612f7b6 Author: Ken McDonell Date: Tue Apr 8 13:33:38 2014 +1000 libpcp/p_auth.c - fix minor compilation warning Arg to isprint() should be (int) not (char). Found on NetBSD. From minnus@buffalo.edu Tue Apr 8 14:53:38 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 8FB9E8057 for ; Tue, 8 Apr 2014 14:53:38 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 172CCAC013 for ; Tue, 8 Apr 2014 12:53:37 -0700 (PDT) X-ASG-Debug-ID: 1396986813-04cbb00dc40d1b0001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id 1eCH1NOrq7IVGu1h for ; Tue, 08 Apr 2014 12:53:33 -0700 (PDT) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 2FD6CAEA; Tue, 8 Apr 2014 15:53:33 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 2184811DCE; Tue, 8 Apr 2014 15:53:33 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id A921011AD6; Tue, 8 Apr 2014 15:52:24 -0400 (EDT) Received: from smtp.buffalo.edu (smtp4.acsu.buffalo.edu [128.205.5.229]) by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id 9404611AD1; Tue, 8 Apr 2014 15:52:24 -0400 (EDT) Received: from gilmour.ccr.buffalo.edu (gilmour.ccr.buffalo.edu [128.205.40.13]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 87B74A6AA; Tue, 8 Apr 2014 15:52:24 -0400 (EDT) Message-ID: <53445377.50301@buffalo.edu> Date: Tue, 08 Apr 2014 15:52:23 -0400 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Ken McDonell , pcp@oss.sgi.com CC: "Eigler, Frank" Subject: Re: pcp updates - slow start PMDAs References: <5341E094.5090304@internode.on.net> X-ASG-Orig-Subj: Re: pcp updates - slow start PMDAs In-Reply-To: <5341E094.5090304@internode.on.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1396986813 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4697 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Ken, On 4/6/14 7:17 PM, Ken McDonell wrote: > This commit will allow all PMDAs (C, Perl and Python) to get early access to pmdaConnect(). > > This should avoid most of the slow start issues where a PMDA is launched but takes some time to discover what metrics and/or instance domains are available ... if this process takes too long, pmcd will assume the initial credentials handshake has timed out and the PMDA will not be a happy camper thereafter. > > By giving the PMDA developer the option to connect to pmcd sooner, this problem is largely avoided. > > What remains is the slow PMDA issue where a subsequent PDU request from a PMAPI client is received from pmcd while the PMDA is "busy" (including slow start) ... this requires much more serious surgery to fix. > > If you've been seeing PMDAs not starting (perhaps sometimes, not all times), this change should help ... feedback and confirmation of this hypothesis would be most valuable. > Based on some initial testing, my slow start case in perl appears to be fixed with this change, thanks very much for fixing this! I'm not sure yet if I will hit the other problem you mention. I will have to do some more testing. Thanks again. Martins From kenj@internode.on.net Tue Apr 8 15:36:21 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 05A2980BE for ; Tue, 8 Apr 2014 15:36:21 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id E6F0C304062 for ; Tue, 8 Apr 2014 13:36:17 -0700 (PDT) X-ASG-Debug-ID: 1396989372-04cb6c77b50dcd0001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id ZW4j8txPDVopVjmR for ; Tue, 08 Apr 2014 13:36:13 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlMeAENdRFN20fC0PGdsb2JhbAANTA6LTrlpgw6BOAMBAQEBOIJaAQEBBEcxARALDgoJFg8JAwIBAgExFAYBDAEHAQGveoUInVgXjmwHhDgBA64NUg Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail05.adl6.internode.on.net with ESMTP; 09 Apr 2014 06:06:11 +0930 Message-ID: <53445E15.3000001@internode.on.net> Date: Wed, 09 Apr 2014 06:37:41 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Martins Innus , pcp@oss.sgi.com CC: "Eigler, Frank" Subject: Re: pcp updates - slow start PMDAs References: <5341E094.5090304@internode.on.net> <53445377.50301@buffalo.edu> X-ASG-Orig-Subj: Re: pcp updates - slow start PMDAs In-Reply-To: <53445377.50301@buffalo.edu> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396989372 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4697 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 09/04/14 05:52, Martins Innus wrote: > ... > Based on some initial testing, my slow start case in perl appears to be > fixed with this change, thanks very much for fixing this! I'm not sure > yet if I will hit the other problem you mention. I will have to do some > more testing. Thanks Martins. This may help any diagnosis you're trying. Slow startup failure is reported in pmcd.log like this ... [Mon Apr 7 06:55:09] pmcd(5065) Warning: pduread: timeout (after 3.000 sec) while attempting to read 12 bytes out of 12 in HDR on fd=38 pmcd: error at initial PDU exchange with slow PMDA: Timeout waiting for a response from PMCD but a PMDA that does not respond to a subsequent request from pmcd in time produces this failure report in pmcd.log ... [Wed Apr 9 06:32:34] pmcd(4426) Warning: pduread: timeout (after 5.000 sec) while attempting to read 12 bytes out of 12 in HDR on fd=28 [Wed Apr 9 06:32:34] pmcd(4426) Info: CleanupAgent ... Cleanup "slow" agent (dom 243): protocol failure for fd=28 The pduread diagnostic is similar (although the 3.000 sec vs 5.000 sec is significant, assuming you've not changed pmcd's default timeout parameters via pmcd.options), but the next line shows the different signatures. From scox@redhat.com Tue Apr 8 16:35:07 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 000D529E28 for ; Tue, 8 Apr 2014 16:35:06 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id BE30A8F8052 for ; Tue, 8 Apr 2014 14:35:03 -0700 (PDT) X-ASG-Debug-ID: 1396992899-04bdf07dca10b90001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id MvSvjgUySlkQucso for ; Tue, 08 Apr 2014 14:34:59 -0700 (PDT) X-Barracuda-Envelope-From: scox@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s38LYwsC031090 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 8 Apr 2014 17:34:58 -0400 Received: from [10.13.129.29] (dhcp129-29.rdu.redhat.com [10.13.129.29]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s38LYvQO018181; Tue, 8 Apr 2014 17:34:57 -0400 Message-ID: <53446B81.4020602@redhat.com> Date: Tue, 08 Apr 2014 17:34:57 -0400 From: Stan Cox User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Tom Yearke , pcp@oss.sgi.com Subject: Re: [pcp] Patch for Python's pmExtractValue Function References: <53342449.7070602@buffalo.edu> X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function In-Reply-To: <53342449.7070602@buffalo.edu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1396992899 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 I am not absolutely certain about the Python 2.6+ availability requirement so I think this will do things in a similar 2.5 supported way. Does this work for you? diff --git a/src/python/pcp/pmapi.py b/src/python/pcp/pmapi.py index 3324187..5d37217 100644 --- a/src/python/pcp/pmapi.py +++ b/src/python/pcp/pmapi.py @@ -1622,6 +1622,15 @@ class pmContext(object): byref(outAtom), outtype) if status < 0: raise pmErr, status + + if outtype == c_api.PM_TYPE_STRING: + # Get address of C string + c_str = outAtom.vp + # Convert to a python string and have result point to it + python_char_array = ctypes.string_at(c_str) + outAtom.cp = cast(python_char_array, c_char_p) + # Free the C string + LIBC.free(c_str) return outAtom @staticmethod From kenj@internode.on.net Tue Apr 8 16:43:31 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D154F804D for ; Tue, 8 Apr 2014 16:43:31 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7502AAC013 for ; Tue, 8 Apr 2014 14:43:28 -0700 (PDT) X-ASG-Debug-ID: 1396993406-04bdf07dc811280001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id ClSA9Fc3Mb380sp5 for ; Tue, 08 Apr 2014 14:43:26 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AmEeAIdsRFN20fC0PGdsb2JhbAANTIcjhDm5bIMECoE4AwEBAQE4gloBAQEEIxVAAQwECxgCAgUWCwICCQMCAQIBMRQGAQwBBwEBr3x2oXQXgSmNQweCb4FJAQOFNJ92iTU Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail05.adl6.internode.on.net with ESMTP; 09 Apr 2014 07:13:25 +0930 Message-ID: <53446DD5.3050109@internode.on.net> Date: Wed, 09 Apr 2014 07:44:53 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott , "Frank Ch. Eigler" CC: pcp@oss.sgi.com Subject: Re: [pcp] pcp updates - Frank's pmmgr exec fix References: <53432999.1030706@internode.on.net> <1960971043.1295159.1396923203345.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] pcp updates - Frank's pmmgr exec fix In-Reply-To: <1960971043.1295159.1396923203345.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396993406 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4699 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 08/04/14 12:13, Nathan Scott wrote: > > > ----- Original Message ----- >> Applied locally and tested. All's well. >> >> Thanks Frank. >> > > Thanks guys. Ken I've also completed the initial cut at all of > the remaining PCP::PMDA doc TODOs, if you could review that and > one/two other small tidy-ups to your changes, that'd be great. PCP::PMDA doc looks good ... I've made some minor changes and will push that later (soonish). Other changes are OK ... I was avoiding changing the copyright notices when the "new" stuff was actually copied from someplace else, so this was sort of intentional rather than an oversight. But the waters are so muddy here (PMDA Install/Remove/GNUmakefile) that it does not matter. From kenj@internode.on.net Tue Apr 8 16:45:30 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id A31F9804D for ; Tue, 8 Apr 2014 16:45:30 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 48D11AC013 for ; Tue, 8 Apr 2014 14:45:30 -0700 (PDT) X-ASG-Debug-ID: 1396993527-04cbb00dc612f80001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id 9ne27QACvjNTJ9fH for ; Tue, 08 Apr 2014 14:45:28 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkYeAIdsRFN20fC0PGdsb2JhbAANTINBiBu+MgMBAQEBOIMZQDANFhgDAgECATEOGQYCAQGvfKJqF48JhCIEmhGUTg Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail05.adl6.internode.on.net with ESMTP; 09 Apr 2014 07:15:26 +0930 Message-ID: <53446E51.8040901@internode.on.net> Date: Wed, 09 Apr 2014 07:46:57 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1396993527 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4699 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev src/perl/PMDA/PMDA.pm | 66 +++++++++++++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 19 deletions(-) commit 233b57971c4e63a64e8777984601af66c0aa4660 Author: Ken McDonell Date: Wed Apr 9 07:46:29 2014 +1000 Perl PMDA - tweak documentation From kenj@internode.on.net Wed Apr 9 01:06:35 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 6388D8140 for ; Wed, 9 Apr 2014 01:06:35 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 42126304067 for ; Tue, 8 Apr 2014 23:06:35 -0700 (PDT) X-ASG-Debug-ID: 1397023589-04cb6c77b525240001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id dEPiKDwy7iVfyqUB for ; Tue, 08 Apr 2014 23:06:30 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkcfAGniRFN20fC0PGdsb2JhbAANTA6LTrlngw6BMQMBAQEBOIJaAQEBBDhAARALDgoJFg8JAwIBAgExFAYNAQcBAbAHonwXjgFrB4Q4AQOjP4pPUg Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 09 Apr 2014 15:36:28 +0930 Message-ID: <5344E3BF.5060605@internode.on.net> Date: Wed, 09 Apr 2014 16:07:59 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Martins Innus CC: PCP Mailing List Subject: Re: [pcp] pmlogger_daily question/patch References: <530D0B02.6080208@buffalo.edu> X-ASG-Orig-Subj: Re: [pcp] pmlogger_daily question/patch In-Reply-To: <530D0B02.6080208@buffalo.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1397023590 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4710 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 26/02/14 08:28, Martins Innus wrote: > Hi, > > We use the attached patch on pmlogger_daily to prevent daily log > merging. We do this so we can use rsync throughout the day to backup > the logs and not take a big hit syncing the daily log once a day. Is > this something that would be acceptable or is there a better way to do it? Martins, I want to expand the -M option to mean, ... - do not rewrite with pmlogrewrite, and - do not merge with pmextract via pmlogger_merge, and - do not rename with pmlogmv [I just committed a change for this optimization] Does that match your intent/hope? It is easier to outlaw all 3 when -M is specififed, rather than just the merge case alone. And any of the above will mess with your rsync. Lemme know. From nscott@redhat.com Wed Apr 9 01:17:51 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 56F928036 for ; Wed, 9 Apr 2014 01:17:51 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 413EC8F8071 for ; Tue, 8 Apr 2014 23:17:51 -0700 (PDT) X-ASG-Debug-ID: 1397024266-04cbb00dc529340001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id FwAqO3Jyqk8W9hOu for ; Tue, 08 Apr 2014 23:17:46 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s396HgTu011039; Wed, 9 Apr 2014 02:17:42 -0400 Date: Wed, 9 Apr 2014 02:17:42 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp@oss.sgi.com Message-ID: <661012632.2277886.1397024262797.JavaMail.zimbra@redhat.com> In-Reply-To: <53446E51.8040901@internode.on.net> References: <53446E51.8040901@internode.on.net> Subject: Re: [pcp] pcp updates MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pcp updates Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates Thread-Index: 78vHyALlVAy+b53d8BxMUkrotq3JeA== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1397024266 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4710 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... > Changes committed to git://oss.sgi.com/kenj/pcp.git dev > > src/perl/PMDA/PMDA.pm | 66 > +++++++++++++++++++++++++++++++++++--------------- > 1 file changed, 47 insertions(+), 19 deletions(-) Thanks Ken, a good set of improvements :) (Heh - pmcd(3)? Hooboy, what was I thinking!) Also on that *cough* obtuse debugging mechanism, and lack of decent argument parsing capabilities in general - we could now do a vastly better job with pmdaGetOptions(1) now available. We could squirrel away parsing of the $ENV (perl) and sys.argv (python) elements entirely from the PMDAs - with no individual PMDA changes. This'll bring in all the usual options that a C PMDA has (-D, -d, etc), automatically. cheers. -- Nathan From nscott@redhat.com Wed Apr 9 01:24:13 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id F0BB38037 for ; Wed, 9 Apr 2014 01:24:12 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id C678C30404E for ; Tue, 8 Apr 2014 23:24:12 -0700 (PDT) X-ASG-Debug-ID: 1397024651-04cb6c77b725a30001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id u2od2wDQ2ZpkFQnC for ; Tue, 08 Apr 2014 23:24:11 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s396O72i011784 for ; Wed, 9 Apr 2014 02:24:10 -0400 Date: Wed, 9 Apr 2014 02:24:07 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp@oss.sgi.com Message-ID: <2118581152.2279569.1397024647922.JavaMail.zimbra@redhat.com> In-Reply-To: <1773429166.2279383.1397024573456.JavaMail.zimbra@redhat.com> Subject: pcp updates: kenj merges, qa MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: kenj merges, qa Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: kenj merges, qa Thread-Index: /iLJzDhswW9sC56Po02s0V51kCSGWQ== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1397024651 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4710 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev man/man1/pmconfig.1 | 13 qa/.gitignore | 2 qa/022 | 22 qa/022.linux.2 | 191 ------ qa/022.linux.3 | 193 ------ qa/022.linux.4 | 198 ------ qa/022.out.linux | 198 ++++++ qa/023 | 3 qa/051 | 19 qa/051.out.2 | 1015 ---------------------------------- qa/051.out.3 | 1016 ---------------------------------- qa/051.out.4 | 1005 ---------------------------------- qa/051.out.5 | 1006 ---------------------------------- qa/051.out.ipv6 | 1006 ++++++++++++++++++++++++++++++++++ qa/051.out.nonipv6 | 1005 ++++++++++++++++++++++++++++++++++ qa/062 | 7 qa/062.out.1 | 1197 ----------------------------------------- qa/062.out.2 | 1197 ----------------------------------------- qa/062.out.ipv6 | 1197 +++++++++++++++++++++++++++++++++++++++++ qa/062.out.nonipv6 | 1197 +++++++++++++++++++++++++++++++++++++++++ qa/066 | 23 qa/066.out.1 | 85 -- qa/066.out.2 | 86 -- qa/066.out.3 | 85 -- qa/066.out.4 | 85 -- qa/066.out.5 | 88 --- qa/066.out.ipv6 | 88 +++ qa/066.out.nonipv6 | 85 ++ qa/067 | 18 qa/067.out.1 | 36 - qa/067.out.2 | 37 - qa/067.out.3 | 36 - qa/067.out.4 | 37 - qa/067.out.ipv6 | 37 + qa/067.out.nonipv6 | 36 + qa/069 | 9 qa/069.out.ipv4 | 73 -- qa/069.out.nonipv6 | 73 ++ qa/172 | 9 qa/172.out.1 | 20 qa/172.out.2 | 22 qa/172.out.ipv6 | 22 qa/172.out.nonipv6 | 20 qa/243 | 24 qa/243.out.1 | 17 qa/243.out.2 | 21 qa/243.out.3 | 18 qa/243.out.4 | 18 qa/243.out.5 | 20 qa/243.out.ipv6 | 20 qa/243.out.nonipv6 | 18 qa/244 | 8 qa/244.out.1 | 109 --- qa/244.out.2 | 112 --- qa/244.out.ipv6 | 112 +++ qa/244.out.nonipv6 | 109 +++ qa/255 | 19 qa/255.out.1 | 279 --------- qa/255.out.2 | 280 --------- qa/255.out.3 | 277 --------- qa/255.out.4 | 267 --------- qa/255.out.5 | 268 --------- qa/255.out.ipv6 | 268 +++++++++ qa/255.out.nonipv6 | 267 +++++++++ qa/364 | 3 qa/365 | 32 - qa/365.out.1 | 40 - qa/365.out.2 | 41 - qa/365.out.3 | 40 - qa/365.out.4 | 40 - qa/365.out.5 | 41 - qa/365.out.ipv6 | 41 + qa/365.out.nonipv6 | 40 + qa/390 | 5 qa/449 | 15 qa/449.out | 93 +++ qa/449.out.1 | 93 --- qa/449.out.2 | 93 --- qa/451 | 5 qa/470 | 5 qa/472 | 5 qa/473 | 5 qa/474 | 5 qa/475 | 5 qa/533 | 4 qa/546 | 3 qa/580 | 3 qa/724 | 16 qa/740 | 4 qa/749 | 9 qa/775 | 16 qa/823 | 3 qa/831 | 4 qa/832 | 3 qa/840 | 9 qa/943 | 11 qa/943.out | 175 +++++ qa/943.out.1 | 171 ----- qa/943.out.2 | 175 ----- qa/944 | 3 qa/946 | 20 qa/common.check | 7 qa/common.secure | 3 src/libpcp/src/p_auth.c | 4 src/perl/PMDA/PMDA.pm | 66 +- src/pmconfig/pmconfig.c | 90 +-- src/pmdas/linux/proc_cpuinfo.c | 7 107 files changed, 6330 insertions(+), 10451 deletions(-) commit cb86aa3f14febd52bdd5ab9a52b07d0534eeaa3a Merge: 7c3f89a 233b579 Author: Nathan Scott Date: Wed Apr 9 16:02:35 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit 7c3f89afe2254d54f2338ec75ca3fcc30dc0d682 Author: Nathan Scott Date: Wed Apr 9 16:01:56 2014 +1000 Sufficient quoting for pmconfig library output shell scripting Make the -L option more like the non-L mode when reporting with the -s option, suitable for use in shell scripts. Opportunity arose to cleanup a great number of shell scripts, making use of pmconfig but also having pcp-version-dependent complexities. In all cases test simplification resulted, and readability was noticably enhanced - win, win. Resolves Fedora BZ #1085401. commit 233b57971c4e63a64e8777984601af66c0aa4660 Author: Ken McDonell Date: Wed Apr 9 07:46:29 2014 +1000 Perl PMDA - tweak documentation commit be49fd8b8cc5239893158ebbe850cd41b32b26e8 Merge: d1bfabb bb9f98e Author: Nathan Scott Date: Tue Apr 8 14:23:07 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit d1bfabb229c33ce26f40a6b3a103bdba1bd24074 Author: Nathan Scott Date: Tue Apr 8 14:22:42 2014 +1000 Resolve a pmdalinux memory leak in cpu:node resolution The map_cpu_nodes function was being called on each and every refresh_proc_cpuinfo call unintentionally. This resulted in the it_set pointer being overwritten with a new calloc chunk each time, leaking its earlier memory. commit bb9f98e7a0864c8b5d2cb49593f1de47c612f7b6 Author: Ken McDonell Date: Tue Apr 8 13:33:38 2014 +1000 libpcp/p_auth.c - fix minor compilation warning Arg to isprint() should be (int) not (char). Found on NetBSD. From kenj@internode.on.net Wed Apr 9 01:54:01 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 972308037 for ; Wed, 9 Apr 2014 01:54:01 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 5BAD08F804B for ; Tue, 8 Apr 2014 23:54:01 -0700 (PDT) X-ASG-Debug-ID: 1397026438-04bdf07dc7266b0001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id dQLF8C5VzGmZFS7j for ; Tue, 08 Apr 2014 23:53:59 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 09 Apr 2014 16:23:31 +0930 Message-ID: <5344EEC2.60202@internode.on.net> Date: Wed, 09 Apr 2014 16:54:58 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott CC: pcp@oss.sgi.com Subject: Re: [pcp] pcp updates References: <53446E51.8040901@internode.on.net> <661012632.2277886.1397024262797.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] pcp updates In-Reply-To: <661012632.2277886.1397024262797.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1397026439 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4711 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 09/04/14 16:17, Nathan Scott wrote: > >> Changes committed to git://oss.sgi.com/kenj/pcp.git dev >> >> src/perl/PMDA/PMDA.pm | 66 > ... > Also on that *cough* obtuse debugging mechanism, and lack of > decent argument parsing capabilities in general - we could now > do a vastly better job with pmdaGetOptions(1) now available. > We could squirrel away parsing of the $ENV (perl) and sys.argv > (python) elements entirely from the PMDAs - with no individual > PMDA changes. This'll bring in all the usual options that a C > PMDA has (-D, -d, etc), automatically. Indeed. But that is way beyond my plodder level Perl skills ... obtuse and old world works for me ... 8^)> From nscott@redhat.com Wed Apr 9 02:02:15 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id CFE568037 for ; Wed, 9 Apr 2014 02:02:15 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id BB931304059 for ; Wed, 9 Apr 2014 00:02:12 -0700 (PDT) X-ASG-Debug-ID: 1397026928-04cbb00dc32aee0001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id 6ytECoHZoR7tVVKj for ; Wed, 09 Apr 2014 00:02:09 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s39728uP002163 for ; Wed, 9 Apr 2014 03:02:08 -0400 Date: Wed, 9 Apr 2014 03:02:08 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <1347416557.2288818.1397026928638.JavaMail.zimbra@redhat.com> Subject: pcp updates: fche merge, URLs MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: fche merge, URLs Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: fche merge, URLs Thread-Index: alJCnQRnLVzoGIaanoNGzkMt2HGajg== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1397026929 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4711 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev CHANGELOG | 2 HACKING | 6 build/aix/pcp.template.in | 2 build/mac/installer-resources/ReadMe.html | 4 build/rpm/fedora.spec | 34 - build/rpm/pcp.spec.in | 36 - build/sun/GNUmakefile | 2 configure | 6 configure.in | 6 debian/.gitignore | 1 debian/control | 377 +++++++++++++++++++ debian/control.master | 4 debian/copyright | 12 man/man1/pcpintro.1 | 4 pcp.lsm.in | 8 src/libpcp/src/check-statics | 583 +++++++++++++++--------------- 16 files changed, 737 insertions(+), 350 deletions(-) commit 0825f3070126c9f0dae1c3ba75ef275708b8abaf Author: Nathan Scott Date: Wed Apr 9 16:49:29 2014 +1000 Begin URL transition over to performancecopilot.org names Focus on docs and build/packaging at this early stage. Noticed the debian/control file was not being included as it was meant to have been, due to .gitignore getting in the way (earlier Makepkgs changes mandated this), so this is fixed up in the process. commit 2156e8232c70610406f1fa43cf0333bc152801b4 Merge: cb86aa3 842b42a Author: Nathan Scott Date: Wed Apr 9 16:30:14 2014 +1000 Merge branch 'fche/for-merge' of ../pcpfans into dev commit 842b42ad7d94ac01f421c4c9776f05e82ceec77c Author: Frank Ch. Eigler Date: Wed Mar 19 23:10:09 2014 -0400 libpcp/src/check-statics: simplify, pessify check-statics has been in some cases overly and in others underly sensitive. In the overly sensitive category, it's too sensitive to compilers' choice of section placement and for static data and suffixing due to function-static / optimization. It seems safe to accept the symbols with any of the usual prefixes and suffixes. In the underly sensitive category, check-statics could exit with a success rc in a variety of conditions, such as being interrupted, or a failure of an inferior command, or even a script syntax error. Let's flip the $sts default around and add a 'set -e' to become very conservative with respect to environmental catastrophes. kenj corrected/tested on mac/*bsd/suse/debian nathans corrected/tested on solaris fche retested on plain old linux From minnus@buffalo.edu Wed Apr 9 07:54:04 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 063CB8112 for ; Wed, 9 Apr 2014 07:54:04 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id DAA9F8F8068 for ; Wed, 9 Apr 2014 05:54:03 -0700 (PDT) X-ASG-Debug-ID: 1397048039-04bdf07dcc3c170001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id cW8ShzORIDBirm3x for ; Wed, 09 Apr 2014 05:53:59 -0700 (PDT) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailB.acsu.buffalo.edu (localmailb.acsu.buffalo.edu [128.205.5.200]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id F1C7C1745; Wed, 9 Apr 2014 08:53:58 -0400 (EDT) Received: from localmailB.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id EDAEB10BA4; Wed, 9 Apr 2014 08:53:58 -0400 (EDT) Received: from localmailB.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailB.acsu.buffalo.edu (Postfix) with ESMTP id 7BDE610BA1; Wed, 9 Apr 2014 08:53:58 -0400 (EDT) Received: from smtp.buffalo.edu (smtp1.acsu.buffalo.edu [128.205.5.253]) by localmailB.acsu.buffalo.edu (Prefixe) with ESMTP id 6EDA410B9D; Wed, 9 Apr 2014 08:53:58 -0400 (EDT) Received: from gilmour.ccr.buffalo.edu (gilmour.ccr.buffalo.edu [128.205.40.13]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 62DC29664; Wed, 9 Apr 2014 08:53:58 -0400 (EDT) Message-ID: <534542E5.1020308@buffalo.edu> Date: Wed, 09 Apr 2014 08:53:57 -0400 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Ken McDonell CC: PCP Mailing List Subject: Re: [pcp] pmlogger_daily question/patch References: <530D0B02.6080208@buffalo.edu> <5344E3BF.5060605@internode.on.net> X-ASG-Orig-Subj: Re: [pcp] pmlogger_daily question/patch In-Reply-To: <5344E3BF.5060605@internode.on.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397048039 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4716 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 4/9/14 2:07 AM, Ken McDonell wrote: > On 26/02/14 08:28, Martins Innus wrote: >> Hi, >> >> We use the attached patch on pmlogger_daily to prevent daily log >> merging. We do this so we can use rsync throughout the day to backup >> the logs and not take a big hit syncing the daily log once a day. Is >> this something that would be acceptable or is there a better way to >> do it? > > Martins, > > I want to expand the -M option to mean, ... > - do not rewrite with pmlogrewrite, and > - do not merge with pmextract via pmlogger_merge, and > - do not rename with pmlogmv [I just committed a change for this > optimization] > > Does that match your intent/hope? > > It is easier to outlaw all 3 when -M is specififed, rather than just > the merge case alone. And any of the above will mess with your rsync. > > Lemme know. Ken, That is perfect, thanks. Martins From kenj@internode.on.net Wed Apr 9 15:24:41 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 7B7237F5F for ; Wed, 9 Apr 2014 15:24:41 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 41464304059 for ; Wed, 9 Apr 2014 13:24:37 -0700 (PDT) X-ASG-Debug-ID: 1397075071-04bdf07dc865860001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id hHPmVUNNQsbGlRky for ; Wed, 09 Apr 2014 13:24:32 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAKyrRVN20fC0/2dsb2JhbAANTINBxh6EGDANFhgDAgECAVgGAgEBsWSjSxeJRYVEhCIEmhOJLosh Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 10 Apr 2014 05:54:31 +0930 Message-ID: <5345ACCF.5050805@internode.on.net> Date: Thu, 10 Apr 2014 06:25:51 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - pmlogger_daily improvements Content-Type: text/plain; charset=ISO-8859-1 X-ASG-Orig-Subj: pcp updates - pmlogger_daily improvements Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397075072 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4732 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Knocks off a couple more items from the last PCP developer's meeting. Changes committed to git://oss.sgi.com/kenj/pcp.git dev HACKING | 28 ++++ man/man1/pmdumplog.1 | 2 man/man1/pmlogcheck.1 | 2 man/man1/pmlogger_check.1 | 231 +++++++++++++++++++++++------------------ man/man1/pmlogmv.1 | 6 - qa/738 | 20 +-- qa/738.out | 4 qa/929 | 81 +++++++++++++- qa/929.out | 37 ++++++ qa/group | 1 src/pmlogger/pmlogger_daily.sh | 192 ++++++++++++++++++++-------------- src/pmlogger/pmlogmv.sh | 8 - 12 files changed, 405 insertions(+), 207 deletions(-) commit e70efd23bfe539306109c401765ab09a600322d2 Author: Ken McDonell Date: Thu Apr 10 06:20:27 2014 +1000 pmlogger_check - add -M option With -M there is no merging, renaming or rewriting of archives. This helps scenarios where rsync (or similar) is being used to incrementally copy the archives to some remote repository. Clean up the man page ... document -M, editorial changes and moving some text around to improve readability. Extend qa/929 to check -M as well as the single archive, so rename don't merge optimization. commit 086e473d0642dfdc97e78b692bb09a09b962971b Author: Ken McDonell Date: Wed Apr 9 14:34:23 2014 +1000 pmlogger_daily - don't merge if not needed [optimization] In the (quite common) case of pmlogger_daily finding one archive that needs to be "merged", use pmlogmv(1) to rename instead of pmlogger_merge(1) (which uses pmlogextract(1)) to copy. commit e2c8242e5abbfcc3fc79613ce762de178bf27e2e Author: Ken McDonell Date: Wed Apr 9 12:13:30 2014 +1000 pmlogmv - change args from -nv to -NV Better match with pmlogger_daily, especially since pmlogger_daily will (soon) be optionally passing -N and -V down to pmlogmv. commit 43022648e059b5d241502cb3f7745bd760f5d8a2 Author: Ken McDonell Date: Wed Apr 9 09:06:53 2014 +1000 HACKING - add some futher explanantions and verbage From kenj@internode.on.net Wed Apr 9 20:48:17 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 61C977F99 for ; Wed, 9 Apr 2014 20:48:17 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id DB2CEAC002 for ; Wed, 9 Apr 2014 18:48:13 -0700 (PDT) X-ASG-Debug-ID: 1397094487-04bdf07dca726d0001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id yRQ9Qa4lDTrd2zsh for ; Wed, 09 Apr 2014 18:48:08 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAP32RVN20fC0/2dsb2JhbAANTINBxiGDWEAwDRYYAwIBAgFYBgIBAbAYo34XjgiBAYQiBK5igVc Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 10 Apr 2014 11:18:07 +0930 Message-ID: <5345F8A7.2090608@internode.on.net> Date: Thu, 10 Apr 2014 11:49:27 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - qa nit (triggered 24 failures for me) Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - qa nit (triggered 24 failures for me) Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397094487 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4742 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/common.check | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit ecfa750e321149dac484cea68fb9a34029737f5a Author: Ken McDonell Date: Thu Apr 10 11:47:40 2014 +1000 qa/common.check - "source" is not a sh keyword I know you really meant "." not "source". From wwwrun@oss.sgi.com Thu Apr 10 12:41:38 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: by oss.sgi.com (Postfix, from userid 30) id CBDF67F94; Thu, 10 Apr 2014 12:41:38 -0500 (CDT) From: bugzilla-daemon@oss.sgi.com To: pcp@oss.sgi.com Subject: [Bug 1050] New: pmdalogger docs lack config file Date: Thu, 10 Apr 2014 17:41:38 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Classification: Unclassified X-Bugzilla-Product: pcp X-Bugzilla-Component: pcp X-Bugzilla-Keywords: X-Bugzilla-Severity: major X-Bugzilla-Who: fche@redhat.com X-Bugzilla-Status: NEW X-Bugzilla-Priority: P5 X-Bugzilla-Assigned-To: pcp@kenj.com.au X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter cc classification Message-ID: Content-Type: multipart/alternative; boundary="1397151698.fcD821.842"; charset="us-ascii" X-Bugzilla-URL: http://oss.sgi.com/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 --1397151698.fcD821.842 Date: Thu, 10 Apr 2014 12:41:38 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" http://oss.sgi.com/bugzilla/show_bug.cgi?id=1050 Bug ID: 1050 Summary: pmdalogger docs lack config file Product: pcp Version: unspecified Hardware: All OS: Linux Status: NEW Severity: major Priority: P5 Component: pcp Assignee: pcp@kenj.com.au Reporter: fche@redhat.com CC: pcp@oss.sgi.com Classification: Unclassified The pmdalogger man page contains the usual pmda boilerplate info, but apprx. nothing about what makes this pmda useful: how it works, how to configure it, how to use it. There is no sample config file, nor a reference to the Install script that can make one. -- You are receiving this mail because: You are on the CC list for the bug. --1397151698.fcD821.842 Date: Thu, 10 Apr 2014 12:41:38 -0500 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8"
Bug ID 1050
Summary pmdalogger docs lack config file
Product pcp
Version unspecified
Hardware All
OS Linux
Status NEW
Severity major
Priority P5
Component pcp
Assignee pcp@kenj.com.au
Reporter fche@redhat.com
CC pcp@oss.sgi.com
Classification Unclassified 

The pmdalogger man page contains the usual pmda boilerplate info,
but apprx. nothing about what makes this pmda useful: how it works,
how to configure it, how to use it.  There is no sample config file,
nor a reference to the Install script that can make one.

You are receiving this mail because:
  • You are on the CC list for the bug.
--1397151698.fcD821.842-- From wwwrun@oss.sgi.com Thu Apr 10 12:45:52 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: by oss.sgi.com (Postfix, from userid 30) id 022717F94; Thu, 10 Apr 2014 12:45:52 -0500 (CDT) From: bugzilla-daemon@oss.sgi.com To: pcp@oss.sgi.com Subject: [Bug 1051] New: pmdalogger should have an tail -F (vs tail -f) mode Date: Thu, 10 Apr 2014 17:45:51 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Classification: Unclassified X-Bugzilla-Product: pcp X-Bugzilla-Component: pcp X-Bugzilla-Keywords: X-Bugzilla-Severity: major X-Bugzilla-Who: fche@redhat.com X-Bugzilla-Status: NEW X-Bugzilla-Priority: P5 X-Bugzilla-Assigned-To: pcp@kenj.com.au X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter cc classification Message-ID: Content-Type: multipart/alternative; boundary="1397151951.74FcF1.1159"; charset="us-ascii" X-Bugzilla-URL: http://oss.sgi.com/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 --1397151951.74FcF1.1159 Date: Thu, 10 Apr 2014 12:45:51 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" http://oss.sgi.com/bugzilla/show_bug.cgi?id=1051 Bug ID: 1051 Summary: pmdalogger should have an tail -F (vs tail -f) mode Product: pcp Version: unspecified Hardware: All OS: Linux Status: NEW Severity: major Priority: P5 Component: pcp Assignee: pcp@kenj.com.au Reporter: fche@redhat.com CC: pcp@oss.sgi.com Classification: Unclassified In the presence of logfile rotation, simply hanging onto an original file descriptor and reading from the end, is not good enough. If the file disappears and reappears, it should be reopened anew. Similarly, if someone truncates the file (so it becomes empty again), we should lseek to the beginning and go from there. -- You are receiving this mail because: You are on the CC list for the bug. --1397151951.74FcF1.1159 Date: Thu, 10 Apr 2014 12:45:51 -0500 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8"
Bug ID 1051
Summary pmdalogger should have an tail -F (vs tail -f) mode
Product pcp
Version unspecified
Hardware All
OS Linux
Status NEW
Severity major
Priority P5
Component pcp
Assignee pcp@kenj.com.au
Reporter fche@redhat.com
CC pcp@oss.sgi.com
Classification Unclassified 

In the presence of logfile rotation, simply hanging onto an original
file descriptor and reading from the end, is not good enough.  If the
file disappears and reappears, it should be reopened anew.  Similarly,
if someone truncates the file (so it becomes empty again), we should
lseek to the beginning and go from there.

You are receiving this mail because:
  • You are on the CC list for the bug.
--1397151951.74FcF1.1159-- From kenj@internode.on.net Thu Apr 10 17:32:19 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 4B1C57F91 for ; Thu, 10 Apr 2014 17:32:19 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 3474B304053 for ; Thu, 10 Apr 2014 15:32:19 -0700 (PDT) X-ASG-Debug-ID: 1397169133-04cbb00dc4b9de0001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id 6bjloMri0sycoGzB for ; Thu, 10 Apr 2014 15:32:13 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqkdAP0aR1N20fC0PGdsb2JhbAANTYNBiB6+IwMBAQEBOIMZQDANFhgDAgECATEnBgIBAbEKo1sXjwmEIgSuYg Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 11 Apr 2014 08:02:07 +0930 Message-ID: <53471C45.8000009@internode.on.net> Date: Fri, 11 Apr 2014 08:33:41 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - qa and lib changes to address valgrind issues Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - qa and lib changes to address valgrind issues Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1397169133 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4768 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/common.check | 2 +- src/libpcp/src/pmns.c | 16 ++++++++++++---- src/libpcp_pmda/src/tree.c | 16 +++++++++++++--- src/pmdas/linux/proc_net_dev.c | 12 +++++++++++- 4 files changed, 37 insertions(+), 9 deletions(-) commit 485860a0fb6f693bbc3f49fb5fdad251bc1907aa Author: Ken McDonell Date: Fri Apr 11 08:30:55 2014 +1000 libpcp_pmda - pmdaTreeChildren tweak for dynamic metrics Tighten the guard to prevent a couple of malloc(0) cases that were causing qa/957 fallout. commit 9b752b675678412cd1c77ba3738ef4df35c67b1c Author: Ken McDonell Date: Fri Apr 11 08:27:27 2014 +1000 libpcp - pmns traversal fixups Fixed a couple of zero-sized malloc() instances in the PM_CONTEXT_LOCAL case expanding the PMNS below a dynamic metric. Tightening the guards prevents the pointless malloc()s. This was causing some qa/957 fallout. commit 0ef492a9287f046542e9f61345af32313bf10f98 Author: Ken McDonell Date: Fri Apr 11 06:17:58 2014 +1000 linux PMDA proc_net_dev.c - small initialization change for valgrind WAR Comment from the code ... * Note: * Initialization of ecmd is not really needed. If the ioctl()s * work, ecmd is filled in ... but valgrind (at least up to * version 3.9.0) does not know about the SIOCETHTOOL ioctl() * and thinks the use of ecmd after this call propagates * uninitialized data in to ioc.speed and ioc.duplex, causing * failures for qa/957 commit ecfa750e321149dac484cea68fb9a34029737f5a Author: Ken McDonell Date: Thu Apr 10 11:47:40 2014 +1000 qa/common.check - "source" is not a sh keyword I know you really meant "." not "source". From kenj@internode.on.net Thu Apr 10 18:07:16 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id E3EC57F93 for ; Thu, 10 Apr 2014 18:07:15 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 6E85CAC002 for ; Thu, 10 Apr 2014 16:07:12 -0700 (PDT) X-ASG-Debug-ID: 1397171229-04cb6c77b6b9670001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id Z5l8LuiAxAZ4dzak for ; Thu, 10 Apr 2014 16:07:10 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqkdACwjR1N20fC0PGdsb2JhbAANTYNBiB6+IwMBAQEBOIMZQDANFhgDAgECATEnBgIBAbEKo1YXjwmEIgSuYg Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 11 Apr 2014 08:37:09 +0930 Message-ID: <5347247B.2050603@internode.on.net> Date: Fri, 11 Apr 2014 09:08:43 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - fix small libpcp memleak Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - fix small libpcp memleak Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1397171229 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4769 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev src/libpcp/src/pmns.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) commit e6dce1ef8d358d5f6fa2b697d9b2b64bcbe65a61 Author: Ken McDonell Date: Fri Apr 11 09:03:56 2014 +1000 libpcp pmns - fix obscure memory leak To see this you'd need to be using PM_CONTEXT_LOCAL for a DSO PMDA with dynamic metrics and then call pmGetChildren() or pmGetChildrenStatus() and the metric at the root of the search needs to be a non-leaf with more than zero children. This corner case is exposed by qa/957. From kenj@internode.on.net Thu Apr 10 21:19:36 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 69AF37F62 for ; Thu, 10 Apr 2014 21:19:36 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id EC2F4AC00D for ; Thu, 10 Apr 2014 19:19:35 -0700 (PDT) X-ASG-Debug-ID: 1397182769-04cbb00dc5c5960001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id Fj932hEbRYQGQLl5 for ; Thu, 10 Apr 2014 19:19:30 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAPNPR1N20fC0/2dsb2JhbAANTYNBxkKDWEAwDRYYAwIBAgFYBgIBAbE8o08XjgiBAYQiBK5igVc Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 11 Apr 2014 11:49:28 +0930 Message-ID: <53475183.3080007@internode.on.net> Date: Fri, 11 Apr 2014 12:20:51 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - numa archive Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - numa archive Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397182769 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4775 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/src/GNUlocaldefs | 2 +- qa/src/mknuma | 37 +++++++++++++++++++++++++++++++++++++ qa/src/numa.0 |binary qa/src/numa.index |binary qa/src/numa.meta |binary 5 files changed, 38 insertions(+), 1 deletion(-) commit c19913150ab58848b2c9e20491daf632f122716d Author: Ken McDonell Date: Fri Apr 11 12:14:41 2014 +1000 qa - numa archive From kenj@internode.on.net Thu Apr 10 23:42:01 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 88DD97F72 for ; Thu, 10 Apr 2014 23:42:01 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 27BE0AC002 for ; Thu, 10 Apr 2014 21:42:00 -0700 (PDT) X-ASG-Debug-ID: 1397191315-04bdf07dc8cb750001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id yYVi0wfANNu1LrDW for ; Thu, 10 Apr 2014 21:41:56 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBADpyR1N20fC0/2dsb2JhbAANTINBxkeDWEAwDRYYAwIBAgFYBgIBAbEro08XjwmEIgSuYg Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 11 Apr 2014 14:11:34 +0930 Message-ID: <534772D1.2060109@internode.on.net> Date: Fri, 11 Apr 2014 14:42:57 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates - qa/929 Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates - qa/929 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397191315 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4777 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/929 | 11 +++++++++++ 1 file changed, 11 insertions(+) commit c2377a0167d0f11283a2bc297dde2878c84163e0 Author: Ken McDonell Date: Fri Apr 11 14:39:34 2014 +1000 qa/929 - stronger filter Juggling with the NOTICES file outside the scope of the test and outside our control (at least easily), so filter out these optional lines. From nscott@redhat.com Fri Apr 11 02:12:56 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 7CAB67F7C for ; Fri, 11 Apr 2014 02:12:56 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 097BEAC002 for ; Fri, 11 Apr 2014 00:12:55 -0700 (PDT) X-ASG-Debug-ID: 1397200369-04bdf07dc8d14c0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id 9MrnmswSopgapVV9 for ; Fri, 11 Apr 2014 00:12:49 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3B7CjHi016435 for ; Fri, 11 Apr 2014 03:12:45 -0400 Date: Fri, 11 Apr 2014 03:12:45 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <1429874976.3777376.1397200365901.JavaMail.zimbra@redhat.com> In-Reply-To: <1821183812.3777009.1397200271225.JavaMail.zimbra@redhat.com> Subject: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa Thread-Index: 6WwY9HytCe+7XPTGoOtG3MpW2ONKLg== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397200369 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4781 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev man/man1/pcp.1 | 105 ++++++++++++++++++--- qa/366.out | 6 + qa/518 | 4 qa/742 | 6 - qa/743 | 35 +++++++ qa/743.out | 54 ++++++++++ qa/900 | 102 ++++++++++++++++++++ qa/900.out | 63 ++++++++++++ qa/929 | 11 ++ qa/991 | 22 ++-- qa/991.out | 18 +-- qa/common.check | 2 qa/group | 2 qa/src/GNUlocaldefs | 4 qa/src/mknuma | 39 +++++++ qa/src/numa.0 |binary qa/src/numa.index |binary qa/src/numa.meta |binary qa/src/pcp-numastat-1-node.0 |binary qa/src/pcp-numastat-1-node.index |binary qa/src/pcp-numastat-1-node.meta |binary src/libpcp/src/getopt.c | 5 - src/libpcp/src/pmns.c | 26 ++++- src/libpcp_pmda/src/open.c | 34 +++++- src/libpcp_pmda/src/tree.c | 16 ++- src/pcp/GNUmakefile | 2 src/pcp/free/pcp-free.1 | 116 ++++++----------------- src/pcp/free/pcp-free.py | 18 --- src/pcp/numastat/.gitignore | 1 src/pcp/numastat/GNUmakefile | 34 ++++++ src/pcp/numastat/pcp-numastat.1 | 57 +++++++++++ src/pcp/numastat/pcp-numastat.py | 157 +++++++++++++++++++++++++++++++ src/pcp/pcp.sh | 194 +++++++++++++++++++++++++++++---------- src/pcp/uptime/pcp-uptime.1 | 95 +++---------------- src/pcp/uptime/pcp-uptime.py | 12 -- src/pmconfig/pmconfig.c | 24 ++++ src/pmdas/linux/proc_net_dev.c | 12 ++ src/pmlogconf/tools/localdefs | 23 ++++ src/pmlogconf/tools/numastat | 4 src/python/pcp/pmapi.py | 12 +- 40 files changed, 1013 insertions(+), 302 deletions(-) commit a797ed2ac8c1cd63150baec13a5be3c0beee931a Author: Nathan Scott Date: Fri Apr 11 16:27:20 2014 +1000 Fix memory leak reiniting PMDA hash table for dynamic metrics Test qa/957 picked up on a memory leak in the handling of the metric table hash lookup optimization, where second/subsequent calls to pmdaRehash would rewrite the hash table, accidentally leaking any entries written to it earlier. commit 2ad345724188fe3ed925e66b5866ac27cf4d64af Author: Nathan Scott Date: Fri Apr 11 15:52:00 2014 +1000 Quote characters special to the shell in pmconfig(1) output commit 7b594dda682fb83fa211220332035b6fc3e77454 Merge: 42145e6 c2377a0 Author: Nathan Scott Date: Fri Apr 11 15:48:49 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit 42145e672b91bfb8a4cef29259640d9aa7493b4a Author: Nathan Scott Date: Fri Apr 11 15:48:38 2014 +1000 Support for pcp(1) running system performance scripts Add in support for pcp(1) to be used to invoke system tools as scripts (like the existing pcp-free, pcp-uptime scripts) with automatic argument passing/parsing. So, no support is needed in the scripts, further simplifying them. Man page updates and qa/900 is added to exercise this. The -p/-P option switch in pcp(1) is to facilitate standard option usage across all tools (the pmie stats are so rarely used this is not expected to be problem in practice). In addition, a pcp-numastat(1) script is added, along with pmlogconf spec, man page, qa/743 and initial archives that exercise it (thanks Ken). With this third script addition, the python options support is looking fairly solid. The last little tweak made here is to automatically enable the archive boundary options, in the library, as it appears all scripts will want to use this by default. commit c2377a0167d0f11283a2bc297dde2878c84163e0 Author: Ken McDonell Date: Fri Apr 11 14:39:34 2014 +1000 qa/929 - stronger filter Juggling with the NOTICES file outside the scope of the test and outside our control (at least easily), so filter out these optional lines. commit c19913150ab58848b2c9e20491daf632f122716d Author: Ken McDonell Date: Fri Apr 11 12:14:41 2014 +1000 qa - numa archive commit e6dce1ef8d358d5f6fa2b697d9b2b64bcbe65a61 Author: Ken McDonell Date: Fri Apr 11 09:03:56 2014 +1000 libpcp pmns - fix obscure memory leak To see this you'd need to be using PM_CONTEXT_LOCAL for a DSO PMDA with dynamic metrics and then call pmGetChildren() or pmGetChildrenStatus() and the metric at the root of the search needs to be a non-leaf with more than zero children. This corner case is exposed by qa/957. commit 485860a0fb6f693bbc3f49fb5fdad251bc1907aa Author: Ken McDonell Date: Fri Apr 11 08:30:55 2014 +1000 libpcp_pmda - pmdaTreeChildren tweak for dynamic metrics Tighten the guard to prevent a couple of malloc(0) cases that were causing qa/957 fallout. commit 9b752b675678412cd1c77ba3738ef4df35c67b1c Author: Ken McDonell Date: Fri Apr 11 08:27:27 2014 +1000 libpcp - pmns traversal fixups Fixed a couple of zero-sized malloc() instances in the PM_CONTEXT_LOCAL case expanding the PMNS below a dynamic metric. Tightening the guards prevents the pointless malloc()s. This was causing some qa/957 fallout. commit 0ef492a9287f046542e9f61345af32313bf10f98 Author: Ken McDonell Date: Fri Apr 11 06:17:58 2014 +1000 linux PMDA proc_net_dev.c - small initialization change for valgrind WAR Comment from the code ... * Note: * Initialization of ecmd is not really needed. If the ioctl()s * work, ecmd is filled in ... but valgrind (at least up to * version 3.9.0) does not know about the SIOCETHTOOL ioctl() * and thinks the use of ecmd after this call propagates * uninitialized data in to ioc.speed and ioc.duplex, causing * failures for qa/957 commit ecfa750e321149dac484cea68fb9a34029737f5a Author: Ken McDonell Date: Thu Apr 10 11:47:40 2014 +1000 qa/common.check - "source" is not a sh keyword I know you really meant "." not "source". From fche@redhat.com Fri Apr 11 09:54:23 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 627007F3F for ; Fri, 11 Apr 2014 09:54:23 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 45D0D304059 for ; Fri, 11 Apr 2014 07:54:20 -0700 (PDT) X-ASG-Debug-ID: 1397228056-04cbb00dc5106630001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id mIa2vLbvAx3r4LqB for ; Fri, 11 Apr 2014 07:54:16 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3BEsFkd007708 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Fri, 11 Apr 2014 10:54:16 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3BEsFVe012109; Fri, 11 Apr 2014 10:54:15 -0400 Received: by fche.csb (Postfix, from userid 2569) id E6186585D8; Fri, 11 Apr 2014 10:54:14 -0400 (EDT) To: Nathan Scott Cc: PCP Mailing List Subject: Re: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa References: <1821183812.3777009.1397200271225.JavaMail.zimbra@redhat.com> <1429874976.3777376.1397200365901.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa From: fche@redhat.com (Frank Ch. Eigler) Date: Fri, 11 Apr 2014 10:54:14 -0400 In-Reply-To: <1429874976.3777376.1397200365901.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Fri, 11 Apr 2014 03:12:45 -0400 (EDT)") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397228056 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - > [...] > commit 42145e672b91bfb8a4cef29259640d9aa7493b4a > Author: Nathan Scott > Date: Fri Apr 11 15:48:38 2014 +1000 > > [...] > Add in support for pcp(1) to be used to invoke system tools > as scripts (like the existing pcp-free, pcp-uptime scripts) > with automatic argument passing/parsing. [...] Just took a look at the new pcp-free.py script; nice and short. Some aspects though concern me: - a python-binding issue where clients must manually manage pmResult memory, as in: 136 result = self.context.pmFetch(pmids) 137 values = self.extract(descs, result) 138 self.context.pmFreeResult(result) Can we hide this within the bindings (implicitly extracting values, and freeing the pmapi level results)? - a coding-style issue, where this client makes assumptions about the scaling values of metrics, specifically report(), which assumes all values[] are in Kbytes. This happens to be true today, but there's no guarantee that metric units/scales will never change. Such apps should instead use pmConvScale to assert/adapt. Perhaps both bullets are of the same kind: the python pmapi bindings should automate metric value fetching / scaling / decoding fuller (and more declaratively if possible). - FChE From wwwrun@oss.sgi.com Fri Apr 11 10:25:54 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: by oss.sgi.com (Postfix, from userid 30) id 2AC147F50; Fri, 11 Apr 2014 10:25:54 -0500 (CDT) From: bugzilla-daemon@oss.sgi.com To: pcp@oss.sgi.com Subject: [Bug 1052] New: acls relying on names should not rely on cached DNS results Date: Fri, 11 Apr 2014 15:25:53 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Classification: Unclassified X-Bugzilla-Product: pcp X-Bugzilla-Component: pcp X-Bugzilla-Keywords: X-Bugzilla-Severity: major X-Bugzilla-Who: fche@redhat.com X-Bugzilla-Status: NEW X-Bugzilla-Priority: P5 X-Bugzilla-Assigned-To: pcp@kenj.com.au X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter cc classification Message-ID: Content-Type: multipart/alternative; boundary="1397229954.c3eF1.21423"; charset="us-ascii" X-Bugzilla-URL: http://oss.sgi.com/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 --1397229954.c3eF1.21423 Date: Fri, 11 Apr 2014 10:25:54 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" http://oss.sgi.com/bugzilla/show_bug.cgi?id=1052 Bug ID: 1052 Summary: acls relying on names should not rely on cached DNS results Product: pcp Version: unspecified Hardware: All OS: Linux Status: NEW Severity: major Priority: P5 Component: pcp Assignee: pcp@kenj.com.au Reporter: fche@redhat.com CC: pcp@oss.sgi.com Classification: Unclassified In libpcp/src/access.c, DNS resolution for ACL entries is done once, at __pmAccAddHost() time (generally the startup time of the daemon). This ignore the possibility of DNS entries changing. Given that there appears to be no posix resolver API that lets apps know the TTL of DNS information they supply, the app can really only use the the DNS data instantaneously. So it seems to me that we would need to do the ACL name->address resolution at __pmAccAddClient time (whenever clients connect). Perhaps we can add a little cache-TTL of our own, if the libc resolvers turn out measurably too slow. Related, getmyhostid() in libpcp/src/access.c shouldn't imagine that there is a single "real IP address" for the host, nor that it is unchanging. -- You are receiving this mail because: You are on the CC list for the bug. --1397229954.c3eF1.21423 Date: Fri, 11 Apr 2014 10:25:54 -0500 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8"
Bug ID 1052
Summary acls relying on names should not rely on cached DNS results
Product pcp
Version unspecified
Hardware All
OS Linux
Status NEW
Severity major
Priority P5
Component pcp
Assignee pcp@kenj.com.au
Reporter fche@redhat.com
CC pcp@oss.sgi.com
Classification Unclassified 

In libpcp/src/access.c, DNS resolution for ACL entries is done
once, at __pmAccAddHost() time (generally the startup time of
the daemon).  This ignore the possibility of DNS entries
changing.  Given that there appears to be no posix resolver API
that lets apps know the TTL of DNS information they supply, the
app can really only use the the DNS data instantaneously.

So it seems to me that we would need to do the ACL name->address
resolution at __pmAccAddClient time (whenever clients connect).
Perhaps we can add a little cache-TTL of our own, if the libc
resolvers turn out measurably too slow.

Related, getmyhostid() in libpcp/src/access.c shouldn't
imagine that there is a single "real IP address" for the host,
nor that it is unchanging.

You are receiving this mail because:
  • You are on the CC list for the bug.
--1397229954.c3eF1.21423-- From minnus@buffalo.edu Fri Apr 11 12:52:14 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id B9FB87F3F for ; Fri, 11 Apr 2014 12:52:14 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 8545E8F8035 for ; Fri, 11 Apr 2014 10:52:11 -0700 (PDT) X-ASG-Debug-ID: 1397238726-04cb6c77b710a590001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id yysYn1YGmd9coX1s for ; Fri, 11 Apr 2014 10:52:07 -0700 (PDT) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailC.acsu.buffalo.edu (localmailc.acsu.buffalo.edu [128.205.5.204]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id C1918BA0 for ; Fri, 11 Apr 2014 13:52:06 -0400 (EDT) Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id BC86ED6CF for ; Fri, 11 Apr 2014 13:52:06 -0400 (EDT) Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailC.acsu.buffalo.edu (Postfix) with ESMTP id 64F62D6CC for ; Fri, 11 Apr 2014 13:52:06 -0400 (EDT) Received: from smtp.buffalo.edu (smtp4.acsu.buffalo.edu [128.205.5.229]) by localmailC.acsu.buffalo.edu (Prefixe) with ESMTP id 5A3C7D6CA for ; Fri, 11 Apr 2014 13:52:06 -0400 (EDT) Received: from gilmour.ccr.buffalo.edu (gilmour.ccr.buffalo.edu [128.205.40.13]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 3AE77A77E for ; Fri, 11 Apr 2014 13:52:06 -0400 (EDT) Message-ID: <53482BC5.6070203@buffalo.edu> Date: Fri, 11 Apr 2014 13:52:05 -0400 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: PCP Mailing List Subject: signal pmlogger Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: signal pmlogger Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397238727 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Hello, I'm trying to find a way to send a signal to pmlogger to log certain metrics immediately. For instance if we have a set of metrics that get logged every 10 minutes but we notice something "interesting" going on with the machine, I'd like to say: "Log those metrics right now" in some way. The only way I could think of was to duplicate these as "once" metrics in the conf file and then send pmlogger a SIGHUP. Is there a better way that wont start a new log file? Thanks Martins From fche@redhat.com Fri Apr 11 14:04:08 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C535B29DF7 for ; Fri, 11 Apr 2014 14:04:08 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 9E4888F8039 for ; Fri, 11 Apr 2014 12:04:05 -0700 (PDT) X-ASG-Debug-ID: 1397243044-04cb6c77b7111210001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id iwWFSAeA2WDmA6uL for ; Fri, 11 Apr 2014 12:04:04 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3BJ4322020801 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Apr 2014 15:04:03 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3BJ42Bs007816; Fri, 11 Apr 2014 15:04:03 -0400 Received: by fche.csb (Postfix, from userid 2569) id 22A44585D8; Fri, 11 Apr 2014 15:04:02 -0400 (EDT) To: Martins Innus Cc: PCP Mailing List Subject: Re: signal pmlogger References: <53482BC5.6070203@buffalo.edu> X-ASG-Orig-Subj: Re: signal pmlogger From: fche@redhat.com (Frank Ch. Eigler) Date: Fri, 11 Apr 2014 15:04:02 -0400 In-Reply-To: <53482BC5.6070203@buffalo.edu> (Martins Innus's message of "Fri, 11 Apr 2014 13:52:05 -0400") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397243044 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 minnus wrote: > [...] I'm trying to find a way to send a signal to pmlogger to log > certain metrics immediately. [...] See the pmlc tool. It can direct a pmlogger instance to log new metrics on demand. (One can imagine a future where this is automated via pmie or other ways.) - FChE From kenj@internode.on.net Fri Apr 11 16:06:03 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id BDB3F7F3F for ; Fri, 11 Apr 2014 16:06:03 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 9CC908F8039 for ; Fri, 11 Apr 2014 14:06:03 -0700 (PDT) X-ASG-Debug-ID: 1397250357-04bdf07dc711ebd0001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id d94EEiMerHw3Ue9u for ; Fri, 11 Apr 2014 14:05:57 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq5nAOFXSFN20WzCPGdsb2JhbABZDoJ4O4MTuS6HMAMCgR0XAwEBAQE4NYIlAQEBBAEBAQUCMBwYCwwBAwIGAxEEAQEBJwcZDhIKAwkIAgQBEgsFh2sOy14XjG+BfQcGhDIEj0meSlIr Received: from ppp118-209-108-194.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.108.194]) by ipmail06.adl2.internode.on.net with ESMTP; 12 Apr 2014 06:35:56 +0930 From: "Ken McDonell" To: "'Frank Ch. Eigler'" , "'Martins Innus'" Cc: "'PCP Mailing List'" References: <53482BC5.6070203@buffalo.edu> In-Reply-To: Subject: RE: [pcp] signal pmlogger Date: Sat, 12 Apr 2014 07:05:52 +1000 X-ASG-Orig-Subj: RE: [pcp] signal pmlogger Message-ID: <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQFjOOwECeWAyGg9kCExpCCyzl4RFwLbJ7mVm85HLYA= Content-Language: en-au X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1397250357 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4800 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header Frank's correct. pmlc was conceived at the same time pmlogger was created for just this purpose ... and the equally important case - after some elapsed time, if we don't know what's going on, we're never going to find out, so stop logging those additional metrics. All of the plumbing already exists for both the turning on and turning off to be driven by pmie (if the triggers to stop/start can be captured by pmie rules). It is all a bit low-level, but has been made to work in the past and will still work. If you need any assistance putting this together, I can help if I know the specific details. > -----Original Message----- > From: pcp-bounces@oss.sgi.com [mailto:pcp-bounces@oss.sgi.com] On > Behalf Of Frank Ch. Eigler > Sent: Saturday, 12 April 2014 5:04 AM > To: Martins Innus > Cc: PCP Mailing List > Subject: Re: [pcp] signal pmlogger > > > minnus wrote: > > > [...] I'm trying to find a way to send a signal to pmlogger to log > > certain metrics immediately. [...] > > See the pmlc tool. It can direct a pmlogger instance to log new metrics on > demand. (One can imagine a future where this is automated via pmie or > other ways.) > > - FChE > > _______________________________________________ > pcp mailing list > pcp@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/pcp From fche@redhat.com Fri Apr 11 18:44:12 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id F2A5E7F51 for ; Fri, 11 Apr 2014 18:44:11 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id DE44C8F8052 for ; Fri, 11 Apr 2014 16:44:11 -0700 (PDT) X-ASG-Debug-ID: 1397259847-04cbb00dc41305d0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id gfh1MNFSJjbg03Nm for ; Fri, 11 Apr 2014 16:44:07 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3BNi1YP013969 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 11 Apr 2014 19:44:02 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3BNi10X007856; Fri, 11 Apr 2014 19:44:01 -0400 Received: by fche.csb (Postfix, from userid 2569) id C925A585D8; Fri, 11 Apr 2014 19:44:00 -0400 (EDT) To: "Ken McDonell" Cc: "'Martins Innus'" , pcp@oss.sgi.com Subject: Re: signal pmlogger References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> X-ASG-Orig-Subj: Re: signal pmlogger From: fche@redhat.com (Frank Ch. Eigler) Date: Fri, 11 Apr 2014 19:44:00 -0400 In-Reply-To: <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> (Ken McDonell's message of "Sat, 12 Apr 2014 07:05:52 +1000") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397259847 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 kenj wrote: > [...] All of the plumbing already exists for both the turning on > and turning off to be driven by pmie (if the triggers to stop/start > can be captured by pmie rules). [...] Note that pmie has no native idea about which pmlogger process (amongst many locally or conceivably on a network) might be appropriate to target a with a pmlc connection. - FChE From kenj@internode.on.net Fri Apr 11 19:36:16 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 950747F4E for ; Fri, 11 Apr 2014 19:36:16 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 2B96EAC003 for ; Fri, 11 Apr 2014 17:36:12 -0700 (PDT) X-ASG-Debug-ID: 1397262970-04cbb00dc3132810001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id u0dKM4cGvt966Jew for ; Fri, 11 Apr 2014 17:36:10 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhiFAICJSFN20WzCPGdsb2JhbABZgwaDUIULu1kDAoEcFwMBAQEBODWCJQEBAQMBCAIeEhwjBQgDAgYDRhkgChQCBB4Fh2QHy1cXjmwHhDgEj0mfHCs Received: from ppp118-209-108-194.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.108.194]) by ipmail06.adl2.internode.on.net with ESMTP; 12 Apr 2014 10:05:49 +0930 From: "Ken McDonell" To: "'Frank Ch. Eigler'" Cc: "'Martins Innus'" , References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> In-Reply-To: Subject: RE: signal pmlogger Date: Sat, 12 Apr 2014 10:35:44 +1000 X-ASG-Orig-Subj: RE: signal pmlogger Message-ID: <001801cf55e7$287fbc60$797f3520$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQFjOOwECeWAyGg9kCExpCCyzl4RFwLbJ7mVAiOVnP0BW5en4ZuyiQtA Content-Language: en-au X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1397262970 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4806 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header Frank notes ... > Note that pmie has no native idea about which pmlogger process (amongst > many locally or conceivably on a network) might be appropriate to target a > with a pmlc connection. Correct. I said it was low-level plumbing ... 8^)> But in lots of cases that matter the pmlogger of interest is easily identified, especially when it is known to be the primary pmlogger on a particular host. From kenj@internode.on.net Sun Apr 13 01:05:18 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C7FD67F58 for ; Sun, 13 Apr 2014 01:05:18 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 8D50C8F8035 for ; Sat, 12 Apr 2014 23:05:15 -0700 (PDT) X-ASG-Debug-ID: 1397369112-04bdf07dca177700001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id muhtFV5mfT9N4DTD for ; Sat, 12 Apr 2014 23:05:12 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvFHAO0nSlN20fC0PGdsb2JhbABYgkJEO4MVwgYXAwEBAQE4NYIsCAIjKTAFBmIgCg4HAQQeBYdrmH+xW48LhCIEj0qfHSuBLCQ Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO bozohorize) ([118.209.240.180]) by ipmail06.adl2.internode.on.net with ESMTP; 13 Apr 2014 15:35:11 +0930 From: "Ken McDonell" To: Subject: pmlogrewrite questions from the developer meeting Date: Sun, 13 Apr 2014 16:05:10 +1000 X-ASG-Orig-Subj: pmlogrewrite questions from the developer meeting Message-ID: <01c501cf56de$54b6d370$fe247a50$@internode.on.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01C6_01CF5732.2663F4E0" X-Mailer: Microsoft Outlook 15.0 Thread-Index: Ac9WkUroZbfqdTajSgqHF7LgHS1u4g== Content-Language: en-au X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1397369112 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4845 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 HTML_MESSAGE BODY: HTML included in message This is a multipart message in MIME format. ------=_NextPart_000_01C6_01CF5732.2663F4E0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit At the last developer meeting, in the context of looking after precious archives files, some concern was raised about the behaviour of the -i (in place) option to pmlogrewrite. I've reviewed the code and the original implementation of -i seems robust to me. 1. temporary files are in the same directory as the input archive, so renaming does not imply any copying, just directory updates 2. the input archive is never overwritten 3. the input archive files is maintained via hard links (using a second set of temporary names) throughout and any error causes the old names to be reinstated Feedback from others would be welcome, but I think the concerns raised at the meeting are not substantiated by the code. Onto the next item . ------=_NextPart_000_01C6_01CF5732.2663F4E0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

At the last developer meeting, in the context of = looking after precious archives files, some concern was raised about the = behaviour of the –i (in place) option to = pmlogrewrite.

 

I’ve reviewed the code and the original = implementation of –i seems robust to me.

 

1.       = temporary files are in the same directory as the = input archive, so renaming does not imply any copying, just directory = updates

2.       the = input archive is never overwritten

3.       the = input archive files is maintained via hard links (using a second set of = temporary names) throughout and any error causes the old names to be = reinstated

 

Feedback from others would be welcome, but I think the = concerns raised at the meeting are not substantiated by the = code.

 

Onto the next item = …

------=_NextPart_000_01C6_01CF5732.2663F4E0-- From kenj@internode.on.net Sun Apr 13 01:12:25 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 6244E7F58 for ; Sun, 13 Apr 2014 01:12:25 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 3EB418F8037 for ; Sat, 12 Apr 2014 23:12:24 -0700 (PDT) X-ASG-Debug-ID: 1397369528-04cbb00dc617b790001-S8gJnT Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id xvwt9geOF9xpvJFD for ; Sat, 12 Apr 2014 23:12:11 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.129 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Au9HAEwqSlN20fC0PGdsb2JhbABYgkJEO4MVwgYXAwEBAQE4NYIsCAIjKTAFBmIgChUBBB4Fh2uZBLFdjh0BAWyEIgSPSoUqmXMrgTU Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO bozohorize) ([118.209.240.180]) by ipmail06.adl2.internode.on.net with ESMTP; 13 Apr 2014 15:42:08 +0930 From: "Ken McDonell" To: Subject: pmlogger -u questions Date: Sun, 13 Apr 2014 16:12:07 +1000 X-ASG-Orig-Subj: pmlogger -u questions Message-ID: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01EA_01CF5733.1E971480" X-Mailer: Microsoft Outlook 15.0 Thread-Index: Ac9Wk0u5sliNSYJNTG+IYRoOQpU0FQ== Content-Language: en-au X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129] X-Barracuda-Start-Time: 1397369531 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4845 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 HTML_MESSAGE BODY: HTML included in message This is a multipart message in MIME format. ------=_NextPart_000_01EA_01CF5733.1E971480 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Another one from the developer's meeting. The -u option to pmlogger is not really unbuffered. We're still using stdio and stdio buffers and stdio default buffer flushing, but we're adding fflush() calls in the correct order (meta data file, data file, index file) at the end of each logical record written to the archive (after each pmFetch). This closes, but does not remove the time window in which the physical files are not consistent and aligned with the end of a logical archive record. -u can be passed to all the pmloggers being managed by pmlogger_check and friends by adding -u to each line in the control file. As I see it, we have 3 options: 1. leave as is, other than some clarification of the -u option in the usage verbage and the man page 2. make -u the default (and introduce some new option, like -b, to change back to the previous buffering policy in case someone is logging very short records and does not like the additional write() calls that -u would imply in this case. Note that the default pmlogger config for the machine I'm composing this mail on writes 8700 byte records every 6 0seconds with a stdio buffer size of 8K, so -u would make very little difference to the write() rate 3. Same as 2. plus really do unbuffered I/O and avoid the stdio copying and not expose odd half records between a normal stdio write on buffer full and the fflush() . we would not need the fflush() in this case. Thoughts? Comments? ------=_NextPart_000_01EA_01CF5733.1E971480 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Another one from the developer’s = meeting.

 

The –u option to pmlogger is not really = unbuffered.  We’re still using stdio and stdio buffers and = stdio default buffer flushing, but we’re adding fflush() calls in = the correct order (meta data file, data file, index file) at the end of = each logical record written to the archive (after each = pmFetch).

 

This closes, but does not remove the time window in = which the physical files are not consistent and aligned with the end of = a logical archive record.

 

-u can be = passed to all the pmloggers being managed by pmlogger_check and friends = by adding –u to each line in the control file.

 

As I see it, = we have 3 options:

 

1.       = leave as is, other than some clarification of = the –u option in the usage verbage and the man = page

2.       = make –u the default (and introduce some = new option, like –b, to change back to the previous buffering = policy in case someone is logging very short records and does not like = the additional write() calls that –u would imply in this = case.  Note that the default pmlogger config for the machine = I’m composing this mail on writes 8700 byte records every 6 = 0seconds with a stdio buffer size of 8K, so –u would make very = little difference to the write() rate

3.       = Same as 2. plus really do unbuffered I/O and = avoid the stdio copying and not expose odd half records between a normal = stdio write on buffer full and the fflush() … we would not need = the fflush() in this case.

 

Thoughts?  = Comments?

------=_NextPart_000_01EA_01CF5733.1E971480-- From nscott@redhat.com Sun Apr 13 19:49:30 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 3AA217F4E for ; Sun, 13 Apr 2014 19:49:30 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 12275304039 for ; Sun, 13 Apr 2014 17:49:26 -0700 (PDT) X-ASG-Debug-ID: 1397436561-04cbb00dc31a6d20001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id LlfbreKwUaOZ9q2u for ; Sun, 13 Apr 2014 17:49:22 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3E0nAeP012880; Sun, 13 Apr 2014 20:49:10 -0400 Date: Sun, 13 Apr 2014 20:49:10 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp@oss.sgi.com Message-ID: <379318588.4720106.1397436550627.JavaMail.zimbra@redhat.com> In-Reply-To: <01c501cf56de$54b6d370$fe247a50$@internode.on.net> References: <01c501cf56de$54b6d370$fe247a50$@internode.on.net> Subject: Re: [pcp] pmlogrewrite questions from the developer meeting MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pmlogrewrite questions from the developer meeting Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogrewrite questions from the developer meeting Thread-Index: Ac9WkUroZbfqdTajSgqHF7LgHS1u4ntiUdPu X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1397436561 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4880 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Ken, ----- Original Message ----- > [..] > 1. temporary files are in the same directory as the input archive, so > renaming does not imply any copying, just directory updates > > 2. the input archive is never overwritten > > 3. the input archive files is maintained via hard links (using a second set > of temporary names) throughout and any error causes the old names to be > reinstated > There is a metadata vs data ordering assumption, I believe. We need to fsync the newly created data files before the rename otherwise we could end up with empty files on-disk - or files-filled-with-zeroes depending on the filesystem implementation (in this case: directory metadata IO & new file inode metadata IO completes up to the open(O_CREAT), but no data writes/allocation happen). > Feedback from others would be welcome, but I think the concerns raised at the > meeting are not substantiated by the code. I suspect I obfuscated the matter by incorrectly thinking that the temporary file creation was happening in some other TMPDIR - I was totally wrong there, as your audit found - apologies for the misunderstanding! cheers. -- Nathan From nscott@redhat.com Sun Apr 13 19:58:30 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 0A6797F4E for ; Sun, 13 Apr 2014 19:58:30 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id DE73C304032 for ; Sun, 13 Apr 2014 17:58:29 -0700 (PDT) X-ASG-Debug-ID: 1397437108-04cb6c77b719a910001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id oukqEFnFvYVPfluq for ; Sun, 13 Apr 2014 17:58:28 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3E0wO7b014145; Sun, 13 Apr 2014 20:58:24 -0400 Date: Sun, 13 Apr 2014 20:58:24 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp@oss.sgi.com Message-ID: <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> In-Reply-To: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> Subject: Re: [pcp] pmlogger -u questions MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pmlogger -u questions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogger -u questions Thread-Index: Ac9Wk0u5sliNSYJNTG+IYRoOQpU0FUwylkIw X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397437108 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4880 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi, ----- Original Message ----- >=20 >=20 > Another one from the developer=E2=80=99s meeting. >=20 I can't remember all of the context of this one, I think it was Frank who pointed out -u but the context is eluding me... >=20 > The =E2=80=93u option to pmlogger is not really unbuffered. We=E2=80=99re= still using stdio > and stdio buffers and stdio default buffer flushing, but we=E2=80=99re ad= ding > fflush() calls in the correct order (meta data file, data file, index fil= e) > at the end of each logical record written to the archive (after each > pmFetch). >=20 *nod* >=20 > This closes, but does not remove the time window in which the physical fi= les > are not consistent and aligned with the end of a logical archive record. >=20 Is our concern about "not consistent" here on-disk or in-memory consistency= ? >=20 > -u can be passed to all the pmloggers being managed by pmlogger_check and > friends by adding =E2=80=93u to each line in the control file. >=20 Not clear who (which tools?/code?) benefit from that, if anyone...? >=20 > As I see it, we have 3 options: > [...] > Thoughts? Comments? >=20 I think we need some demonstrated, concrete, actual issues here - its all a bit too hypothetical at this stage - a (QA?) tool that attempts to read fro= m the end of a growing log file for starters, and the clear existence of some problems, before we start fixing them. Do we have such a QA tool already? (reading the cases covered in pmGetArchiveEnd in libpcp suggests we might?) thanks. -- Nathan From nscott@redhat.com Sun Apr 13 20:42:35 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id B67207F4E for ; Sun, 13 Apr 2014 20:42:35 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 9C74F304039 for ; Sun, 13 Apr 2014 18:42:35 -0700 (PDT) X-ASG-Debug-ID: 1397439753-04cbb00dc61a8d10001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id UENvaL8iw0prttpo for ; Sun, 13 Apr 2014 18:42:34 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3E1gWAm031316; Sun, 13 Apr 2014 21:42:32 -0400 Date: Sun, 13 Apr 2014 21:42:32 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Stan Cox , Tom Yearke Cc: pcp@oss.sgi.com Message-ID: <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> In-Reply-To: <53446B81.4020602@redhat.com> References: <53342449.7070602@buffalo.edu> <53446B81.4020602@redhat.com> Subject: Re: [pcp] Patch for Python's pmExtractValue Function MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: Patch for Python's pmExtractValue Function Thread-Index: zSSXPdwCLDu48dHVEfJtpPoRaqVGcg== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1397439753 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4880 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain Hi guys, ----- Original Message ----- > I am not absolutely certain about the Python 2.6+ availability > requirement so I think this will do things in a similar 2.5 supported > way. Does this work for you? > We're a day or two out from a release, any help I can offer here? Do we have a reproducible test case I can try out, demonstrating the problem? (perhaps a little py script I can use, and then add a bit of valgrind sauce via /usr/bin/python to observe the leak?) thanks! -- Nathan From kenj@internode.on.net Sun Apr 13 21:07:50 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id F0F0E7F4E for ; Sun, 13 Apr 2014 21:07:49 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id DBD35304032 for ; Sun, 13 Apr 2014 19:07:46 -0700 (PDT) X-ASG-Debug-ID: 1397441264-04cb6c77b719d4f0001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id P5yImAz4toJP23uP for ; Sun, 13 Apr 2014 19:07:45 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApQBAGNCS1N20fC0/2dsb2JhbAANTYcovHmDDoE1gxkBAQEDASMPAQU8BAEQCxgCAgUWCwICCQMCAQIBRQYNAQcBAYdwp2F2oi0XgSmNRQeCb4FJAQOuZw Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 14 Apr 2014 11:37:19 +0930 Message-ID: <534B4330.1060008@internode.on.net> Date: Mon, 14 Apr 2014 12:08:48 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott CC: pcp@oss.sgi.com Subject: Re: [pcp] pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] pmlogger -u questions In-Reply-To: <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397441264 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4881 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 14/04/14 10:58, Nathan Scott wrote: > ... > I can't remember all of the context of this one, I think it was Frank > who pointed out -u but the context is eluding me... As the archives are being created, the stdio buffering in pmlogger (before we even get to the block layer / buffer cache in the kernel) guarantees that if pmlogger dies unexpectedly, or someone else tries to read the archive as it is being written, there is a 99.95% chance (2047 in 2048) of the archive appearing to be corrupted (assuming 8K stdio buffers, and we always write records that are a whole number of 4-byte words, and assuming my pending truncated != corrupted change has not been applied). With -u the chance becomes 0% of the archive appearing truncated in the absence of a system crash. >> This closes, but does not remove the time window in which the physical files >> are not consistent and aligned with the end of a logical archive record. >> > > Is our concern about "not consistent" here on-disk or in-memory consistency? Not consistent if pmlogger dies or someone tries to read the archive while it is being written. It is not an on-disk issue. It is an issue between pmlogger and the block layer / buffer cache of the kernel. >> -u can be passed to all the pmloggers being managed by pmlogger_check and >> friends by adding –u to each line in the control file. >> > > Not clear who (which tools?/code?) benefit from that, if anyone...? All the loggers run by pmlogger_{check,daily} are candidate beneficiaries. >> >> As I see it, we have 3 options: >> [...] >> Thoughts? Comments? >> > > I think we need some demonstrated, concrete, actual issues here - its all a > bit too hypothetical at this stage - a (QA?) tool that attempts to read from > the end of a growing log file for starters, and the clear existence of some > problems, before we start fixing them. Do we have such a QA tool already? > (reading the cases covered in pmGetArchiveEnd in libpcp suggests we might?) The issue is real, but I'm obviously not explaining it well enough. I am not sure there will be QA coverage, because the issue is a design problem, rather than a bug in implementation. Let me try with an annotated example. pmlogger is running and at some point, does a pmFetch which returns a PDU of 8700 bytes and a change in an instance domain, so the new instance domain requires 400 bytes to be written to the .meta file. Now with the default stdio buffering ... 1. the data PDU has to be split somewhere on an 8K boundary, so the head of the PDU is written to the .0 file and the tail of the PDU stays in the stdio buffer within pmlogger 2. either none or some of the new instance domain remains in the stdio buffers within pmlogger ... making the metadata either truncated or inconsistent with the archive data file Neither of these problems are new. But as we inch towards a unified context of some sort, and before that pmNewContext accepting a directory name as an argument for PM_CONTEXT_ARCHIVE it is going to become a more pressing matter. From fche@redhat.com Sun Apr 13 21:13:20 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 8A0CE7F4E for ; Sun, 13 Apr 2014 21:13:20 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 788118F8033 for ; Sun, 13 Apr 2014 19:13:17 -0700 (PDT) X-ASG-Debug-ID: 1397441596-04cb6c77b519d810001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id bHtv9WntjKu7TCZa for ; Sun, 13 Apr 2014 19:13:16 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3E2DE6s024303 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Sun, 13 Apr 2014 22:13:15 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3E2DE81011702 for ; Sun, 13 Apr 2014 22:13:14 -0400 Received: by fche.csb (Postfix, from userid 2569) id EBE4258183; Sun, 13 Apr 2014 22:13:13 -0400 (EDT) Date: Sun, 13 Apr 2014 22:13:13 -0400 From: "Frank Ch. Eigler" To: pcp developers Subject: graphite interfacing prototype Message-ID: <20140414021313.GH14108@redhat.com> X-ASG-Orig-Subj: graphite interfacing prototype Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397441596 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - The pcpfans.git repo fche/graphite branch has some early results [1]. [1] https://web.elastic.org/~fche/blog3/archive/2014/04/13/pcp-and-graphite It required correcting a few problems in the python bindings, which would be really handy to pull from the fche/for-merge branch into release 3.9.2. commit 1d235890e44e53d738a6ecc05b08cc2bc7ff2e1b (HEAD, origin/fche/for-merge, fche/for-merge) Author: Frank Ch. Eigler Date: Sat Apr 12 23:57:34 2014 -0400 pcp pmapi: fix pmConvScale parametrization The python binding to pmConvScale should be given a general pmUnits struct as a target scale, as at the C PMAPI level, not a parameter that limits us to "1 $space"-units. While in the vicinity, fix pmLookupName's exception message to identify the metrics that failed resolution, instead of an internal magic python char* rune. commit 3749b110f1d4e3dd5245779a68a21efdd98df969 Author: Frank Ch. Eigler Date: Sat Apr 12 22:00:28 2014 -0400 python pmapi.c: Don't crash on exceptions from options_callback() In case the python routine being invoked exits with an exception (result is NULL), we shouldn't try to Py_DECREF the bad boy. For that matter, we shouldn't clear such exceptions with PyErr_Print() at all, but pass them through to the calling python VM. This cleanup is left for later. commit 163db2e7ae96437a33ca8a0535af8bde35e95a34 Author: Frank Ch. Eigler Date: Sat Apr 12 21:46:51 2014 -0400 libpcp __pmSetProgname: copy incoming string __pmSetProgname, undocumented but widely loved, is used to being given system argv[0] as a parameter, which it has saved verbatim under the assumption that the underlying string can never change. This assumption is broken by the python pmoptions bindings, wherein random python string copies are processed, and occasionally handed to __pmSetProgname. This can easily corrupt pmUsageMessage(). In order to make it safer for mankind, __pmSetProgname henceforth copies the incoming const char*'s with strdup(). - FChE From fche@redhat.com Sun Apr 13 21:59:52 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 1F30D7F4E for ; Sun, 13 Apr 2014 21:59:52 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 023CC304032 for ; Sun, 13 Apr 2014 19:59:51 -0700 (PDT) X-ASG-Debug-ID: 1397444387-04cb6c77b619f380001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id NJCzjy6xCf1Ttxf6 for ; Sun, 13 Apr 2014 19:59:48 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3E2xhSA011748 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 13 Apr 2014 22:59:43 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3E2xhW1025364; Sun, 13 Apr 2014 22:59:43 -0400 Received: by fche.csb (Postfix, from userid 2569) id 8134758183; Sun, 13 Apr 2014 22:59:42 -0400 (EDT) To: Ken McDonell Cc: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> X-ASG-Orig-Subj: Re: pmlogger -u questions From: fche@redhat.com (Frank Ch. Eigler) Date: Sun, 13 Apr 2014 22:59:42 -0400 In-Reply-To: <534B4330.1060008@internode.on.net> (Ken McDonell's message of "Mon, 14 Apr 2014 12:08:48 +1000") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397444388 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 kenj wrote: > [...] > With -u the chance becomes 0% of the archive appearing truncated in > the absence of a system crash. Well, almost -- with the fwrite(3) data going out in dribs & drabs already, all those pre-fflush(3) moments can make the file appear truncated to another reader. (One might see that with a abort() inserted between the fwrite's and fflush's.) The -u option is at least useful for a lesser level of correctness, namely satisfying the pcp-archive.5 invariant that metadata must be present for metric values in the .0 file, by fflush()ing the .meta files before writing into the archive. This -u is not sufficient to protect the data from system crashes; one'd need fsync(2) syscalls in there too. It could be colocated with the -u fflush()es, or left to the fche/fsync-prototype fclose(). > Not consistent if pmlogger dies or someone tries to read the archive > while it is being written. It is not an on-disk issue. [...] (It is, to the extent that some kernel-level write(2)s could occur in sequences that are inconsistent.) >> Not clear who (which tools?/code?) benefit from that, if anyone...? > > All the loggers run by pmlogger_{check,daily} are candidate beneficiaries. ... as is anyone who loves their data. :-) With the present scheme, it's not hard to find pmlogger-generated archives that PMAPI refuses to open. I've got a bunch here, whether resulting from an untimely pmlogger exit or a system crash/reboot. (Note that our own tools sometimes SIGKILL an intransigent pmlogger.) (By the way, the same thing happens to systemd journals on my machines/VMs with some regularity, and those become write-offs after a "recovery" consisting of just moving the corrupt files out of place and letting them rot until GC.) - FChE From tyearke@buffalo.edu Sun Apr 13 23:29:43 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 0673D7F4E for ; Sun, 13 Apr 2014 23:29:43 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id EB38E8F8035 for ; Sun, 13 Apr 2014 21:29:39 -0700 (PDT) X-ASG-Debug-ID: 1397449778-04cb6c77b41a2090001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id 5ufUBIWysLvxZFS0 for ; Sun, 13 Apr 2014 21:29:38 -0700 (PDT) X-Barracuda-Envelope-From: tyearke@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id CF082D84; Mon, 14 Apr 2014 00:29:37 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id C4BB212EA4; Mon, 14 Apr 2014 00:29:37 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id D980712EA1; Mon, 14 Apr 2014 00:29:36 -0400 (EDT) Received: from smtp.buffalo.edu (smtp4.acsu.buffalo.edu [128.205.5.229]) by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id BBC4812EA0; Mon, 14 Apr 2014 00:29:36 -0400 (EDT) Received: from [192.168.1.76] (cpe-74-77-213-47.buffalo.res.rr.com [74.77.213.47]) (Authenticated sender: tyearke@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 76E7AA1AD; Mon, 14 Apr 2014 00:29:36 -0400 (EDT) Message-ID: <534B6428.1090709@buffalo.edu> Date: Mon, 14 Apr 2014 00:29:28 -0400 From: Tom Yearke User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott , Stan Cox CC: pcp@oss.sgi.com Subject: Re: [pcp] Patch for Python's pmExtractValue Function References: <53342449.7070602@buffalo.edu> <53446B81.4020602@redhat.com> <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function In-Reply-To: <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: XX: 28% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397449778 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4884 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Sorry about the delay in getting back to you all. I've had a number of other issues come up over the last few days that have prevented me from more fully investigating the issue, which I had hoped to do before replying but have not yet. Pointing to a Python string (instead of a C string instantiated by Python) works for my purposes. The only side effect this introduces, I believe, is that the string buffer pointed to is now an immutable one, but I can't imagine anyone's Python scripts involve modifying the string buffer that a pmAtomValue points to. Unless someone's doing some low-level manipulation with ctypes, I like this solution better than my own, since it avoids some expensive string copying. The only issue that I'm concerned with in this new solution is that, as in my first patch, the value of outAtom.vp is being passed to LIBC.free. In later testing of my first solution, I found that this method of freeing the C string led to seg faults when multiple processes using large amounts of memory were run, suggesting something isn't translated properly when 64-bit memory addresses are in use. The issue was resolved by using a ctypes char pointer (obtained by doing c_char_p.from_buffer_copy(outAtom, pmAtomValue.cp.offset)) instead of a Python integer (obtained by calling outAtom.vp). This is where the compatibility issue mentioned comes in - from_buffer_copy is only available in Python 2.6+. From what I recall in investigating this fix, I believe from_buffer_copy may be a convenience function for a series of functions available in 2.5 to accomplish the same task. I have not yet found what that series of functions was, but perhaps the ctypes source code contains it. I am hoping to finally get back to this issue in the morning, my timezone, but if you'd like to take a look in the meantime, I would welcome it. I'm afraid I don't have a script that's not tightly coupled to other resources to share, but calling pmExtractValue with a pmResult for a string metric, followed by losing the reference to the returned pmAtomValue, will create the leak. The example code at the top of pmapi.py should be a good starting point. Tom On 4/13/2014 9:42 PM, Nathan Scott wrote: > Hi guys, > > ----- Original Message ----- >> I am not absolutely certain about the Python 2.6+ availability >> requirement so I think this will do things in a similar 2.5 supported >> way. Does this work for you? >> > We're a day or two out from a release, any help I can offer here? > Do we have a reproducible test case I can try out, demonstrating > the problem? (perhaps a little py script I can use, and then add > a bit of valgrind sauce via /usr/bin/python to observe the leak?) > > thanks! > > -- > Nathan From nscott@redhat.com Sun Apr 13 23:50:33 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 720697F4E for ; Sun, 13 Apr 2014 23:50:33 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 60C7C304039 for ; Sun, 13 Apr 2014 21:50:30 -0700 (PDT) X-ASG-Debug-ID: 1397451028-04cb6c77b61a29f0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id zB6yUZWv7Ce1BF9y for ; Sun, 13 Apr 2014 21:50:29 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3E4oQfR009890; Mon, 14 Apr 2014 00:50:26 -0400 Date: Mon, 14 Apr 2014 00:50:26 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: PCP Mailing List Message-ID: <2067252072.4937914.1397451026337.JavaMail.zimbra@redhat.com> In-Reply-To: References: <1821183812.3777009.1397200271225.JavaMail.zimbra@redhat.com> <1429874976.3777376.1397200365901.JavaMail.zimbra@redhat.com> Subject: python API quirks (was Re: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa) MIME-Version: 1.0 X-ASG-Orig-Subj: python API quirks (was Re: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: python API quirks (was Re: pcp updates: kenj merge, numastat, memleak fixes, qa, qa, qa) Thread-Index: Sn1qviGNjFeBjnmQsoW9sGC0YMK8Hg== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397451029 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4885 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ----- Original Message ----- > > Just took a look at the new pcp-free.py script; nice and short. > Some aspects though concern me: > > - a python-binding issue where clients must manually manage > pmResult memory, as in: Ayup, this concerns me also. > 136 result = self.context.pmFetch(pmids) > 137 values = self.extract(descs, result) > 138 self.context.pmFreeResult(result) > > Can we hide this within the bindings (implicitly extracting > values, and freeing the pmapi level results)? Not easily, having tried a couple of things. My first thoughts were to attempt to handle this in the destructor, but thats problematic (we should *also* do that I think, but its not enough). The problem is we often need to fetch in a loop, and the destructor is not going to be triggered until some time after that, after the result has gone out of scope (and after we've leaked oodles of memory from repeated alloc's of the pmResult structure). I believe a more clean, pythonic approach will be to use the "with" keyword... something like: with self.context.pmFetch(pmids) as result: values = self.extract(descs, result) self.report(values) This evidently calls the __exit__ handler at the right time - we'd need to add one of those. However, it all falls down there - the return from pmapi.pmFetch is a pmResult ctypes *pointer*, not a pmResult. This is a synthesized type, not one of the types we define. Hence there's nowhere to implement this __exit__ routine. Maybe we can create a new class inheriting the POINTER(pmResult) class and return that from pmFetch, then have an __exit__ on that class... not sure, further experimentation required. > - a coding-style issue, where this client makes assumptions about the > scaling values of metrics, specifically report(), which assumes all > values[] are in Kbytes. This happens to be true today, but there's > no guarantee that metric units/scales will never change. Such apps > should instead use pmConvScale to assert/adapt. *nod*. In practice that is exceedingly unlikely for these particular metrics (kernel ABI is fixed, literally thousands of tools would break if it changed), but you're certainly right in general - I shall go make this change pronto. > [...] the python pmapi bindings > should automate metric value fetching / scaling / decoding fuller > (and more declaratively if possible). Definitely. I believe the original python API author was intending that the "pmcc" module would aid in solving these sorts of issues. Handling of rate conversion for counters is another candidate for abstraction... anything of a level of complexity beyond these simple tools (which don't even need to rate convert), should really be via an API. I think Stan went with a new pmsubsys module to abstract some of the atop / collectl common code in this way. We are at risk of producing more and more APIs above the tools doing the same things, differently. This happened with the C tools, heh, you'd think we'd learn. :) Doesn't overly affect these super-simple tools, but building up to the next level of complexity we need to start thinking about these things. cheers. -- Nathan From nscott@redhat.com Mon Apr 14 03:17:56 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 9E58D7F4E for ; Mon, 14 Apr 2014 03:17:56 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 6004A304032 for ; Mon, 14 Apr 2014 01:17:53 -0700 (PDT) X-ASG-Debug-ID: 1397463468-04bdf07dcc1b4da0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id yeSF4Vou02UrLVzf for ; Mon, 14 Apr 2014 01:17:48 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3E8HlC1004918; Mon, 14 Apr 2014 04:17:47 -0400 Date: Mon, 14 Apr 2014 04:17:47 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: pcp developers Message-ID: <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> In-Reply-To: <20140414021313.GH14108@redhat.com> References: <20140414021313.GH14108@redhat.com> Subject: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) MIME-Version: 1.0 X-ASG-Orig-Subj: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Content-Type: multipart/mixed; boundary="----=_Part_5020373_1108477399.1397463467939" X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Thread-Index: jBeK0Ew1KmDcoyOtnTrqoVkZkNU/SA== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397463468 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4889 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ------=_Part_5020373_1108477399.1397463467939 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi Frank! ----- Original Message ----- > Hi - > [...] Thanks for looking into the graphite support, haven't reviewed it yet but have fwd'd your post on to several people who have asked for this and feedback has been universally positive & grateful! :) > It required correcting a few problems in the python bindings, which > would be really handy to pull from the fche/for-merge branch into > release 3.9.2. BTW, there are several changes in fche/for-merge that you've not put in the summary - any reason for that? Some of them look useful and possibly merge-ready too. Others, maybe not ready yet ... hmmm, who knows? It makes it impossible to just git pull into the dev branch, anyway, which slows my workflow. Comments inline... (I'm still hopeful Ken will review the new qa/666 and offer suggestions - 6 minutes is still a really, really long time for a test to run - its >5 minutes longer than most other tests and about double the next slowest test. And *that* other test is already a big problem IMO so I'm very reluctant to add more QA laggards). > commit 1d235890e44e53d738a6ecc05b08cc2bc7ff2e1b (HEAD, origin/fche/for-merge, > fche/for-merge) > Author: Frank Ch. Eigler > Date: Sat Apr 12 23:57:34 2014 -0400 > > pcp pmapi: fix pmConvScale parametrization > > The python binding to pmConvScale should be given a general pmUnits > struct as a target scale, as at the C PMAPI level, not a parameter > that limits us to "1 $space"-units. Good find, however this is an API breaking change. A fully backward compatible solution would be to use type(units) and switch behaviour based on the type of the final parameter passed in. > While in the vicinity, fix pmLookupName's exception message to > identify the metrics that failed resolution, instead of an internal > magic python char* rune. Hmm, not so sure about this - looking inside the libpcp code, errors returned here are not related to the names at all, right? (they are runtime errors, so things like PDU I/O errors, and so on). Passing out the names doesn't really help to understand the failure, AFAICT? It looks like we should just treat this one the same as all the other pmErr exceptions (IOW, neither pmid/names list passed out). Name lookup failures (i.e. failure related to the names themselves, not other errors) are the area of auto-reporting via "relaxed mode", which appears to be fine. (or did it not suit/help here?) What was the error case you came across here, OOC? > commit 3749b110f1d4e3dd5245779a68a21efdd98df969 > Author: Frank Ch. Eigler > Date: Sat Apr 12 22:00:28 2014 -0400 > > python pmapi.c: Don't crash on exceptions from options_callback() > > In case the python routine being invoked exits with an exception > (result is NULL), we shouldn't try to Py_DECREF the bad boy. Good catch! > commit 163db2e7ae96437a33ca8a0535af8bde35e95a34 > Author: Frank Ch. Eigler > Date: Sat Apr 12 21:46:51 2014 -0400 > > libpcp __pmSetProgname: copy incoming string > > __pmSetProgname, undocumented but widely loved, is used to being given > system argv[0] as a parameter, which it has saved verbatim under the > assumption that the underlying string can never change. This assumption > is broken by the python pmoptions bindings, wherein random python string > copies are processed, and occasionally handed to __pmSetProgname. This > can easily corrupt pmUsageMessage(). Wow, amazed that I have not ever observed this! Do any/all of tests 739, 741, 742, 743, 991 fail for you? Its clearly busted (the python code, I mean) - good diagnosis, though I'm not super keen on the fix... > In order to make it safer for mankind, __pmSetProgname henceforth copies > the incoming const char*'s with strdup(). So, I'm trying to ignore the fact that you're once again introducing GNU coding style, into the middle of libpcp here (seriously? oh the humanity!). But lets be more objective - all the people using valgrind on C PMAPI tools will gnash their teeth with this change, as they will start to get warnings now. We need to be absolutely sure this is the only way to fix this, and there appear to be other good options (below). If we were to make this kind of change the qa/valgrind-suppressions file would need to be updated. Were there any failing QA tests after this change? Shame on myself/Ken if not...? [ Thats not rhetorical - could well be no new failures - I think we're reporting only "definitely lost" memory, not "maybe lost" in QA, so this would possibly sneak under the radar. Anyway, just a note - there exists a PCP valgrind suppressions file - in case of any future changes of this ilk (e.g. pmGetConfig behaves like this, so its in there). ] At the end of the day, this is a python API problem, ideally we'd solve it there and not affect the C or other language tools. Does the patch attached resolve the problem you observed? Do we have QA coverage of the problem with those existing tests I wrote (above) or do we need to add something more? (any hints re increasing chances of hitting it?) thanks! -- Nathan ------=_Part_5020373_1108477399.1397463467939 Content-Type: text/x-patch; name=patchlet Content-Disposition: attachment; filename=patchlet Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9weXRob24vcG1hcGkuYyBiL3NyYy9weXRob24vcG1hcGkuYwppbmRl eCA3MTVkNjM0Li5iMWViNTNkIDEwMDY0NAotLS0gYS9zcmMvcHl0aG9uL3BtYXBpLmMKKysrIGIv c3JjL3B5dGhvbi9wbWFwaS5jCkBAIC01OTUsNyArNTk1LDE2IEBAIGdldE9wdGlvbnNGcm9tTGlz dChQeU9iamVjdCAqc2VsZiwgUHlPYmplY3QgKmFyZ3MsIFB5T2JqZWN0ICprZXl3b3JkcykKIAog ICAgIGZvciAoaSA9IDA7IGkgPCBhcmdjOyBpKyspIHsKIAlQeU9iamVjdCAqcHlhcmcgPSBQeUxp c3RfR0VUX0lURU0ocHlhcmd2LCBpKTsKLQlhcmd2W2ldID0gUHlTdHJpbmdfQXNTdHJpbmcocHlh cmcpOworCWNoYXIgKnN0cmluZyA9IFB5U3RyaW5nX0FzU3RyaW5nKHB5YXJnKTsKKworCS8qIGFy Z3ZbMF0gcGFyYW1ldGVyIHdpbGwgYmUgdXNlZCBmb3IgcG1Qcm9nbmFtZSwgc28gbmVlZCB0bwor CSAqIGVuc3VyZSB0aGUgbWVtb3J5IHRoYXQgYmFja3MgaXQgd2lsbCBiZSB3aXRoIHVzIGZvcmV2 ZXIuCisgICAgICAgICAqLworCWlmIChhcmdjID09IDAgJiYgKHN0cmluZyA9IHN0cmR1cChzdHJp bmcpKSA9PSBOVUxMKSB7CisJICAgIFB5X0RFQ1JFRihweWFyZ3YpOworCSAgICByZXR1cm4gUHlF cnJfTm9NZW1vcnkoKTsKKwl9CisJYXJndltpXSA9IHN0cmluZzsKICAgICB9CiAKICAgICBpZiAo b3ZlcnJpZGVzQ2FsbGJhY2spCg== ------=_Part_5020373_1108477399.1397463467939-- From nscott@redhat.com Mon Apr 14 03:22:16 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 19C717F4E for ; Mon, 14 Apr 2014 03:22:16 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id A4427AC002 for ; Mon, 14 Apr 2014 01:22:15 -0700 (PDT) X-ASG-Debug-ID: 1397463733-04cb6c77b71aa810001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id q35XEArc6I27cb57 for ; Mon, 14 Apr 2014 01:22:13 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3E8MB1Q019349; Mon, 14 Apr 2014 04:22:11 -0400 Date: Mon, 14 Apr 2014 04:22:11 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Tom Yearke Cc: Stan Cox , pcp@oss.sgi.com Message-ID: <379826388.5021539.1397463731656.JavaMail.zimbra@redhat.com> In-Reply-To: <534B6428.1090709@buffalo.edu> References: <53342449.7070602@buffalo.edu> <53446B81.4020602@redhat.com> <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> <534B6428.1090709@buffalo.edu> Subject: Re: [pcp] Patch for Python's pmExtractValue Function MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: Patch for Python's pmExtractValue Function Thread-Index: UCpDh1hRz0EgEFWXJitOpbxecJfRQg== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1397463733 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4890 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Tom, ----- Original Message ----- > Sorry about the delay in getting back to you all. I've had a number of > other issues come up over the last few days that have prevented me from > more fully investigating the issue, which I had hoped to do before > replying but have not yet. No problem at all - I believe Stan has strained his back and has been offline a fair bit as well. > I am hoping to finally get back to this issue in the morning, my > timezone, but if you'd like to take a look in the meantime, I would > welcome it. I'm afraid I don't have a script that's not tightly coupled > to other resources to share, but calling pmExtractValue with a pmResult > for a string metric, followed by losing the reference to the returned > pmAtomValue, will create the leak. The example code at the top of > pmapi.py should be a good starting point. OK, thanks! I've run out of time for today, but hope to look into it more deeply tomorrow. cheers. -- Nathan From nscott@redhat.com Mon Apr 14 05:12:06 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 1D9F97F4E for ; Mon, 14 Apr 2014 05:12:06 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 8D867AC005 for ; Mon, 14 Apr 2014 03:12:02 -0700 (PDT) X-ASG-Debug-ID: 1397470317-04cbb00dc61bbf50001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id xk5fAt3ywK9xhvPQ for ; Mon, 14 Apr 2014 03:11:57 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3EABruP007612 for ; Mon, 14 Apr 2014 06:11:53 -0400 Date: Mon, 14 Apr 2014 06:11:53 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp developers Message-ID: <319687965.5116130.1397470313357.JavaMail.zimbra@redhat.com> In-Reply-To: <371414768.5114834.1397470051067.JavaMail.zimbra@redhat.com> Subject: pcp updates: fche cherrys, minor self.fixes MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: fche cherrys, minor self.fixes Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: fche cherrys, minor self.fixes Thread-Index: L/Nx/UZHm2RVy3aPfoYdADVqEH72+g== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1397470317 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4892 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev qa/721 | 1 - qa/721.out | 8 ++++---- qa/src/test_pcp.python | 6 +++++- src/dbpmda/src/pmda.c | 2 +- src/pcp/pcp.sh | 8 ++++++++ src/pmconfig/pmconfig.c | 4 ++-- src/python/pcp/pmapi.py | 13 ++++++++----- src/python/pmapi.c | 6 +++++- 8 files changed, 33 insertions(+), 15 deletions(-) commit f704c70f6aec02006139b54b2964568bd99cb279 Author: Nathan Scott Date: Mon Apr 14 20:03:03 2014 +1000 pcp pmapi: fix pmConvScale parametrization The python binding to pmConvScale should be given a general pmUnits struct as a target scale, as at the C PMAPI level, whereas the API wrapping to date limits us to "1 $space"-units. [ This commit is a reworked version of Franks earlier fix, but this time preserving compatibility with earlier versions of the API. ] commit 2cc49939dcfe291743649aafe689aa66e99163e4 Author: Frank Ch. Eigler Date: Sat Apr 12 22:00:28 2014 -0400 python pmapi.c: Don't crash on exceptions from options_callback() In case the python routine being invoked exits with an exception (result is NULL), we shouldn't try to Py_DECREF the bad boy. For that matter, we shouldn't clear such exceptions with PyErr_Print() at all, but pass them through to the calling python VM. This cleanup is left for later. commit 33a1899da93b7f8e92eb58c54f1295dfceeb5c8a Author: Frank Ch. Eigler Date: Fri Apr 11 18:30:35 2014 -0400 dbpmda: use context#0 for outgoing attr PDUs The attr command was unique in terms of inventing use of getpid() as a substitute context# for sending to a pipe-pmda. This makes it impossible to set the attribute data for the main context #0 across which fetches etc. are performed. Now we hard-code #0, as the other similar operations do. commit e488231b3f92ccc716b77ee1e42257715045e5e2 Author: Nathan Scott Date: Mon Apr 14 12:24:18 2014 +1000 Add the list of commands to pcp(1) usage message output The wrapped pcp-prefixed system commands will not be on the standard paths (pcp-free(1) and friends). Thus its handy to get a list of all that are available using the pcp(1) command usage message - Make It So. commit 87c30aed7628c1cac9f731ef5924c4dda2128e5a Author: Nathan Scott Date: Mon Apr 14 11:27:10 2014 +1000 Further improvements to the pmconfig string quoting Frank sagely points out pmconfig was not quoting backslash itself. Further testing revealed several of the characters being quoted should not have been (in particular, all forms of braces), as these would remain in the final string, even after expansion by the shell. The set there now appears to be correct after a fair bit of testing of all of the usual suspect characters. From nscott@redhat.com Mon Apr 14 05:14:02 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 435DA7F4E for ; Mon, 14 Apr 2014 05:14:02 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 20E6A8F8033 for ; Mon, 14 Apr 2014 03:14:01 -0700 (PDT) X-ASG-Debug-ID: 1397470440-04bdf07dca1ba510001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id bAJ0oPbgW8DNMimq for ; Mon, 14 Apr 2014 03:14:00 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3EAE0cV008111; Mon, 14 Apr 2014 06:14:00 -0400 Date: Mon, 14 Apr 2014 06:14:00 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: pcp developers Message-ID: <1417963059.5116963.1397470440481.JavaMail.zimbra@redhat.com> In-Reply-To: <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> Subject: Re: [pcp] fche/for-merge updates (was Re: graphite interfacing prototype) MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] fche/for-merge updates (was Re: graphite interfacing prototype) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Thread-Index: jBeK0Ew1KmDcoyOtnTrqoVkZkNU/SD4FtrxP X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1397470440 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4892 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ----- Original Message ----- > > [...] > > Author: Frank Ch. Eigler > > Date: Sat Apr 12 23:57:34 2014 -0400 > > > > pcp pmapi: fix pmConvScale parametrization > > > > The python binding to pmConvScale should be given a general pmUnits > > struct as a target scale, as at the C PMAPI level, not a parameter > > that limits us to "1 $space"-units. > > Good find, however this is an API breaking change. A fully backward > compatible solution would be to use type(units) and switch behaviour > based on the type of the final parameter passed in. I've fixed this up, updated the QA code, tested & merged it now BTW, please double check it though - thanks. cheers. -- Nathan From fche@redhat.com Mon Apr 14 09:17:38 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id CDA857F4E for ; Mon, 14 Apr 2014 09:17:38 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 8C5C5304067 for ; Mon, 14 Apr 2014 07:17:35 -0700 (PDT) X-ASG-Debug-ID: 1397485054-04bdf07dca1cb2d0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id rghnskK2XpIUtjbY for ; Mon, 14 Apr 2014 07:17:34 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3EEHYKs019252 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 14 Apr 2014 10:17:34 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3EEHXwA004427; Mon, 14 Apr 2014 10:17:33 -0400 Received: by fche.csb (Postfix, from userid 2569) id D052E58183; Mon, 14 Apr 2014 10:17:32 -0400 (EDT) Date: Mon, 14 Apr 2014 10:17:32 -0400 From: "Frank Ch. Eigler" To: Nathan Scott Cc: pcp developers Subject: Re: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Message-ID: <20140414141732.GI14108@redhat.com> X-ASG-Orig-Subj: Re: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397485054 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Nathan - > [...] > BTW, there are several changes in fche/for-merge that you've not put > in the summary - any reason for that? Some of them look useful and > possibly merge-ready too. Others, maybe not ready yet ... hmmm, who > knows? It makes it impossible to just git pull into the dev branch, > anyway, which slows my workflow. I considered all the for-merge bits ready to go, just didn't want to advertise them in the context of the python work. > Comments inline... (I'm still hopeful Ken will review the new qa/666 > and offer suggestions - 6 minutes is still a really, really long time > for a test to run - its >5 minutes longer than most other tests and > about double the next slowest test. And *that* other test is already > a big problem IMO so I'm very reluctant to add more QA laggards). Fair enough, but I'm reluctant to keep working on it more without a definite "good enough" target. (Of course one could reduce test coverage to save time, but that's not dignified.) > [...] > > While in the vicinity, fix pmLookupName's exception message to > > identify the metrics that failed resolution, instead of an internal > > magic python char* rune. > > Hmm, not so sure about this - looking inside the libpcp code, errors > returned here are not related to the names at all, right? (they are > runtime errors, so things like PDU I/O errors, and so on). In this particular case, the error parameter was for an informative name, but the object passed back was the ctypes char* rather than the python string, which made the error message unnecessarily opaque. > Passing out the names doesn't really help to understand the failure, > AFAICT? It looks like we should just treat this one the same as all > the other pmErr exceptions (IOW, neither pmid/names list passed out). > [...] There are many cases where the pcp/pmapi.py code does something like: raise pmErr, (status, nameA) In these cases, returning legible parameters (a native python object rather than nameA) would be helpful. > What was the error case you came across here, OOC? Sending bad metric names to pcp-graphite.py, either before or after the PMNS traversal code-additions. > [...] > > commit 163db2e7ae96437a33ca8a0535af8bde35e95a34 > > Author: Frank Ch. Eigler > > Date: Sat Apr 12 21:46:51 2014 -0400 > > > > libpcp __pmSetProgname: copy incoming string > [...] > > Wow, amazed that I have not ever observed this! Do any/all of > tests 739, 741, 742, 743, 991 fail for you? [...] No. > > In order to make it safer for mankind, __pmSetProgname henceforth copies > > the incoming const char*'s with strdup(). > > So, I'm trying to ignore the fact that you're once again introducing > GNU coding style, into the middle of libpcp here (seriously? oh the > humanity!). Oops, you've uncovered my secret scheme to eventually convert the whole codebase! > But lets be more objective - all the people using valgrind on C > PMAPI tools will gnash their teeth with this change, as they will > start to get warnings now. [...] If we were to make this kind of > change the qa/valgrind-suppressions file would need to be updated. Did you see an actual warning from this change? For programs that call __pmSetProgname only once (as is typical), there is no "leak" in that a global variable maintains a reference to the object. I can't get valgrind to even mention it. > [...] At the end of the day, this is a python API problem, ideally > we'd solve it there and not affect the C or other language tools. I disagree. There is no guarantee that a program's original argv[0] pointer needs to point at a stable string. > Does the patch attached resolve the problem you observed? I'm not sure it's sufficient, and IMHO the patch is worse: it still introduces the same kind of leak, and is now far removed from the place in libpcp that holds onto that pointer and makes the poor assumption. > Do we have QA coverage of the problem with those existing tests I > wrote (above) or do we need to add something more? (any hints re > increasing chances of hitting it?) You could strcpy(argv[0], "bozo the clown"); in a C PMAPI program sometime after __pmSetProgname(). The only other patch in fche/for-merge was this: commit e54d33f7fde4208a01107cd807292e9aaeffd3f6 Author: Frank Ch. Eigler Date: Sat Apr 12 14:32:46 2014 -0400 pcp.sh: add Available Commands: list to usage message, like already documented I see that you have your own version in /dev now. FWIW, I like this one better because its logic matches that of what is done later in pcp.sh, namely the ordering and executability checking, and does not litter in $tmp. YMMV, HTH, HAND, & c. - FChE From scox@redhat.com Mon Apr 14 09:35:46 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D593D7F4E for ; Mon, 14 Apr 2014 09:35:45 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7B781AC002 for ; Mon, 14 Apr 2014 07:35:42 -0700 (PDT) X-ASG-Debug-ID: 1397486141-04bdf07dca1cc310001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id BSilEAZGxv9esyj5 for ; Mon, 14 Apr 2014 07:35:41 -0700 (PDT) X-Barracuda-Envelope-From: scox@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3EEZeTp002744 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2014 10:35:40 -0400 Received: from [10.13.129.29] (dhcp129-29.rdu.redhat.com [10.13.129.29]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3EEZd5r003597; Mon, 14 Apr 2014 10:35:39 -0400 Message-ID: <534BF23B.80009@redhat.com> Date: Mon, 14 Apr 2014 10:35:39 -0400 From: Stan Cox User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Nathan Scott CC: pcp@oss.sgi.com, Tom Yearke Subject: Re: [pcp] Patch for Python's pmExtractValue Function References: <53342449.7070602@buffalo.edu> <53446B81.4020602@redhat.com> <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> <534B6428.1090709@buffalo.edu> <379826388.5021539.1397463731656.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function In-Reply-To: <379826388.5021539.1397463731656.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397486141 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 > I believe Stan has strained his back Torn oblique; but I am running on all cylinders now (finally!). I will poke at this today, which is EDT for me. From tyearke@buffalo.edu Mon Apr 14 11:19:16 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 0F0637F37 for ; Mon, 14 Apr 2014 11:19:16 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 9C831AC002 for ; Mon, 14 Apr 2014 09:19:12 -0700 (PDT) X-ASG-Debug-ID: 1397492347-04bdf07dcc1d39d0001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id z8FOJeBdnYxGLxvN for ; Mon, 14 Apr 2014 09:19:07 -0700 (PDT) X-Barracuda-Envelope-From: tyearke@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailC.acsu.buffalo.edu (localmailc.acsu.buffalo.edu [128.205.5.204]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id EAA2847A; Mon, 14 Apr 2014 12:19:06 -0400 (EDT) Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id E584FC10A; Mon, 14 Apr 2014 12:19:06 -0400 (EDT) Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailC.acsu.buffalo.edu (Postfix) with ESMTP id C2779C102; Mon, 14 Apr 2014 12:19:05 -0400 (EDT) Received: from smtp.buffalo.edu (smtp4.acsu.buffalo.edu [128.205.5.229]) by localmailC.acsu.buffalo.edu (Prefixe) with ESMTP id B59D1C101; Mon, 14 Apr 2014 12:19:05 -0400 (EDT) Received: from everly.ccr.buffalo.edu (everly.ccr.buffalo.edu [128.205.40.21]) (Authenticated sender: tyearke@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id A92D7A3E8; Mon, 14 Apr 2014 12:19:05 -0400 (EDT) Message-ID: <534C0A79.7020501@buffalo.edu> Date: Mon, 14 Apr 2014 12:19:05 -0400 From: Tom Yearke User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Stan Cox , Nathan Scott CC: pcp@oss.sgi.com Subject: Re: [pcp] Patch for Python's pmExtractValue Function References: <53342449.7070602@buffalo.edu> <53446B81.4020602@redhat.com> <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> <534B6428.1090709@buffalo.edu> <379826388.5021539.1397463731656.JavaMail.zimbra@redhat.com> <534BF23B.80009@redhat.com> X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function In-Reply-To: <534BF23B.80009@redhat.com> Content-Type: multipart/mixed; boundary="------------000403000708010803080806" X-PM-EL-Spam-Prob: : 9% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397492347 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4901 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- This is a multi-part message in MIME format. --------------000403000708010803080806 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Ow! Glad you're doing better! I've created a patch for the new solution that obtains the C string address in a hopefully-safe, Python 2.5-friendly manner. It will take a bit longer to test for the seg fault problem, but it's passed the small scale tests I've done, so I wanted to get it out ASAP for feedback. I did notice some unusual valgrind messages for both the patched and unpatched new solution, which did not occur for any of the other solutions I've attempted. A series of errors like the following are printed: ==14442== Invalid read of size 8 ==14442== at 0x4A08BE8: memcpy (mc_replace_strmem.c:882) ==14442== by 0x458D26: PyString_FromString (stringobject.c:144) ==14442== by 0x44CFEA: PyObject_GenericGetAttr (object.c:1424) ==14442== by 0x49D216: PyEval_EvalFrameEx (ceval.c:2082) ==14442== by 0x4A00B7: PyEval_EvalFrameEx (ceval.c:3836) ==14442== by 0x4A00B7: PyEval_EvalFrameEx (ceval.c:3836) ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) ==14442== by 0x49F372: PyEval_EvalFrameEx (ceval.c:3846) ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) ==14442== by 0x4A1271: PyEval_EvalCode (ceval.c:541) ==14442== by 0x4C0D5D: PyRun_FileExFlags (pythonrun.c:1351) ==14442== by 0x4C0F73: PyRun_SimpleFileExFlags (pythonrun.c:941) ==14442== Address 0x10ff1ca8 is 40 bytes inside a block of size 67 free'd ==14442== at 0x4A063F0: free (vg_replace_malloc.c:446) ==14442== by 0x4F6A21: frame_dealloc (frameobject.c:420) ==14442== by 0x4A00DD: PyEval_EvalFrameEx (ceval.c:3838) ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) ==14442== by 0x49F372: PyEval_EvalFrameEx (ceval.c:3846) ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) ==14442== by 0x4A1271: PyEval_EvalCode (ceval.c:541) ==14442== by 0x4C0D5D: PyRun_FileExFlags (pythonrun.c:1351) ==14442== by 0x4C0F73: PyRun_SimpleFileExFlags (pythonrun.c:941) ==14442== by 0x413EB7: Py_Main (main.c:577) ==14442== by 0x3F7821ED1C: (below main) (in /lib64/libc-2.12.so) It doesn't seem to occur for every string fetch, and the new code doesn't appear to impact the contents of the returned strings, but I thought it might be important to share. On 4/14/14, 10:35 AM, Stan Cox wrote: >> I believe Stan has strained his back > > Torn oblique; but I am running on all cylinders now (finally!). I > will poke at this today, which is EDT for me. > --------------000403000708010803080806 Content-Type: text/plain; charset=UTF-8; name="patch.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="patch.diff" RnJvbSBlNTU2MTBlMmJkOTE5NzE3ODE3YzYxYjI3M2E5NzRhNDgwMzE4ODJiIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBUb20gWWVhcmtlIDx0eWVhcmtlQGJ1ZmZhbG8uZWR1 PgpEYXRlOiBNb24sIDE0IEFwciAyMDE0IDEyOjA0OjEzIC0wNDAwClN1YmplY3Q6IFtQQVRD SF0gcHl0aG9uL3BtYXBpLnB5IC0gZnJlZSBDIHN0cmluZyB1c2luZyBwb2ludGVyIGZyb20g b3V0QXRvbS5jcAogaW5zdGVhZCBvZiBhZGRyZXNzIGZyb20gb3V0QXRvbS52cCAoZml4ZXMg cG9zc2libGUgc2VnIGZhdWx0KQoKLS0tCiBzcmMvcHl0aG9uL3BjcC9wbWFwaS5weSB8IDEw ICsrKysrKy0tLS0KIDEgZmlsZSBjaGFuZ2VkLCA2IGluc2VydGlvbnMoKyksIDQgZGVsZXRp b25zKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL3B5dGhvbi9wY3AvcG1hcGkucHkgYi9zcmMvcHl0 aG9uL3BjcC9wbWFwaS5weQppbmRleCBmZjY5MjJmLi40OWRjNTgzIDEwMDY0NAotLS0gYS9z cmMvcHl0aG9uL3BjcC9wbWFwaS5weQorKysgYi9zcmMvcHl0aG9uL3BjcC9wbWFwaS5weQpA QCAtODEsNyArODEsNyBAQCBmcm9tIGN0eXBlcyBpbXBvcnQgY19jaGFyLCBjX2ludCwgY191 aW50LCBjX2xvbmcsIGNfY2hhcl9wLCBjX3ZvaWRfcAogZnJvbSBjdHlwZXMgaW1wb3J0IGNf bG9uZ2xvbmcsIGNfdWxvbmdsb25nLCBjX2Zsb2F0LCBjX2RvdWJsZQogZnJvbSBjdHlwZXMg aW1wb3J0IENETEwsIFBPSU5URVIsIENGVU5DVFlQRSwgU3RydWN0dXJlLCBVbmlvbgogZnJv bSBjdHlwZXMgaW1wb3J0IGFkZHJlc3NvZiwgcG9pbnRlciwgc2l6ZW9mLCBjYXN0LCBieXJl ZgotZnJvbSBjdHlwZXMgaW1wb3J0IGNyZWF0ZV9zdHJpbmdfYnVmZmVyCitmcm9tIGN0eXBl cyBpbXBvcnQgY3JlYXRlX3N0cmluZ19idWZmZXIsIG1lbW1vdmUKIGZyb20gY3R5cGVzLnV0 aWwgaW1wb3J0IGZpbmRfbGlicmFyeQogCiAjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKQEAg LTE2NjMsMTAgKzE2NjMsMTIgQEAgY2xhc3MgcG1Db250ZXh0KG9iamVjdCk6CiAgICAgICAg ICAgICByYWlzZSBwbUVyciwgc3RhdHVzCiAKICAgICAgICAgaWYgb3V0dHlwZSA9PSBjX2Fw aS5QTV9UWVBFX1NUUklORzoKLSAgICAgICAgICAgICMgR2V0IGFkZHJlc3Mgb2YgQyBzdHJp bmcKLSAgICAgICAgICAgIGNfc3RyID0gb3V0QXRvbS52cAorICAgICAgICAgICAgIyBHZXQg cG9pbnRlciB0byBDIHN0cmluZworICAgICAgICAgICAgY19zdHIgPSBjX2NoYXJfcCgpCisg ICAgICAgICAgICBtZW1tb3ZlKGJ5cmVmKGNfc3RyKSwgYWRkcmVzc29mKG91dEF0b20pICsg cG1BdG9tVmFsdWUuY3Aub2Zmc2V0LAorICAgICAgICAgICAgICAgICAgICBzaXplb2YoY19j aGFyX3ApKQogICAgICAgICAgICAgIyBDb252ZXJ0IHRvIGEgcHl0aG9uIHN0cmluZyBhbmQg aGF2ZSByZXN1bHQgcG9pbnQgdG8gaXQKLSAgICAgICAgICAgIHB5dGhvbl9jaGFyX2FycmF5 ID0gY3R5cGVzLnN0cmluZ19hdChjX3N0cikKKyAgICAgICAgICAgIHB5dGhvbl9jaGFyX2Fy cmF5ID0gb3V0QXRvbS5jcAogICAgICAgICAgICAgb3V0QXRvbS5jcCA9IGNhc3QocHl0aG9u X2NoYXJfYXJyYXksIGNfY2hhcl9wKQogICAgICAgICAgICAgIyBGcmVlIHRoZSBDIHN0cmlu ZwogICAgICAgICAgICAgTElCQy5mcmVlKGNfc3RyKQotLSAKMS44LjQuMgoK --------------000403000708010803080806-- From minnus@buffalo.edu Mon Apr 14 14:27:24 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B5EB27F37 for ; Mon, 14 Apr 2014 14:27:24 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 24EF3AC004 for ; Mon, 14 Apr 2014 12:27:20 -0700 (PDT) X-ASG-Debug-ID: 1397503640-04cb6c243806680001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id eHXGCTUr30YRabLA for ; Mon, 14 Apr 2014 12:27:20 -0700 (PDT) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 6FBCA921; Mon, 14 Apr 2014 15:27:19 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 65860D3CE; Mon, 14 Apr 2014 15:27:19 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id 7CF76D3C8; Mon, 14 Apr 2014 15:27:18 -0400 (EDT) Received: from smtp.buffalo.edu (smtp3.acsu.buffalo.edu [128.205.5.226]) by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id 6BCEED3C5; Mon, 14 Apr 2014 15:27:18 -0400 (EDT) Received: from gilmour.ccr.buffalo.edu (gilmour.ccr.buffalo.edu [128.205.40.13]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 53E9E94CC; Mon, 14 Apr 2014 15:27:18 -0400 (EDT) Message-ID: <534C3695.4050306@buffalo.edu> Date: Mon, 14 Apr 2014 15:27:17 -0400 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Ken McDonell , "'Frank Ch. Eigler'" CC: 'PCP Mailing List' Subject: Re: [pcp] signal pmlogger References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> X-ASG-Orig-Subj: Re: [pcp] signal pmlogger In-Reply-To: <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397503640 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4906 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Ken/Frank, Thanks, I played with pmlc a little bit and have a couple more questions. I may have simplified my initial description too much. Assume a setup where we have a manually crafted pmlogger config with hundreds of metrics being logged over several different time intervals: Some at 1 minute, 10 minutes and 30 minutes. If I start some compute process, I want to make sure that I log all of the above described metrics at least once over the life of that process, but otherwise keep the logging schedule unchanged. If the process runs for less than 30 minutes, currently I will get anywhere from a subset to none of the metrics logged, depending on the length of the compute process. Playing with pmlc, i can do a: log mandatory on once to force a logging event when the compute process starts, but then the logger stops its regular logging schedule. So I assume then I have to write a script that parses the logger config file and re-enables the logging as it was with pmlc commands? I didn't see a save/load config set of commands from pmlc unless I missed it. I can go down that path if that is the right way to do it. But I was hoping for some sort of "log this list of metrics right now, just once, but don't otherwise change pmlogger" command. This is all on the local host running just a primary logger. pmie support down the road might be interesting, but for now I know when I want these logging events to occur. Thanks Martins On 4/11/14 5:05 PM, Ken McDonell wrote: > Frank's correct. pmlc was conceived at the same time pmlogger was created > for just this purpose ... and the equally important case - after some > elapsed time, if we don't know what's going on, we're never going to find > out, so stop logging those additional metrics. > > All of the plumbing already exists for both the turning on and turning off > to be driven by pmie (if the triggers to stop/start can be captured by pmie > rules). It is all a bit low-level, but has been made to work in the past > and will still work. > > If you need any assistance putting this together, I can help if I know the > specific details. > >> -----Original Message----- >> From: pcp-bounces@oss.sgi.com [mailto:pcp-bounces@oss.sgi.com] On >> Behalf Of Frank Ch. Eigler >> Sent: Saturday, 12 April 2014 5:04 AM >> To: Martins Innus >> Cc: PCP Mailing List >> Subject: Re: [pcp] signal pmlogger >> >> >> minnus wrote: >> >>> [...] I'm trying to find a way to send a signal to pmlogger to log >>> certain metrics immediately. [...] >> See the pmlc tool. It can direct a pmlogger instance to log new metrics > on >> demand. (One can imagine a future where this is automated via pmie or >> other ways.) >> >> - FChE >> >> _______________________________________________ From tyearke@buffalo.edu Mon Apr 14 14:27:25 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 5AD3C7F37 for ; Mon, 14 Apr 2014 14:27:25 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 055E88F8033 for ; Mon, 14 Apr 2014 12:27:21 -0700 (PDT) X-ASG-Debug-ID: 1397503635-04bdf0455206440001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id EFHFPV546D7DZNFS for ; Mon, 14 Apr 2014 12:27:15 -0700 (PDT) X-Barracuda-Envelope-From: tyearke@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailC.acsu.buffalo.edu (localmailc.acsu.buffalo.edu [128.205.5.204]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 4FA1991B; Mon, 14 Apr 2014 15:27:15 -0400 (EDT) Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 46E6C5058; Mon, 14 Apr 2014 15:27:15 -0400 (EDT) Received: from localmailC.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailC.acsu.buffalo.edu (Postfix) with ESMTP id E26ADCF25; Mon, 14 Apr 2014 15:25:50 -0400 (EDT) Received: from smtp.buffalo.edu (smtp2.acsu.buffalo.edu [128.205.5.254]) by localmailC.acsu.buffalo.edu (Prefixe) with ESMTP id D425BCF24; Mon, 14 Apr 2014 15:25:50 -0400 (EDT) Received: from everly.ccr.buffalo.edu (everly.ccr.buffalo.edu [128.205.40.21]) (Authenticated sender: tyearke@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 8B70C9A7B; Mon, 14 Apr 2014 15:25:50 -0400 (EDT) Message-ID: <534C363D.8020001@buffalo.edu> Date: Mon, 14 Apr 2014 15:25:49 -0400 From: Tom Yearke User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Stan Cox , Nathan Scott CC: pcp@oss.sgi.com Subject: Re: [pcp] Patch for Python's pmExtractValue Function References: <53342449.7070602@buffalo.edu> <53446B81.4020602@redhat.com> <235383721.4734118.1397439752269.JavaMail.zimbra@redhat.com> <534B6428.1090709@buffalo.edu> <379826388.5021539.1397463731656.JavaMail.zimbra@redhat.com> <534BF23B.80009@redhat.com> <534C0A79.7020501@buffalo.edu> X-ASG-Orig-Subj: Re: [pcp] Patch for Python's pmExtractValue Function In-Reply-To: <534C0A79.7020501@buffalo.edu> Content-Type: multipart/mixed; boundary="------------080705060309090109070100" X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397503635 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4906 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 HTML_MESSAGE BODY: HTML included in message This is a multi-part message in MIME format. --------------080705060309090109070100 Content-Type: multipart/alternative; boundary="------------030802030205040500010301" --------------030802030205040500010301 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Attached is a revised patch which addresses the valgrind errors. Just to see how the new Python string solution works step-by-step, I recreated it using some test structures and code in an interactive Python shell. What I found was that the final string stored in a test ctypes union (like pmAtomValue) would often be empty when the original string was not. I'm not sure why the solution would work in my PCP scripts and not in my test shell, but removing the cast to c_char_p seems to have produced a solution that works for valgrind and all situations I've thrown at it so far. I still need to perform some large-scale tests of this revised solution, but I'm hopeful it will work as expected. On 4/14/14, 12:19 PM, Tom Yearke wrote: > Ow! Glad you're doing better! > > I've created a patch for the new solution that obtains the C string > address in a hopefully-safe, Python 2.5-friendly manner. It will take > a bit longer to test for the seg fault problem, but it's passed the > small scale tests I've done, so I wanted to get it out ASAP for feedback. > > I did notice some unusual valgrind messages for both the patched and > unpatched new solution, which did not occur for any of the other > solutions I've attempted. A series of errors like the following are > printed: > > ==14442== Invalid read of size 8 > ==14442== at 0x4A08BE8: memcpy (mc_replace_strmem.c:882) > ==14442== by 0x458D26: PyString_FromString (stringobject.c:144) > ==14442== by 0x44CFEA: PyObject_GenericGetAttr (object.c:1424) > ==14442== by 0x49D216: PyEval_EvalFrameEx (ceval.c:2082) > ==14442== by 0x4A00B7: PyEval_EvalFrameEx (ceval.c:3836) > ==14442== by 0x4A00B7: PyEval_EvalFrameEx (ceval.c:3836) > ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) > ==14442== by 0x49F372: PyEval_EvalFrameEx (ceval.c:3846) > ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) > ==14442== by 0x4A1271: PyEval_EvalCode (ceval.c:541) > ==14442== by 0x4C0D5D: PyRun_FileExFlags (pythonrun.c:1351) > ==14442== by 0x4C0F73: PyRun_SimpleFileExFlags (pythonrun.c:941) > ==14442== Address 0x10ff1ca8 is 40 bytes inside a block of size 67 > free'd > ==14442== at 0x4A063F0: free (vg_replace_malloc.c:446) > ==14442== by 0x4F6A21: frame_dealloc (frameobject.c:420) > ==14442== by 0x4A00DD: PyEval_EvalFrameEx (ceval.c:3838) > ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) > ==14442== by 0x49F372: PyEval_EvalFrameEx (ceval.c:3846) > ==14442== by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000) > ==14442== by 0x4A1271: PyEval_EvalCode (ceval.c:541) > ==14442== by 0x4C0D5D: PyRun_FileExFlags (pythonrun.c:1351) > ==14442== by 0x4C0F73: PyRun_SimpleFileExFlags (pythonrun.c:941) > ==14442== by 0x413EB7: Py_Main (main.c:577) > ==14442== by 0x3F7821ED1C: (below main) (in /lib64/libc-2.12.so) > > It doesn't seem to occur for every string fetch, and the new code > doesn't appear to impact the contents of the returned strings, but I > thought it might be important to share. > > On 4/14/14, 10:35 AM, Stan Cox wrote: >>> I believe Stan has strained his back >> >> Torn oblique; but I am running on all cylinders now (finally!). I >> will poke at this today, which is EDT for me. >> > > > > _______________________________________________ > pcp mailing list > pcp@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/pcp --------------030802030205040500010301 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Attached is a revised patch which addresses the valgrind errors.

Just to see how the new Python string solution works step-by-step, I recreated it using some test structures and code in an interactive Python shell. What I found was that the final string stored in a test ctypes union (like pmAtomValue) would often be empty when the original string was not. I'm not sure why the solution would work in my PCP scripts and not in my test shell, but removing the cast to c_char_p seems to have produced a solution that works for valgrind and all situations I've thrown at it so far.

I still need to perform some large-scale tests of this revised solution, but I'm hopeful it will work as expected.

On 4/14/14, 12:19 PM, Tom Yearke wrote:
Ow! Glad you're doing better!

I've created a patch for the new solution that obtains the C string address in a hopefully-safe, Python 2.5-friendly manner. It will take a bit longer to test for the seg fault problem, but it's passed the small scale tests I've done, so I wanted to get it out ASAP for feedback.

I did notice some unusual valgrind messages for both the patched and unpatched new solution, which did not occur for any of the other solutions I've attempted. A series of errors like the following are printed:

==14442== Invalid read of size 8
==14442==    at 0x4A08BE8: memcpy (mc_replace_strmem.c:882)
==14442==    by 0x458D26: PyString_FromString (stringobject.c:144)
==14442==    by 0x44CFEA: PyObject_GenericGetAttr (object.c:1424)
==14442==    by 0x49D216: PyEval_EvalFrameEx (ceval.c:2082)
==14442==    by 0x4A00B7: PyEval_EvalFrameEx (ceval.c:3836)
==14442==    by 0x4A00B7: PyEval_EvalFrameEx (ceval.c:3836)
==14442==    by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000)
==14442==    by 0x49F372: PyEval_EvalFrameEx (ceval.c:3846)
==14442==    by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000)
==14442==    by 0x4A1271: PyEval_EvalCode (ceval.c:541)
==14442==    by 0x4C0D5D: PyRun_FileExFlags (pythonrun.c:1351)
==14442==    by 0x4C0F73: PyRun_SimpleFileExFlags (pythonrun.c:941)
==14442==  Address 0x10ff1ca8 is 40 bytes inside a block of size 67 free'd
==14442==    at 0x4A063F0: free (vg_replace_malloc.c:446)
==14442==    by 0x4F6A21: frame_dealloc (frameobject.c:420)
==14442==    by 0x4A00DD: PyEval_EvalFrameEx (ceval.c:3838)
==14442==    by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000)
==14442==    by 0x49F372: PyEval_EvalFrameEx (ceval.c:3846)
==14442==    by 0x4A119F: PyEval_EvalCodeEx (ceval.c:3000)
==14442==    by 0x4A1271: PyEval_EvalCode (ceval.c:541)
==14442==    by 0x4C0D5D: PyRun_FileExFlags (pythonrun.c:1351)
==14442==    by 0x4C0F73: PyRun_SimpleFileExFlags (pythonrun.c:941)
==14442==    by 0x413EB7: Py_Main (main.c:577)
==14442==    by 0x3F7821ED1C: (below main) (in /lib64/libc-2.12.so)

It doesn't seem to occur for every string fetch, and the new code doesn't appear to impact the contents of the returned strings, but I thought it might be important to share.

On 4/14/14, 10:35 AM, Stan Cox wrote:
 I believe Stan has strained his back

Torn oblique;  but I am running on all cylinders now (finally!). I will poke at this today, which is EDT for me. 




_______________________________________________
pcp mailing list
pcp@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/pcp

--------------030802030205040500010301-- --------------080705060309090109070100 Content-Type: text/plain; charset=UTF-8; name="patch.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="patch.diff" >From 7e8c4f12352929cf93346237112b1e9542e13bff Mon Sep 17 00:00:00 2001 From: Tom Yearke Date: Mon, 14 Apr 2014 14:49:45 -0400 Subject: [PATCH] python/pmapi.py - free C string using pointer from outAtom.cp instead of address from outAtom.vp (fixes possible seg fault), resolve valgrind read errors --- src/python/pcp/pmapi.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/python/pcp/pmapi.py b/src/python/pcp/pmapi.py index ff6922f..9b2c04a 100644 --- a/src/python/pcp/pmapi.py +++ b/src/python/pcp/pmapi.py @@ -81,7 +81,7 @@ from ctypes import c_char, c_int, c_uint, c_long, c_char_p, c_void_p from ctypes import c_longlong, c_ulonglong, c_float, c_double from ctypes import CDLL, POINTER, CFUNCTYPE, Structure, Union from ctypes import addressof, pointer, sizeof, cast, byref -from ctypes import create_string_buffer +from ctypes import create_string_buffer, memmove from ctypes.util import find_library ############################################################################## @@ -1663,11 +1663,12 @@ class pmContext(object): raise pmErr, status if outtype == c_api.PM_TYPE_STRING: - # Get address of C string - c_str = outAtom.vp + # Get pointer to C string + c_str = c_char_p() + memmove(byref(c_str), addressof(outAtom) + pmAtomValue.cp.offset, + sizeof(c_char_p)) # Convert to a python string and have result point to it - python_char_array = ctypes.string_at(c_str) - outAtom.cp = cast(python_char_array, c_char_p) + outAtom.cp = outAtom.cp # Free the C string LIBC.free(c_str) return outAtom -- 1.8.4.2 --------------080705060309090109070100-- From kenj@internode.on.net Mon Apr 14 16:01:41 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id C6DD07F37 for ; Mon, 14 Apr 2014 16:01:41 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 46283AC006 for ; Mon, 14 Apr 2014 14:01:38 -0700 (PDT) X-ASG-Debug-ID: 1397509292-04cb6c24380c390001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id qr2x6jNSg4KalkFQ for ; Mon, 14 Apr 2014 14:01:33 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAGpLTFN20fC0/2dsb2JhbAANTA7ELIMOgT2DGQEBAQQ4QAEQCw4KCRYPCQMCAQIBRQYBDAEHAQGwEqN3F45uB4Q4AQOuFVI Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 15 Apr 2014 06:31:32 +0930 Message-ID: <534C4D07.8020604@internode.on.net> Date: Tue, 15 Apr 2014 07:03:03 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Martins Innus , "'Frank Ch. Eigler'" CC: 'PCP Mailing List' Subject: Re: [pcp] signal pmlogger References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> <534C3695.4050306@buffalo.edu> X-ASG-Orig-Subj: Re: [pcp] signal pmlogger In-Reply-To: <534C3695.4050306@buffalo.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397509293 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4909 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 15/04/14 05:27, Martins Innus wrote: > ... > > Playing with pmlc, i can do a: > > log mandatory on once > > to force a logging event when the compute process starts, but then the > logger stops its regular logging schedule. So I assume then I have to > write a script that parses the logger config file and re-enables the > logging as it was with pmlc commands? I didn't see a save/load config > set of commands from pmlc unless I missed it. > > I can go down that path if that is the right way to do it. But I was > hoping for some sort of "log this list of metrics right now, just once, > but don't otherwise change pmlogger" command. Thanks for the insight Martins. Well this is truly interesting. Your use case is something that has not previously been encountered in over a decade of belting PCP in production environments. I find that both surprising and exciting. I can't see a way to achieve what you want with the single pmlogger process and the capabilities today. This is because the log mandatory pmlc command updates the logging state for all the named metrics within pmlogger, so what you describe is what I'd expect to happen. The options are: 1. extend pmlc and pmlogger to have an additional set of metrics that can have duration and logging state that co-exists with the config file ones at startup (so the state change from pmlc is independent of, not a replacement of, the original state) ... it would be nice to have some short hand notation for turning all of these additional metrics off in one command and extend the "once" duration to allow N times. 2. Have your script that is currently calling pmlc actually launch a _new_ pmlogger to capture just the metrics you want at the start or end of the computation or with some particular frequency during the computation. Then at the end of the day, you can use pmlogextract to merge together the archive from the default system logging (or whatever pmlogger you're pointing pmlc at) with one or more archives from your computation runs. The merged archive will be the same as if 1. was implemented. Given you can do 2. today, and it would take me a couple of days to get 1. to a prototype state, I'd vote for 2. And in the mean time we should consider the discussion to see if we can refine 1. into something that is general useful, robust and easy to use. From kenj@internode.on.net Mon Apr 14 16:14:08 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id ED59D7F37 for ; Mon, 14 Apr 2014 16:14:07 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id ABD6E8F8037 for ; Mon, 14 Apr 2014 14:14:04 -0700 (PDT) X-ASG-Debug-ID: 1397510042-04bdf045520d780001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id QEttEPMT5wOzKz2n for ; Mon, 14 Apr 2014 14:14:02 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAC9PTFN20fC0/2dsb2JhbAANTMQ6gw6BPYMZAQEBAwE4QAEFCwsYCRYPCQMCAQIBRQYNAQcBAReHWagco34Xjm4HhDgBA5R0mXM Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 15 Apr 2014 06:44:01 +0930 Message-ID: <534C4FF4.5000304@internode.on.net> Date: Tue, 15 Apr 2014 07:15:32 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> X-ASG-Orig-Subj: Re: pmlogger -u questions In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397510042 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4910 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On 14/04/14 12:59, Frank Ch. Eigler wrote: > > kenj wrote: > >> [...] >> With -u the chance becomes 0% of the archive appearing truncated in >> the absence of a system crash. > > Well, almost -- with the fwrite(3) data going out in dribs & drabs > already, all those pre-fflush(3) moments can make the file appear > truncated to another reader. (One might see that with a abort() > inserted between the fwrite's and fflush's.) Which is why I was suggesting abandoning stdio buffering for the -u case and making this the default. > This -u is not sufficient to protect the data from system crashes; > one'd need fsync(2) syscalls in there too. It could be colocated with > the -u fflush()es, or left to the fche/fsync-prototype fclose(). A simpler solution might be to offer O_SYNC I/O as an option. >> Not consistent if pmlogger dies or someone tries to read the archive >> while it is being written. It is not an on-disk issue. [...] > > (It is, to the extent that some kernel-level write(2)s could occur in > sequences that are inconsistent.) If I control the write(2) directly (no stdio as I'm suggesting) then the writes are guaranteed to be in a consistent order ... the code already does this above stdio, it is just the stdio buffering that masks some of the writes, and writes data that is not aligned to the logical boundaries of the archive records. > ... With the present scheme, > it's not hard to find pmlogger-generated archives that PMAPI refuses > to open. I've got a bunch here, whether resulting from an untimely > pmlogger exit or a system crash/reboot. (Note that our own tools > sometimes SIGKILL an intransigent pmlogger.) I'll bet SIGKILL has the highest probability. And the "truncated is not corrupted" change (that is independent of -u issues) would address many of these I would expect. Can we agree that -u semantics and no stdio should be the default behaviour going forward? And I'll do the "truncated is not corrupted" change as a separate work item. From fche@redhat.com Mon Apr 14 16:25:57 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 93FA87F37 for ; Mon, 14 Apr 2014 16:25:57 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 59DBF304053 for ; Mon, 14 Apr 2014 14:25:57 -0700 (PDT) X-ASG-Debug-ID: 1397510755-04bdf045530e440001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id G4hE33CXyjyCqPic for ; Mon, 14 Apr 2014 14:25:56 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3ELPqUn021899 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2014 17:25:52 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3ELPquY023060; Mon, 14 Apr 2014 17:25:52 -0400 Received: by fche.csb (Postfix, from userid 2569) id 6EAE658183; Mon, 14 Apr 2014 17:25:51 -0400 (EDT) Date: Mon, 14 Apr 2014 17:25:51 -0400 From: "Frank Ch. Eigler" To: Ken McDonell Cc: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions Message-ID: <20140414212551.GK14108@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <534C4FF4.5000304@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397510756 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - On Tue, Apr 15, 2014 at 07:15:32AM +1000, Ken McDonell wrote: > [...] Can we agree that -u semantics and no stdio should be the > default behaviour going forward? [...] That would make sense to me (leaving room for future O_SYNC and/or fsync(2) at related times). - FChE From kenj@internode.on.net Mon Apr 14 17:44:47 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 7E1D47F37 for ; Mon, 14 Apr 2014 17:44:47 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 68B57304059 for ; Mon, 14 Apr 2014 15:44:44 -0700 (PDT) X-ASG-Debug-ID: 1397515478-04cbb06e9d14160001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id FUMuPKEW7563vK2i for ; Mon, 14 Apr 2014 15:44:39 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAAZkTFN20fC0/2dsb2JhbAANS8Q3gw6BPYMZAQEBBDhAARALGAkWDwkDAgECAUUGDQEHAQGwIqNnF45uB4Q4AQOuZw Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 15 Apr 2014 08:14:38 +0930 Message-ID: <534C6531.6050502@internode.on.net> Date: Tue, 15 Apr 2014 08:46:09 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions In-Reply-To: <20140414212551.GK14108@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397515478 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4914 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On 15/04/14 07:25, Frank Ch. Eigler wrote: > Hi - > > On Tue, Apr 15, 2014 at 07:15:32AM +1000, Ken McDonell wrote: > >> [...] Can we agree that -u semantics and no stdio should be the >> default behaviour going forward? [...] > > That would make sense to me ... Having actually made the change now, I like this even more ... in the simplest implementation it is a one line change in __pmLogNewFile() to add setvbuf(f, NULL, _IONBF, 0); and then everyone who creates an archive (pmlogger, pmlogrewite, pmlogextract, pmlogreduce, ...) gets unconditional unbuffered I/O. To get the logical record semantics I need a little more hackery as the trailer 4 bytes is currently done in a separate fwrite(). And the metadata writes are all fragmented, so I may need to do something smarter there. From kenj@internode.on.net Mon Apr 14 17:51:17 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id CAA3B7F37 for ; Mon, 14 Apr 2014 17:51:17 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 68A71AC003 for ; Mon, 14 Apr 2014 15:51:14 -0700 (PDT) X-ASG-Debug-ID: 1397515872-04bdf0455413d40001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id lFlBg7RM6IJMeI8k for ; Mon, 14 Apr 2014 15:51:12 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApQBAHRlTFN20fC0/2dsb2JhbAANS4covQ+DDoE9gxkBAQEEIxVAARALGAICBRYLAgIJAwIBAgFFBg0BBwEBsCV2onIXgSmNRQeCb4FJAQOuZw Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 15 Apr 2014 08:21:12 +0930 Message-ID: <534C66BB.6050206@internode.on.net> Date: Tue, 15 Apr 2014 08:52:43 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott CC: pcp@oss.sgi.com Subject: Re: [pcp] pmlogrewrite questions from the developer meeting References: <01c501cf56de$54b6d370$fe247a50$@internode.on.net> <379318588.4720106.1397436550627.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] pmlogrewrite questions from the developer meeting In-Reply-To: <379318588.4720106.1397436550627.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397515872 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4915 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 14/04/14 10:49, Nathan Scott wrote: > ... > There is a metadata vs data ordering assumption, I believe. We need to fsync > the newly created data files before the rename otherwise we could end up with > empty files on-disk - or files-filled-with-zeroes depending on the filesystem > implementation (in this case: directory metadata IO & new file inode metadata > IO completes up to the open(O_CREAT), but no data writes/allocation happen). Nathan, So before exit(), you're sugesting fsync() on each of the data files, and I think I need fsync() on the container directory as well. But we should be consistent. If this is the "right" way to do it then surely all applications that can write PCP archives should do the same thing. I am not against doing this, although if one was concerned at this level then I suspect an option to enforce O_SYNC might be better to guarantee on disk for all writes, not just flushing everything at exit, but we should choose one policy for writing PCP archives and implement it consistently throughout the PCP ecosystem. From fche@redhat.com Mon Apr 14 18:18:33 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 833847F37 for ; Mon, 14 Apr 2014 18:18:33 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 719298F8035 for ; Mon, 14 Apr 2014 16:18:30 -0700 (PDT) X-ASG-Debug-ID: 1397517506-04cb6c243614fe0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id jCZqhrtE2qRaLyuR for ; Mon, 14 Apr 2014 16:18:26 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3ENIMeU025142 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2014 19:18:22 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3ENILsh028438; Mon, 14 Apr 2014 19:18:21 -0400 Received: by fche.csb (Postfix, from userid 2569) id 08F9958183; Mon, 14 Apr 2014 19:18:20 -0400 (EDT) Date: Mon, 14 Apr 2014 19:18:20 -0400 From: "Frank Ch. Eigler" To: Ken McDonell Cc: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions Message-ID: <20140414231820.GL14108@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <534C6531.6050502@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397517506 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - On Tue, Apr 15, 2014 at 08:46:09AM +1000, Ken McDonell wrote: > [...] > Having actually made the change now, I like this even more ... in the > simplest implementation it is a one line change in __pmLogNewFile() to add > setvbuf(f, NULL, _IONBF, 0); We should do a round of analysis (strace or stap or whatever) to see how the change would affect i/o syscall traffic, just to confirm that we write records in nice big chunks. (If not, another possibility would be to adjust the stdio buffering dynamically, based on the actual size of the logger data records.) - FChE From nscott@redhat.com Mon Apr 14 18:34:00 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 73F4A7F37 for ; Mon, 14 Apr 2014 18:34:00 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 05D21AC002 for ; Mon, 14 Apr 2014 16:33:56 -0700 (PDT) X-ASG-Debug-ID: 1397518435-04bdf04555164b0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id WnwGQfZkbfh46TdP for ; Mon, 14 Apr 2014 16:33:55 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3ENXoxS013691; Mon, 14 Apr 2014 19:33:50 -0400 Date: Mon, 14 Apr 2014 19:33:50 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp@oss.sgi.com Message-ID: <944351754.5545062.1397518430382.JavaMail.zimbra@redhat.com> In-Reply-To: <534C66BB.6050206@internode.on.net> References: <01c501cf56de$54b6d370$fe247a50$@internode.on.net> <379318588.4720106.1397436550627.JavaMail.zimbra@redhat.com> <534C66BB.6050206@internode.on.net> Subject: Re: [pcp] pmlogrewrite questions from the developer meeting MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pmlogrewrite questions from the developer meeting Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogrewrite questions from the developer meeting Thread-Index: HswyodS32DUcoBLWIlNI/+Vre1lG8g== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397518435 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4916 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Ken, ----- Original Message ----- > On 14/04/14 10:49, Nathan Scott wrote: > > ... > > There is a metadata vs data ordering assumption, I believe. We need to > > fsync the newly created data files before the rename > [...] > > Nathan, > > So before exit(), you're sugesting fsync() on each of the data files, > and I think I need fsync() on the container directory as well. Just before the renames, we need to ensure the new data is on the platter. So, fsync the new files before renaming, and we're done. We do not need to fsync again, later, on exit - this is simply unnecessary latency and doesn't fix the problem. > But we should be consistent. If this is the "right" way to do it then > surely all applications that can write PCP archives should do the same > thing. It is only this rename-over-the-top situation that requires a preceding fsync to close that complete-data-loss gap. > I am not against doing this, although if one was concerned at this level > then I suspect an option to enforce O_SYNC might be better to guarantee > on disk for all writes, not just flushing everything at exit, but we > should choose one policy for writing PCP archives and implement it > consistently throughout the PCP ecosystem. O_SYNC? Yikes, I prefer not - there will be widespread introduction of latency on write; and preferably not just-fsync-everything-always-to-be-sure either. These approaches introduce potentially very large delays, in new and sometimes highly undesirable places. And it'd be all so unnecessary. Worse, they *still* leave the window for total data loss (although of course they drastically diminish the time window it can happen). Everything is fine as-is, except for this rename situation which has a small potential-total-loss time window without fsync. A surgical incision is what we need here, not a shotgun. cheers. -- Nathan From kenj@internode.on.net Mon Apr 14 18:38:45 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id AE69D7F37 for ; Mon, 14 Apr 2014 18:38:45 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 88E338F8035 for ; Mon, 14 Apr 2014 16:38:45 -0700 (PDT) X-ASG-Debug-ID: 1397518723-04cbb06e9b17460001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id HKBIzkCvluHlqMqA for ; Mon, 14 Apr 2014 16:38:43 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBACpxTFN20fC0/2dsb2JhbAANS8Q3gw6BOYMZAQEBBDhAARALGAkWDwkDAgECAUUGDQEHAQGwI6NtF45uB4Q4AQOuZw Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 15 Apr 2014 09:08:17 +0930 Message-ID: <534C71C2.9080600@internode.on.net> Date: Tue, 15 Apr 2014 09:39:46 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <20140414231820.GL14108@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions In-Reply-To: <20140414231820.GL14108@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397518723 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4916 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On 15/04/14 09:18, Frank Ch. Eigler wrote: > Hi - > > On Tue, Apr 15, 2014 at 08:46:09AM +1000, Ken McDonell wrote: >> [...] >> Having actually made the change now, I like this even more ... in the >> simplest implementation it is a one line change in __pmLogNewFile() to add >> setvbuf(f, NULL, _IONBF, 0); > > We should do a round of analysis (strace or stap or whatever) to see > how the change would affect i/o syscall traffic, just to confirm that > we write records in nice big chunks. (If not, another possibility > would be to adjust the stdio buffering dynamically, based on the > actual size of the logger data records.) Yep, already doing that. Before with default config on my workstation ... every 60 secs I see 2 writes of 4096 and/or 8192 bytes (can't explain why as yet)/ After with same config ... every 60 secs I see 2 writes of 11244 and 4 bytes (the latter is the trailer that I am confident I can force into the first write ... but this is more than a 1 line change as the semantics of __pmLogPutResult have to change in a subtle way). From nscott@redhat.com Mon Apr 14 18:43:02 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id E5C847F37 for ; Mon, 14 Apr 2014 18:43:02 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id CFBC78F8035 for ; Mon, 14 Apr 2014 16:43:02 -0700 (PDT) X-ASG-Debug-ID: 1397518981-04cb6c2439165d0001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id Tzny8USQns4xmFXk for ; Mon, 14 Apr 2014 16:43:01 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3ENgvU9014428; Mon, 14 Apr 2014 19:42:57 -0400 Date: Mon, 14 Apr 2014 19:42:57 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: "Frank Ch. Eigler" , pcp@oss.sgi.com Message-ID: <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> In-Reply-To: <534C6531.6050502@internode.on.net> References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> Subject: Re: pmlogger -u questions MIME-Version: 1.0 X-ASG-Orig-Subj: Re: pmlogger -u questions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogger -u questions Thread-Index: cLJSwv2cAafSQrN8f3Ma9zF0sbVEeg== X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1397518981 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4916 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... ----- Original Message ----- > On 15/04/14 07:25, Frank Ch. Eigler wrote: > > Hi - > > > > On Tue, Apr 15, 2014 at 07:15:32AM +1000, Ken McDonell wrote: > > > >> [...] Can we agree that -u semantics and no stdio should be the > >> default behaviour going forward? [...] > > > > That would make sense to me ... > > Having actually made the change now, I like this even more ... in the > simplest implementation it is a one line change in __pmLogNewFile() to add > > setvbuf(f, NULL, _IONBF, 0); > > and then everyone who creates an archive (pmlogger, pmlogrewite, > pmlogextract, pmlogreduce, ...) gets unconditional unbuffered I/O. > > To get the logical record semantics I need a little more hackery as the > trailer 4 bytes is currently done in a separate fwrite(). And the > metadata writes are all fragmented, so I may need to do something > smarter there. > I concur with Frank here - definitely warrants some deep cost analysis - it sounds risky WRT introducing performance regression. What is the I/O size we generally submit from pmlogger data writes, OOC? This may be an opportunity to improve too, by gathering larger consistent chunks before submitting I/O... cheers. -- Nathan From fche@redhat.com Mon Apr 14 19:30:51 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B97357F37 for ; Mon, 14 Apr 2014 19:30:51 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 5B98FAC002 for ; Mon, 14 Apr 2014 17:30:48 -0700 (PDT) X-ASG-Debug-ID: 1397521832-04cb6c243918de0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id dSLSql0Cx1iwNVcQ for ; Mon, 14 Apr 2014 17:30:47 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3F0TrXD005348 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Apr 2014 20:29:53 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3F0Tr6g012086; Mon, 14 Apr 2014 20:29:53 -0400 Received: by fche.csb (Postfix, from userid 2569) id B248A58183; Mon, 14 Apr 2014 20:29:52 -0400 (EDT) Date: Mon, 14 Apr 2014 20:29:52 -0400 From: "Frank Ch. Eigler" To: Nathan Scott Cc: Ken McDonell , pcp@oss.sgi.com Subject: Re: pmlogger -u questions Message-ID: <20140415002952.GM14108@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397521832 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - On Mon, Apr 14, 2014 at 07:42:57PM -0400, Nathan Scott wrote: > [...] > I concur with Frank here - definitely warrants some deep cost analysis - > it sounds risky WRT introducing performance regression. I wouldn't over-worry about it. The PMCD PDU's are already being relayed to volume files in one chunk, so that's great. Getting the trailer bytes in there would be dandy. Beyond that, we'd be actually saving some effort in that we'd eliminate one copy (by not going into stdio buffers). > What is the I/O size we generally submit from pmlogger data writes, > OOC? [...] Ken reported seeing alternating 4/8K write(2) syscalls coming, which makes sense for 12K-ish log record batches going into stdio. - FChE From nscott@redhat.com Mon Apr 14 19:37:57 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 715777F37 for ; Mon, 14 Apr 2014 19:37:57 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 53ECF304032 for ; Mon, 14 Apr 2014 17:37:54 -0700 (PDT) X-ASG-Debug-ID: 1397522272-04cb6c2436194e0001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id lLiQxq34ljVchVaC for ; Mon, 14 Apr 2014 17:37:53 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3F0bmNc026372; Mon, 14 Apr 2014 20:37:48 -0400 Date: Mon, 14 Apr 2014 20:37:48 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: Ken McDonell , pcp@oss.sgi.com Message-ID: <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> In-Reply-To: <20140415002952.GM14108@redhat.com> References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> <20140415002952.GM14108@redhat.com> Subject: Re: pmlogger -u questions MIME-Version: 1.0 X-ASG-Orig-Subj: Re: pmlogger -u questions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogger -u questions Thread-Index: zr+aoYGrmGFihtg7M+V0FborNNN7lQ== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1397522273 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4917 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ----- Original Message ----- > Hi - > > On Mon, Apr 14, 2014 at 07:42:57PM -0400, Nathan Scott wrote: > > [...] > > I concur with Frank here - definitely warrants some deep cost analysis - > > it sounds risky WRT introducing performance regression. > > I wouldn't over-worry about it. The PMCD PDU's are already being > relayed to volume files in one chunk, so that's great. Getting the > trailer bytes in there would be dandy. Beyond that, we'd be actually > saving some effort in that we'd eliminate one copy (by not going into > stdio buffers). > > > What is the I/O size we generally submit from pmlogger data writes, > > OOC? [...] > > Ken reported seeing alternating 4/8K write(2) syscalls coming, which > makes sense for 12K-ish log record batches going into stdio. Awesome, thanks guys - you had me at hello^W"I wouldn't over-worry". ;) cheers. -- Nathan From nscott@redhat.com Mon Apr 14 21:56:58 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 8F1637F37 for ; Mon, 14 Apr 2014 21:56:58 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 49A65304051 for ; Mon, 14 Apr 2014 19:56:58 -0700 (PDT) X-ASG-Debug-ID: 1397530615-04bdf04554223c0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id CjO2b5Qm6VlqL6nK for ; Mon, 14 Apr 2014 19:56:56 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3F2uscj006426; Mon, 14 Apr 2014 22:56:54 -0400 Date: Mon, 14 Apr 2014 22:56:54 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp developers Cc: Tom Yearke Message-ID: <1648738368.5584160.1397530614435.JavaMail.zimbra@redhat.com> In-Reply-To: <697504469.5583964.1397530481571.JavaMail.zimbra@redhat.com> Subject: pcp updates: pmdarpm, python API - scox merge (plus tyearke fix) MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: pmdarpm, python API - scox merge (plus tyearke fix) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: pmdarpm, python API - scox merge (plus tyearke fix) Thread-Index: Vgy53PKcsjuQwNx8rhfjYh9vk8hSqw== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397530615 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4920 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev qa/750 | 57 +- qa/750.out | 30 - qa/751 | 49 ++ qa/751.out | 13 qa/752 | 9 qa/752.out | 52 ++ qa/900 | 1 qa/group | 1 qa/src/test_pcp.python | 104 +++- src/libpcp/src/check-statics | 1 src/libpcp/src/getdate.y | 963 ++++++++++++++++++++----------------------- src/libpcp/src/internal.h | 2 src/libpcp/src/rtime.c | 10 src/pmdas/rpm/GNUmakefile | 3 src/pmdas/rpm/migrate.conf | 18 src/pmdas/rpm/rpm.c | 12 src/pmdas/rpm/rpm.h | 16 src/python/pcp/pmapi.py | 83 +++ 18 files changed, 799 insertions(+), 625 deletions(-) commit 5f9b4d87741cf68bbb655201a604a3335fbcdb6a Author: Nathan Scott Date: Tue Apr 15 12:44:37 2014 +1000 Test python derived metrics with archives as well commit b62db54614671a0f8e5cd18a367c7877aa6e097f Author: Nathan Scott Date: Tue Apr 15 12:34:56 2014 +1000 QA python improvements after followup discussion with scox Split the newly added pmdarpm metric log rewrite testing into a new test (751) to keep 750 simpler and self-contained. Correct a cut+paste typo in the python derived metrics support where the derived metric name was used recursively in its own definition. commit b17f96b083653930fef1398bd2536c67bda42741 Author: Tom Yearke Date: Tue Apr 15 11:05:53 2014 +1000 python/pmapi.py - free C string using pointer from outAtom.cp instead of address from outAtom.vp (fixes possible seg fault), resolve valgrind read errors commit a7dd7d5f998a5becb8d2742a38851b14f76760a4 Author: Nathan Scott Date: Tue Apr 15 10:12:48 2014 +1000 Ensure test script is executable in qa/900 commit 5be56892ab6207e20b737f0a25ecb8d5ee51bd1d Author: Stan Cox Date: Mon Apr 14 17:06:22 2014 -0400 Convert extracted C strings to python strings * pmapi.py (pmExtractValue): Improve string conversion. commit c83c0dd4a9dd758f78f83998b680c8eddfe732c9 Author: Stan Cox Date: Thu Apr 10 16:30:17 2014 -0400 Add pmRegisterDerived, pmLoadDerivedConfig, pmDerivedErrStr to python bindings. * pmapi.py (pmRegisterDerived, pmLoadDerivedConfig, pmDerivedErrStr): New. * test_pcp.python (test_pcp): Test it. commit 5f0f9504bd1cfbe7a6832fa1e3b7f9adbe875f98 Author: Stan Cox Date: Wed Apr 9 13:43:31 2014 -0400 Convert extracted C strings to python strings * pmapi.py (pmExtractValue): Convert C strings to python strings. * test_pcp.python (test_pcp): Test it. commit 9ae592498404a96a0ea5707163761ba0c7ab02db Author: Stan Cox Date: Fri Mar 28 14:53:30 2014 -0400 Refine datetime test results a bit commit 221b0372e09290c27c319474145b23400b0f5dc6 Author: Stan Cox Date: Wed Mar 26 16:56:35 2014 -0400 Set rpm.size from RPMTAG_LONGSIZE * rpm.c (metrictab, rpm_extract_metadata): Set rpm.size from RPMTAG_LONGSIZE * rpm.h (metadata): Rename size to longsize, same name as rpm tag. * (qa/750, qa/750.out): Test pmlogrewrite of rpm/migrate.conf * migrate.conf: Only migrate rpm.size commit a2fac2ac198287dcf2b46cecfb344e3ebe6a0668 Author: Stan Cox Date: Thu Mar 13 16:15:45 2014 -0400 Bump rpm numeric metric fields to 64 bits; migrate the logs. * rpm/migrate.conf: New file. * rpm/GNUmakefile: Install migrate.conf * rpm/(rpm.h, rpm.c): Bump rpm numeric metric fields to 64 bits. From nscott@redhat.com Tue Apr 15 00:40:02 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B5DEB7F37 for ; Tue, 15 Apr 2014 00:40:02 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 3F47EAC002 for ; Mon, 14 Apr 2014 22:39:59 -0700 (PDT) X-ASG-Debug-ID: 1397540394-04cbb06e9b2c6e0001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id jFhEgKiufG9IDa3P for ; Mon, 14 Apr 2014 22:39:54 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3F5dnmZ026351; Tue, 15 Apr 2014 01:39:49 -0400 Date: Tue, 15 Apr 2014 01:39:49 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp@oss.sgi.com Message-ID: <158034809.5621684.1397540389674.JavaMail.zimbra@redhat.com> In-Reply-To: <534B4330.1060008@internode.on.net> References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> Subject: Re: [pcp] pmlogger -u questions MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pmlogger -u questions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogger -u questions Thread-Index: E/7p6T51djhmAHFVAZxUUDuBSn0z/w== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397540394 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4924 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Ken, ----- Original Message ----- > [...] > The issue is real, but I'm obviously not explaining it well enough. > [...] > Let me try with an annotated example. BTW - thanks for the detailed explanation, this got me on the same page as you now I think. > am not sure there will be QA coverage, because the issue is a design > problem, rather than a bug in implementation. OK. It sounds possible that "kill -9" on an active logger, followed by replay/logcheck may be able to induce the failures of the current code, and demonstrate quantifiable improvements. Maybe not reliable failure cases, but with sufficient iterations/entropy/gamma-rays, we should be able to build confidence in any changes. > Neither of these problems are new. But as we inch towards a unified > context of some sort, and before that pmNewContext accepting a directory > name as an argument for PM_CONTEXT_ARCHIVE it is going to become a more > pressing matter. Ayup, sounds good. -- Nathan From nscott@redhat.com Tue Apr 15 00:56:42 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 1B5BA7F37 for ; Tue, 15 Apr 2014 00:56:42 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id DF30B304053 for ; Mon, 14 Apr 2014 22:56:38 -0700 (PDT) X-ASG-Debug-ID: 1397541397-04bdf045522c890001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id d37kTMCEv6vOlxWi for ; Mon, 14 Apr 2014 22:56:37 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3F5uXJT028505; Tue, 15 Apr 2014 01:56:33 -0400 Date: Tue, 15 Apr 2014 01:56:33 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell , Martins Innus , "Frank Ch. Eigler" Cc: PCP Mailing List Message-ID: <2083518993.5624332.1397541393227.JavaMail.zimbra@redhat.com> In-Reply-To: <534C4D07.8020604@internode.on.net> References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> <534C3695.4050306@buffalo.edu> <534C4D07.8020604@internode.on.net> Subject: Re: [pcp] signal pmlogger MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] signal pmlogger Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: signal pmlogger Thread-Index: O/jpi/Wmp0CChzaTZsQyTQuwQ2NZEQ== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1397541397 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4924 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi guys, ----- Original Message ----- > On 15/04/14 05:27, Martins Innus wrote: > > ... > > Thanks for the insight Martins. > > Well this is truly interesting. Your use case is something that has not > previously been encountered in over a decade of belting PCP in > production environments. I find that both surprising and exciting. :) > I can't see a way to achieve what you want with the single pmlogger > process and the capabilities today. This is because the log mandatory > pmlc command updates the logging state for all the named metrics within > pmlogger, so what you describe is what I'd expect to happen. > > The options are: > > 1. extend pmlc and pmlogger to have an additional set of metrics that > can have duration and logging state that co-exists with the config file > ones at startup (so the state change from pmlc is independent of, not a > replacement of, the original state) ... it would be nice to have some > short hand notation for turning all of these additional metrics off in > one command and extend the "once" duration to allow N times. > ... > Given you can do 2. today, and it would take me a couple of days to get > 1. to a prototype state, I'd vote for 2. > > And in the mean time we should consider the discussion to see if we can > refine 1. into something that is general useful, robust and easy to use. > I like the direction #1 is headed also. I wonder if a concept of groups of metrics would be useful here (IOW, user defined, named groups) - so not only the one additional set. There may be different problem scenarios that someone is trying to focus on, problem A might want to catch a set of proc metrics, problem B maybe some mem state, problem C a third set ... switching on/off these groups via pmlc commands that pmie could latch onto simply via "enable setB, interval X", "disable setB" and so on, could be quite neat. cheers. -- Nathan From nscott@redhat.com Tue Apr 15 02:40:45 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id EBD7A7F3F for ; Tue, 15 Apr 2014 02:40:45 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id BDD708F8040 for ; Tue, 15 Apr 2014 00:40:42 -0700 (PDT) X-ASG-Debug-ID: 1397547640-04bdf0455434be0001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id mi1smLdOE8VfvB1t for ; Tue, 15 Apr 2014 00:40:40 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3F7edtm025852; Tue, 15 Apr 2014 03:40:39 -0400 Date: Tue, 15 Apr 2014 03:40:39 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: pcp developers Message-ID: <1862784899.5665840.1397547639744.JavaMail.zimbra@redhat.com> In-Reply-To: <20140414141732.GI14108@redhat.com> References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> <20140414141732.GI14108@redhat.com> Subject: Re: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) MIME-Version: 1.0 X-ASG-Orig-Subj: Re: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Thread-Index: 0xDy3Qc4moF8Is5QV5VenSjN58ntFQ== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1397547640 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4926 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ----- Original Message ----- > > [...] > > BTW, there are several changes in fche/for-merge that you've not put > > in the summary - any reason for that? [...] > > I considered all the for-merge bits ready to go, just didn't want > to advertise them in the context of the python work. (nor in any other context either, though) The guidelines in HACKING suggest this is a good, common methodology to follow (2nd paragraph, last sentence). Otherwise, confusion and misinterpretation-of-intent will result on my end - I am but a simple lad, easily confused. Sending the report to the list gives everyone else a good opportunity to review too (not just me all the time). Thanks for helping out with that! > > Comments inline... (I'm still hopeful Ken will review the new qa/666 > > and offer suggestions - 6 minutes is still a really, really long time > > for a test to run - its >5 minutes longer than most other tests and > > about double the next slowest test. And *that* other test is already > > a big problem IMO so I'm very reluctant to add more QA laggards). > > Fair enough, but I'm reluctant to keep working on it more without a > definite "good enough" target. As mentioned on IRC, splitting the test into smaller parts may be one strategy. For reference, the pmlogger check/daily tests are exercising similar sorts of functionality (sorta) and there are ~15 tests there, which are in the several-seconds to one-or-two-minutes-max range. Those are the kinds of timeframes and number of tests I'd hope to see for an important daemon like pmmgr. Some valgrind testing would be a welcome addition too - its long-running, makes sense to check memory use. And so on. Like all the other daemons - use those other tests as references for what is normal here. > (Of course one could reduce test coverage to save time, > but that's not dignified.) *nod* - goes without saying. Everyone loses if you do that. > > [...] > > Hmm, not so sure about this - looking inside the libpcp code, errors > > returned here are not related to the names at all, right? (they are > > runtime errors, so things like PDU I/O errors, and so on). > > In this particular case, the error parameter was for an informative > name, but the object passed back was the ctypes char* rather than the > python string, which made the error message unnecessarily opaque. Understood. My question, though, was does passing back out parameters passed in help at all here, not whether the existing code was useful. > > Passing out the names doesn't really help to understand the failure, > > AFAICT? It looks like we should just treat this one the same as all > > the other pmErr exceptions (IOW, neither pmid/names list passed out). > > [...] > > What was the error case you came across here, OOC? > > Sending bad metric names to pcp-graphite.py, either before or after > the PMNS traversal code-additions. Right - ah, I see whats happened here. I was specifically asking about pmLookupName, thinking it should be going through the second error code handling path on name lookup failure, where it already returns the names as a result of this (from pmLookupName(3))... The result from pmLookupName will be the number of names translated in the absence of errors, else an error code less than zero. When errors are encountered, the corresponding value in pmidlist will be PM_ID_NULL. I think you are seeing a case where every single name fails to resolve, right? This case (I think? hence my unanswered questions), is not distinguished from other errors like networking errors, etc. For this latter case, its probably unhelpful to misguide the user with an error message containing details irrelevant to the failure. To avoid this, the list could be conditionally appended if PM_ERR_NAME is the error. Could you try that out with your test case? (and please make a qa test addition to exercise these cases - both Stan and I have been hacking in here, it appears to be a problematic area). Thanks! Beware though - it could be a really long list of names - are we sure we want to do this? (may result in potentially long list of names that we dump on stderr, typically). Not sure, not sure at all. > > [...] > > > commit 163db2e7ae96437a33ca8a0535af8bde35e95a34 > > > Author: Frank Ch. Eigler > > > Date: Sat Apr 12 21:46:51 2014 -0400 > > > > > > libpcp __pmSetProgname: copy incoming string > > [...] > > > > Wow, amazed that I have not ever observed this! Do any/all of > > tests 739, 741, 742, 743, 991 fail for you? [...] > > No. OK - so we have no tests that demonstrate the problem, on any python version / test platforms? That should be addressed. > > But lets be more objective - all the people using valgrind on C > > PMAPI tools will gnash their teeth with this change, as they will > > start to get warnings now. [...] If we were to make this kind of > > change the qa/valgrind-suppressions file would need to be updated. > > Did you see an actual warning from this change? For programs that > call __pmSetProgname only once (as is typical), there is no "leak" > in that a global variable maintains a reference to the object. I > can't get valgrind to even mention it. Nope, this assertion was based on the way pmGetConfig behaves and is known to trigger valgrind warnings. But on review, that is slightly different and the permanent pointer reference will indeed keep any valgrind warnings at bay - you're right! (sorry, my mistake) > > Does the patch attached resolve the problem you observed? > > I'm not sure it's sufficient, and IMHO the patch is worse So ... you didn't try it out for me on your test case? (please do & let me know, and could you add the python test case to the testsuite please? this discussion is made more difficult because we're talking about abstract instead of concrete test cases). libpcp has taken this pragmatic approach for so many years - I think its just fine. There are no performance tools that I've come across that would ever want to change their argv[0] after starting. So, I'm going for "practicality beats purity" and limiting the scope of the change, particularly as we are so close to a release. > [...] > pcp.sh: add Available Commands: list to usage message, like already > documented > > I see that you have your own version in /dev now. FWIW, I like this > one better because its logic matches that of what is done later in > pcp.sh, namely the ordering and executability checking, and does > not litter in $tmp. YMMV, HTH, HAND, & c. > *nod* - I really like the way your variant checks the executable bits, I've observed that problem as well. I'm not seeing value in ensuring the path evaluation order is maintained from elsewhere - keeping the list sorted makes it easier to find things in it. The other version also neatly formats the output over multiple lines, handy as the list grows also. This script already uses numerous tmpfiles and cleans 'em up already, so I wouldn't worry too much about that. So in the end, I've taken another look though and have attempted to produce a best-of-both-worlds approach for this release. cheers. -- Nathan From nscott@redhat.com Tue Apr 15 02:44:35 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C7D7D7F3F for ; Tue, 15 Apr 2014 02:44:35 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id B380E8F8052 for ; Tue, 15 Apr 2014 00:44:35 -0700 (PDT) X-ASG-Debug-ID: 1397547874-04cbb06e9b35e60001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id WvZrMENePO6RHo5O for ; Tue, 15 Apr 2014 00:44:34 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3F7iXG7006284 for ; Tue, 15 Apr 2014 03:44:34 -0400 Date: Tue, 15 Apr 2014 03:44:33 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp developers Message-ID: <1145066309.5666647.1397547873972.JavaMail.zimbra@redhat.com> In-Reply-To: <493273703.5666148.1397547717194.JavaMail.zimbra@redhat.com> Subject: pcp updates: last few 3.9.2 tidbits, hopefully MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: last few 3.9.2 tidbits, hopefully Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: last few 3.9.2 tidbits, hopefully Thread-Index: Z7rQjhHMYdomEJmhcyi7OGSJsJxJyg== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1397547874 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4926 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://oss.sgi.com/pcp/pcp.git dev CHANGELOG | 39 +++++++++++++++++++++++++++++++++++++-- build/rpm/fedora.spec | 6 ++++-- debian/changelog | 2 +- src/pcp/pcp.sh | 11 +++++++---- src/python/pmapi.c | 11 ++++++++++- 5 files changed, 59 insertions(+), 10 deletions(-) commit b48f3c54cf913f670297a2a78bc27dbc875c11f3 Author: Nathan Scott Date: Tue Apr 15 17:40:30 2014 +1000 Update changelogs and administrivia for release 3.9.2 commit d16d56eea934b655d7d738c3eff49c1273d5cee4 Author: Nathan Scott Date: Tue Apr 15 16:38:11 2014 +1000 python api: ensure the memory backing pmProgname persists commit bfaeacae9fbcdc10214c729687da68fdeb17c6cb Author: Nathan Scott Date: Tue Apr 15 15:29:22 2014 +1000 Implement command permission checking for pcp(1) usage Follow up on Franks idea of checking that commands have an appropriate mode before adding them to the list of available commands that pcp(1) usage message will report. From minnus@buffalo.edu Tue Apr 15 08:28:08 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 9F45C7F3F for ; Tue, 15 Apr 2014 08:28:08 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 6385E8F8037 for ; Tue, 15 Apr 2014 06:28:05 -0700 (PDT) X-ASG-Debug-ID: 1397568480-04bdf045524cf30001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id qb2GaFaD2tGNnw1u for ; Tue, 15 Apr 2014 06:28:00 -0700 (PDT) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailD.acsu.buffalo.edu (localmaild.acsu.buffalo.edu [128.205.5.208]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 2AFB7A2E; Tue, 15 Apr 2014 09:28:00 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 2028610349; Tue, 15 Apr 2014 09:28:00 -0400 (EDT) Received: from localmailD.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailD.acsu.buffalo.edu (Postfix) with ESMTP id 63CF91033E; Tue, 15 Apr 2014 09:27:59 -0400 (EDT) Received: from smtp.buffalo.edu (smtp4.acsu.buffalo.edu [128.205.5.229]) by localmailD.acsu.buffalo.edu (Prefixe) with ESMTP id 50B161033D; Tue, 15 Apr 2014 09:27:59 -0400 (EDT) Received: from gilmour.ccr.buffalo.edu (gilmour.ccr.buffalo.edu [128.205.40.13]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 4162CAA8A; Tue, 15 Apr 2014 09:27:59 -0400 (EDT) Message-ID: <534D33DE.4050807@buffalo.edu> Date: Tue, 15 Apr 2014 09:27:58 -0400 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Ken McDonell , "'Frank Ch. Eigler'" CC: 'PCP Mailing List' Subject: Re: [pcp] signal pmlogger References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> <534C3695.4050306@buffalo.edu> <534C4D07.8020604@internode.on.net> X-ASG-Orig-Subj: Re: [pcp] signal pmlogger In-Reply-To: <534C4D07.8020604@internode.on.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397568480 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4934 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Ken, >> >> I can go down that path if that is the right way to do it. But I was >> hoping for some sort of "log this list of metrics right now, just once, >> but don't otherwise change pmlogger" command. > > Thanks for the insight Martins. > > Well this is truly interesting. Your use case is something that has > not previously been encountered in over a decade of belting PCP in > production environments. I find that both surprising and exciting. > Ha! > I can't see a way to achieve what you want with the single pmlogger > process and the capabilities today. This is because the log mandatory > pmlc command updates the logging state for all the named metrics > within pmlogger, so what you describe is what I'd expect to happen. > > The options are: > > 1. extend pmlc and pmlogger to have an additional set of metrics that > can have duration and logging state that co-exists with the config > file ones at startup (so the state change from pmlc is independent of, > not a replacement of, the original state) ... it would be nice to have > some short hand notation for turning all of these additional metrics > off in one command and extend the "once" duration to allow N times. > > 2. Have your script that is currently calling pmlc actually launch a > _new_ pmlogger to capture just the metrics you want at the start or > end of the computation or with some particular frequency during the > computation. Then at the end of the day, you can use pmlogextract to > merge together the archive from the default system logging (or > whatever pmlogger you're pointing pmlc at) with one or more archives > from your computation runs. The merged archive will be the same as if > 1. was implemented. Thanks for the ideas. I'll work on 2 for now and test 1 as it evolves. Thanks Martins From minnus@buffalo.edu Tue Apr 15 08:31:46 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 745197F3F for ; Tue, 15 Apr 2014 08:31:46 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 0059FAC002 for ; Tue, 15 Apr 2014 06:31:42 -0700 (PDT) X-ASG-Debug-ID: 1397568700-04cb6c24384a310001-S8gJnT Received: from mtareserve1.acsu.buffalo.edu (mtareserve12.acsu.buffalo.edu [128.205.6.33]) by cuda.sgi.com with ESMTP id 4Fqt11F7SXRQFPz8 for ; Tue, 15 Apr 2014 06:31:40 -0700 (PDT) X-Barracuda-Envelope-From: minnus@buffalo.edu X-Barracuda-Apparent-Source-IP: 128.205.6.33 Received: from localmailA.acsu.buffalo.edu (localmaila.acsu.buffalo.edu [128.205.5.196]) by mtareserve1.acsu.buffalo.edu (Postfix) with ESMTP id 0E2E3A9A; Tue, 15 Apr 2014 09:31:40 -0400 (EDT) Received: from localmailA.acsu.buffalo.edu (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 08122F8A9; Tue, 15 Apr 2014 09:31:40 -0400 (EDT) Received: from localmailA.acsu.buffalo.edu (localhost [127.0.0.1]) by localmailA.acsu.buffalo.edu (Postfix) with ESMTP id 7879AF8A5; Tue, 15 Apr 2014 09:31:39 -0400 (EDT) Received: from smtp.buffalo.edu (smtp2.acsu.buffalo.edu [128.205.5.254]) by localmailA.acsu.buffalo.edu (Prefixe) with ESMTP id 43940F8A4; Tue, 15 Apr 2014 09:31:39 -0400 (EDT) Received: from gilmour.ccr.buffalo.edu (gilmour.ccr.buffalo.edu [128.205.40.13]) (Authenticated sender: minnus@buffalo.edu) by smtp.buffalo.edu (Postfix) with ESMTPSA id 14F059BD6; Tue, 15 Apr 2014 09:31:39 -0400 (EDT) Message-ID: <534D34BA.8030708@buffalo.edu> Date: Tue, 15 Apr 2014 09:31:38 -0400 From: Martins Innus User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott , Ken McDonell , "Frank Ch. Eigler" CC: PCP Mailing List Subject: Re: [pcp] signal pmlogger References: <53482BC5.6070203@buffalo.edu> <01c401cf55c9$d4322fc0$7c968f40$@internode.on.net> <534C3695.4050306@buffalo.edu> <534C4D07.8020604@internode.on.net> <2083518993.5624332.1397541393227.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] signal pmlogger In-Reply-To: <2083518993.5624332.1397541393227.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-PM-EL-Spam-Prob: : 8% X-Barracuda-Connect: mtareserve12.acsu.buffalo.edu[128.205.6.33] X-Barracuda-Start-Time: 1397568700 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4934 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Nathan, On 4/15/14 1:56 AM, Nathan Scott wrote: >> >> >> And in the mean time we should consider the discussion to see if we can >> refine 1. into something that is general useful, robust and easy to use. >> > I like the direction #1 is headed also. > > I wonder if a concept of groups of metrics would be useful here (IOW, user > defined, named groups) - so not only the one additional set. There may be > different problem scenarios that someone is trying to focus on, problem A > might want to catch a set of proc metrics, problem B maybe some mem state, > problem C a third set ... switching on/off these groups via pmlc commands > that pmie could latch onto simply via "enable setB, interval X", "disable > setB" and so on, could be quite neat. > Yeah, multiple named groups would be great. Martins From fche@redhat.com Tue Apr 15 11:51:53 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 688737F3F for ; Tue, 15 Apr 2014 11:51:53 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id ED289AC002 for ; Tue, 15 Apr 2014 09:51:52 -0700 (PDT) X-ASG-Debug-ID: 1397580707-04cb6c24375d6c0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id TzyTjWB5PebQzIGK for ; Tue, 15 Apr 2014 09:51:48 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3FGpln5010350 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 15 Apr 2014 12:51:47 -0400 Received: from fche.csb (vpn-55-53.rdu2.redhat.com [10.10.55.53]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3FGple1031675; Tue, 15 Apr 2014 12:51:47 -0400 Received: by fche.csb (Postfix, from userid 2569) id AB92F58114; Tue, 15 Apr 2014 12:51:46 -0400 (EDT) Date: Tue, 15 Apr 2014 12:51:46 -0400 From: "Frank Ch. Eigler" To: Nathan Scott Cc: pcp developers Subject: Re: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Message-ID: <20140415165146.GN14108@redhat.com> X-ASG-Orig-Subj: Re: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> <20140414141732.GI14108@redhat.com> <1862784899.5665840.1397547639744.JavaMail.zimbra@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1862784899.5665840.1397547639744.JavaMail.zimbra@redhat.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397580708 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi - On Tue, Apr 15, 2014 at 03:40:39AM -0400, Nathan Scott wrote: > [...] > > Fair enough, but I'm reluctant to keep working on [666] more without a > > definite "good enough" target. > > As mentioned on IRC, splitting the test into smaller parts may be one > strategy. [...] I see, so the problem is that the single test case takes 6 minutes, but you'd be fine it if broken into three test cases they took 2 minutes each? > [...] > To avoid this, the list could be conditionally appended if PM_ERR_NAME > is the error. Could you try that out with your test case? [...] > Beware though - it could be a really long list of names - are we sure > we want to do this? (may result in potentially long list of names that > we dump on stderr, typically). Not sure, not sure at all. There is no requirement that the pmErr exception object produce a super-long textual report. The __str__ method could abbreviate. > > > Wow, amazed that I have not ever observed this! Do any/all of > > > tests 739, 741, 742, 743, 991 fail for you? [...] > > > > No. > > OK - so we have no tests that demonstrate the problem, on any python > version / test platforms? That should be addressed. Sure, but that's simply because those test cases don't print the usage text, so they don't notice any corruption of the mistakenly-assumed-stable pmProgname char*. > > > Does the patch attached resolve the problem you observed? > > > > I'm not sure it's sufficient, and IMHO the patch is worse > > So ... you didn't try it out for me on your test case? (please do & > let me know, and could you add the python test case to the testsuite > please? this discussion is made more difficult because we're talking > about abstract instead of concrete test cases). Sorry. The problem seemed perfectly concrete, small, and was incidental to the development going on elsewhere so I didn't bother create a stripped-down test case at the time. > libpcp has taken this pragmatic approach for so many years - I think > its just fine. There are no performance tools that I've come across > that would ever want to change their argv[0] after starting. So, I'm > going for "practicality beats purity" and limiting the scope of the > change, particularly as we are so close to a release. FWIW, I see nothing "impractical" about strdup, which your own patch does in another context, but opinions vary so. Anyway, if we are to stay with this assumption, then it should be documented so if anybody's future PCP application uses PMAPI and decides to change their argv[] and get corrupted error and usage messages or logfile names, we can throw the book at them. > [...] So in the end, I've taken another look though and have > attempted to produce a best-of-both-worlds approach for this > release. (IMHO, this much discussion & revision is out of proportion with one small working patch.) - FChE From pcp-announce-bounces@oss.sgi.com Tue Apr 15 20:30:37 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from oss.sgi.com (localhost [IPv6:::1]) by oss.sgi.com (Postfix) with ESMTP id 854927F50; Tue, 15 Apr 2014 20:30:37 -0500 (CDT) X-Original-To: pcp-announce@oss.sgi.com Delivered-To: pcp-announce@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 816B97F3F for ; Tue, 15 Apr 2014 20:30:35 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 5BDB5304082 for ; Tue, 15 Apr 2014 18:30:35 -0700 (PDT) X-ASG-Debug-ID: 1397611829-04cb6c24396e4b0001-87ZIJf Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id XAKVqrLpAMIsiJbM for ; Tue, 15 Apr 2014 18:30:29 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3G1UT3k018583 for ; Tue, 15 Apr 2014 21:30:29 -0400 Date: Tue, 15 Apr 2014 21:30:29 -0400 (EDT) From: Nathan Scott To: pcp-announce@oss.sgi.com Message-ID: <1991451814.6172545.1397611829159.JavaMail.zimbra@redhat.com> In-Reply-To: <2028330622.6169168.1397610820758.JavaMail.zimbra@redhat.com> MIME-Version: 1.0 X-ASG-Orig-Subj: Performance Co-Pilot pcp-3.9.2 release X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: Performance Co-Pilot pcp-3.9.2 release Thread-Index: JsITgRzn/iGh8DRmrKb+CPRTeFeL4A== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1397611829 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-BRTS-Evidence: performancecopilot.org X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4954 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Subject: [pcp-announce] Performance Co-Pilot pcp-3.9.2 release X-BeenThere: pcp-announce@oss.sgi.com X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Nathan Scott List-Id: pcp announcements List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pcp-announce-bounces@oss.sgi.com Sender: pcp-announce-bounces@oss.sgi.com Hi all, The latest version of PCP has been tagged and uploaded to all the usual locations. This release comes with a shiny new URL for the project: http://www.performancecopilot.org/ Many thanks to the efforts of the oss.sgi.com sysadmins who made it a reality, Russell and Trev! This release has a distinctly pythonic flavour. Special thanks to first-time contributor Tom Yearke, hacking in that area too. An interface allowing slow PMDAs to avoid the initial timeout with pmcd(1) is introduced, improvements to the RPM PMDA, new tools, the next batch of long options updates. Almost too many changes to list ... pcp-3.9.2 (15 April 2014) - python api: direct support for creating derived metrics - python api: fix C strings memory leak in pmExtractValue - python api: fix pmConvScale parameterization - python api: pmLookupName API mode allowing partial failure - python api: improvements to the pmErr exceptions class - python api: support auto-command-line-parsing in scripts - python api: switch to thread-safe pmGetContextHostname - pcp: support for scripted pcp(1) child commands, adding in pcp-free(1), pcp-uptime(1) and pcp-numastat(1) to get the ball rolling (python scripts) - pmlogmv: new utility to atomically move/rename archives - pmconfig: improvements to quoting for unusual versions - pmdaproc.sh: allow Install/Remove from any directory - libpcp: pmgetopt_r interface for use by collector tools - libpcp_pmda: slow-start PMDA changes - libpcp_pmda: fix a memory leak dealing in dynamic metrics, with PMDAs using the optional hashed metric table method. - libpcp_pmda: long option command line processing interface - libpcp_pmda: handle POSIXLY_CORRECT arguments internally - pmcd: remove POSIXLY_CORRECT env modifications - dbpmda: remove POSIXLY_CORRECT env modifications - pmlogger_check: add a no-merging-renaming-rewriting option - pmlogger_daily: don't merge archives if it is not needed (optimization) - perl pmda api: add documentation for PCP::PMDA interfaces - pmdalinux: fix a memory leak in cpu:node name resolution - pmmgr: fix daemon invocation quirk for some sh variants - pmdarpm: improvements to concurrent rpmdb access - pmdarpm: rpm.size metric now 64bit, matching rpmdb changes - pmdaproc.sh: add $perl_args and $python_args - long command line options support: pmdamailq, pmns utilities, newhelp, pmcd, pmcd_wait, pmcpp, pmdate, pmdbg, pmerr, pmhostname, pmieconf, collectl2pcp, pmlc, pmmgr, pmproxy, pmwebd. - Makepkgs changes to support source tarball builds via git - HACKING file added, describing PCP development methodology Enjoy! -- Nathan _______________________________________________ pcp-announce mailing list pcp-announce@oss.sgi.com http://oss.sgi.com/mailman/listinfo/pcp-announce From nscott@redhat.com Tue Apr 15 22:22:06 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 9CA847F3F for ; Tue, 15 Apr 2014 22:22:06 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 337ECAC001 for ; Tue, 15 Apr 2014 20:22:03 -0700 (PDT) X-ASG-Debug-ID: 1397618518-04cb6c2436712c0001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id AiiHx74KTlyWlJ6l for ; Tue, 15 Apr 2014 20:21:58 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3G3Lv21006924; Tue, 15 Apr 2014 23:21:57 -0400 Date: Tue, 15 Apr 2014 23:21:57 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: pcp developers Message-ID: <1213182091.6235824.1397618517836.JavaMail.zimbra@redhat.com> In-Reply-To: <20140415165146.GN14108@redhat.com> References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> <20140414141732.GI14108@redhat.com> <1862784899.5665840.1397547639744.JavaMail.zimbra@redhat.com> <20140415165146.GN14108@redhat.com> Subject: Re: fche/for-merge updates MIME-Version: 1.0 X-ASG-Orig-Subj: Re: fche/for-merge updates Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: fche/for-merge updates Thread-Index: VxBcIL5X8k7Uc9CNmIry9kf3qol4Uw== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1397618518 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4957 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ----- Original Message ----- > [...] I didn't bother create a stripped-down test case at the time. Nor when asked repeatedly, nor now, nor tomorrow I'd imagine. While trying to get these patches merged I got a vibe less "collaboration" and more "steamroller". Let's continue to work on that - it's a two way street, and the tests are very near and dear to me. > (IMHO, this much discussion & revision is out of proportion [...] I'm sorry you feel that way. I was trying to manage risk, and explain what I was doing as clearly as possible, with a number of changes that arrived unexpectedly in the lead up to a release. I think we both have very different ideas about being conservative and managing risk. The API breaking change, the lack of testing, a change to a global variable affecting every PCP program, ever - for me, these things represent high risk. But we can manage those differences by - ensuring planned release dates are well known in advance; (tick) - whenever in doubt, or asked, provide test cases showing problems; - continue refining HACKING guidelines to keep our thinking in sync; - reviewing and merging each others code, and merging to dev early on in each release cycle. Let's do it better, together, next time. On that note, I'm away for several days, recharging the batteries - see y'all in about a week! cheers. -- Nathan From fche@redhat.com Wed Apr 16 19:11:54 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2806C7F4E for ; Wed, 16 Apr 2014 19:11:54 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id E4D6130408B for ; Wed, 16 Apr 2014 17:11:50 -0700 (PDT) X-ASG-Debug-ID: 1397693506-04bdf04554c94a0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id WXomooVo25jUDOGD for ; Wed, 16 Apr 2014 17:11:46 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3H0Bh5p012819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 16 Apr 2014 20:11:43 -0400 Received: from fche.csb (vpn-61-243.rdu2.redhat.com [10.10.61.243]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3H0BgOa016937; Wed, 16 Apr 2014 20:11:43 -0400 Received: by fche.csb (Postfix, from userid 2569) id 4415D581C7; Wed, 16 Apr 2014 20:11:42 -0400 (EDT) To: Ken McDonell Cc: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogrewrite questions from the developer meeting References: <01c501cf56de$54b6d370$fe247a50$@internode.on.net> <379318588.4720106.1397436550627.JavaMail.zimbra@redhat.com> <534C66BB.6050206@internode.on.net> X-ASG-Orig-Subj: Re: pmlogrewrite questions from the developer meeting From: fche@redhat.com (Frank Ch. Eigler) Date: Wed, 16 Apr 2014 20:11:42 -0400 In-Reply-To: <534C66BB.6050206@internode.on.net> (Ken McDonell's message of "Tue, 15 Apr 2014 08:52:43 +1000") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397693506 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - kenj wrote: > [...] So before exit(), you're sugesting fsync() on each of the > data files, and I think I need fsync() on the container directory as > well. One should do it not just before program exit, but earlier, just before the unlink of the input files. That way, we have some guarantees that the output file will be on disk before the input files get nuked. (If we were to fsync only at the exit(), then the unlinks could be flushed to disk quickly by the kernel, the new files might not be flushed fully yet, and a crash at this point would lose all copies of the data.) > But we should be consistent. If this is the "right" way to do it > then surely all applications that can write PCP archives should do > the same thing. (That was what I proposed with the fche/fsync branch.) > I am not against doing this, although if one was concerned at this > level then I suspect an option to enforce O_SYNC might be better to > guarantee on disk for all writes, not just flushing everything at > exit, but we should choose one policy for writing PCP archives and > implement it consistently throughout the PCP ecosystem. There is a spectrum, but yes, toolset-wide consistent aim at a point on the spectrum sounds appropriate. 1) a mode where archive data gets fsync()d (or O_SYNC'd) at every incremental write (from pmlogger), so that all times are durable and any time is good time for crashing. This of course means accepting potentially more i/o latency. 2a) a mode where archive data gets fsync()d at close (from all bulk archive creation tools), so that process exits correspond to durable states. 2b) a mode where archive data gets fsync()d from only a few archive creation tools deemed special. not-3) a mode where archive data gets fsync()d asynchronously ( receiving a SIGfoo or pmlc flush), except that this doesn't provide any synchronization (completion information) back, so doesn't identify a durability point; a client might as well system("/bin/sync"); 3) a mode where we we don't care about crash-robustness (status quo) Perhaps this deserves to be a pcp.conf level option. - FChE From irem.hazar@yonetici-asistanligi.com Wed Apr 16 23:13:40 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id CF7BF7F56 for ; Wed, 16 Apr 2014 23:13:40 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 985858F8054 for ; Wed, 16 Apr 2014 21:13:40 -0700 (PDT) X-ASG-Debug-ID: 1397708014-04bdf04554d6770001-S8gJnT Received: from we210.intomaniaser.com (we210.intomaniaser.com [176.53.45.210]) by cuda.sgi.com with ESMTP id RoCGh9tvLWvjS5JU for ; Wed, 16 Apr 2014 21:13:35 -0700 (PDT) X-Barracuda-Envelope-From: irem.hazar@yonetici-asistanligi.com X-Barracuda-Apparent-Source-IP: 176.53.45.210 Received: by we130.intomaniaser.com (PowerMTA(TM) v3.5r16) id h9ta960mnfgu for ; Thu, 17 Apr 2014 06:59:49 +0300 (envelope-from ) From: "=?utf-8?B?xLByZW0gSGF6YXI=?=" Subject: =?utf-8?B?U2VydGlmaWthbMSxIFnDtm5ldGljaSBBc2lzdGFubMSxxJ/EsSBFxJ9pdGltIFByb2dyYW3EsSBIay4=?= To: "pcp" X-ASG-Orig-Subj: =?utf-8?B?U2VydGlmaWthbMSxIFnDtm5ldGljaSBBc2lzdGFubMSxxJ/EsSBFxJ9pdGltIFByb2dyYW3EsSBIay4=?= Content-Type: multipart/alternative; boundary="afuXHK=_xeUtOYWIcYkp23ihDmlQjGUIbd" MIME-Version: 1.0 Organization: iremhaz Date: Thu, 17 Apr 2014 07:01:27 +0300 Message-ID: <0.0.E.EB2.1CF59F17A4F9EDA.0@we130.intomaniaser.com> X-Barracuda-Connect: we210.intomaniaser.com[176.53.45.210] X-Barracuda-Start-Time: 1397708014 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.4992 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 HTML_MESSAGE BODY: HTML included in message This is a multi-part message in MIME format --afuXHK=_xeUtOYWIcYkp23ihDmlQjGUIbd Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline =EF=BB=BFMerhaba, =20 Her ay yo=C4=9Fun ilgi g=C3=B6ren =E2=80=9CSertifikal=C4=B1 Y=C3=B6net= ici Asistanl=C4=B1=C4=9F=C4=B1 E=C4=9Fitim Program=C4=B1=E2=80=9D 19 N= isan 2014 tarihinde Sn. Mustafa =C5=9Eahin=E2=80=99in ak=C4=B1lda kal=C4= =B1c=C4=B1 anlat=C4=B1m=C4=B1yla yeniden d=C3=BCzenlenecektir. San=C4=B1= lan=C4=B1n aksine, bir =C5=9Firketin i=C5=9Fleyi=C5=9Finde =C3=B6nemli= bir role sahip olan y=C3=B6netici asistanl=C4=B1=C4=9F=C4=B1na ait p=C3= =BCf noktalar=C4=B1n=C4=B1 yakalayabilece=C4=9Finiz bu e=C4=9Fitim pro= gram=C4=B1n=C4=B1 ka=C3=A7=C4=B1rmay=C4=B1n. Program hakk=C4=B1nda det= ayl=C4=B1 bilgi almak ve online kay=C4=B1t ger=C3=A7ekle=C5=9Ftirmek i= =C3=A7in buraya t=C4=B1klayabilir veya 0216 334 45 40 =E2=80=93 0850 2= 25 61 30 numaral=C4=B1 telefonlardan bize ula=C5=9Fabilirsiniz. =20 Programa ait i=C3=A7erik a=C5=9Fa=C4=9F=C4=B1da bilginize sunulmu=C5=9F= tur Y=C3=B6netici Asistanl=C4=B1=C4=9F=C4=B1 Nedir ? Y=C3=B6netici Asistanl=C4=B1=C4=9F=C4=B1 ve Organizasyon =C4=B0li=C5=9F= kisi Y=C3=B6netici Asistanlar=C4=B1 ile Y=C3=B6neticilerin ve =C3=9Cst Y=C3= =B6netimin =C3=87al=C4=B1=C5=9Fma =C4=B0li=C5=9Fkisi Ofis Y=C3=B6netimi ve Organizasyon Randevu, Seyahat ve Toplant=C4=B1lar=C4=B1n Organize Edilmesi ve Plan= lanmas=C4=B1 Y=C3=B6netici Asistanl=C4=B1=C4=9F=C4=B1nda Yetki Devri ve =C4=B0nisiy= atif Kullanabilme Kurum =C4=B0=C3=A7i =C4=B0leti=C5=9Fim ve =C4=B0leti=C5=9Fim Engelleri= ni A=C5=9Fma: Bilgilendirme, Dinleme, Planlama Y=C3=B6netici Asistanlar=C4=B1 i=C3=A7in Telefonda =C4=B0leti=C5=9Fimi= n =C3=96nemi ve =C4=B0leti=C5=9Fim Becerilerini Artt=C4=B1rma Dinleme Becerileri ve Soru Sorma Teknikleri Beden Dilinin G=C3=BCc=C3=BC: Jestler ve Mimikler, Ki=C5=9Fileraras=C4= =B1 Mesafe Toplant=C4=B1 Y=C3=B6netimi, Zaman Y=C3=B6netimi ve Planlama *** E=C4=9Fitim vaka =C3=A7al=C4=B1=C5=9Fmalar=C4=B1 ve uygulama test= leri ile desteklenecektir. =20 Yakla=C5=9Fan Etkinliklerimiz: [ 25/04/2014 ] M=C3=BC=C5=9Fteri Memnuniyeti Maksimizasyonu [ 26/04/2014 ] Excel'de Finans Uygulamalar=C4=B1 E=C4=9Fitimi [ 26/04/2014 ] Telefonda Etkili =C4=B0leti=C5=9Fim Becerileri [ 26/04/2014 ] =C4=B0=C5=9F Ya=C5=9Fam=C4=B1nda =C3=96fke ve Stres Kon= trol=C3=BC =20 =C4=B0yi =C3=A7al=C4=B1=C5=9Fmalar, =20 Y.E.D.=C4=B0.T.E.P.E A.K.A.D.E.M.=C4=B0 =C4=B0rem Hazar | Pazarlama G=C3=BCzeltepe Cad. Suyolu =C3=87=C4=B1kmaz=C4=B1 No:2 =C3=87engelk=C3=B6y / =C3=9Csk=C3=BCdar / =C4=B0stanbul T: 0216 334 45 40 / 0850 225 61 30 G: 0533 041 37 91 --afuXHK=_xeUtOYWIcYkp23ihDmlQjGUIbd Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline =EF=BB=BF

Merhaba,
&nbs= p;
Her ay yo=C4=9Fun ilgi g=C3=B6ren =E2=80=9CSertifikal=C4= =B1 Y=C3=B6netici Asistanl=C4=B1=C4=9F=C4=B1 E=C4=9Fitim Program=C4=B1= =E2=80=9D 19 Nisan 2014 tarihinde Sn= =2E Mustafa =C5=9Eahin=E2=80=99in ak=C4=B1lda kal=C4=B1c=C4=B1= anlat=C4=B1m=C4=B1yla yeniden d=C3=BCzenlenecektir. San=C4=B1lan=C4=B1= n aksine, bir =C5=9Firketin i=C5=9Fleyi=C5=9Finde =C3=B6nemli bir role= sahip olan y=C3=B6netici asistanl=C4=B1=C4=9F=C4=B1na ait p=C3=BCf no= ktalar=C4=B1n=C4=B1 yakalayabilece=C4=9Finiz bu e=C4=9Fitim program=C4= =B1n=C4=B1 ka=C3=A7=C4=B1rmay=C4=B1n. Program hakk=C4=B1nda d= etayl=C4=B1 bilgi almak ve online kay=C4=B1t ger=C3=A7ekle=C5=9Ftirmek= i=C3=A7in buraya t=C4=B1klayabilir veya 0216 334 45 40 =E2=80=93 0850 225 61 30 numaral=C4=B1 telefonlardan bize ula=C5=9Fabilirsiniz.
 
= Programa ait i=C3=A7erik a=C5=9Fa=C4=9F=C4=B1da bilginize sunu= lmu=C5=9Ftur
Y=C3=B6netici Asistanl=C4=B1=C4=9F=C4=B1 Nedi= r ?
Y=C3=B6netici Asistanl=C4=B1=C4=9F=C4=B1 ve Organizasyon =C4=B0= li=C5=9Fkisi
Y=C3=B6netici Asistanlar=C4=B1 ile Y=C3=B6neticilerin = ve =C3=9Cst Y=C3=B6netimin  =C3=87al=C4=B1=C5=9Fma =C4=B0li=C5=9F= kisi
Ofis Y=C3=B6netimi ve Organizasyon
Randevu, Seyahat ve Topl= ant=C4=B1lar=C4=B1n  Organize Edilmesi ve Planlanmas=C4=B1
Y=C3= =B6netici Asistanl=C4=B1=C4=9F=C4=B1nda Yetki Devri ve =C4=B0nisiyatif=   Kullanabilme
Kurum =C4=B0=C3=A7i =C4=B0leti=C5=9Fim ve =C4=B0= leti=C5=9Fim Engellerini A=C5=9Fma: Bilgilendirme, Dinleme, PlanlamaY=C3=B6netici Asistanlar=C4=B1 i=C3=A7in Telefonda =C4=B0leti=C5=9Fi= min =C3=96nemi  ve =C4=B0leti=C5=9Fim Becerilerini Artt=C4=B1rma<= BR>Dinleme Becerileri ve Soru Sorma Teknikleri
Beden Dilinin G=C3=BC= c=C3=BC: Jestler ve Mimikler, Ki=C5=9Fileraras=C4=B1 Mesafe
Toplant= =C4=B1 Y=C3=B6netimi, Zaman Y=C3=B6netimi ve Planlama
*** E=C4=9Fit= im  vaka =C3=A7al=C4=B1=C5=9Fmalar=C4=B1 ve uygulama testleri&nbs= p; ile desteklenecektir.
 
Yakla=C5=9Fan Etkinlikle= rimiz:
[ 25/04/2014 ] = M=C3=BC=C5=9Fteri Memnuniyeti Maksimizasyonu
[ 26/04/2014 ] Excel'de Finans Uygulamalar=C4=B1= E=C4=9Fitimi
[ 26/04/= 2014 ] Telefonda Etkili =C4=B0leti=C5=9Fim Becerileri
[ 26/04/2014 ] =C4=B0=C5=9F Ya=C5=9Fam= =C4=B1nda =C3=96fke ve Stres Kontrol=C3=BC
 
=C4= =B0yi =C3=A7al=C4=B1=C5=9Fmalar,
 
Y.<= SPAN style=3D"FONT-SIZE: 12pt; mso-fareast-font-family: 'Times New Rom= an'; mso-ansi-language: TR; mso-fareast-language: TR; mso-bidi-languag= e: AR-SA">E.D.=C4=B0.T.E.P.E   A.K.A.D.E.M.=C4=B0= =C4=B0rem Hazar | Pazarlama
G=C3=BCzeltepe Cad. Suyolu =C3=87=C4=B1= kmaz=C4=B1 No:2
=C3=87engelk=C3=B6y / =C3=9Csk=C3=BCdar / =C4=B0sta= nbul
T: 0216 334 45 40 / 0850 225 61 30
G: 0533 041 37 91=

--afuXHK=_xeUtOYWIcYkp23ihDmlQjGUIbd-- From SRS0=9GzJRh=ZR=info.com=info@yourhostingaccount.com Thu Apr 17 12:32:12 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 79FA37F54 for ; Thu, 17 Apr 2014 12:32:12 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id EED50AC003 for ; Thu, 17 Apr 2014 10:32:11 -0700 (PDT) X-ASG-Debug-ID: 1397755926-04cb6c2439111af0001-S8gJnT Received: from mailout04.yourhostingaccount.com (mailout04.yourhostingaccount.com [65.254.254.66]) by cuda.sgi.com with ESMTP id pcXDTFCFEGJNu322 for ; Thu, 17 Apr 2014 10:32:06 -0700 (PDT) X-Barracuda-Envelope-From: SRS0=9GzJRh=ZR=info.com=info@yourhostingaccount.com X-Barracuda-Apparent-Source-IP: 65.254.254.66 Received: from mailscan07.yourhostingaccount.com ([10.1.15.7] helo=mailscan07.yourhostingaccount.com) by mailout04.yourhostingaccount.com with esmtp (Exim) id 1WaqAR-0005Nn-1l for pcp@oss.sgi.com; Thu, 17 Apr 2014 13:32:07 -0400 Received: from impout02.yourhostingaccount.com ([10.1.55.2] helo=impout02.yourhostingaccount.com) by mailscan07.yourhostingaccount.com with esmtp (Exim) id 1WaqAO-0007cA-7g; Thu, 17 Apr 2014 13:32:04 -0400 Received: from webmail03.yourhostingaccount.com ([10.1.16.3]) by impout02.yourhostingaccount.com with NO UCE id r5Y31n00D03xsMW015Y4iC; Thu, 17 Apr 2014 13:32:04 -0400 X-Authority-Analysis: v=2.0 cv=Y4J6Q2iN c=1 sm=1 a=XjXZBYodlRNFlKnqxEWOUQ==:17 a=9cW_t1CCXrUA:10 a=Q7zus9ReCAYA:10 a=BwVQVwQTPqIA:10 a=8nJEP1OIZ-IA:10 a=BqjLhEBqAAAA:8 a=_FZyd2kkAAAA:8 a=qasVEpY2AAAA:20 a=QxoRV1d26v0qjZ_5mPQA:9 a=wPNLvfGTeEIA:10 a=m01mqmSmUL0A:10 a=N9cELZsY5McA:10 a=qQATh_Prau8A:10 a=P6/a/9HhigYI3usPxuwpQA==:117 X-EN-OrigOutIP: 10.1.16.3 X-EN-IMPSID: r5Y31n00D03xsMW015Y4iC Received: from [127.0.0.1] (helo=email.powweb.com) by webmail03.yourhostingaccount.com with esmtp (Exim) id 1WaqA7-000113-LR; Thu, 17 Apr 2014 13:31:47 -0400 Received: from 41.203.67.130 (SquirrelMail authenticated user harnold@harnold.powweb.com) by email.powweb.com with HTTP; Thu, 17 Apr 2014 13:31:47 -0400 Message-ID: Date: Thu, 17 Apr 2014 13:31:47 -0400 Subject: Webmel From: "Webmel" X-ASG-Orig-Subj: Webmel User-Agent: SquirrelMail/1.4.19 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Sender: "Webmel" X-Barracuda-Connect: mailout04.yourhostingaccount.com[65.254.254.66] X-Barracuda-Start-Time: 1397755926 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.21 X-Barracuda-Spam-Status: No, SCORE=1.21 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=MISSING_HEADERS, TO_CC_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5010 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 1.21 MISSING_HEADERS Missing To: header 0.00 TO_CC_NONE No To: or Cc: header To: undisclosed-recipients:; Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux messages jusqu' ce que vous mettez niveau votre quota de courriel. Copiez le lien ci-dessous et remplissez le formulaire pour mettre niveau votre compte. https://www.formstack.com/forms/?1728681-4402ZNwJGN Administrateur systme 192.168.0.1 From kenj@internode.on.net Thu Apr 17 17:26:32 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 31D107F52 for ; Thu, 17 Apr 2014 17:26:32 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 0A70F8F8035 for ; Thu, 17 Apr 2014 15:26:28 -0700 (PDT) X-ASG-Debug-ID: 1397773584-04cb6c2439127c00001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id BgBoVUfMzbgHOu1s for ; Thu, 17 Apr 2014 15:26:24 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApQBAFNUUFN20fC0/2dsb2JhbAANTIcrvUSDEYE+gxoBAQQjFUABEAsaAgUWCwICCQMCAQIBRQYBDAEHAQGvfnajLheBKY05B4JvgUkBA65+ Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail07.adl2.internode.on.net with ESMTP; 18 Apr 2014 07:56:23 +0930 Message-ID: <53505571.6050900@internode.on.net> Date: Fri, 18 Apr 2014 08:28:01 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott , "Frank Ch. Eigler" CC: pcp@oss.sgi.com Subject: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> <20140415002952.GM14108@redhat.com> <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions In-Reply-To: <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397773584 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5018 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- I have completed the first round of this work ... as usual the final implementation is a little messier than first expected ... - changing the semantics of __pmLogPutResult to allow the trailer to be written in the same write as the header and pmResult was a little messy - unbuffered I/O for the metadata and index files is a bad idea (see note below) The testing (all QA passes, so no functional regressions) reported below shows less writes and about the same CPU burn for the typical cases ... so resource wise a break even, but we get the better semantics. For the extreme small logging config case there are a lot more writes and a small increase in CPU burn ... both as a consequence of the external log record being much smaller than the stdio buffer. The results are less dramatic than I had expected because any change in the metadata and the emitting of an index record forces the data stream to be flushed. Test cases 10,000 samples at 10msec intervals. short - sample.colour default - pmlogconf generated default.config long - all linux metrics (not proc) plus sampledso new I/O + unbuffered writes for the data file, one write per pmResult (includes header, pmResult and trailer) + buffered writes for the metadata and the index (these have always been fflush() synced with the data file, and making them unbuffered adds CPU time and exposes fragmented logical records externally ... so a loss+loss situation) CPU #writes Avg write size (sec) (bytes) short - 3.9.2 1.15 142 3946 short - new I/O 1.30 10004 56 default - 3.9.2 3.77 20001 5623 default - new I/O 3.73 10004 11571 long - 3.9.2 26.5 20001 20824 long - new I/O 26.9 10004 42409 ~ From fche@redhat.com Thu Apr 17 17:51:59 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 44E9B7F52 for ; Thu, 17 Apr 2014 17:51:59 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 160CE304066 for ; Thu, 17 Apr 2014 15:51:55 -0700 (PDT) X-ASG-Debug-ID: 1397775114-04cb6c2438128f00001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Et4TJDM6IlNTL5QA for ; Thu, 17 Apr 2014 15:51:55 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3HMppcC026617 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 17 Apr 2014 18:51:51 -0400 Received: from fche.csb (vpn-61-243.rdu2.redhat.com [10.10.61.243]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3HMppK0029021; Thu, 17 Apr 2014 18:51:51 -0400 Received: by fche.csb (Postfix, from userid 2569) id AA25858259; Thu, 17 Apr 2014 18:51:50 -0400 (EDT) Date: Thu, 17 Apr 2014 18:51:50 -0400 From: "Frank Ch. Eigler" To: Ken McDonell Cc: Nathan Scott , pcp@oss.sgi.com Subject: Re: pmlogger -u questions Message-ID: <20140417225150.GA8368@redhat.com> X-ASG-Orig-Subj: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> <20140415002952.GM14108@redhat.com> <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> <53505571.6050900@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53505571.6050900@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1397775115 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - > I have completed the first round of this work ... as usual the final > implementation is a little messier than first expected ... Looks good! > [...] > CPU #writes Avg write size > (sec) (bytes) > short - 3.9.2 1.15 142 3946 > short - new I/O 1.30 10004 56 > [...] It would be informative to know how this translates to disk i/o. We don't use O_SYNC or fsync at this point, so it would be nice to be reassured that we're not triggering many more physical I/Os than before. (We'd just be pushing buffering to the kernel.) - FChE From kenj@internode.on.net Thu Apr 17 21:46:10 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B83867F53 for ; Thu, 17 Apr 2014 21:46:10 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 3A2EAAC001 for ; Thu, 17 Apr 2014 19:46:06 -0700 (PDT) X-ASG-Debug-ID: 1397789161-04cb6c24391343b0001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id PHcYdVq3FQBuvAPR for ; Thu, 17 Apr 2014 19:46:01 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArUMAIeQUFN5LGqv/2dsb2JhbABZgwY7xDsDgSMXdIIsCAIeEhwwBQYXSyAfAQQTCwWHa5ljsmCOf4QiBI9Tm2qBP4IEKw Received: from ppp121-44-106-175.lns20.syd6.internode.on.net (HELO bozohorize) ([121.44.106.175]) by ipmail07.adl2.internode.on.net with ESMTP; 18 Apr 2014 12:15:59 +0930 From: "Ken McDonell" To: Subject: pcp updates - archive I/O Date: Fri, 18 Apr 2014 12:45:44 +1000 X-ASG-Orig-Subj: pcp updates - archive I/O Message-ID: <004a01cf5ab0$540ff6b0$fc2fe410$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: Ac9asDxybQBOwKEVQICTJj4I8+pWNQ== Content-Language: en-au X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397789161 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5025 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/038 | 1 src/libpcp/src/logutil.c | 55 +- src/libpcp/src/p_result.c | 63 +- src/pmlogextract/pmlogextract.c | 6 src/pmlogreduce/scan.c | 7 ... commit 61308844be8caa5bf8cde3857f31012c970b92ad Merge: f7e19c9 ada57af Author: Ken McDonell Date: Fri Apr 18 12:11:31 2014 +1000 Merge branch 'archio' into dev Improved archive I/O features. commit f7e19c9b43dc7bf263418f2da3d4b3cd214fcb3a Merge: 7a0b9fe 6cfd9c0 Author: Ken McDonell Date: Fri Apr 18 12:10:30 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/pcp/pcp into dev commit ada57af7f86706729209bfe555a8fe957792ff53 Author: Ken McDonell Date: Fri Apr 18 11:59:52 2014 +1000 pmlogger I/O changes - part 2 This commit makes further changes to improve the I/O behaviour when archives are being created. 1. revert to buffered I/O for the metadata and index files ... unbuffered I/O here was a signficant hit and reduced (rather than improved) the semantic integrity of the external files 2. change the semantics for the internal __pmLogPutResult() routine so that the buffer containing the PDU is large enough to allow __pmLogPutResult() set the record length in the end of the buffer and then output all of the logical record (header, pmResult & trailer) in a single write() 3. consequent changes to the users of __pmLogPutResult() in the light of 2. commit 78782332f1ee9d027a6f9d8e3ccba59f806f6502 Author: Ken McDonell Date: Wed Apr 16 09:19:11 2014 +1000 qa/038 - capture a little more diagnostic detail in 038.full commit 7320e673ce8d4b00d19a05656992c067a6d531e4 Author: Ken McDonell Date: Tue Apr 15 10:13:28 2014 +1000 libpcp - __pmLogNewFile() - make writes unbuffered One line change to add setvbuf(f, NULL, _IONBF, 0) call after fopen(.., "w"); From kenj@kenj.com.au Fri Apr 18 00:53:44 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 63BC97F5D for ; Fri, 18 Apr 2014 00:53:44 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 41E78304070 for ; Thu, 17 Apr 2014 22:53:41 -0700 (PDT) X-ASG-Debug-ID: 1397800417-04cbb06e9a13cb60001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id PekBq6tnpsfan9G5 for ; Thu, 17 Apr 2014 22:53:38 -0700 (PDT) X-Barracuda-Envelope-From: kenj@kenj.com.au X-Barracuda-Apparent-Source-IP: 150.101.137.131 Received: from ppp121-44-106-175.lns20.syd6.internode.on.net (HELO bozo-vm.localdomain) ([121.44.106.175]) by ipmail07.adl2.internode.on.net with ESMTP; 18 Apr 2014 15:23:36 +0930 Received: by bozo-vm.localdomain (Postfix, from userid 1000) id 4EBDBA5347; Fri, 18 Apr 2014 15:53:21 +1000 (EST) To: pcp@oss.sgi.com Subject: pcp updates - pmlogmv Message-Id: <20140418055322.4EBDBA5347@bozo-vm.localdomain> X-ASG-Orig-Subj: pcp updates - pmlogmv Date: Fri, 18 Apr 2014 15:53:21 +1000 (EST) From: kenj@kenj.com.au (Ken McDonell) X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1397800417 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5029 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/738.out | 2 - qa/898 | 65 ++++++++++++++++++++++++++++++++++++++++++++++++ qa/898.out | 55 ++++++++++++++++++++++++++++++++++++++++ src/pmlogger/pmlogmv.sh | 62 ++++++++++++++++++++++++++++++++++----------- 4 files changed, 168 insertions(+), 16 deletions(-) commit e85a6cfb549d1da09cbc51c96c21586e3af83517 Author: Ken McDonell Date: Fri Apr 18 15:43:01 2014 +1000 pmlogmv - fix a couple of corner cases Validation of input arguments was deficient in a couple of ways ... - oldname.NNN was assumed to be data file, but it could be the basename for a data file such as oldname.NNN.0 - if oldname was the prefix of multiple archives, then this was not discovered until the linking had begun ... it should be checked earlier and trigger a more specific error message Both isues found when using pmlogmv in anger! qa/898 added to address the inadequate testing coverage that these issues also exposed. From kenj@internode.on.net Sat Apr 19 16:00:52 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 5CCC37F61 for ; Sat, 19 Apr 2014 16:00:52 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 07A82AC005 for ; Sat, 19 Apr 2014 14:00:48 -0700 (PDT) X-ASG-Debug-ID: 1397941246-04bdf045521beff0001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id LwbaU5SiNvzuvfXi for ; Sat, 19 Apr 2014 14:00:46 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aj57AM3jUlN20VG+PGdsb2JhbABYgwaDZIUKvDgDAoEbFwMBAQEBODWCJQEBBQgCHhIcIwwBAwIGAxUBKQcZIAoDEQIEEwsFiDDNMheNf2MHhDgEj1SfLCs Received: from ppp118-209-81-190.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.81.190]) by ipmail05.adl6.internode.on.net with ESMTP; 20 Apr 2014 06:30:20 +0930 From: "Ken McDonell" To: "'Frank Ch. Eigler'" Cc: "'Nathan Scott'" , References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <534B4330.1060008@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> <20140415002952.GM14108@redhat.com> <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> <53505571.6050900@internode.on.net> <20140417225150.GA8368@redhat.com> In-Reply-To: <20140417225150.GA8368@redhat.com> Subject: RE: pmlogger -u questions Date: Sun, 20 Apr 2014 07:00:16 +1000 X-ASG-Orig-Subj: RE: pmlogger -u questions Message-ID: <001f01cf5c12$6024b9a0$206e2ce0$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQHyaZW6bsfhxrCcQijEIAaglOJyoQJu8AsmAgYFuvACIJ/JvAJssPlKAdGT6JEC87Bg0AEG2a6TAUsG6vEBt36cDAGruMnSmjdqkUA= Content-Language: en-au X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1397941246 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5072 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header > -----Original Message----- > From: Frank Ch. Eigler [mailto:fche@redhat.com] > Sent: Friday, 18 April 2014 8:52 AM > ... > It would be informative to know how this translates to disk i/o. We don't use > O_SYNC or fsync at this point, so it would be nice to be reassured that we're > not triggering many more physical I/Os than before. (We'd just be pushing > buffering to the kernel.) These tests are not really long enough, and it is hard to isolate physical writes cause by pmlogger from the general background noise and whatever the kernel is using as the algorithm du jour to flush dirty blocks to disk. And we are not using anything to trigger more aggressive or pre-emptive I/O as you note. However I did do one quick test and the number of disk writes writes for the short case was 290 for the 3.9.2 version and 265 for the new version ... given the factors outside my control, I'd be willing to declare this a "draw" (the expected result) and certainly does provides no support for the hypothesis that something really bad has gone wrong here. From nwf@hum.ku.dk Mon Apr 21 06:47:02 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 8394A7F3F for ; Mon, 21 Apr 2014 06:47:02 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 5705A8F8040 for ; Mon, 21 Apr 2014 04:46:59 -0700 (PDT) X-ASG-Debug-ID: 1398080813-04bdf0455221b5c0001-S8gJnT Received: from mailgate.sc.ku.dk (mailgate.sc.ku.dk [192.38.29.215]) by cuda.sgi.com with ESMTP id SUTzQ9N9cOHJRPBg (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 21 Apr 2014 04:46:54 -0700 (PDT) X-Barracuda-Envelope-From: nwf@hum.ku.dk X-Barracuda-Apparent-Source-IP: 192.38.29.215 Received: from localhost (localhost [127.0.0.1]) by mailgate.sc.ku.dk (Postfix) with ESMTP id 78782200A07F; Mon, 21 Apr 2014 13:46:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at sc.ku.dk Received: from mailgate.sc.ku.dk ([127.0.0.1]) by localhost (mailgate.sc.ku.dk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JNY_wZIon6ly; Mon, 21 Apr 2014 13:46:50 +0200 (CEST) Received: from post.hum.ku.dk (excas1.sc.ku.dk [130.225.200.137]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "post.hum.ku.dk", Issuer "GeoTrust SSL CA" (verified OK)) by mailgate.sc.ku.dk (Postfix) with ESMTPS; Mon, 21 Apr 2014 13:46:50 +0200 (CEST) Received: from exmbx1.hum2005.hum.ku.dk ([fe80::bdef:f841:cc92:b6b1]) by excas1.hum2005.hum.ku.dk ([::1]) with mapi id 14.03.0181.006; Mon, 21 Apr 2014 13:46:41 +0200 From: Niels Werner Frederiksen Subject: Webmel Thread-Topic: Webmel X-ASG-Orig-Subj: Webmel Thread-Index: Ac9dV1wjt/3pWmOXTAm4imPPyrjp0Q== Date: Mon, 21 Apr 2014 11:46:40 +0000 Message-ID: Accept-Language: da-DK, en-US Content-Language: da-DK X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [188.227.182.22] Content-Type: multipart/alternative; boundary="_000_D5E2A3D24642354EB842EE73CE642C5C28566598exmbx1hum2005hu_" MIME-Version: 1.0 X-Barracuda-Connect: mailgate.sc.ku.dk[192.38.29.215] X-Barracuda-Start-Time: 1398080813 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.23 X-Barracuda-Spam-Status: No, SCORE=1.23 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE, MISSING_HEADERS, THREAD_INDEX, THREAD_TOPIC, TO_CC_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5113 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 1.21 MISSING_HEADERS Missing To: header 0.00 HTML_MESSAGE BODY: HTML included in message 0.00 TO_CC_NONE No To: or Cc: header To: undisclosed-recipients:; --_000_D5E2A3D24642354EB842EE73CE642C5C28566598exmbx1hum2005hu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux messages jusqu'=E0 ce que vous mettez =E0 niveau votre quota de courriel. Copiez le lien ci-dessous et remplissez le formulaire pour mettre =E0 nivea= u votre compte. https://www.formstack.com/forms/?1728681-4402ZNwJGN Administrateur syst=E8me 192.168.0.1 --_000_D5E2A3D24642354EB842EE73CE642C5C28566598exmbx1hum2005hu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux mes= sages
jusqu'=E0 ce que vous mettez =E0 niveau votre
quota de courriel.

Copiez le lien ci-dessous et remplissez le formulaire pour mettre =E0 nivea= u
votre compte.


https://www.formstack.com/forms/?1728681-440= 2ZNwJGN

Administrateur syst=E8me
192.168.0.1

--_000_D5E2A3D24642354EB842EE73CE642C5C28566598exmbx1hum2005hu_-- From maxwellrafiq@gmail.com Mon Apr 21 10:51:15 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE, LOTS_OF_MONEY,T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 508FC7F3F for ; Mon, 21 Apr 2014 10:51:15 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 21C6C8F8039 for ; Mon, 21 Apr 2014 08:51:12 -0700 (PDT) X-ASG-Debug-ID: 1398095469-04bdf04552226270001-S8gJnT Received: from mail-wi0-f196.google.com (mail-wi0-f196.google.com [209.85.212.196]) by cuda.sgi.com with ESMTP id ak4NzErNHMjtRua7 (version=TLSv1 cipher=RC4-SHA bits=128 verify=NO) for ; Mon, 21 Apr 2014 08:51:10 -0700 (PDT) X-Barracuda-Envelope-From: maxwellrafiq@gmail.com X-Barracuda-Apparent-Source-IP: 209.85.212.196 X-Barracuda-IPDD: Level1 [gmail.com/209.85.212.196] Received: by mail-wi0-f196.google.com with SMTP id d1so676129wiv.11 for ; Mon, 21 Apr 2014 08:51:08 -0700 (PDT) X-Barracuda-IPDD: Level1 [gmail.com/209.85.212.196] X-Barracuda-IPDD: Level1 [gmail.com/209.85.212.196] DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:date:message-id:subject:from:to :content-type; bh=h3elaEQvc3edMDVF5iwLxAoHTMzgdsAufEVI1WXkqQs=; b=ROy5xtkQ2XuNG94yS8vrG68DtVrEvOx23PG/TmSkes43L6nYmtpJqujiWioXFjdirX DFj95IQ/77etS8xhkvt78GnhNIrK7vFsUjy1FJ0docMPw5wA2zlKCzR9jLMyeA81dLWu mjY4SZqIxmtwAj5rrn5OssXY+c6GGAZtcqqw2gmVtOhl7IRbJVLX7VBKIZMoMzY3Fcns JwpxbkRA75sj49yOKWyR/eMoW9t2J0hiu+HYdpggDpZZZmr82QBju5MmPQdKN0d1erfD EVBZSmcmis+AMcUMnF8Zr2bYWAbZaGbNgsxZgHMzskT9/a5wPTn8awC9waMW80S7LcMS m8Yg== MIME-Version: 1.0 X-Received: by 10.180.74.210 with SMTP id w18mr14689973wiv.10.1398095468537; Mon, 21 Apr 2014 08:51:08 -0700 (PDT) Reply-To: smithawaawala@gmail.com Sender: maxwellrafiq@gmail.com Received: by 10.216.67.19 with HTTP; Mon, 21 Apr 2014 08:51:08 -0700 (PDT) Date: Mon, 21 Apr 2014 17:51:08 +0200 X-Google-Sender-Auth: DaVjtSx3cjAaaG215ua8g90lZCo Message-ID: Subject: I want to transfer an abandoned USD5.5Million to your Bank account. From: SMITH AWALA X-ASG-Orig-Subj: I want to transfer an abandoned USD5.5Million to your Bank account. To: undisclosed-recipients:; Content-Type: multipart/alternative; boundary=f46d043749fbc3b50504f78f773f X-Barracuda-Connect: mail-wi0-f196.google.com[209.85.212.196] X-Barracuda-Start-Time: 1398095469 X-Barracuda-Encrypted: RC4-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-BRTS-Evidence: 214bb443db4a466af69ef481bb566088-135-txt X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=ADVANCE_FEE_1, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5118 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.00 HTML_MESSAGE BODY: HTML included in message 0.00 ADVANCE_FEE_1 Appears to be advance fee fraud (Nigerian 419) --f46d043749fbc3b50504f78f773f Content-Type: text/plain; charset=UTF-8 -- Hi Friend, I`m a Banker. I want to transfer an abandoned USD5.5Million to your Bank account. 40% will be for you while 60% for me. --f46d043749fbc3b50504f78f773f Content-Type: text/html; charset=UTF-8


--
Hi Friend, I`m a Banker. I want to transfer an abandoned USD5.5Million to your Bank account. 40% will be for you while 60% for me.
--f46d043749fbc3b50504f78f773f-- From nscott@redhat.com Mon Apr 21 23:09:53 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 088517F3F for ; Mon, 21 Apr 2014 23:09:53 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 99B58AC008 for ; Mon, 21 Apr 2014 21:09:52 -0700 (PDT) X-ASG-Debug-ID: 1398139786-04bdf04552245f40001-S8gJnT Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id hrQHdezXGHwyRbP9 for ; Mon, 21 Apr 2014 21:09:46 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.25 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3M49hv1007919; Tue, 22 Apr 2014 00:09:43 -0400 Date: Tue, 22 Apr 2014 00:09:43 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: "Frank Ch. Eigler" , pcp@oss.sgi.com Message-ID: <2125677089.8342041.1398139783815.JavaMail.zimbra@redhat.com> In-Reply-To: <53505571.6050900@internode.on.net> References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <534C4FF4.5000304@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> <20140415002952.GM14108@redhat.com> <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> <53505571.6050900@internode.on.net> Subject: Re: pmlogger -u questions MIME-Version: 1.0 X-ASG-Orig-Subj: Re: pmlogger -u questions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogger -u questions Thread-Index: NwiHeUGviz9pBLv0TeUl4PtZ/xSv+A== X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25] X-Barracuda-Start-Time: 1398139786 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5138 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... ----- Original Message ----- > I have completed the first round of this work ... as usual the final > implementation is a little messier than first expected ... :) > - changing the semantics of __pmLogPutResult to allow the trailer to be > written in the same write as the header and pmResult was a little messy Yeah, hmm, oh well. > The testing (all QA passes, so no functional regressions) reported below > shows less writes and about the same CPU burn for the typical cases ... > so resource wise a break even, but we get the better semantics. For the Yep, agreed. Actually I would say the log write I/O patterns appear to be noticeably improved for the more realistic sizes, and well worth the slight CPU overhead. However, given this: > Test cases 10,000 samples at 10msec intervals. Could we re-create that table at a 10sec logged interval? Perhaps just for 100 samples - that'd certainly satisfy my curiosity anyway. > ... > CPU #writes Avg write size > (sec) (bytes) > short - 3.9.2 1.15 142 3946 > short - new I/O 1.30 10004 56 > default - 3.9.2 3.77 20001 5623 > default - new I/O 3.73 10004 11571 > long - 3.9.2 26.5 20001 20824 > long - new I/O 26.9 10004 42409 Nicely done. Slightly off-topic, but in your dev branch the pmlogmv change looked a little unexpected re the change in one output message, to my eye anyway. Also there was one comment I had to re-read to grok - do these small tweaks seem OK & match the intention? diff --git a/qa/738.out b/qa/738.out index 23d3b71..5e6473e 100644 --- a/qa/738.out +++ b/qa/738.out @@ -6,7 +6,7 @@ Usage: pmlogmv [-NV] oldname newname exit status 1 pmlogmv: Error: cannot find any files for the input archive (foo) exit status 1 -pmlogmv: Error: cannot find .metadata file for the input archive (foo) +pmlogmv: Error: cannot find metadata file for the input archive (foo) ... ls data ... foo.0 exit status 1 pmlogmv: Error: cannot find any data files for the input archive (foo) diff --git a/src/libpcp/src/logutil.c b/src/libpcp/src/logutil.c index 37510d0..f9e3c6e 100644 --- a/src/libpcp/src/logutil.c +++ b/src/libpcp/src/logutil.c @@ -522,8 +522,8 @@ __pmLogNewFile(const char *base, int vol) /* * Want unbuffered I/O for the data volumes ... the metadata * already has a fflush() after the batch (if any) of changes - * from the current pmResult, and temporal index also has - * fflush()ing everytime it is written to. + * from the current pmResult, and the temporal index also has + * fflush() issued every time it is written to. */ if (vol != PM_LOG_VOL_META && vol != PM_LOG_VOL_TI) setvbuf(f, NULL, _IONBF, 0); diff --git a/src/pmlogger/pmlogmv.sh b/src/pmlogger/pmlogmv.sh index 1c372e6..dc22677 100755 --- a/src/pmlogger/pmlogmv.sh +++ b/src/pmlogger/pmlogmv.sh @@ -172,7 +172,7 @@ if grep -q '.meta$' $tmp.old then : else - echo >&2 "pmlogmv: Error: cannot find .metadata file for the input archive ($old)" + echo >&2 "pmlogmv: Error: cannot find metadata file for the input archive ($old)" ls -l "$old"* exit fi cheers. -- Nathan From nscott@redhat.com Tue Apr 22 02:36:18 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id D9C277F3F for ; Tue, 22 Apr 2014 02:36:18 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id BB7798F8040 for ; Tue, 22 Apr 2014 00:36:18 -0700 (PDT) X-ASG-Debug-ID: 1398152172-04cb6c2438241b40001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id BgHcg8FXhYDEG2jS for ; Tue, 22 Apr 2014 00:36:12 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3M7aC5Z009024 for ; Tue, 22 Apr 2014 03:36:12 -0400 Date: Tue, 22 Apr 2014 03:36:12 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp developers Message-ID: <2103055230.8481757.1398152172348.JavaMail.zimbra@redhat.com> In-Reply-To: <62074033.8479698.1398152020878.JavaMail.zimbra@redhat.com> Subject: pcp updates: log I/O, logmv, dumplog longopts, qa MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: log I/O, logmv, dumplog longopts, qa Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: log I/O, logmv, dumplog longopts, qa Thread-Index: q3HWD5mdo2A4kGCrJIJVYykhTnekgw== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1398152172 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5142 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.performancecopilot.org/pcp/pcp.git dev VERSION.pcp | 2 build/rpm/fedora.spec | 7 debian/changelog | 6 qa/.gitignore | 1 qa/038 | 1 qa/195.out | 8 qa/417.out | 20 +- qa/438 | 8 qa/438.out | 129 +++++++++++++++ qa/438.out.1 | 103 ------------ qa/438.out.2 | 129 --------------- qa/555 | 15 + qa/738.out | 2 qa/898 | 65 +++++++ qa/898.out | 55 ++++++ src/libpcp/src/logutil.c | 55 +++--- src/libpcp/src/p_result.c | 63 ++++--- src/pcp/free/pcp-free.py | 4 src/pmdumplog/pmdumplog.c | 339 +++++++++++++++------------------------- src/pmlogextract/pmlogextract.c | 6 src/pmlogger/pmlogmv.sh | 62 +++++-- src/pmlogreduce/scan.c | 7 22 files changed, 556 insertions(+), 531 deletions(-) commit 90170fe6c9498398ed994d1f5b06ceef62d2b9cd Merge: 073793a e85a6cf Author: Nathan Scott Date: Tue Apr 22 11:37:47 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit 073793a9d6ab8bef112da7fe264a2ff731eeb092 Author: Nathan Scott Date: Tue Apr 22 09:03:24 2014 +1000 Convert pmdumplog to using long options via pmGetOptions commit e85a6cfb549d1da09cbc51c96c21586e3af83517 Author: Ken McDonell Date: Fri Apr 18 15:43:01 2014 +1000 pmlogmv - fix a couple of corner cases Validation of input arguments was deficient in a couple of ways ... - oldname.NNN was assumed to be data file, but it could be the basename for a data file such as oldname.NNN.0 - if oldname was the prefix of multiple archives, then this was not discovered until the linking had begun ... it should be checked earlier and trigger a more specific error message Both isues found when using pmlogmv in anger! qa/898 added to address the inadequate testing coverage that these issues also exposed. commit 61308844be8caa5bf8cde3857f31012c970b92ad Merge: f7e19c9 ada57af Author: Ken McDonell Date: Fri Apr 18 12:11:31 2014 +1000 Merge branch 'archio' into dev Improved archive I/O features. commit f7e19c9b43dc7bf263418f2da3d4b3cd214fcb3a Merge: 7a0b9fe 6cfd9c0 Author: Ken McDonell Date: Fri Apr 18 12:10:30 2014 +1000 Merge branch 'dev' of git://oss.sgi.com/pcp/pcp into dev commit ada57af7f86706729209bfe555a8fe957792ff53 Author: Ken McDonell Date: Fri Apr 18 11:59:52 2014 +1000 pmlogger I/O changes - part 2 This commit makes further changes to improve the I/O behaviour when archives are being created. 1. revert to buffered I/O for the metadata and index files ... unbuffered I/O here was a signficant hit and reduced (rather than improved) the semantic integrity of the external files 2. change the semantics for the internal __pmLogPutResult() routine so that the buffer containing the PDU is large enough to allow __pmLogPutResult() set the record length in the end of the buffer and then output all of the logical record (header, pmResult & trailer) in a single write() 3. consequent changes to the users of __pmLogPutResult() in the light of 2. commit 6cfd9c0f42429a20007c9367a29eb1ff89973b54 Author: Nathan Scott Date: Wed Apr 16 13:36:14 2014 +1000 Ensure memory metrics are converted to kilobytes in pcp-free Implements Franks review point that pcp-free should make sure that its working with kilobytes, instead of assuming that the metric descriptor from the kernel PMDA will never change. A no-op for the present, but good future-proofing nonetheless. commit c2f0a562c66baf28a7c81679ec1b2b7901428c14 Author: Nathan Scott Date: Wed Apr 16 13:26:51 2014 +1000 Prepare next dev branch, make a call on next release date commit 78782332f1ee9d027a6f9d8e3ccba59f806f6502 Author: Ken McDonell Date: Wed Apr 16 09:19:11 2014 +1000 qa/038 - capture a little more diagnostic detail in 038.full commit 116c98a850dad1d1b3924e9e40742b245b9161be Author: Nathan Scott Date: Tue Apr 15 18:07:52 2014 +1000 Update contact details in fedora spec for daves builds commit 7320e673ce8d4b00d19a05656992c067a6d531e4 Author: Ken McDonell Date: Tue Apr 15 10:13:28 2014 +1000 libpcp - __pmLogNewFile() - make writes unbuffered One line change to add setvbuf(f, NULL, _IONBF, 0) call after fopen(.., "w"); commit 7a0b9fe524b6b13979fa99adf760d5c5096512d5 Author: Ken McDonell Date: Fri Apr 11 19:32:16 2014 +1000 qa/555 - better diags and improved upstart detection From fche@redhat.com Tue Apr 22 14:38:42 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 145D67F3F for ; Tue, 22 Apr 2014 14:38:42 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7B04AAC00D for ; Tue, 22 Apr 2014 12:38:38 -0700 (PDT) X-ASG-Debug-ID: 1398195516-04cb6c2438267090001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id i3nBW1PF6nsBMgLB for ; Tue, 22 Apr 2014 12:38:37 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3MJcagF028493 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 22 Apr 2014 15:38:36 -0400 Received: from fche.csb (vpn-49-173.rdu2.redhat.com [10.10.49.173]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3MJcaGB014694 for ; Tue, 22 Apr 2014 15:38:36 -0400 Received: by fche.csb (Postfix, from userid 2569) id 6506258153; Tue, 22 Apr 2014 15:38:35 -0400 (EDT) To: pcp developers Subject: ABI preservation vs. internal functions, was Re: pcp updates: log I/O, logmv, dumplog longopts, qa References: <62074033.8479698.1398152020878.JavaMail.zimbra@redhat.com> <2103055230.8481757.1398152172348.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: ABI preservation vs. internal functions, was Re: pcp updates: log I/O, logmv, dumplog longopts, qa From: fche@redhat.com (Frank Ch. Eigler) Date: Tue, 22 Apr 2014 15:38:35 -0400 In-Reply-To: <2103055230.8481757.1398152172348.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Tue, 22 Apr 2014 03:36:12 -0400 (EDT)") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398195517 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 > commit ada57af7f86706729209bfe555a8fe957792ff53 > Author: Ken McDonell > Date: Fri Apr 18 11:59:52 2014 +1000 > > pmlogger I/O changes - part 2 > > This commit makes further changes to improve the I/O behaviour when > archives are being created. > > [...] > 2. change the semantics for the internal __pmLogPutResult() routine > so that the buffer containing the PDU is large enough to allow > __pmLogPutResult() set the record length in the end of the buffer > and then output all of the logical record (header, pmResult & > trailer) in a single write() > [...] This change is backward-incompatible, so any older binaries would likely result in memory corruption when run against the new shared library. We could say that it doesn't matter, that the symbol is "internal", etc. Or we could fix it. The function with the new requirement could be renamed __pmLogPutResult2. The older __pmLogPutResult symbol could be replaced by a wrapper that does an internal memory-copy, or something else safe. We could even inoculate against such future problems to some extent. "internal" utility functions could be formally identified (since the "__" naming prefix is not that). They could be excluded from shared libraries and installed documentation, and built only as .a files. The export files for future versions of PCP shared libraries could be a lot smaller. - FChE From nscott@redhat.com Tue Apr 22 15:39:56 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 2A5107F3F for ; Tue, 22 Apr 2014 15:39:56 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id ADF33AC00F for ; Tue, 22 Apr 2014 13:39:52 -0700 (PDT) X-ASG-Debug-ID: 1398199190-04cb6c2439268d20001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id HvjFvDzfM6ZCAYUK for ; Tue, 22 Apr 2014 13:39:51 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3MKdlAE004887; Tue, 22 Apr 2014 16:39:48 -0400 Date: Tue, 22 Apr 2014 16:39:47 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: pcp@oss.sgi.com Message-ID: <1736718933.9019660.1398199187948.JavaMail.zimbra@redhat.com> In-Reply-To: <2125677089.8342041.1398139783815.JavaMail.zimbra@redhat.com> References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <20140414212551.GK14108@redhat.com> <534C6531.6050502@internode.on.net> <155006091.5545657.1397518977813.JavaMail.zimbra@redhat.com> <20140415002952.GM14108@redhat.com> <216112516.5558630.1397522268210.JavaMail.zimbra@redhat.com> <53505571.6050900@internode.on.net> <2125677089.8342041.1398139783815.JavaMail.zimbra@redhat.com> Subject: Re: pmlogger -u questions MIME-Version: 1.0 X-ASG-Orig-Subj: Re: pmlogger -u questions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pmlogger -u questions Thread-Index: NwiHeUGviz9pBLv0TeUl4PtZ/xSv+KMrqN6d X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1398199191 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5161 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... ----- Original Message ----- > ----- Original Message ----- > [...] > Yep, agreed. Actually I would say the log write I/O patterns appear to > be noticeably improved for the more realistic sizes, and well worth the > slight CPU overhead. > > However, given this: > > Test cases 10,000 samples at 10msec intervals. > > Could we re-create that table at a 10sec logged interval? Perhaps > just for 100 samples - that'd certainly satisfy my curiosity anyway. Pondering further overnight, and after some code digging - there's no point in doing this, I think - please ignore. The answers will be (effectively) the same AIUI, as there is nothing in the __pmOptFetch APIs that'll change behaviour based on a different delta - same number of writes should result. There was an earlier comment about difficulty in discerning physical I/O resulting from pmlogger writes from those issued by the rest of the system. One way to isolate that (somewhat) is to write to a separate unused device. This device can then be targetted by blktrace(1). Other factors still play an overwhelming role though - filesystem characteristics, like delalloc vs. allocating space when pmlogger issues the write(2), memory utilisation, and so on. cheers. -- Nathan From kenj@internode.on.net Tue Apr 22 19:56:21 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 85C5F7F3F for ; Tue, 22 Apr 2014 19:56:21 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 18259AC00A for ; Tue, 22 Apr 2014 17:56:20 -0700 (PDT) X-ASG-Debug-ID: 1398214577-04cbb06e9a26eed0001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id z5Ef0fSWVYVI0SzS for ; Tue, 22 Apr 2014 17:56:18 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhUCALkOV1OvLVMinGdsb2JhbABZyF0DAoEpDgEBAQEBCBQJPIIlAQEEAQgCHhJEBwEDAgYDFQElBAcZLRECBAESCwWIKQfPOReOXYQ4BI9WnV6BUSs Received: from mail.messagemedia.com.au (HELO bozohorize) ([175.45.83.34]) by ipmail04.adl6.internode.on.net with ESMTP; 23 Apr 2014 10:26:17 +0930 From: "Ken McDonell" To: "'Frank Ch. Eigler'" , "'pcp developers'" References: <62074033.8479698.1398152020878.JavaMail.zimbra@redhat.com> <2103055230.8481757.1398152172348.JavaMail.zimbra@redhat.com> In-Reply-To: Subject: RE: [pcp] ABI preservation vs. internal functions, was Re: pcp updates: log I/O, logmv, dumplog longopts, qa Date: Wed, 23 Apr 2014 10:56:17 +1000 X-ASG-Orig-Subj: RE: [pcp] ABI preservation vs. internal functions, was Re: pcp updates: log I/O, logmv, dumplog longopts, qa Message-ID: <000101cf5e8e$d64fba70$82ef2f50$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQJXQV7Rp4jsNT4i3OFV9zvNJT2bBQFWae0/AR7VNjmZ+tkq8A== Content-Language: en-au X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1398214577 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5169 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== > -----Original Message----- > From: pcp-bounces@oss.sgi.com [mailto:pcp-bounces@oss.sgi.com] On > Behalf Of Frank Ch. Eigler > Sent: Wednesday, 23 April 2014 5:39 AM > ... > This change is backward-incompatible, so any older binaries would likely > result in memory corruption when run against the new shared library. We > could say that it doesn't matter, that the symbol is "internal", etc. Frank is correct. This is a case where I was being lazy and not "doing as I say". My only excuse is I fixed all the uses of __pmLogPutResult in the PCP source, and could not imagine anyone else using this method. But, that's not the "right" attitude. > Or we could fix it. The function with the new requirement could be renamed > __pmLogPutResult2. The older __pmLogPutResult symbol could be replaced > by a wrapper that does an internal memory-copy, or something else safe. Yep +1 for __pmLogPutResult2 ... I'll fix it. We don't need anything different, just the old implementation ... > We could even inoculate against such future problems to some extent. > "internal" utility functions could be formally identified (since the "__" naming > prefix is not that). They could be excluded from shared libraries and installed > documentation, and built only as .a files. > The export files for future versions of PCP shared libraries could be a lot > smaller. Sounds like more work ... if I'd done this correctly the first time, we wouldn't need to be having this discussion. From fche@redhat.com Tue Apr 22 20:01:56 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 2510E7F3F for ; Tue, 22 Apr 2014 20:01:56 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 12DAF8F8049 for ; Tue, 22 Apr 2014 18:01:52 -0700 (PDT) X-ASG-Debug-ID: 1398214911-04cbb06e9c26f130001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id KD4WoAtREVE86xbH for ; Tue, 22 Apr 2014 18:01:52 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3N11mjU019017 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 22 Apr 2014 21:01:48 -0400 Received: from fche.csb (vpn-49-173.rdu2.redhat.com [10.10.49.173]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3N11ljR003878; Tue, 22 Apr 2014 21:01:48 -0400 Received: by fche.csb (Postfix, from userid 2569) id 4906458153; Tue, 22 Apr 2014 21:01:47 -0400 (EDT) Date: Tue, 22 Apr 2014 21:01:47 -0400 From: "Frank Ch. Eigler" To: Ken McDonell Cc: "'pcp developers'" Subject: Re: [pcp] ABI preservation vs. internal functions, was Re: pcp updates: log I/O, logmv, dumplog longopts, qa Message-ID: <20140423010147.GA18206@redhat.com> X-ASG-Orig-Subj: Re: [pcp] ABI preservation vs. internal functions, was Re: pcp updates: log I/O, logmv, dumplog longopts, qa References: <62074033.8479698.1398152020878.JavaMail.zimbra@redhat.com> <2103055230.8481757.1398152172348.JavaMail.zimbra@redhat.com> <000101cf5e8e$d64fba70$82ef2f50$@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000101cf5e8e$d64fba70$82ef2f50$@internode.on.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398214912 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Ken - On Wed, Apr 23, 2014 at 10:56:17AM +1000, Ken McDonell wrote: > [...] > Yep +1 for __pmLogPutResult2 ... I'll fix it. Cool. > We don't need anything different, just the old implementation ... (Right; though reuse of the new code could make the code simpler, and just penalize older binaries with some memory alloc/copy overheads.) > > We could even inoculate against such future problems to some extent. > > "internal" utility functions could be formally identified [...] > Sounds like more work ... if I'd done this correctly the first time, we > wouldn't need to be having this discussion. Yeah, it's more of a long-term suggestion. At this time, the effective API exported by all our .so's is way, way bigger than the core items documented in the programmer's guide books, or even the man pages. (I'd find it handy to have a place for utility functions that are available to be reused amongst current-generation pcp binaries, but that are not meant for use across versions or by out-of-tree tools, so would be excluded from shared library / ABI consideration.) - FChE From nwf@hum.ku.dk Wed Apr 23 03:23:10 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 03D4C7F3F for ; Wed, 23 Apr 2014 03:23:10 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 956A2AC00A for ; Wed, 23 Apr 2014 01:23:06 -0700 (PDT) X-ASG-Debug-ID: 1398241380-04bdf0455228c170001-S8gJnT Received: from mailgate2.sc.ku.dk ([192.38.29.235]) by cuda.sgi.com with ESMTP id HDZlLmUZ8OAfXCGu (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 23 Apr 2014 01:23:01 -0700 (PDT) X-Barracuda-Envelope-From: nwf@hum.ku.dk X-Barracuda-Apparent-Source-IP: 192.38.29.235 Received: from localhost (localhost [127.0.0.1]) by mailgate2.sc.ku.dk (Postfix) with ESMTP id 62176100AA4E; Wed, 23 Apr 2014 10:22:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at sc.ku.dk Received: from mailgate2.sc.ku.dk ([127.0.0.1]) by localhost (mailgate2.sc.ku.dk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lzZ96dENqrDC; Wed, 23 Apr 2014 10:22:57 +0200 (CEST) Received: from post.hum.ku.dk (excas1.sc.ku.dk [130.225.200.137]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "post.hum.ku.dk", Issuer "GeoTrust SSL CA" (verified OK)) by mailgate2.sc.ku.dk (Postfix) with ESMTPS; Wed, 23 Apr 2014 10:22:57 +0200 (CEST) Received: from exmbx1.hum2005.hum.ku.dk ([fe80::bdef:f841:cc92:b6b1]) by excas1.hum2005.hum.ku.dk ([::1]) with mapi id 14.03.0181.006; Wed, 23 Apr 2014 10:22:47 +0200 From: Niels Werner Frederiksen Subject: =?iso-8859-1?Q?Mise_=E0_niveau_du_Webmel?= Thread-Topic: =?iso-8859-1?Q?Mise_=E0_niveau_du_Webmel?= X-ASG-Orig-Subj: =?iso-8859-1?Q?Mise_=E0_niveau_du_Webmel?= Thread-Index: Ac9ezTSoHxc2lPCiQp2wX6DppUwaDg== Date: Wed, 23 Apr 2014 08:22:46 +0000 Message-ID: Accept-Language: da-DK, en-US Content-Language: da-DK X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [41.203.67.131] Content-Type: multipart/alternative; boundary="_000_D5E2A3D24642354EB842EE73CE642C5C28569442exmbx1hum2005hu_" MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[192.38.29.235] X-Barracuda-Start-Time: 1398241381 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.33 X-Barracuda-Spam-Status: No, SCORE=1.33 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE, MISSING_HEADERS, RDNS_NONE, THREAD_INDEX, THREAD_TOPIC, TO_CC_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5177 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 1.21 MISSING_HEADERS Missing To: header 0.00 HTML_MESSAGE BODY: HTML included in message 0.00 TO_CC_NONE No To: or Cc: header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS To: undisclosed-recipients:; --_000_D5E2A3D24642354EB842EE73CE642C5C28569442exmbx1hum2005hu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux messages j= usqu'=E0 ce que vous mettez =E0 niveau votre quota de courriel. Copiez le lien ci-dessous et remplissez le formulaire pour mettre =E0 nivea= u votre compte. https://www.formstack.com/forms/?1731705-YgLCimnCqW Administrateur syst=E8me 192.168.0.1 --_000_D5E2A3D24642354EB842EE73CE642C5C28569442exmbx1hum2005hu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux message= s jusqu'=E0 ce que vous mettez =E0 niveau votre
quota de courriel.

Copiez le lien ci-dessous et remplissez le formulaire pour mettre =E0 ni= veau votre compte.


https://www= .formstack.com/forms/?1731705-YgLCimnCqW

Administrateur syst=E8me
192.168.0.1

--_000_D5E2A3D24642354EB842EE73CE642C5C28569442exmbx1hum2005hu_-- From brolley@redhat.com Wed Apr 23 12:35:23 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 82EA67F52 for ; Wed, 23 Apr 2014 12:35:23 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 150FDAC00E for ; Wed, 23 Apr 2014 10:35:19 -0700 (PDT) X-ASG-Debug-ID: 1398274515-04bdf045552a9850001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 6eHv6aoceOCZ6lIB for ; Wed, 23 Apr 2014 10:35:15 -0700 (PDT) X-Barracuda-Envelope-From: brolley@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3NHZF4Y026683 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 23 Apr 2014 13:35:15 -0400 Received: from [10.10.51.125] (vpn-51-125.rdu2.redhat.com [10.10.51.125]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3NHZExS012595 for ; Wed, 23 Apr 2014 13:35:14 -0400 Message-ID: <5357FA37.2070202@redhat.com> Date: Wed, 23 Apr 2014 13:36:55 -0400 From: Dave Brolley User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: PCP Mailing List Subject: PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API References: <360594769.5718335.1381470346334.JavaMail.root@redhat.com> X-ASG-Orig-Subj: PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API In-Reply-To: <360594769.5718335.1381470346334.JavaMail.root@redhat.com> X-Forwarded-Message-Id: <360594769.5718335.1381470346334.JavaMail.root@redhat.com> Content-Type: multipart/alternative; boundary="------------050002070201000702080905" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398274515 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 This is a multi-part message in MIME format. --------------050002070201000702080905 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit These commits, on the brolley/dev branch of pcpfans remove the use of inet-specific APIs (i.e. no ipv6) in favour of the libpcp I/O APIs (which do support IPv6, where available). The original message from Nathan which inspired this can be found below. Some __pm level functions which were previously static were moved to impl.h in order to accomplish this. A local qa run shows no regressions. Dave ------------------------------------------------------------------ commit fa6700099d746c3b72764de74d9606f7abe61505 Author: Dave Brolley Date: Tue Apr 22 16:55:45 2014 -0400 Cisco PMDA: Use the __pm*() socket APIs throughout. commit a08c81563dd4bb2b187ce8de9b364b2be2bf2789 Author: Dave Brolley Date: Tue Apr 22 16:15:52 2014 -0400 src/telnet-probe/telnet-probe.c: Use the __pm*() socket APIs throughout. commit a1920dec79e58eca519e3af62c1690242fc71d16 Author: Dave Brolley Date: Tue Apr 22 15:26:18 2014 -0400 src/pmie/src/stomp.c: Use the __pm*() socket APIs throughout. commit a7100289652d54661b1b40625e48fe902b9ad98e Author: Dave Brolley Date: Tue Apr 22 15:25:42 2014 -0400 Don't call __pmCloseSocket() on fds which indicate error (i.e. < 0). commit 7bbc80366dddf9b9e38d50ba7ee4e467573ef7ca Author: Dave Brolley Date: Tue Apr 22 14:44:58 2014 -0400 http_fetcher.c: Use __pm*() socket APIs throughout. commit a72b8b5dabf00148c2962ae90f81533a17041248 Author: Dave Brolley Date: Tue Apr 22 12:12:55 2014 -0400 src/pmlogger/src/preamble.c: Use __pmgetAddrInfo() instead of gethostbyname(). commit ca59cbabb7e37e080c171fcb64e064b526ffdfdc Author: Dave Brolley Date: Fri Apr 18 14:34:11 2014 -0400 libpcp_trace/src/trace.c: Remove use of gethostbyname(). Use __pm* I/O API throughout instead. commit cc930ea1994d1d028a186520d789d2a61196009f Author: Dave Brolley Date: Mon Mar 31 11:47:45 2014 -0400 Fix typo in debugging message: gethostbyname -> __pmGetAddrInfo. commit 5758ab54cffc4811b77d3092b2c2148f81ab0b31 Author: Dave Brolley Date: Mon Mar 31 11:37:52 2014 -0400 qa chkacc[123]: Remove references to inet_aton() and gethostbyname(). Always use the __pmSockaddr* API. Also removed PCP_VER checking throughout. -------- Original Message -------- Subject: IPv6 issues Date: Fri, 11 Oct 2013 01:45:46 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Dave Brolley FYI - rpmdiff came up with this list of remaining IPv6 lurking issues. Guess we should slowly clean these up over time? cheers. VERIFY pcp usr/bin/pmie on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp usr/bin/pmlogger on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp usr/libexec/pcp/bin/telnet-probe on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp var/lib/pcp/pmdas/apache/pmdaapache on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp var/lib/pcp/pmdas/cisco/parse on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp var/lib/pcp/pmdas/cisco/pmdacisco on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp var/lib/pcp/pmdas/cisco/probe on all architectures uses function gethostbyname, which may impact IPv6 support VERIFY pcp var/lib/pcp/pmdas/linux/pmda_linux.so on all architectures uses function inet_ntoa, which may impact IPv6 support VERIFY pcp var/lib/pcp/pmdas/linux/pmdalinux on all architectures uses function inet_ntoa, which may impact IPv6 support VERIFY pcp-libs usr/lib/libpcp.so.3 on i686 ppc s390 uses function gethostbyname, which may impact IPv6 support VERIFY pcp-libs usr/lib/libpcp.so.3 on i686 ppc s390 uses function inet_ntoa, which may impact IPv6 support VERIFY pcp-libs usr/lib/libpcp_trace.so.2 on i686 ppc s390 uses function gethostbyname, which may impact IPv6 support VERIFY pcp-libs usr/lib64/libpcp.so.3 on x86_64 ppc64 s390x uses function gethostbyname, which may impact IPv6 support VERIFY pcp-libs usr/lib64/libpcp.so.3 on x86_64 ppc64 s390x uses function inet_ntoa, which may impact IPv6 support VERIFY pcp-libs usr/lib64/libpcp_trace.so.2 on x86_64 ppc64 s390x uses function gethostbyname, which may impact IPv6 support -- Nathan --------------050002070201000702080905 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit These commits, on the brolley/dev branch of pcpfans remove the use of inet-specific APIs (i.e. no ipv6) in favour of the libpcp I/O APIs (which do support IPv6, where available).

The original message from Nathan which inspired this can be found below.

Some __pm level functions which were previously static were moved to impl.h in order to accomplish this.

A local qa run shows no regressions.

Dave

------------------------------------------------------------------

commit fa6700099d746c3b72764de74d9606f7abe61505
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Apr 22 16:55:45 2014 -0400

    Cisco PMDA: Use the __pm*() socket APIs throughout.

commit a08c81563dd4bb2b187ce8de9b364b2be2bf2789
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Apr 22 16:15:52 2014 -0400

    src/telnet-probe/telnet-probe.c: Use the __pm*() socket APIs throughout.

commit a1920dec79e58eca519e3af62c1690242fc71d16
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Apr 22 15:26:18 2014 -0400

    src/pmie/src/stomp.c: Use the __pm*() socket APIs throughout.

commit a7100289652d54661b1b40625e48fe902b9ad98e
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Apr 22 15:25:42 2014 -0400

    Don't call __pmCloseSocket() on fds which indicate error (i.e. < 0).

commit 7bbc80366dddf9b9e38d50ba7ee4e467573ef7ca
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Apr 22 14:44:58 2014 -0400

    http_fetcher.c: Use __pm*() socket APIs throughout.

commit a72b8b5dabf00148c2962ae90f81533a17041248
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Apr 22 12:12:55 2014 -0400

    src/pmlogger/src/preamble.c: Use __pmgetAddrInfo()
   
    instead of gethostbyname().

commit ca59cbabb7e37e080c171fcb64e064b526ffdfdc
Author: Dave Brolley <brolley@redhat.com>
Date:   Fri Apr 18 14:34:11 2014 -0400

    libpcp_trace/src/trace.c: Remove use of gethostbyname().
   
    Use __pm* I/O API throughout instead.

commit cc930ea1994d1d028a186520d789d2a61196009f
Author: Dave Brolley <brolley@redhat.com>
Date:   Mon Mar 31 11:47:45 2014 -0400

    Fix typo in debugging message: gethostbyname -> __pmGetAddrInfo.

commit 5758ab54cffc4811b77d3092b2c2148f81ab0b31
Author: Dave Brolley <brolley@redhat.com>
Date:   Mon Mar 31 11:37:52 2014 -0400

    qa chkacc[123]: Remove references to inet_aton() and gethostbyname().
   
    Always use the __pmSockaddr* API.
    Also removed PCP_VER checking throughout.



-------- Original Message --------
Subject: IPv6 issues
Date: Fri, 11 Oct 2013 01:45:46 -0400 (EDT)
From: Nathan Scott <nathans@redhat.com>
Reply-To: Nathan Scott <nathans@redhat.com>
To: Dave Brolley <brolley@redhat.com> 


FYI - rpmdiff came up with this list of remaining IPv6 lurking issues.
Guess we should slowly clean these up over time?

cheers.



VERIFY	 	pcp	 	usr/bin/pmie on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	usr/bin/pmlogger on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	usr/libexec/pcp/bin/telnet-probe on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	var/lib/pcp/pmdas/apache/pmdaapache on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	var/lib/pcp/pmdas/cisco/parse on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	var/lib/pcp/pmdas/cisco/pmdacisco on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	var/lib/pcp/pmdas/cisco/probe on all architectures uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp	 	var/lib/pcp/pmdas/linux/pmda_linux.so on all architectures uses function inet_ntoa, which may impact IPv6 support
VERIFY	 	pcp	 	var/lib/pcp/pmdas/linux/pmdalinux on all architectures uses function inet_ntoa, which may impact IPv6 support
VERIFY	 	pcp-libs	 	usr/lib/libpcp.so.3 on i686 ppc s390 uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp-libs	 	usr/lib/libpcp.so.3 on i686 ppc s390 uses function inet_ntoa, which may impact IPv6 support
VERIFY	 	pcp-libs	 	usr/lib/libpcp_trace.so.2 on i686 ppc s390 uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp-libs	 	usr/lib64/libpcp.so.3 on x86_64 ppc64 s390x uses function gethostbyname, which may impact IPv6 support
VERIFY	 	pcp-libs	 	usr/lib64/libpcp.so.3 on x86_64 ppc64 s390x uses function inet_ntoa, which may impact IPv6 support
VERIFY	 	pcp-libs	 	usr/lib64/libpcp_trace.so.2 on x86_64 ppc64 s390x uses function gethostbyname, which may impact IPv6 support


--
Nathan


--------------050002070201000702080905-- From brolley@redhat.com Wed Apr 23 12:38:03 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 669057F52 for ; Wed, 23 Apr 2014 12:38:03 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id EC20EAC014 for ; Wed, 23 Apr 2014 10:38:02 -0700 (PDT) X-ASG-Debug-ID: 1398274680-04bdf045542a9c60001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Iq509z0Pm1WpM1or for ; Wed, 23 Apr 2014 10:38:00 -0700 (PDT) X-Barracuda-Envelope-From: brolley@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3NHc0eQ013778 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 23 Apr 2014 13:38:00 -0400 Received: from [10.10.51.125] (vpn-51-125.rdu2.redhat.com [10.10.51.125]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3NHbxER014401 for ; Wed, 23 Apr 2014 13:37:59 -0400 Message-ID: <5357FADD.9090909@redhat.com> Date: Wed, 23 Apr 2014 13:39:41 -0400 From: Dave Brolley User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: PCP Mailing List Subject: PCP Updates: Compilation Error When Buiding on Ubuntu 14.04 (kernel: 3.13) Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: PCP Updates: Compilation Error When Buiding on Ubuntu 14.04 (kernel: 3.13) Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398274680 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 This came in on IRC last Friday, April 18. As usual, on the brolley/dev branch of pcpfans. Dave ------------------------------------------------------------------------------------ commit fdeeea289ec75039c17c2aec037e82b9730a56c5 Author: Dave Brolley Date: Fri Apr 18 14:52:42 2014 -0400 src/libpcp/src/getdate.y: Conflicting declaration and definition of yylex() union YYSTYPE; static int yylex (union YYSTYPE *, parser_control *); [ ... ] static int yylex(YYSTYPE * lvalp, parser_control * pc) {...} Need 'union YYSTYPE *' in the definition. From nwf@hum.ku.dk Wed Apr 23 14:42:09 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id F02DC7F52 for ; Wed, 23 Apr 2014 14:42:08 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id CADDE8F8054 for ; Wed, 23 Apr 2014 12:42:08 -0700 (PDT) X-ASG-Debug-ID: 1398282123-04cb6c243829fba0001-S8gJnT Received: from mailgate2.sc.ku.dk (mailgate2.sc.ku.dk [192.38.29.235]) by cuda.sgi.com with ESMTP id RVFCJmDFcIqgpKlw (version=TLSv1 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 23 Apr 2014 12:42:04 -0700 (PDT) X-Barracuda-Envelope-From: nwf@hum.ku.dk X-Barracuda-Apparent-Source-IP: 192.38.29.235 Received: from localhost (localhost [127.0.0.1]) by mailgate2.sc.ku.dk (Postfix) with ESMTP id BD39C100AA48; Wed, 23 Apr 2014 21:42:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at sc.ku.dk Received: from mailgate2.sc.ku.dk ([127.0.0.1]) by localhost (mailgate2.sc.ku.dk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K4Djhq7HRlb2; Wed, 23 Apr 2014 21:41:58 +0200 (CEST) Received: from post.hum.ku.dk (excas1.sc.ku.dk [130.225.200.137]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "post.hum.ku.dk", Issuer "GeoTrust SSL CA" (verified OK)) by mailgate2.sc.ku.dk (Postfix) with ESMTPS; Wed, 23 Apr 2014 21:41:57 +0200 (CEST) Received: from exmbx1.hum2005.hum.ku.dk ([fe80::bdef:f841:cc92:b6b1]) by excas1.hum2005.hum.ku.dk ([::1]) with mapi id 14.03.0181.006; Wed, 23 Apr 2014 21:41:47 +0200 From: Niels Werner Frederiksen Subject: =?iso-8859-1?Q?Mise_=E0_niveau_du_Webmel?= Thread-Topic: =?iso-8859-1?Q?Mise_=E0_niveau_du_Webmel?= X-ASG-Orig-Subj: =?iso-8859-1?Q?Mise_=E0_niveau_du_Webmel?= Thread-Index: Ac9fLA9pzXuCs1vKQbGQkI+mUHONAA== Date: Wed, 23 Apr 2014 19:41:46 +0000 Message-ID: Accept-Language: da-DK, en-US Content-Language: da-DK X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [41.203.67.141] Content-Type: multipart/alternative; boundary="_000_D5E2A3D24642354EB842EE73CE642C5C2857526Dexmbx1hum2005hu_" MIME-Version: 1.0 X-Barracuda-Connect: mailgate2.sc.ku.dk[192.38.29.235] X-Barracuda-Start-Time: 1398282124 X-Barracuda-Encrypted: AES256-SHA X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.23 X-Barracuda-Spam-Status: No, SCORE=1.23 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=HTML_MESSAGE, MISSING_HEADERS, THREAD_INDEX, THREAD_TOPIC, TO_CC_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5192 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 1.21 MISSING_HEADERS Missing To: header 0.00 HTML_MESSAGE BODY: HTML included in message 0.00 TO_CC_NONE No To: or Cc: header To: undisclosed-recipients:; --_000_D5E2A3D24642354EB842EE73CE642C5C2857526Dexmbx1hum2005hu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux messages j= usqu'=E0 ce que vous mettez =E0 niveau votre quota de courriel. Copiez le lien ci-dessous et remplissez le formulaire pour mettre =E0 nivea= u votre compte. https://www.formstack.com/forms/?1732174-XcprLnLOaP Administrateur syst=E8me 192.168.0.1 --_000_D5E2A3D24642354EB842EE73CE642C5C2857526Dexmbx1hum2005hu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux messa= ges jusqu'=E0 ce que vous mettez =E0 niveau votre
quota de courriel.

Copiez le lien ci-dessous et remplissez le formulaire pour mettre =E0 = niveau votre compte.


https://www.formstack.com/forms/?1732174-XcprLnLOaP

Administrateur syst=E8me
192.168.0.1
--_000_D5E2A3D24642354EB842EE73CE642C5C2857526Dexmbx1hum2005hu_-- From nscott@redhat.com Wed Apr 23 22:49:47 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 700807F52 for ; Wed, 23 Apr 2014 22:49:47 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id F357FAC007 for ; Wed, 23 Apr 2014 20:49:43 -0700 (PDT) X-ASG-Debug-ID: 1398311378-04cb6c2b7a1eeb0001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id eOvX71iQCVNGN0Xi for ; Wed, 23 Apr 2014 20:49:38 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3O3ncZO019389; Wed, 23 Apr 2014 23:49:38 -0400 Date: Wed, 23 Apr 2014 23:49:38 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Dave Brolley Cc: PCP Mailing List Message-ID: <1204878828.10178271.1398311378294.JavaMail.zimbra@redhat.com> In-Reply-To: <5357FA37.2070202@redhat.com> References: <360594769.5718335.1381470346334.JavaMail.root@redhat.com> <5357FA37.2070202@redhat.com> Subject: Re: [pcp] PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API Thread-Index: /mrqpUfkWEGCg4AOTAlP521FpOpGhA== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1398311378 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5203 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ----- Original Message ----- > These commits, on the brolley/dev branch of pcpfans remove the use of > inet-specific APIs (i.e. no ipv6) in favour of the libpcp I/O APIs (which do > support IPv6, where available). > > The original message from Nathan which inspired this can be found below. > > Some __pm level functions which were previously static were moved to impl.h > in order to accomplish this. > > A local qa run shows no regressions. Looking good Dave. I hit one regression over in test 589 - looks like telnet-probe is not quite right. Diff below, I've push the merge into my own dev branch on oss; its the same as yours with a pmdacisco merge conflict sorted out. The failure looks like a non-blocking I/O issue of some kind during connect(2). $ diff 589.out 589.out.bad 12,13c12,13 < exit status is 0 (should be 0) < exit status is 0 (should be 0) --- > exit status is 1 (should be 0) > exit status is 1 (should be 0) 16c16 < connect: Connection refused --- > connect: Operation now in progress 19c19 < connect: Connection refused --- > connect: Operation now in progress This test requires a remote host, but even without one you should be able to reproduce it using localhost/22 (see 2nd part of 589) - lemme know if you have no luck and I'll dig deeper. cheers. -- Nathan From nscott@redhat.com Wed Apr 23 23:00:59 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 85D007F52 for ; Wed, 23 Apr 2014 23:00:59 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id EC45CAC007 for ; Wed, 23 Apr 2014 21:00:58 -0700 (PDT) X-ASG-Debug-ID: 1398312056-04cbb04b911fb90001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id NtBOpGzT4lxawFK5 for ; Wed, 23 Apr 2014 21:00:56 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3O40upV032181 for ; Thu, 24 Apr 2014 00:00:56 -0400 Date: Thu, 24 Apr 2014 00:00:56 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: PCP Mailing List Message-ID: <751138293.10179218.1398312056455.JavaMail.zimbra@redhat.com> In-Reply-To: <51235430.10178448.1398311553701.JavaMail.zimbra@redhat.com> Subject: pcp updates: log tools longopts, pmdacisco MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: log tools longopts, pmdacisco Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: log tools longopts, pmdacisco Thread-Index: sK/Som4BuLtsVjYP3GZqQuydKAB2Ag== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1398312056 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5204 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.performancecopilot.org/pcp.git dev qa/.gitignore | 2 qa/363 | 13 - qa/363.out | 134 ++++++++++++++++ qa/363.out.1 | 144 ----------------- qa/363.out.2 | 134 ---------------- qa/379 | 4 qa/417.out | 2 qa/440.out | 16 - qa/492 | 10 - qa/492.out | 122 ++++++++++++++ qa/492.out.1 | 128 --------------- qa/492.out.2 | 128 --------------- src/pmdas/cisco/.gitignore | 2 src/pmdas/cisco/GNUmakefile | 20 -- src/pmdas/cisco/parse.sh | 3 src/pmdas/cisco/pmda.c | 97 ++++++----- src/pmdas/cisco/probe.c | 15 + src/pmlogcheck/.gitignore | 1 src/pmlogcheck/pmlogcheck.c | 91 +++++----- src/pmlogextract/pmlogextract.c | 154 ++++++------------ src/pmlogger/src/pmlogger.c | 178 +++++++++++---------- src/pmloglabel/pmloglabel.c | 71 ++++---- src/pmlogreduce/GNUmakefile | 3 src/pmlogreduce/pmlogreduce.c | 186 +++++++++++++++++----- src/pmlogreduce/pmlogreduce.h | 4 src/pmlogreduce/util.c | 132 --------------- src/pmlogrewrite/pmlogrewrite.c | 127 +++++++-------- src/pmlogsummary/pmlogcheck.c | 284 +++++++++++---------------------- src/pmlogsummary/pmlogsummary.c | 335 ++++++++++++++-------------------------- 29 files changed, 1009 insertions(+), 1531 deletions(-) commit 8da53d2672b84c1221aa4d2aa8f54a8d648cf86d Author: Nathan Scott Date: Thu Apr 24 11:10:29 2014 +1000 Rework the parse mode in pmdacisco for QA testing efficiency Change the Cisco PMDA parse mode to tackle intermittent failure in test qa/379. Simply moving a diagnostic to before a thread is started removed the main race. But, digging into it, found the test running significantly longer than it should due to many (unresolvable) DNS lookups - so, allow those to be disabled too for QA, making it run alot quicker (1 minute -> 1 second). Instead of building pmdacisco twice (second time for parse), we now use a shell script front for pmdacisco and a single binary. commit 5b9c468bd476be4ab12c1a78566b87ff7f202cca Author: Nathan Scott Date: Thu Apr 24 11:04:22 2014 +1000 Convert PCP archive tools over to long options interfaces This coverts pmlogcheck, pmlogextract, pmlogger, pmloglabel, pmlogreduce, pmlogrewrite and pmlogsummary to making use of the long option interfaces. From nscott@redhat.com Wed Apr 23 23:06:18 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 4EF0B7F52 for ; Wed, 23 Apr 2014 23:06:18 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 2FD0C8F804B for ; Wed, 23 Apr 2014 21:06:15 -0700 (PDT) X-ASG-Debug-ID: 1398312373-04cb6c2b7d1fed0001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id hkloHHjd0X6gfVmQ for ; Wed, 23 Apr 2014 21:06:14 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3O46AVR001219; Thu, 24 Apr 2014 00:06:10 -0400 Date: Thu, 24 Apr 2014 00:06:10 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: Ken McDonell Cc: PCP Mailing List Message-ID: <572140880.10180616.1398312370352.JavaMail.zimbra@redhat.com> In-Reply-To: <751138293.10179218.1398312056455.JavaMail.zimbra@redhat.com> References: <751138293.10179218.1398312056455.JavaMail.zimbra@redhat.com> Subject: Re: [pcp] pcp updates: log tools longopts, pmdacisco MIME-Version: 1.0 X-ASG-Orig-Subj: Re: [pcp] pcp updates: log tools longopts, pmdacisco Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: log tools longopts, pmdacisco Thread-Index: sK/Som4BuLtsVjYP3GZqQuydKAB2AmZDp2og X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1398312373 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5203 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Hi Ken, Could I get you to pass an especially critical eyeball over the following commit please? I think everything still works exactly as before there (except qa/379 completes quickly for me now). I left the (newly added) debugging options undocumented on purpose (like -D usually is). thanks! ----- Original Message ----- > Changes committed to git://git.performancecopilot.org/pcp.git dev > > qa/379 | 4 > [...] > src/pmdas/cisco/.gitignore | 2 > src/pmdas/cisco/GNUmakefile | 20 -- > src/pmdas/cisco/parse.sh | 3 > src/pmdas/cisco/pmda.c | 97 ++++++----- > src/pmdas/cisco/probe.c | 15 + > [...] > commit 8da53d2672b84c1221aa4d2aa8f54a8d648cf86d > Author: Nathan Scott > Date: Thu Apr 24 11:10:29 2014 +1000 > > Rework the parse mode in pmdacisco for QA testing efficiency > > Change the Cisco PMDA parse mode to tackle intermittent failure > in test qa/379. Simply moving a diagnostic to before a thread > is started removed the main race. But, digging into it, found > the test running significantly longer than it should due to many > (unresolvable) DNS lookups - so, allow those to be disabled too > for QA, making it run alot quicker (1 minute -> 1 second). > > Instead of building pmdacisco twice (second time for parse), we > now use a shell script front for pmdacisco and a single binary. From nscott@redhat.com Thu Apr 24 03:46:26 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 481C17F52 for ; Thu, 24 Apr 2014 03:46:26 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 287238F8039 for ; Thu, 24 Apr 2014 01:46:22 -0700 (PDT) X-ASG-Debug-ID: 1398329177-04cbb04b9229900001-S8gJnT Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id aWCn4J23Hx3k3Mbg for ; Thu, 24 Apr 2014 01:46:17 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.24 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3O8kHYR005672; Thu, 24 Apr 2014 04:46:17 -0400 Date: Thu, 24 Apr 2014 04:46:17 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: "Frank Ch. Eigler" Cc: pcp developers Message-ID: <254689105.10367537.1398329177136.JavaMail.zimbra@redhat.com> In-Reply-To: <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> Subject: Re: graphite interfacing prototype MIME-Version: 1.0 X-ASG-Orig-Subj: Re: graphite interfacing prototype Content-Type: multipart/mixed; boundary="----=_Part_10367535_104339014.1398329177134" X-Originating-IP: [10.5.82.12] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: fche/for-merge updates (was Re: [pcp] graphite interfacing prototype) Thread-Index: jBeK0Ew1KmDcoyOtnTrqoVkZkNU/SIbm4y// X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24] X-Barracuda-Start-Time: 1398329177 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.03 X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5208 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain ------=_Part_10367535_104339014.1398329177134 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- Original Message ----- > ----- Original Message ----- > > Hi - > > [...] > > Thanks for looking into the graphite support, haven't reviewed it yet > but have fwd'd your post on to several people who have asked for this > and feedback has been universally positive & grateful! :) > Thanks for sending this early for review - here's some initial feedback... Usage: The explanatory text usage message issue is a missing piece of functionality in the underlying APIs I s'pose - the attached patch may be one way we could tackle this (untested at this stage, but I'll happily take that on if you think this approach might suit the need here?) Units: The handling of units conversion feels problematic. We have potentially large numbers of very different metrics we'll want to pump through this into graphite, but just the one units specification for all of them (AIUI) via: self.opts.pmSetLongOption("units", 1, 'u', '', "rescale all metric units (e.g., \"mbyte/sec\")") It seems like we'll need a more expressive configuration mechanism. Instead of having the command line specify a single units and then many metrics, how about a config file which allows all metrics to be specified, and also provides options for conversion of individual metrics to specific units? Also, the default could be to convert to canonical units across the board (seconds, bytes, etc) - this is what pmie does IIRC. Daemon: Following on from that, I imagine users will want to run this as a daemon, often, feeding data constantly (hence the auto-socket-reconnect you've added I guess?). Thus, a configuration file seems appropriate, but we'll also need to think about init scripts down the track as well. Rate Conversion: The "mbyte/sec" reference in the above code line I quoted suggests rate conversion of counter metrics can be done, but I don't see that anywhere. We'll need to add in some memory about the previous fetch - aaand hello complexity. :| At this point, I starting to wonder if maybe this needs to be encapsulated and supported by external module (see below) since its getting complex for what we'd like to be a simple script. The existing (unused) pmcc module seems like it'll be a suitable helper module for this script, to handle that side of things (and perhaps lots more, over time) - it has the concept of the-previous-sampled-value already. It is still simplistic though - noones using it yet AFAICT, so you will be breaking new ground there I think. Archives: I think this will just work, as is now - in the last round of changes, the time window handling and pmSetMode was added in as default-to-on, so this: # self.opts.pmSetLongOptionArchive() # -a FILE -- not supported yet; need -S/etc. controls should just work, and you should be able to just add in the start/finish/align options, and they should automatically do the right thing. This will help when it comes to automated testing - but it'll also be useful for handling exporting in the case that the graphite server was done, during an outage or something (we used to hit that with our data warehouse exporting, all the time - in a previously life). Generality: Marko pointed us toward Sensu, which looks like it has a very similar interface to Graphite for its importing of new metrics. This makes me think we should push as much of the config file handling as we can into the shared pmcc module, so we can write an exporter for that tool also (and likely others later), sharing as much code as possible. Callbacks maybe (or inheritance, or whatever), for setting behaviour when converting metric/instance names to local conventions, would be a good way to go. Instances: While we're talking instance names, I noticed the call to pmNameIndom in the code that cracks open the fetch result. Thats a round trip for each instance name - caching this is a good idea. Perhaps pmcc could be taught to help us out there as well (its caching values already). Maybe something it already does, not sure - haven't looked closely enough - but it would be handy. Testing: Its very early to have added this, I fully understand - just want to give suggestions though. As mentioned earlier, the archive mode could help here - a test which replays from an archive, and runs nc(1) to capture the output would be nice and deterministic. OTOH, instead of nc(1) it may be better to write a python nc-alike script so both pickled and text modes can be QA'd? Unrelated, but any config file bits that we can make generic and add to pmcc.py would be testable by a targetting script, like Stan did with the pmapi API test. I think there is an early attempt at a pmcc test, but its gonna need some love, and is not run as part of the automated QA yet - it'd be great to get it going, since we ship that module already. :| Packaging: I was a bit surprised to see this done as a pcp(1) command, its more a "native" PCP command than a wrapped system command - I'd recommend it live in /usr/bin - front-and-centre, such that people get it on their default path for nice easy access (rather than hidden away in a non-name-conflicting place like PCP_BINADM_DIR). We also have several import scripts (collectl2pcp, mrtg2pcp, and so on) - this is the first export script, so a name like /usr/bin/pcp2graphite would be ideal. As you mention in the blog post, a graphite2pcp might well pop into existence at some point too. These have tended to be packaged in separate RPMs - we could either go for one pcp-graphite or separate graphite2pcp and pcp2graphite packages here. Since an init script seems likely, and its only really for graphite folks, I'd prefer not to have it reside in the core pcp package. pmParseUnitStr: Yep, agreed - we should do this in libpcp as you say, and percolate it up through the python layers. Oh finally, I usually forget then Stan reminds me - pylint. It'll complain about the length of the __init__ function for sure, probably a bunch of other stuff that I'm not noticing, its fairly picky. cheers. -- Nathan ------=_Part_10367535_104339014.1398329177134 Content-Type: text/x-patch; name=getopts-descriptions.patch Content-Disposition: attachment; filename=getopts-descriptions.patch Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9pbmNsdWRlL3BjcC9wbWFwaS5oIGIvc3JjL2luY2x1ZGUvcGNwL3Bt YXBpLmgKaW5kZXggYmQ2YzU2NS4uY2RhZTBhMyAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvcGNw L3BtYXBpLmgKKysrIGIvc3JjL2luY2x1ZGUvcGNwL3BtYXBpLmgKQEAgLTYwOCw2ICs2MDgsNyBA QCBleHRlcm4gY2hhciAqcG1HZXRDb25maWcoY29uc3QgY2hhciAqKTsKIAogI2RlZmluZSBQTUFQ SV9PUFRJT05TCSJBOmE6RDpnaDpuOk86cDpTOnM6VDp0OlZaOno/IgogI2RlZmluZSBQTUFQSV9P UFRJT05TX0hFQURFUihzKQl7ICIiLCAwLCAnLScsIDAsIChzKSB9CisjZGVmaW5lIFBNQVBJX09Q VElPTlNfVEVYVChzKQl7ICIiLCAwLCAnKycsIDAsIChzKSB9CiAjZGVmaW5lIFBNQVBJX09QVElP TlNfRU5ECXsgTlVMTCwgMCwgMCwgMCwgTlVMTCB9CiAKICNkZWZpbmUgUE1PUFRfQUxJR04JeyAi YWxpZ24iLAkxLCAnQScsCSJUSU1FIiwgXApkaWZmIC0tZ2l0IGEvc3JjL2xpYnBjcC9zcmMvZ2V0 b3B0LmMgYi9zcmMvbGlicGNwL3NyYy9nZXRvcHQuYwppbmRleCA1ZjlkMzdiLi4wMzJjZGQ5IDEw MDY0NAotLS0gYS9zcmMvbGlicGNwL3NyYy9nZXRvcHQuYworKysgYi9zcmMvbGlicGNwL3NyYy9n ZXRvcHQuYwpAQCAtNzU5LDYgKzc1OSwxMCBAQCBwbVVzYWdlTWVzc2FnZShwbU9wdGlvbnMgKm9w dHMpCiAJICAgIHBtcHJpbnRmKCJcbiVzOlxuIiwgb3B0aW9uLT5tZXNzYWdlKTsKIAkgICAgY29u dGludWU7CiAgICAgICAgIH0KKwlpZiAob3B0aW9uLT5zaG9ydF9vcHQgPT0gJysnKSB7CS8qIGRl c2NyaXB0aXZlIHRleHQgKi8KKwkgICAgcG1wcmludGYoIiVzXG4iLCBvcHRpb24tPm1lc3NhZ2Up OworCSAgICBjb250aW51ZTsKKyAgICAgICAgfQogCiAJbWVzc2FnZSA9IG9wdGlvbi0+YXJnbmFt ZSA/IG9wdGlvbi0+YXJnbmFtZSA6ICI/IjsKIAlpZiAob3B0aW9uLT5sb25nX29wdCAmJiBvcHRp b24tPmxvbmdfb3B0WzBdICE9ICdcMCcpIHsKZGlmZiAtLWdpdCBhL3NyYy9weXRob24vcGNwL3Bt YXBpLnB5IGIvc3JjL3B5dGhvbi9wY3AvcG1hcGkucHkKaW5kZXggY2QxMDk4OC4uNDI5NTRiNSAx MDA2NDQKLS0tIGEvc3JjL3B5dGhvbi9wY3AvcG1hcGkucHkKKysrIGIvc3JjL3B5dGhvbi9wY3Av cG1hcGkucHkKQEAgLTc1MCw2ICs3NTAsMTAgQEAgY2xhc3MgcG1PcHRpb25zKG9iamVjdCk6CiAg ICAgICAgICIiIiBBZGQgYSBuZXcgc2VjdGlvbiBoZWFkaW5nIGludG8gdGhlIGxvbmcgb3B0aW9u IHVzYWdlIG1lc3NhZ2UgIiIiCiAgICAgICAgIHJldHVybiBjX2FwaS5wbVNldExvbmdPcHRpb25I ZWFkZXIoaGVhZGluZykKIAorICAgIGRlZiBwbVNldExvbmdPcHRpb25EZXNjcmlwdGlvbihzZWxm LCB0ZXh0KToKKyAgICAgICAgIiIiIEFkZCBzb21lIGRlc2NyaXB0aXZlIHRleHQgaW50byB0aGUg bG9uZyBvcHRpb24gdXNhZ2UgbWVzc2FnZSAiIiIKKyAgICAgICAgcmV0dXJuIGNfYXBpLnBtU2V0 TG9uZ09wdGlvbkRlc2NyaXB0aW9uKHRleHQpCisKICAgICBkZWYgcG1TZXRMb25nT3B0aW9uQWxp Z24oc2VsZik6CiAgICAgICAgICIiIiBBZGQgc3VwcG9ydCBmb3IgLUEvLS1hbGlnbiBpbnRvIFBN QVBJIG1vbml0b3IgdG9vbCAiIiIKICAgICAgICAgcmV0dXJuIGNfYXBpLnBtU2V0TG9uZ09wdGlv bkFsaWduKCkKZGlmZiAtLWdpdCBhL3NyYy9weXRob24vcG1hcGkuYyBiL3NyYy9weXRob24vcG1h cGkuYwppbmRleCA1YmM5ZjVmLi43NDg0N2VkIDEwMDY0NAotLS0gYS9zcmMvcHl0aG9uL3BtYXBp LmMKKysrIGIvc3JjL3B5dGhvbi9wbWFwaS5jCkBAIC0yMDEsNiArMjAxLDI3IEBAIHNldExvbmdP cHRpb25IZWFkZXIoUHlPYmplY3QgKnNlbGYsIFB5T2JqZWN0ICphcmdzLCBQeU9iamVjdCAqa2V5 d29yZHMpCiB9CiAKIHN0YXRpYyBQeU9iamVjdCAqCitzZXRMb25nT3B0aW9uRGVzY3JpcHRpb24o UHlPYmplY3QgKnNlbGYsIFB5T2JqZWN0ICphcmdzLCBQeU9iamVjdCAqa2V5d29yZHMpCit7Cisg ICAgcG1Mb25nT3B0aW9ucyB0ZXh0ID0gUE1BUElfT1BUSU9OU19ERVNDUklQVElPTigiIik7Cisg ICAgY2hhciAqa2V5d29yZF9saXN0W10gPSB7InRleHQiLCBOVUxMfTsKKworICAgIGlmICghUHlB cmdfUGFyc2VUdXBsZUFuZEtleXdvcmRzKGFyZ3MsIGtleXdvcmRzLAorCQkJInM6cG1TZXRMb25n T3B0aW9uRGVzY3JpcHRpb24iLCBrZXl3b3JkX2xpc3QsCisJCQkmdGV4dC5tZXNzYWdlKSkKKwly ZXR1cm4gTlVMTDsKKyAgICBpZiAoKHRleHQubWVzc2FnZSA9IHN0cmR1cCh0ZXh0Lm1lc3NhZ2Up KSA9PSBOVUxMKQorCXJldHVybiBQeUVycl9Ob01lbW9yeSgpOworCisgICAgaWYgKGFkZExvbmdP cHRpb24oJnRleHQsIDApIDwgMCkgeworCWZyZWUoKGNoYXIgKil0ZXh0Lm1lc3NhZ2UpOworCXJl dHVybiBQeUVycl9Ob01lbW9yeSgpOworICAgIH0KKyAgICBQeV9JTkNSRUYoUHlfTm9uZSk7Cisg ICAgcmV0dXJuIFB5X05vbmU7Cit9CisKK3N0YXRpYyBQeU9iamVjdCAqCiBhZGRMb25nT3B0aW9u T2JqZWN0KHBtTG9uZ09wdGlvbnMgKm9wdGlvbikKIHsKICAgICBpZiAoYWRkTG9uZ09wdGlvbihv cHRpb24sIDEpIDwgMCkKQEAgLTg3Myw2ICs4OTQsOSBAQCBzdGF0aWMgUHlNZXRob2REZWYgbWV0 aG9kc1tdID0gewogICAgIHsgLm1sX25hbWUgPSAicG1TZXRMb25nT3B0aW9uSGVhZGVyIiwKIAku bWxfbWV0aCA9IChQeUNGdW5jdGlvbikgc2V0TG9uZ09wdGlvbkhlYWRlciwKICAgICAgICAgLm1s X2ZsYWdzID0gTUVUSF9WQVJBUkdTIHwgTUVUSF9LRVlXT1JEUyB9LAorICAgIHsgLm1sX25hbWUg PSAicG1TZXRMb25nT3B0aW9uRGVzY3JpcHRpb24iLAorCS5tbF9tZXRoID0gKFB5Q0Z1bmN0aW9u KSBzZXRMb25nT3B0aW9uRGVzY3JpcHRpb24sCisgICAgICAgIC5tbF9mbGFncyA9IE1FVEhfVkFS QVJHUyB8IE1FVEhfS0VZV09SRFMgfSwKICAgICB7IC5tbF9uYW1lID0gInBtU2V0TG9uZ09wdGlv biIsCiAJLm1sX21ldGggPSAoUHlDRnVuY3Rpb24pIHNldExvbmdPcHRpb24sCiAgICAgICAgIC5t bF9mbGFncyA9IE1FVEhfVkFSQVJHUyB8IE1FVEhfS0VZV09SRFMgfSwK ------=_Part_10367535_104339014.1398329177134-- From SRS0=e0nMTe=ZY=info.com=info@yourhostingaccount.com Thu Apr 24 04:23:17 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 0655C7F51 for ; Thu, 24 Apr 2014 04:23:17 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id A555DAC016 for ; Thu, 24 Apr 2014 02:23:16 -0700 (PDT) X-ASG-Debug-ID: 1398331391-04bdf05c7529870001-S8gJnT Received: from mailout13.yourhostingaccount.com (mailout13.yourhostingaccount.com [65.254.253.107]) by cuda.sgi.com with ESMTP id kG1Oi77RMz2Dq7ET for ; Thu, 24 Apr 2014 02:23:12 -0700 (PDT) X-Barracuda-Envelope-From: SRS0=e0nMTe=ZY=info.com=info@yourhostingaccount.com X-Barracuda-Apparent-Source-IP: 65.254.253.107 Received: from mailscan06.yourhostingaccount.com ([10.1.15.6] helo=mailscan06.yourhostingaccount.com) by mailout13.yourhostingaccount.com with esmtp (Exim) id 1WdFs7-0000zS-Gs for pcp@oss.sgi.com; Thu, 24 Apr 2014 05:23:11 -0400 Received: from impout02.yourhostingaccount.com ([10.1.55.2] helo=impout02.yourhostingaccount.com) by mailscan06.yourhostingaccount.com with esmtp (Exim) id 1WdFrz-0008Lr-Lo; Thu, 24 Apr 2014 05:23:03 -0400 Received: from webmail07.yourhostingaccount.com ([10.1.16.7]) by impout02.yourhostingaccount.com with NO UCE id tlP11n00m098Zia01lP1jC; Thu, 24 Apr 2014 05:23:02 -0400 X-Authority-Analysis: v=2.0 cv=XZQLPfF5 c=1 sm=1 a=kfqE/iRSlp6rKD2zGfQtkg==:17 a=9cW_t1CCXrUA:10 a=Q7zus9ReCAYA:10 a=wHWL7Wht9LMA:10 a=BwVQVwQTPqIA:10 a=jPJDawAOAc8A:10 a=8nJEP1OIZ-IA:10 a=BqjLhEBqAAAA:8 a=_FZyd2kkAAAA:8 a=9LoNXIGrAAAA:20 a=lPIDSOEwlpviVq_WiGkA:9 a=wPNLvfGTeEIA:10 a=m01mqmSmUL0A:10 a=N9cELZsY5McA:10 a=qQATh_Prau8A:10 a=qnR/rrf4y7rOidud/oaubA==:117 X-EN-OrigOutIP: 10.1.16.7 X-EN-IMPSID: tlP11n00m098Zia01lP1jC Received: from [127.0.0.1] (helo=email.powweb.com) by webmail07.yourhostingaccount.com with esmtp (Exim) id 1WdFrp-0002j5-89; Thu, 24 Apr 2014 05:22:53 -0400 Received: from 41.203.67.137 (SquirrelMail authenticated user harnold@harnold.powweb.com) by email.powweb.com with HTTP; Thu, 24 Apr 2014 05:22:53 -0400 Message-ID: <69d61e4320035efc8f9001afbce6b5a1.squirrel@email.powweb.com> Date: Thu, 24 Apr 2014 05:22:53 -0400 Subject: Mise =?iso-8859-1?Q?=E0_niveau_du_Webmel?= From: "Webmel Account Support" X-ASG-Orig-Subj: Mise =?iso-8859-1?Q?=E0_niveau_du_Webmel?= User-Agent: SquirrelMail/1.4.19 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Sender: "Webmel Account Support" X-Barracuda-Connect: mailout13.yourhostingaccount.com[65.254.253.107] X-Barracuda-Start-Time: 1398331391 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 1.21 X-Barracuda-Spam-Status: No, SCORE=1.21 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=MISSING_HEADERS, TO_CC_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5209 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 1.21 MISSING_HEADERS Missing To: header 0.00 TO_CC_NONE No To: or Cc: header To: undisclosed-recipients:; Vous ne serez pas en mesure d'envoyer ou de recevoir de nouveaux messages jusqu' ce que vous mettez niveau votrequota de courriel. Copiez le lien ci-dessous et remplissez le formulaire pour mettre niveau votre compte. https://www.formstack.com/forms/?1732174-XcprLnLOaP Administrateur systme192.168.0.1 From fche@redhat.com Thu Apr 24 14:29:15 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 036117F52 for ; Thu, 24 Apr 2014 14:29:15 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id D52C78F8040 for ; Thu, 24 Apr 2014 12:29:11 -0700 (PDT) X-ASG-Debug-ID: 1398367747-04cb6c58bc21ab0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id MT1Xy7iuDOLx2pKO for ; Thu, 24 Apr 2014 12:29:07 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3OJT6T4009032 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 24 Apr 2014 15:29:07 -0400 Received: from fche.csb (vpn-51-135.rdu2.redhat.com [10.10.51.135]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3OJT6Ne001285; Thu, 24 Apr 2014 15:29:06 -0400 Received: by fche.csb (Postfix, from userid 2569) id 1349F58154; Thu, 24 Apr 2014 15:29:05 -0400 (EDT) Date: Thu, 24 Apr 2014 15:29:05 -0400 From: "Frank Ch. Eigler" To: Nathan Scott Cc: pcp developers Subject: Re: graphite interfacing prototype Message-ID: <20140424192905.GA3575@redhat.com> X-ASG-Orig-Subj: Re: graphite interfacing prototype References: <20140414021313.GH14108@redhat.com> <466664093.5020375.1397463467942.JavaMail.zimbra@redhat.com> <254689105.10367537.1398329177136.JavaMail.zimbra@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <254689105.10367537.1398329177136.JavaMail.zimbra@redhat.com> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398367747 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Nathan - > [...] > Thanks for sending this early for review - here's some > initial feedback... (Technically, it wasn't even an RFC yet!) Thanks for your suggestions, every single one a good point, and will incorporate them. - FChE From brolley@redhat.com Thu Apr 24 16:34:39 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 048127F52 for ; Thu, 24 Apr 2014 16:34:39 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id BAF0C8F8052 for ; Thu, 24 Apr 2014 14:34:35 -0700 (PDT) X-ASG-Debug-ID: 1398375274-04cbb04b9060940001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id sDKh8CUsGKJOdxp4 for ; Thu, 24 Apr 2014 14:34:35 -0700 (PDT) X-Barracuda-Envelope-From: brolley@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3OLYY5U018427 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 24 Apr 2014 17:34:34 -0400 Received: from [10.15.16.200] ([10.15.16.200]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3OLYX4m011471; Thu, 24 Apr 2014 17:34:34 -0400 Message-ID: <535983D0.40509@redhat.com> Date: Thu, 24 Apr 2014 17:36:16 -0400 From: Dave Brolley User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott CC: PCP Mailing List Subject: Re: [pcp] PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API References: <360594769.5718335.1381470346334.JavaMail.root@redhat.com> <5357FA37.2070202@redhat.com> <1204878828.10178271.1398311378294.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API In-Reply-To: <1204878828.10178271.1398311378294.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398375275 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On 04/23/2014 11:49 PM, Nathan Scott wrote: > > ----- Original Message ----- >> These commits, on the brolley/dev branch of pcpfans remove the use of >> inet-specific APIs (i.e. no ipv6) in favour of the libpcp I/O APIs (which do >> support IPv6, where available). >> >> The original message from Nathan which inspired this can be found below. >> >> Some __pm level functions which were previously static were moved to impl.h >> in order to accomplish this. >> >> A local qa run shows no regressions. > Looking good Dave. I hit one regression over in test 589 - looks like > telnet-probe is not quite right. Diff below, I've push the merge into > my own dev branch on oss; its the same as yours with a pmdacisco merge > conflict sorted out. The failure looks like a non-blocking I/O issue > of some kind during connect(2). > I've investigated and fixed this problem. Currently doing a new qa run. Dave From brolley@redhat.com Thu Apr 24 21:15:04 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 108DD7F52 for ; Thu, 24 Apr 2014 21:15:04 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 86530AC007 for ; Thu, 24 Apr 2014 19:15:00 -0700 (PDT) X-ASG-Debug-ID: 1398392099-04cbb04b916e740001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id PGLv3YTEPpkwu4hS for ; Thu, 24 Apr 2014 19:14:59 -0700 (PDT) X-Barracuda-Envelope-From: brolley@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3P2EwcS013382 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 24 Apr 2014 22:14:59 -0400 Received: from [10.10.48.5] (vpn-48-5.rdu2.redhat.com [10.10.48.5]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3P2EvSt014387; Thu, 24 Apr 2014 22:14:58 -0400 Message-ID: <5359C589.4000700@redhat.com> Date: Thu, 24 Apr 2014 22:16:41 -0400 From: Dave Brolley User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nathan Scott CC: PCP Mailing List Subject: Re: [pcp] PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API References: <360594769.5718335.1381470346334.JavaMail.root@redhat.com> <5357FA37.2070202@redhat.com> <1204878828.10178271.1398311378294.JavaMail.zimbra@redhat.com> X-ASG-Orig-Subj: Re: [pcp] PCP Updates -- Remove inet-specific API Use in Favour of the libpcp I/O API In-Reply-To: <1204878828.10178271.1398311378294.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398392099 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On 04/23/2014 11:49 PM, Nathan Scott wrote: > Looking good Dave. I hit one regression over in test 589 - looks like > telnet-probe is not quite right. Diff below, I've push the merge into > my own dev branch on oss; its the same as yours with a pmdacisco merge > conflict sorted out. The failure looks like a non-blocking I/O issue > of some kind during connect(2). $ diff 589.out 589.out.bad 12,13c12,13 > < exit status is 0 (should be 0) < exit status is 0 (should be 0) --- I just pushed these to the brolley/dev branch of pcpfans. You might want to double check my resoltion of the cisco pmda conflicts. The pmda.cisco group of qa tests ran ok for me. Dave commit 8e560e72809f988621631dbf26e38414469813bc Merge: 2522e16 8da53d2 Author: Dave Brolley Date: Thu Apr 24 22:01:17 2014 -0400 Merge remote-tracking branch 'origin/dev' into brolley/dev Conflicts: src/pmdas/cisco/pmda.c src/pmdas/cisco/probe.c commit 2522e16f324ab130c0812c1f33110b252a8d0685 Author: Dave Brolley Date: Thu Apr 24 21:47:27 2014 -0400 Improve error handling for callers of __pmConnectTo(). From kenj@kenj.com.au Thu Apr 24 23:24:29 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 631A47F52 for ; Thu, 24 Apr 2014 23:24:29 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 47684304043 for ; Thu, 24 Apr 2014 21:24:29 -0700 (PDT) X-ASG-Debug-ID: 1398399863-04cb6c58bc3f990001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id ojpyYWORsOqWAnpm for ; Thu, 24 Apr 2014 21:24:23 -0700 (PDT) X-Barracuda-Envelope-From: kenj@kenj.com.au X-Barracuda-Apparent-Source-IP: 150.101.137.143 Received: from ppp118-209-90-57.lns20.mel4.internode.on.net (HELO bozo-vm.localdomain) ([118.209.90.57]) by ipmail05.adl6.internode.on.net with ESMTP; 25 Apr 2014 13:54:21 +0930 Received: by bozo-vm.localdomain (Postfix, from userid 1000) id 40379A5273; Fri, 25 Apr 2014 14:24:03 +1000 (EST) To: pcp@oss.sgi.com Subject: pcp updates - repair ABI breakage and QA Message-Id: <20140425042403.40379A5273@bozo-vm.localdomain> X-ASG-Orig-Subj: pcp updates - repair ABI breakage and QA Date: Fri, 25 Apr 2014 14:24:03 +1000 (EST) From: kenj@kenj.com.au (Ken McDonell) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1398399863 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5234 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/.gitignore | 3 qa/061 | 10 qa/061.out | 1153 ++++++++++++++++++++++++++++++++++++++++ qa/061.out.1 | 1133 --------------------------------------- qa/061.out.2 | 1153 ---------------------------------------- qa/502 | 1 qa/928 | 44 + qa/928.out | 135 ++++ qa/group | 1 qa/pmdas/slow/.gitignore | 2 qa/pmdas/slow_python/.gitignore | 2 qa/src/.gitignore | 1 qa/src/GNUlocaldefs | 2 qa/src/chkputlogresult.c | 169 +++++ src/include/pcp/impl.h | 1 src/libpcp/src/exports | 6 src/libpcp/src/logutil.c | 85 ++ src/libpcp/src/p_result.c | 183 +++++- src/libpcp_import/src/archive.c | 7 src/pmlogextract/pmlogextract.c | 14 src/pmlogger/src/callback.c | 10 src/pmlogger/src/preamble.c | 2 src/pmlogreduce/pmlogreduce.c | 4 src/pmlogreduce/scan.c | 6 src/pmlogrewrite/result.c | 4 25 files changed, 1772 insertions(+), 2359 deletions(-) commit 4f987b5cf5b2a600f170a6d943e81e075490d954 Author: Ken McDonell Date: Fri Apr 25 14:16:00 2014 +1000 .gitignore house keeping commit c897891caac7f9f4d4507c2e6403323066072cc3 Author: Ken McDonell Date: Fri Apr 25 13:57:23 2014 +1000 qa/061 - track diagnostics changes in __pmLogPutResult Also drop support for an old PCP version and go with one output file for the current PCP version. commit 7d2d3ec47f9e98edb1ae0b3ef58025b86a5569a3 Author: Ken McDonell Date: Fri Apr 25 13:56:05 2014 +1000 qa/502 - track minor change in __pmLogPutResult diagnostics commit 9cdb4714b54952f62de3223124d31ae6dba5c271 Author: Ken McDonell Date: Fri Apr 25 13:52:03 2014 +1000 qa/928 (new) - check 2 variants of __pmLogPutResult() Verify __pmLogPutresult() and __pmLogPutresult2() generate identical archives. commit bfedb60fba11e213f2d204de91fe6d67eb62c5aa Author: Ken McDonell Date: Fri Apr 25 13:44:53 2014 +1000 libpcp - redo __pmPutLogResult change to preserve ABI semantics The new code is in __pmPutLogResult2(). Both the old and the new routines are available in libpcp. Converted all use within PCP to __pmPutLogResult2(). commit ac9f1ad4fbaa6373eb6bf6ffee124b93d742a56a Author: Ken McDonell Date: Fri Apr 25 13:41:27 2014 +1000 libpcp - additional diagnostics for __pmDecodeResult Added diags under the control of (pmDebug & DBG_TRACE_PDU) && (pmDebug & DBG_TRACE_DESPERATE) for all the corrupt PDU cases to help diagnose issues. From kenj@kenj.com.au Thu Apr 24 23:42:51 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id BE9997F52 for ; Thu, 24 Apr 2014 23:42:50 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 978A68F8039 for ; Thu, 24 Apr 2014 21:42:50 -0700 (PDT) X-ASG-Debug-ID: 1398400968-04bdf05c7379710001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id z5mGFDSUdh5pNpAC for ; Thu, 24 Apr 2014 21:42:48 -0700 (PDT) X-Barracuda-Envelope-From: kenj@kenj.com.au X-Barracuda-Apparent-Source-IP: 150.101.137.143 Received: from ppp118-209-90-57.lns20.mel4.internode.on.net (HELO bozo-vm.localdomain) ([118.209.90.57]) by ipmail05.adl6.internode.on.net with ESMTP; 25 Apr 2014 14:12:45 +0930 Received: by bozo-vm.localdomain (Postfix, from userid 1000) id 101B2A5273; Fri, 25 Apr 2014 14:42:27 +1000 (EST) To: pcp@oss.sgi.com Subject: pcp updates - merge and reconcile conflicts Message-Id: <20140425044228.101B2A5273@bozo-vm.localdomain> X-ASG-Orig-Subj: pcp updates - merge and reconcile conflicts Date: Fri, 25 Apr 2014 14:42:27 +1000 (EST) From: kenj@kenj.com.au (Ken McDonell) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1398400968 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5234 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev qa/492.out.2 | 128 --------------- src/pmlogrewrite/pmlogrewrite.c | 127 +++++++------- ... commit e16b3fb7e3e91ea133da0dba31d1f122831b1cac Merge: 4f987b5 8da53d2 Author: Ken McDonell Date: Fri Apr 25 14:39:01 2014 +1000 Merge branch 'dev' of git://git.performancecopilot.org/pcp into dev Conflicts: qa/492.out.2 src/pmlogrewrite/pmlogrewrite.c Drop 492.out.2 Reconcile Ken's and Nathan's overlapping printf changes for logrewrite.c. From kenj@internode.on.net Fri Apr 25 01:54:02 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2BF017F52 for ; Fri, 25 Apr 2014 01:54:02 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 0198C304043 for ; Thu, 24 Apr 2014 23:53:58 -0700 (PDT) X-ASG-Debug-ID: 1398408833-04bdf05c737f3e0001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id dvDuTn0JyCGr4LLL for ; Thu, 24 Apr 2014 23:53:53 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AutDABgGWlN20Vo5PGdsb2JhbABZgwaDOlGEOb0DgQ8XAwEBAQE4NYIlAQEFCAIZBS4YCwwBAwIGAxEEAQEDAiMDAgIZIAoDCQgCBBMLBYgwp0ejUBeBKY0wBwaCaYFKBI9hn0Mr Received: from ppp118-209-90-57.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.90.57]) by ipmail05.adl6.internode.on.net with ESMTP; 25 Apr 2014 16:23:51 +0930 From: "Ken McDonell" To: "'Nathan Scott'" Cc: "'PCP Mailing List'" References: <751138293.10179218.1398312056455.JavaMail.zimbra@redhat.com> <572140880.10180616.1398312370352.JavaMail.zimbra@redhat.com> In-Reply-To: <572140880.10180616.1398312370352.JavaMail.zimbra@redhat.com> Subject: RE: [pcp] pcp updates: log tools longopts, pmdacisco Date: Fri, 25 Apr 2014 16:53:51 +1000 X-ASG-Orig-Subj: RE: [pcp] pcp updates: log tools longopts, pmdacisco Message-ID: <003a01cf6053$1f3e91a0$5dbbb4e0$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQLDMayrwmazM9FS8MciuzC4BG3SVwI9xuDSmShUcUA= Content-Language: en-au X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1398408833 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5237 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header > -----Original Message----- > From: Nathan Scott [mailto:nathans@redhat.com] > Sent: Thursday, 24 April 2014 2:06 PM > To: Ken McDonell > Cc: PCP Mailing List > Subject: Re: [pcp] pcp updates: log tools longopts, pmdacisco >=20 > Hi Ken, >=20 > Could I get you to pass an especially critical eyeball over the = following commit > please? =20 > ... By inspection, looks good to me ... builds and passes QA here. From kenj@internode.on.net Fri Apr 25 02:21:27 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 43D867F52 for ; Fri, 25 Apr 2014 02:21:27 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id C0D67AC007 for ; Fri, 25 Apr 2014 00:21:23 -0700 (PDT) X-ASG-Debug-ID: 1398410477-04cb6c2b7d7f900001-S8gJnT Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id amUXB67xVGcnWKKV for ; Fri, 25 Apr 2014 00:21:18 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.143 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AudDAPwLWlN20Vo5PGdsb2JhbABZgwaDOlGEOb0DgQ8XAwEBAQE4NYIlAQEFCAIZBS4jDAEDAgYDFQEEAiMDAgIZIAoDEQIEEwsFiDCnSqNSF4EpjTAHgm+BSgSPYZ9DKw Received: from ppp118-209-90-57.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.90.57]) by ipmail05.adl6.internode.on.net with ESMTP; 25 Apr 2014 16:51:17 +0930 From: "Ken McDonell" To: "'Nathan Scott'" Cc: References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <158034809.5621684.1397540389674.JavaMail.zimbra@redhat.com> In-Reply-To: <158034809.5621684.1397540389674.JavaMail.zimbra@redhat.com> Subject: RE: [pcp] pmlogger -u questions Date: Fri, 25 Apr 2014 17:21:14 +1000 X-ASG-Orig-Subj: RE: [pcp] pmlogger -u questions Message-ID: <005201cf6056$f3de4d80$db9ae880$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQHyaZW6bsfhxrCcQijEIAaglOJyoQLjeeRRAm7wCyYBhaK/UpqlGL6Q Content-Language: en-au X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1398410478 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5237 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header > -----Original Message----- > From: Nathan Scott [mailto:nathans@redhat.com] > Sent: Tuesday, 15 April 2014 3:40 PM > ... > OK. It sounds possible that "kill -9" on an active logger, followed = by > replay/logcheck may be able to induce the failures of the current = code, and > demonstrate quantifiable improvements. Maybe not reliable failure = cases, > but with sufficient iterations/entropy/gamma-rays, we should be able = to > build confidence in any changes. The script below produces 100% failures with the older pmlogger and = libpcp, and 100% passes (no "corrupt" archives) with the newer bits = where pmlogger writes unbuffered and one write per pmResult. Mark this one as "done", Ma ... 8^)> #!/bin/sh # . /etc/pcp.env #tmp=3D/var/tmp/$$ #trap "rm -f $tmp.*; exit 0" 0 1 2 3 tmp=3Dtmp rm -f $tmp.* cat <$tmp.config log mandatory on default { sample.colour } End-of-File nfail=3D0 for i in 1 2 3 4 5 6 7 8 9 10 do rm -f $tmp.0 $tmp.meta $tmp.index pmlogger -c $tmp.config -l $tmp.log -t 10msec -s 10000 -r $tmp 2>&1 = & sleep 3 kill -KILL $! wait if pmlogcheck $tmp then echo OK else nfail=3D`expr $nfail + 1` fi done echo echo "Failed $nfail of 10 attempts." From fche@redhat.com Fri Apr 25 06:09:40 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 388327F52 for ; Fri, 25 Apr 2014 06:09:40 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 1E5B58F8049 for ; Fri, 25 Apr 2014 04:09:39 -0700 (PDT) X-ASG-Debug-ID: 1398424175-04cb6c58bc55480001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id JEJx3ChxEmigaQSs for ; Fri, 25 Apr 2014 04:09:36 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3PB9VTZ006843 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 25 Apr 2014 07:09:31 -0400 Received: from fche.csb (vpn-51-135.rdu2.redhat.com [10.10.51.135]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s3PB9Ux7000782; Fri, 25 Apr 2014 07:09:30 -0400 Received: by fche.csb (Postfix, from userid 2569) id 0BEA858153; Fri, 25 Apr 2014 07:09:29 -0400 (EDT) To: "Ken McDonell" Cc: "'Nathan Scott'" , pcp@oss.sgi.com Subject: Re: pmlogger -u questions References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <158034809.5621684.1397540389674.JavaMail.zimbra@redhat.com> <005201cf6056$f3de4d80$db9ae880$@internode.on.net> X-ASG-Orig-Subj: Re: pmlogger -u questions From: fche@redhat.com (Frank Ch. Eigler) Date: Fri, 25 Apr 2014 07:09:29 -0400 In-Reply-To: <005201cf6056$f3de4d80$db9ae880$@internode.on.net> (Ken McDonell's message of "Fri, 25 Apr 2014 17:21:14 +1000") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398424175 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 "Ken McDonell" writes: > [...] The script below produces 100% failures with the older > pmlogger and libpcp, and 100% passes (no "corrupt" archives) with > the newer bits where pmlogger writes unbuffered and one write per > pmResult. Excellent! > log mandatory on default { > sample.colour > } For added exercise of the metadata/fflush code, this might give a thorough workout: log mandatory on default { proc } (with some serious forking/etc. going on in the background). - FChE From kenj@kenj.com.au Sun Apr 27 16:42:32 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 77D237F3F for ; Sun, 27 Apr 2014 16:42:32 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 6A6B4304039 for ; Sun, 27 Apr 2014 14:42:32 -0700 (PDT) X-ASG-Debug-ID: 1398634946-04cbb03cc495c40001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id FYBjClFSQOQnfGju for ; Sun, 27 Apr 2014 14:42:27 -0700 (PDT) X-Barracuda-Envelope-From: kenj@kenj.com.au X-Barracuda-Apparent-Source-IP: 150.101.137.131 Received: from ppp118-209-90-57.lns20.mel4.internode.on.net (HELO bozo-vm.localdomain) ([118.209.90.57]) by ipmail07.adl2.internode.on.net with ESMTP; 28 Apr 2014 07:12:25 +0930 Received: by bozo-vm.localdomain (Postfix, from userid 1000) id E24CCA59D9; Mon, 28 Apr 2014 07:42:06 +1000 (EST) To: pcp@oss.sgi.com Subject: pcp updates - pmlogger I/O all unbuffered now Message-Id: <20140427214206.E24CCA59D9@bozo-vm.localdomain> X-ASG-Orig-Subj: pcp updates - pmlogger I/O all unbuffered now Date: Mon, 28 Apr 2014 07:42:06 +1000 (EST) From: kenj@kenj.com.au (Ken McDonell) X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1398634946 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5294 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev man/man1/pmlc.1 | 5 + man/man1/pmlogger.1 | 23 ++++- qa/099.out | 1 qa/102.out | 9 -- qa/139.out | 1 qa/179.out | 2 qa/322 | 17 +++- qa/492.out | 2 qa/510.out | 1 qa/518 | 14 +++ qa/928.out | 74 ++++++++++++++++- qa/group | 2 qa/src/chkputlogresult.c | 14 +++ src/libpcp/src/logmeta.c | 184 ++++++++++++++++++++++---------------------- src/libpcp/src/logutil.c | 79 ++++++++++++------ src/pmlc/pmlc.c | 1 src/pmlogger/src/callback.c | 12 -- src/pmlogger/src/dopdu.c | 4 src/pmlogger/src/logger.h | 7 - src/pmlogger/src/pmlogger.c | 41 +-------- src/pmlogger/src/ports.c | 5 - src/pmlogger/src/preamble.c | 2 22 files changed, 287 insertions(+), 213 deletions(-) commit 20dde126ce6a40c4a7b39bfff661e1074cc603b0 Author: Ken McDonell Date: Mon Apr 28 07:40:13 2014 +1000 qa/518 - dodge other pmie's running at the same time Filter pcp -P output to pick pmie lines for just the pmie launched by qa/518. commit 61450c26fe10e7cbc860f220cb7e10722ebe92e3 Author: Ken McDonell Date: Mon Apr 28 07:17:35 2014 +1000 qa/492 - track change in error message format for pmlogrewrite commit 1c85848f3c64a93acc930d505c995601fd3783ab Author: Ken McDonell Date: Sun Apr 27 19:15:25 2014 +1000 pmlc - retire flush command Recent changes tp pmlogger mean the flush operation is no longer needed. Removed reference to "flush command from help text (so lots of cosmetic QA changes), toned down the man page description and turned the implementation to a no-op. The pmlc "flush" command is still (sliently) supported to provide backwards compatibility for scripts using pmlc. commit 9a980fdfd31f900adde8db11c505dbf22c293e54 Author: Ken McDonell Date: Sun Apr 27 17:06:13 2014 +1000 libpcp - __pmLogPut*() all unbuffered writes now Changes to __pmLogPutDesc, __pmLogPutInDom and __pmLogWriteLabel so that each record to an archive file is written in a single fwrite() and using unbuffered I/O for each file. commit 7357a7e8cf96b7e18f8b8cc0777d00e0d8a1894a Author: Ken McDonell Date: Sun Apr 27 17:02:09 2014 +1000 pmlogger - turn all flushing operations into no-ops All pmlogger writes are now unbuffered, with one fwrite() per logical record for each of the data, metadata and index files. So we can retire - the -u command line option - the SIGUSR1 handler - support for the flush command from pmlc All are preserved for backwards compatibility, but they are now no-ops that are either not documented any more or described as no longer useful. commit e93c254ed3b0e9165faf1d56a5122cb78db06928 Author: Ken McDonell Date: Sun Apr 27 16:59:05 2014 +1000 qa/928 - extend chkputlogresult Make sure the test program also calls __pmLogPutInDom() so we can exercise all of the archive fwrite() paths in libpcp. commit bf081792016e9fd0e1db6359056c35855d8fbd9f Author: Ken McDonell Date: Sun Apr 27 16:58:24 2014 +1000 qa/group - qa/510 exercises pmlc as well From kenj@internode.on.net Sun Apr 27 17:49:15 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 3E28E7F3F for ; Sun, 27 Apr 2014 17:49:15 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 31F58304048 for ; Sun, 27 Apr 2014 15:49:14 -0700 (PDT) X-ASG-Debug-ID: 1398638951-04cbb03cc6997b0001-S8gJnT Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id buwtaxDfunqrLJ9t for ; Sun, 27 Apr 2014 15:49:12 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.131 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AsEMAPiIXVN20Vo5/2dsb2JhbABZgwaDOsJLAwKBChd0giUBAQUIAh4SHCMMAQMCBgMVASkHGSANEQIEEwsFiDDJIReOWQeEOQSPZ5wDg0Mr Received: from ppp118-209-90-57.lns20.mel4.internode.on.net (HELO bozohorize) ([118.209.90.57]) by ipmail07.adl2.internode.on.net with ESMTP; 28 Apr 2014 08:18:51 +0930 From: "Ken McDonell" To: "'Frank Ch. Eigler'" Cc: "'Nathan Scott'" , References: <01e901cf56df$4ce97de0$e6bc79a0$@internode.on.net> <1665962954.4723287.1397437104781.JavaMail.zimbra@redhat.com> <534B4330.1060008@internode.on.net> <158034809.5621684.1397540389674.JavaMail.zimbra@redhat.com> <005201cf6056$f3de4d80$db9ae880$@internode.on.net> In-Reply-To: Subject: RE: pmlogger -u questions Date: Mon, 28 Apr 2014 08:48:36 +1000 X-ASG-Orig-Subj: RE: pmlogger -u questions Message-ID: <002201cf626a$dce6c380$96b44a80$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQHyaZW6bsfhxrCcQijEIAaglOJyoQLjeeRRAm7wCyYBhaK/UgJ4eF4BAKxoj7uajNVlsA== Content-Language: en-au X-Barracuda-Connect: ipmail07.adl2.internode.on.net[150.101.137.131] X-Barracuda-Start-Time: 1398638951 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5296 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header > -----Original Message----- > From: Frank Ch. Eigler [mailto:fche@redhat.com] > Sent: Friday, 25 April 2014 9:09 PM > ... > For added exercise of the metadata/fflush code, this might give a thorough > workout: > > log mandatory on default { > proc > } > > (with some serious forking/etc. going on in the background). I would expect this to produce _less_ failures in the old code ... because the indom will change with every fetch, which causes the metadata file to be written to, which triggers a fflush() of all the output files. However, it does expose a remaining (as of Fri) issue with the new code ... I had deliberately not converted the metadata and index writes to unbuffered and one logical record per fwrite(), and of course this example exposed a failure of the killer script (about 1 in 20 failures). So I've committed changes to make all the archive I/O unbuffered and use one fwrite() per logical record, which means there is no need for any flush operations (which used fflush() alone). What this means is I've moved the memmove() from stdio into the libpcp routines (so no real change in work), and we do more write() calls for small pmResults, pmDesc + name records and small pmInDoms, but fewer write() calls for (the more common) larger pmResults and larger pmInDoms. With these changes (a) there is no real difference in CPU load from the numbers I reported before (b) the killer script passes solidly, even for the proc metrics case From nscott@redhat.com Mon Apr 28 04:08:54 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 681D87F3F for ; Mon, 28 Apr 2014 04:08:54 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 70E3F304043 for ; Mon, 28 Apr 2014 02:08:54 -0700 (PDT) X-ASG-Debug-ID: 1398676128-04cbb03cc7bc830001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id wgtzz9AVjfyhqdEo for ; Mon, 28 Apr 2014 02:08:48 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3S98mZO018319 for ; Mon, 28 Apr 2014 05:08:48 -0400 Date: Mon, 28 Apr 2014 05:08:47 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp developers Message-ID: <1210764389.12875531.1398676127917.JavaMail.zimbra@redhat.com> In-Reply-To: <1148434840.12875406.1398676089040.JavaMail.zimbra@redhat.com> Subject: pcp updates: kenj/brolley merge MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: kenj/brolley merge Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: kenj/brolley merge Thread-Index: spP9IcvJZ7XtODAY8hjo9Rqmy+EmFA== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1398676128 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5306 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.performancecopilot.org/pcp.git dev build/rpm/fedora.spec | 5 man/man1/pmlc.1 | 5 man/man1/pmlogger.1 | 23 qa/.gitignore | 3 qa/061 | 10 qa/061.out | 1153 ++++++++++++++++++++++++++++++++++++++++ qa/061.out.1 | 1133 --------------------------------------- qa/061.out.2 | 1153 ---------------------------------------- qa/099.out | 1 qa/102.out | 9 qa/139.out | 1 qa/179.out | 2 qa/322 | 17 qa/492.out | 2 qa/492.out.2 | 2 qa/502 | 1 qa/510.out | 1 qa/518 | 14 qa/928 | 44 + qa/928.out | 209 +++++++ qa/group | 3 qa/pmdas/slow/.gitignore | 2 qa/pmdas/slow_python/.gitignore | 2 qa/src/.gitignore | 1 qa/src/GNUlocaldefs | 2 qa/src/chkputlogresult.c | 183 ++++++ src/include/pcp/impl.h | 1 src/libpcp/src/exports | 6 src/libpcp/src/logmeta.c | 184 +++--- src/libpcp/src/logutil.c | 164 ++++- src/libpcp/src/p_result.c | 183 +++++- src/libpcp_import/src/archive.c | 7 src/pmlc/pmlc.c | 1 src/pmlogextract/pmlogextract.c | 14 src/pmlogger/src/callback.c | 22 src/pmlogger/src/dopdu.c | 4 src/pmlogger/src/logger.h | 7 src/pmlogger/src/pmlogger.c | 41 - src/pmlogger/src/ports.c | 5 src/pmlogger/src/preamble.c | 4 src/pmlogreduce/pmlogreduce.c | 4 src/pmlogreduce/scan.c | 6 src/pmlogrewrite/pmlogrewrite.c | 52 + src/pmlogrewrite/result.c | 4 44 files changed, 2107 insertions(+), 2583 deletions(-) commit 9b60b01c64019b7e55ca9d8590aaea6bffa0fba5 Merge: 55057d0 20dde12 Author: Nathan Scott Date: Mon Apr 28 19:06:23 2014 +1000 Merge branch 'dev' of git://git.performancecopilot.org/kenj/pcp into dev Conflicts: src/libpcp/src/exports commit 55057d0baaa5042885245f491842746c10b385ce Merge: eb1cc9c 40c211b Author: Nathan Scott Date: Mon Apr 28 16:26:03 2014 +1000 Merge branch 'brolley-merge' into dev commit eb1cc9c1889ed12bef6cde5a443ac05af038a518 Author: Nathan Scott Date: Mon Apr 28 16:20:29 2014 +1000 Fix couple of warnings issued today by fedora.spec builds commit 20dde126ce6a40c4a7b39bfff661e1074cc603b0 Author: Ken McDonell Date: Mon Apr 28 07:40:13 2014 +1000 qa/518 - dodge other pmie's running at the same time Filter pcp -P output to pick pmie lines for just the pmie launched by qa/518. commit 61450c26fe10e7cbc860f220cb7e10722ebe92e3 Author: Ken McDonell Date: Mon Apr 28 07:17:35 2014 +1000 qa/492 - track change in error message format for pmlogrewrite commit 1c85848f3c64a93acc930d505c995601fd3783ab Author: Ken McDonell Date: Sun Apr 27 19:15:25 2014 +1000 pmlc - retire flush command Recent changes tp pmlogger mean the flush operation is no longer needed. Removed reference to "flush command from help text (so lots of cosmetic QA changes), toned down the man page description and turned the implementation to a no-op. The pmlc "flush" command is still (sliently) supported to provide backwards compatibility for scripts using pmlc. commit 9a980fdfd31f900adde8db11c505dbf22c293e54 Author: Ken McDonell Date: Sun Apr 27 17:06:13 2014 +1000 libpcp - __pmLogPut*() all unbuffered writes now Changes to __pmLogPutDesc, __pmLogPutInDom and __pmLogWriteLabel so that each record to an archive file is written in a single fwrite() and using unbuffered I/O for each file. commit 7357a7e8cf96b7e18f8b8cc0777d00e0d8a1894a Author: Ken McDonell Date: Sun Apr 27 17:02:09 2014 +1000 pmlogger - turn all flushing operations into no-ops All pmlogger writes are now unbuffered, with one fwrite() per logical record for each of the data, metadata and index files. So we can retire - the -u command line option - the SIGUSR1 handler - support for the flush command from pmlc All are preserved for backwards compatibility, but they are now no-ops that are either not documented any more or described as no longer useful. commit e93c254ed3b0e9165faf1d56a5122cb78db06928 Author: Ken McDonell Date: Sun Apr 27 16:59:05 2014 +1000 qa/928 - extend chkputlogresult Make sure the test program also calls __pmLogPutInDom() so we can exercise all of the archive fwrite() paths in libpcp. commit bf081792016e9fd0e1db6359056c35855d8fbd9f Author: Ken McDonell Date: Sun Apr 27 16:58:24 2014 +1000 qa/group - qa/510 exercises pmlc as well commit e16b3fb7e3e91ea133da0dba31d1f122831b1cac Merge: 4f987b5 8da53d2 Author: Ken McDonell Date: Fri Apr 25 14:39:01 2014 +1000 Merge branch 'dev' of git://git.performancecopilot.org/pcp into dev Conflicts: qa/492.out.2 src/pmlogrewrite/pmlogrewrite.c Drop 492.out.2 Reconcile Ken's and Nathan's overlapping printf changes for logrewrite.c. commit 4f987b5cf5b2a600f170a6d943e81e075490d954 Author: Ken McDonell Date: Fri Apr 25 14:16:00 2014 +1000 .gitignore house keeping commit c897891caac7f9f4d4507c2e6403323066072cc3 Author: Ken McDonell Date: Fri Apr 25 13:57:23 2014 +1000 qa/061 - track diagnostics changes in __pmLogPutResult Also drop support for an old PCP version and go with one output file for the current PCP version. commit 7d2d3ec47f9e98edb1ae0b3ef58025b86a5569a3 Author: Ken McDonell Date: Fri Apr 25 13:56:05 2014 +1000 qa/502 - track minor change in __pmLogPutResult diagnostics commit 9cdb4714b54952f62de3223124d31ae6dba5c271 Author: Ken McDonell Date: Fri Apr 25 13:52:03 2014 +1000 qa/928 (new) - check 2 variants of __pmLogPutResult() Verify __pmLogPutresult() and __pmLogPutresult2() generate identical archives. commit bfedb60fba11e213f2d204de91fe6d67eb62c5aa Author: Ken McDonell Date: Fri Apr 25 13:44:53 2014 +1000 libpcp - redo __pmPutLogResult change to preserve ABI semantics The new code is in __pmPutLogResult2(). Both the old and the new routines are available in libpcp. Converted all use within PCP to __pmPutLogResult2(). commit ac9f1ad4fbaa6373eb6bf6ffee124b93d742a56a Author: Ken McDonell Date: Fri Apr 25 13:41:27 2014 +1000 libpcp - additional diagnostics for __pmDecodeResult Added diags under the control of (pmDebug & DBG_TRACE_PDU) && (pmDebug & DBG_TRACE_DESPERATE) for all the corrupt PDU cases to help diagnose issues. commit 740bdbb203e2a96c9324a00873f3b2b501e79da2 Merge: e85a6cf 93afbb3 Author: Ken McDonell Date: Wed Apr 23 12:55:34 2014 +1000 Merge branch 'oops' into dev commit 93afbb3a4ea166aa39b1c4d3cd2f21835c8ad280 Author: Ken McDonell Date: Wed Apr 23 12:44:03 2014 +1000 pmlogrewrite - fsync() change Be safer about the output archive before renaming starts. Also fixed some diagnostic consistency (and hence the qa/492 change). From nscott@redhat.com Mon Apr 28 19:31:01 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 46A4229DF8 for ; Mon, 28 Apr 2014 19:31:01 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 370A68F8039 for ; Mon, 28 Apr 2014 17:30:57 -0700 (PDT) X-ASG-Debug-ID: 1398731451-04cb6c7291109140001-S8gJnT Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id DsKAe3nYPMBzFrZV for ; Mon, 28 Apr 2014 17:30:51 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.39 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3T0Uo8T020940 for ; Mon, 28 Apr 2014 20:30:51 -0400 Date: Mon, 28 Apr 2014 20:30:50 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp@oss.sgi.com Message-ID: <179965100.13569049.1398731450822.JavaMail.zimbra@redhat.com> In-Reply-To: <1500075656.13568901.1398731412288.JavaMail.zimbra@redhat.com> Subject: pcp updates: docs, getopts MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: docs, getopts Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: docs, getopts Thread-Index: P6FPxvzuw2aEkOXouPkzhmGLj2F1vg== X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39] X-Barracuda-Start-Time: 1398731451 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5336 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.performancecopilot.org/pcp.git dev README | 2 +- build/rpm/fedora.spec | 7 +++++++ man/man3/pmgetoptions.3 | 6 +++++- qa/492.out | 12 ++++++++++++ qa/739.out | 4 ++++ qa/README | 10 +++++----- qa/src/test_pcp_options.python | 2 ++ src/include/pcp/pmapi.h | 1 + src/libpcp/src/getopt.c | 4 ++++ src/perl/LogImport/LogImport.pm | 6 +++--- src/perl/LogSummary/LogSummary.pm | 6 +++--- src/perl/MMV/MMV.pm | 6 +++--- src/perl/PMDA/PMDA.pm | 4 ++-- src/pmatop/pmatop.py | 2 +- src/pmcollectl/pmcollectl.py | 2 +- src/pmlogrewrite/pmlogrewrite.c | 2 ++ src/python/pcp/pmapi.py | 6 +++++- src/python/pcp/pmsubsys.py | 2 +- src/python/pmapi.c | 24 ++++++++++++++++++++++++ src/python/setup.py | 4 ++-- 20 files changed, 88 insertions(+), 24 deletions(-) commit 6cef9631abfdce195efdf710785f88bd2b055c28 Author: Nathan Scott Date: Tue Apr 29 09:47:07 2014 +1000 Add a facility for injecting free-form text into usage messages commit 4dc9cedde5e916abe03ef7e079654e4fc5769ccd Author: Nathan Scott Date: Tue Apr 29 08:51:12 2014 +1000 Merge back the EPEL6 spec change causing ongoing merge conflicts commit 48d644b27ba862e0424871ac876914a09b669185 Author: Nathan Scott Date: Tue Apr 29 08:31:07 2014 +1000 Transition remaining URLs over to performancecopilot.org names From michele@acksyn.org Tue Apr 29 09:15:08 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id B3C2629DF8 for ; Tue, 29 Apr 2014 09:15:08 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 957098F8035 for ; Tue, 29 Apr 2014 07:15:08 -0700 (PDT) X-ASG-Debug-ID: 1398780903-04cb6c7290142f10001-S8gJnT Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id sDUVHTiYI4VCW9VX for ; Tue, 29 Apr 2014 07:15:04 -0700 (PDT) X-Barracuda-Envelope-From: michele@acksyn.org X-Barracuda-Apparent-Source-IP: 5.9.7.26 Received: from localhost (localhost [127.0.0.1]) by palahniuk.acksyn.org (Postfix) with ESMTP id 6520B27708 for ; Tue, 29 Apr 2014 10:15:03 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h= user-agent:content-disposition:content-type:content-type :mime-version:message-id:subject:subject:from:from:date:date :received:received; s=2010; t=1398780903; bh=dUi43A8GDasam7RJZhy RDya38pww07ZzswQWBbFgXdE=; b=FIoOi5v+p5o1I0/pd7IvvCoS0ZI7siPnXse PX9STJX55qa9eO4PdDpoEnEM47CRNGvZNxO+1VkR3J47WOVRMBIIRvTDyhDh32h3 EwfW+ANNSjCJCp+oBvEjdg1eHEyI/sMPpy/7jLFZ8+BFEK0+SWK76xORTzn5WvFp Fs3vbyLo= Received: from palahniuk.acksyn.org ([127.0.0.1]) by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id dWZzzMFriACd for ; Tue, 29 Apr 2014 10:15:03 -0400 (EDT) Received: from localhost (97e50ded.skybroadband.com [151.229.13.237]) by palahniuk.acksyn.org (Postfix) with ESMTPSA id 984B925E61 for ; Tue, 29 Apr 2014 10:15:02 -0400 (EDT) Date: Tue, 29 Apr 2014 15:15:02 +0100 From: Michele Baldessari To: pcp@oss.sgi.com Subject: sosreport and pcp Message-ID: <20140429141502.GD15985@marquez.int.rhx> X-ASG-Orig-Subj: sosreport and pcp MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2012-12-30) X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26] X-Barracuda-Start-Time: 1398780904 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5358 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature Hi all, I am in the process of adding support for PCP in sosreport[1]. Here is the list of files that, from a first glance, sosreport should collect in order to be able to troubleshoot PCP questions/issues: /var/log/pcp, /var/lib/pcp/config, /etc/pcp, /etc/pcp.conf, /etc/pcp.env, /etc/pcp.sh Are there any other files or directory or output of commands that you can think of that are needed when troubleshooting PCP? I'd imagine that extra carefulness needs to be taken for /var/log/pcp in order to avoid collecting stuff (logger data?) that is bigger than X unless explicitely asked for. I assume they can grow moderately big, although I don't have any real-world data on that. Thanks for any inputs and or feedback on this. Kind regards, Michele [1] https://github.com/sosreport/sos -- Michele Baldessari C2A5 9DA3 9961 4FFB E01B D0BC DDD4 DCCB 7515 5C6D From fche@redhat.com Tue Apr 29 14:28:25 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 5686B29DF8 for ; Tue, 29 Apr 2014 14:28:25 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 3F4D48F8035 for ; Tue, 29 Apr 2014 12:28:25 -0700 (PDT) X-ASG-Debug-ID: 1398799701-04cb6c729015d9a0001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id qwUyGPtsQP8dzy1S for ; Tue, 29 Apr 2014 12:28:21 -0700 (PDT) X-Barracuda-Envelope-From: fche@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3TJSJjS022955 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Apr 2014 15:28:19 -0400 Received: from fche.csb (vpn-48-176.rdu2.redhat.com [10.10.48.176]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3TJSJvK019666; Tue, 29 Apr 2014 15:28:19 -0400 Received: by fche.csb (Postfix, from userid 2569) id 9630F58117; Tue, 29 Apr 2014 15:28:18 -0400 (EDT) To: Michele Baldessari Cc: pcp@oss.sgi.com Subject: Re: sosreport and pcp References: <20140429141502.GD15985@marquez.int.rhx> X-ASG-Orig-Subj: Re: sosreport and pcp From: fche@redhat.com (Frank Ch. Eigler) Date: Tue, 29 Apr 2014 15:28:18 -0400 In-Reply-To: <20140429141502.GD15985@marquez.int.rhx> (Michele Baldessari's message of "Tue, 29 Apr 2014 15:15:02 +0100") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398799701 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Hi, Michele - michele wrote: > [...] > I am in the process of adding support for PCP in sosreport[1]. [...] > /var/log/pcp, /var/lib/pcp/config, /etc/pcp, > /etc/pcp.conf, /etc/pcp.env, /etc/pcp.sh Note that /etc/pcp.conf is an input to .env / .sh; the latter are just non-adustable shell scripts. The variables set inside /etc/pcp.conf can redirect the location of many other bits. For example, instead of hardcoding the /var/log/pcp directory name, sosreport -might- consider getting the PCP_LOG_DIR value out of /etc/pcp.conf. > [...] I'd imagine that extra carefulness needs to be taken for > /var/log/pcp in order to avoid collecting stuff (logger data?) that > is bigger than X unless explicitely asked for. I assume they can > grow moderately big, although I don't have any real-world data on > that. The bulk archives (*.[0-9]*, .meta, .index files) certainly grow big: 10-20 MB per day per host, kept by default for 14 days. It can blow up multiplicatively for longer-than-default or multiple-host logging. OTOH, the files are highly (90%+) compressible, and provide a good detailed performance overview of the host(s). Other files in there are small and valuable for diagnosing problems with pcp itself. - FChE From kenj@internode.on.net Tue Apr 29 16:03:46 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 869BC29DF8 for ; Tue, 29 Apr 2014 16:03:46 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 6B5048F8033 for ; Tue, 29 Apr 2014 14:03:43 -0700 (PDT) X-ASG-Debug-ID: 1398805417-04cbb03cc4164340001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id FVGY8k6R00HVI4MM for ; Tue, 29 Apr 2014 14:03:37 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Au4WAKUSYFN20fC0PGdsb2JhbAANTItKvzoDAQEBATiDWT0WGAMCAQIBMRoNCAEBrW6kHxeTDwSvNQ Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 30 Apr 2014 06:33:36 +0930 Message-ID: <5360142E.4030408@internode.on.net> Date: Wed, 30 Apr 2014 07:05:50 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: PCP Mailing List Subject: build breakage telnet-probe Content-Type: text/plain; charset=ISO-8859-1 X-ASG-Orig-Subj: build breakage telnet-probe Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1398805417 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5369 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Heads up ... I have a fix coming. This is a recent regression in the main tree (which has been pulled into my tree) ... telnet-probe.o: In function `main': /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:70: undefined reference to `__pmGetAddrInfo' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:78: undefined reference to `__pmHostEntGetSockAddr' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:83: undefined reference to `__pmCreateSocket' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:94: undefined reference to `__pmConnectTo' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:95: undefined reference to `__pmSockAddrFree' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:110: undefined reference to `__pmFD_ZERO' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:111: undefined reference to `__pmFD_SET' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:112: undefined reference to `__pmSelectWrite' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:125: undefined reference to `__pmCloseSocket' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:80: undefined reference to `__pmHostEntGetSockAddr' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:82: undefined reference to `__pmSockAddrIsInet' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:84: undefined reference to `__pmSockAddrIsIPv6' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:85: undefined reference to `__pmCreateIPv6Socket' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:89: undefined reference to `__pmSockAddrFree' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:118: undefined reference to `__pmConnectCheckError' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:129: undefined reference to `__pmHostEntFree' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:129: undefined reference to `__pmHostEntFree' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:132: undefined reference to `__pmConnectRestoreFlags' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:152: undefined reference to `__pmWrite' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:152: undefined reference to `__pmWrite' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:161: undefined reference to `__pmRead' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:171: undefined reference to `__pmSocketClosed' /home/kenj/src/pcp/src/telnet-probe/telnet-probe.c:129: undefined reference to `__pmHostEntFree' collect2: error: ld returned 1 exit status make[1]: *** [telnet-probe] Error 1 make: *** [default_pcp] Error 2 From kenj@internode.on.net Tue Apr 29 16:26:07 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 0660629DF8 for ; Tue, 29 Apr 2014 16:26:07 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id DD6078F8035 for ; Tue, 29 Apr 2014 14:26:06 -0700 (PDT) X-ASG-Debug-ID: 1398806764-04cbb03cc6165e70001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id RbOEcAnF0WzuMBGp for ; Tue, 29 Apr 2014 14:26:04 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsWAH8YYFN20fC0PGdsb2JhbAANTINVh3W9f4E6AwEBAQE4gloBAQEEOEARCxgJFg8JAwIBAgExFBMIAQGtbaQjF4xUggIWhCMElR6aFw Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 30 Apr 2014 06:55:46 +0930 Message-ID: <53601960.5000503@internode.on.net> Date: Wed, 30 Apr 2014 07:28:00 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: Re: [pcp] sosreport and pcp References: <20140429141502.GD15985@marquez.int.rhx> X-ASG-Orig-Subj: Re: [pcp] sosreport and pcp In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1398806764 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5370 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- On 30/04/14 05:28, Frank Ch. Eigler wrote: > > Hi, Michele - > > michele wrote: > >> [...] >> I am in the process of adding support for PCP in sosreport[1]. [...] >> /var/log/pcp, /var/lib/pcp/config, /etc/pcp, >> /etc/pcp.conf, /etc/pcp.env, /etc/pcp.sh > > Note that /etc/pcp.conf is an input to .env / .sh; the latter are just > non-adustable shell scripts. The variables set inside /etc/pcp.conf > can redirect the location of many other bits. For example, instead of > hardcoding the /var/log/pcp directory name, sosreport -might- consider > getting the PCP_LOG_DIR value out of /etc/pcp.conf. Further to Frank's suggestions ... 1. /etc/pcp.env is not that useful to collect 2. use /etc/pcp.conf (or source /etc/pcp.env) in your collector script to drive the inventory of collection artifacts, e.g. use $PCP_SYSCONF_DIR instead of /etc/pcp and use $PCP_LOG_DIR instead of /var/log/pcp and use $PCP_VAR_DIR/config instead of /var/lib/pcp/config 3. also of value would be the contents of the $PCP_VAR_DIR/pmns directory 4. /etc/pcp.sh is probably not useful in this context 5. if you're interested in current state, capturing the output from the pcp(1) command would be useful 6. in $PCP_LOG_DIR, always collect the contents of the pmcd subdirectory and the NOTICES* files > >> [...] I'd imagine that extra carefulness needs to be taken for >> /var/log/pcp in order to avoid collecting stuff (logger data?) that >> is bigger than X unless explicitely asked for. I assume they can >> grow moderately big, although I don't have any real-world data on >> that. > > The bulk archives (*.[0-9]*, .meta, .index files) certainly grow big: > 10-20 MB per day per host, kept by default for 14 days. It can blow > up multiplicatively for longer-than-default or multiple-host > logging. OTOH, the files are highly (90%+) compressible, and provide > a good detailed performance overview of the host(s). If you're able to able to use heuristics to refine the selection here (there are PCP archives below the $PCP_LOG_DIR/pmlogger and $PCP_LOG_DIR/pmmgr directories), ... a. pick all the *.log files b. restrict the subdirectories to names that match hostname(1) (the others are remote machines, and less useful for sosreport I presume) c. there is basically a currently being written archive and then archives for the last N days ... the current and yesterday is probably most useful if space becomes an issue (name matches on a pattern like *DDDDMMYY* will work, pmdate -1d '%Y%m%d' is helpful here to get yesterday's date pattern, else used find ... -mtime -1) If you need someone to review / check a collection or exercise your changes just let us know. From kenj@internode.on.net Tue Apr 29 17:08:29 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 7922529DF8 for ; Tue, 29 Apr 2014 17:08:29 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 6A7DE304032 for ; Tue, 29 Apr 2014 15:08:26 -0700 (PDT) X-ASG-Debug-ID: 1398809300-04cb6c1fcc17640001-S8gJnT Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id pS5YA9j4HSXHaXL9 for ; Tue, 29 Apr 2014 15:08:20 -0700 (PDT) X-Barracuda-Envelope-From: kenj@internode.on.net X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvQWABwiYFN20fC0PGdsb2JhbAANTINVh3W/OQMBAQEBOIMZQDANFhgDAgECATEOGQYCAQGtdKRKF41rgQGEIwSvNYFX Received: from ppp118-209-240-180.lns20.mel6.internode.on.net (HELO [192.168.1.100]) ([118.209.240.180]) by ipmail04.adl6.internode.on.net with ESMTP; 30 Apr 2014 07:38:01 +0930 Message-ID: <53602348.9040903@internode.on.net> Date: Wed, 30 Apr 2014 08:10:16 +1000 From: Ken McDonell User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: pcp updates Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-ASG-Orig-Subj: pcp updates Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1398809300 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5371 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Changes committed to git://oss.sgi.com/kenj/pcp.git dev src/telnet-probe/GNUmakefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 4e6cb71bb910fb4b660eea3c22905ab42aefa368 Author: Ken McDonell Date: Wed Apr 30 07:05:03 2014 +1000 telnet-probe/GNUmakefile - used LLDLIBS not LLDFLAGS for -lpcp From mgoodwin@redhat.com Tue Apr 29 19:14:04 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 518D529DF8 for ; Tue, 29 Apr 2014 19:14:04 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 0E5108F8035 for ; Tue, 29 Apr 2014 17:14:01 -0700 (PDT) X-ASG-Debug-ID: 1398816836-04bdf02b8b174f10001-S8gJnT Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id vwF9D7CcTlyiwAaf for ; Tue, 29 Apr 2014 17:13:57 -0700 (PDT) X-Barracuda-Envelope-From: mgoodwin@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3U0DtLD018537 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Apr 2014 20:13:55 -0400 Received: from [10.64.48.37] (vpn1-48-37.bne.redhat.com [10.64.48.37]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3U0DoDT014932; Tue, 29 Apr 2014 20:13:52 -0400 Message-ID: <5360403D.6010806@redhat.com> Date: Wed, 30 Apr 2014 10:13:49 +1000 From: Mark Goodwin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Michele Baldessari CC: pcp@oss.sgi.com, "Bryn M. Reeves" Subject: Re: [pcp] sosreport and pcp References: <20140429141502.GD15985@marquez.int.rhx> X-ASG-Orig-Subj: Re: [pcp] sosreport and pcp In-Reply-To: <20140429141502.GD15985@marquez.int.rhx> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1398816837 X-Barracuda-URL: http://192.48.157.11:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On 04/30/2014 12:15 AM, Michele Baldessari wrote: > Hi all, > > I am in the process of adding support for PCP in sosreport[1]. Here is >... > > [1] https://github.com/sosreport/sos > Bryn Reeves already started on a PCP plugin in branch 'bmr-pcp-plugin' see his commit : commit 78d84262bc289ad934ab689ef113b0fc1e86e0f1 Author: Bryn M. Reeves Date: Thu Dec 13 16:56:48 2012 +0000 Add basic performance co-pilot (pcp) module Add a module to collect PCP logs and configuration files from /var Basically it just grabs /var/log/pcp and /var/lib/pcp/config so it's fairly rudimentary compared to some of the suggestions posted in earlier responses. Not sure where Bryn has merged this, if anywhere so far. [cc: bmr@redhat.com] Regards -- Mark From nscott@redhat.com Tue Apr 29 21:19:52 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 2211529DF8 for ; Tue, 29 Apr 2014 21:19:52 -0500 (CDT) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 9D00DAC003 for ; Tue, 29 Apr 2014 19:19:48 -0700 (PDT) X-ASG-Debug-ID: 1398824382-04cbb03cc61757c0001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id 4RQLxsn9yHoKwaAP for ; Tue, 29 Apr 2014 19:19:42 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3U2JgQi028642 for ; Tue, 29 Apr 2014 22:19:42 -0400 Date: Tue, 29 Apr 2014 22:19:42 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp@oss.sgi.com Message-ID: <975519277.14570054.1398824382103.JavaMail.zimbra@redhat.com> In-Reply-To: <1540518257.14569924.1398824271511.JavaMail.zimbra@redhat.com> Subject: pcp updates: kenj fixes, more pmda longopt conversions MIME-Version: 1.0 X-ASG-Orig-Subj: pcp updates: kenj fixes, more pmda longopt conversions Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp updates: kenj fixes, more pmda longopt conversions Thread-Index: SCbGHsO7nY8B6KClYJ/amEqU2vK79A== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1398824382 X-Barracuda-URL: http://192.48.176.25:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5378 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.performancecopilot.org/pcp.git dev qa/709.out | 8 +-- src/include/pcp/pmda.h | 1 src/pmcollectl/pmcollectl.py | 22 +++++---- src/pmdas/apache/apache.c | 73 +++++++++++++++++--------------- src/pmdas/bash/bash.c | 93 ++++++++++++++++++++---------------------- src/pmdas/gfs2/pmda.c | 37 ++++++++-------- src/pmdas/sample/src/pmda.c | 72 ++++++++++++++------------------ src/pmdas/simple/simple.c | 58 ++++++++++++-------------- src/pmdas/trivial/trivial.c | 48 +++++++++------------ src/pmlogreduce/pmlogreduce.c | 2 src/python/pmda.c | 2 src/telnet-probe/GNUmakefile | 2 12 files changed, 203 insertions(+), 215 deletions(-) commit 3d0e372ce1fc61a8d29b835fe768020923cdd7b9 Author: Nathan Scott Date: Wed Apr 30 12:06:49 2014 +1000 Use correct options macros for PMDAs, mainly cosmetic commit e94dcc19592b92fcf2eda1e2ac36fffd47e45510 Author: Nathan Scott Date: Wed Apr 30 12:06:15 2014 +1000 Update test qa/709 output to match collectl more closely commit 07eaee72771b7f4429774189894ea06871274905 Author: Nathan Scott Date: Wed Apr 30 10:26:08 2014 +1000 Long options for sample,simple,trivial PMDAs commit a88c50b3c29aca6fbe6f4d9c5f8b48e690afcbf4 Author: Nathan Scott Date: Wed Apr 30 09:40:52 2014 +1000 Convert pmdagfs2 over to using long options commit eca0f29d8667ab376bc6aacdc34ca9ae7aa77f9c Merge: bd124d5 4e6cb71 Author: Nathan Scott Date: Wed Apr 30 08:37:23 2014 +1000 Merge branch 'dev' of git://git.performancecopilot.org/kenj/pcp into dev commit 4e6cb71bb910fb4b660eea3c22905ab42aefa368 Author: Ken McDonell Date: Wed Apr 30 07:05:03 2014 +1000 telnet-probe/GNUmakefile - used LLDLIBS not LLDFLAGS for -lpcp commit 38b55de98ed2c5225e368dfe30983d1c9ce4f30b Merge: 70a5ec5 6cef963 Author: Ken McDonell Date: Tue Apr 29 19:49:16 2014 +1000 Merge branch 'dev' of git://git.performancecopilot.org/pcp into dev commit bd124d5ec46c70c42d07f5b8ae2fdfecc2322087 Author: Nathan Scott Date: Tue Apr 29 17:42:18 2014 +1000 Convert pmdaapache and pmdabash to long option processing Added in support for PMDA header to use text extension to usage messages, then used it in Apache PMDA conversion. commit 70a5ec50d82f7a18947e558de0896c526d922c6a Merge: c6a5cd6 9b60b01 Author: Ken McDonell Date: Mon Apr 28 21:52:02 2014 +1000 Merge branch 'dev' of git://git.performancecopilot.org/pcp into dev commit c6a5cd6aa79318fe158967e8e0dd666aa05a43cc Author: Ken McDonell Date: Mon Apr 28 14:23:36 2014 +1000 pmlogreduce - fix minor compilation warning From nscott@redhat.com Tue Apr 29 21:24:46 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id A52FE29DF8 for ; Tue, 29 Apr 2014 21:24:46 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 83423304062 for ; Tue, 29 Apr 2014 19:24:46 -0700 (PDT) X-ASG-Debug-ID: 1398824681-04cb6c728f1764d0001-S8gJnT Received: from mx5-phx2.redhat.com (mx5-phx2.redhat.com [209.132.183.37]) by cuda.sgi.com with ESMTP id hweWsUHykyrjMXji for ; Tue, 29 Apr 2014 19:24:41 -0700 (PDT) X-Barracuda-Envelope-From: nscott@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.37 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3U2Oejg029640 for ; Tue, 29 Apr 2014 22:24:41 -0400 Date: Tue, 29 Apr 2014 22:24:40 -0400 (EDT) From: Nathan Scott Reply-To: Nathan Scott To: pcp@oss.sgi.com Message-ID: <105058849.14571538.1398824680845.JavaMail.zimbra@redhat.com> Subject: pcp-gui updates: books MIME-Version: 1.0 X-ASG-Orig-Subj: pcp-gui updates: books Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF17 (Linux)/8.0.6_GA_5922) Thread-Topic: pcp-gui updates: books Thread-Index: er35whSHlHoMUP1GWCkar8l68naTpw== X-Barracuda-Connect: mx5-phx2.redhat.com[209.132.183.37] X-Barracuda-Start-Time: 1398824681 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.02 X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5378 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... Changes committed to git://git.performancecopilot.org/pcp/pcp-gui.git dev books/PCP_PG/pcp-programmers-guide.pdf |binary books/PCP_PG/pcp-programmers-guide.xml | 136 ++++++++++--------- books/PCP_TCS/pcp-tutorials-and-case-studies.pdf |binary books/PCP_TCS/pcp-tutorials-and-case-studies.xml | 8 - books/PCP_UAG/pcp-users-and-administrators-guide.pdf |binary books/PCP_UAG/pcp-users-and-administrators-guide.xml | 7 6 files changed, 82 insertions(+), 69 deletions(-) commit b7d0d4597150019b0f0f4d56c41ac8741016a29f Author: Nathan Scott Date: Wed Apr 30 12:00:37 2014 +1000 Document automated command line processing in python/C clients In addition to command line processing updates, the programmers guide is updated to cover the pmGetContextHostName_r interface. commit 5407fef8c07f109800781f7868d8cf765222b060 Author: Nathan Scott Date: Wed Apr 30 11:19:44 2014 +1000 Update the programmers guide to cover PMDA argument parsing commit ea730238fccc1174cd15d56e923295e7648b9f27 Author: Nathan Scott Date: Wed Apr 30 10:49:31 2014 +1000 Update docbook URLs and datestamps From michele@acksyn.org Wed Apr 30 02:49:33 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 0D46A7F37 for ; Wed, 30 Apr 2014 02:49:33 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 09C06304062 for ; Wed, 30 Apr 2014 00:49:32 -0700 (PDT) X-ASG-Debug-ID: 1398844168-04cb6c728f186430001-S8gJnT Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id oDwxLO9vvOAbw7Vx for ; Wed, 30 Apr 2014 00:49:28 -0700 (PDT) X-Barracuda-Envelope-From: michele@acksyn.org X-Barracuda-Apparent-Source-IP: 5.9.7.26 Received: from localhost (localhost [127.0.0.1]) by palahniuk.acksyn.org (Postfix) with ESMTP id A9C7127761; Wed, 30 Apr 2014 03:49:27 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h= user-agent:in-reply-to:content-disposition:content-type :content-type:mime-version:references:message-id:subject:subject :from:from:date:date:received:received; s=2010; t=1398844167; bh=9DTWnMbmpZVJ9C8g9vt+oIDStX9feEpSYpP0Cq+oP08=; b=eVwTc9P0cm29 XrEgvxmJnP/YL27pSiefRBRjGs3va8hy3kAy4Q28/N2zJirNqo2FBzV4JrSv6beY v6nj9Uqm8AdkjyLZOMGZKk68r1EvsXWfGITWd63tYpHu+YlJanGVG7KCPL9kluRr FZWlY+73SV4eYLYq0CmeTLqjzaSijKg= Received: from palahniuk.acksyn.org ([127.0.0.1]) by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tvoG5bXNZ-3i; Wed, 30 Apr 2014 03:49:27 -0400 (EDT) Received: from localhost (97e50ded.skybroadband.com [151.229.13.237]) by palahniuk.acksyn.org (Postfix) with ESMTPSA id 9891425E61; Wed, 30 Apr 2014 03:49:26 -0400 (EDT) Date: Wed, 30 Apr 2014 08:49:26 +0100 From: Michele Baldessari To: "Frank Ch. Eigler" Cc: pcp@oss.sgi.com Subject: Re: sosreport and pcp Message-ID: <20140430074926.GF15985@marquez.int.rhx> X-ASG-Orig-Subj: Re: sosreport and pcp References: <20140429141502.GD15985@marquez.int.rhx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2012-12-30) X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26] X-Barracuda-Start-Time: 1398844168 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5385 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature Hi Frank, On Tue, Apr 29, 2014 at 03:28:18PM -0400, Frank Ch. Eigler wrote: > michele wrote: > > [...] > > I am in the process of adding support for PCP in sosreport[1]. [...] > > /var/log/pcp, /var/lib/pcp/config, /etc/pcp, > > /etc/pcp.conf, /etc/pcp.env, /etc/pcp.sh > > Note that /etc/pcp.conf is an input to .env / .sh; the latter are just > non-adustable shell scripts. The variables set inside /etc/pcp.conf > can redirect the location of many other bits. For example, instead of > hardcoding the /var/log/pcp directory name, sosreport -might- consider > getting the PCP_LOG_DIR value out of /etc/pcp.conf. good idea, I did this[1] so customized cases will be covered. > > [...] I'd imagine that extra carefulness needs to be taken for > > /var/log/pcp in order to avoid collecting stuff (logger data?) that > > is bigger than X unless explicitely asked for. I assume they can > > grow moderately big, although I don't have any real-world data on > > that. > > The bulk archives (*.[0-9]*, .meta, .index files) certainly grow big: > 10-20 MB per day per host, kept by default for 14 days. It can blow > up multiplicatively for longer-than-default or multiple-host > logging. OTOH, the files are highly (90%+) compressible, and provide > a good detailed performance overview of the host(s). > > Other files in there are small and valuable for diagnosing problems > with pcp itself. Ok. I've added an sensible default upper limit (100MB) and a user can override via an option. Thanks again and regards, Michele [1] https://github.com/mbaldessari/sosreport/blob/pcp-plugin/sos/plugins/pcp.py -- Michele Baldessari C2A5 9DA3 9961 4FFB E01B D0BC DDD4 DCCB 7515 5C6D From michele@acksyn.org Wed Apr 30 02:53:50 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id DCD3F7F37 for ; Wed, 30 Apr 2014 02:53:50 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id D73018F8039 for ; Wed, 30 Apr 2014 00:53:47 -0700 (PDT) X-ASG-Debug-ID: 1398844426-04cb6c7290186800001-S8gJnT Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id graZ0d9IGgVJ9PHt for ; Wed, 30 Apr 2014 00:53:47 -0700 (PDT) X-Barracuda-Envelope-From: michele@acksyn.org X-Barracuda-Apparent-Source-IP: 5.9.7.26 Received: from localhost (localhost [127.0.0.1]) by palahniuk.acksyn.org (Postfix) with ESMTP id 3B1EE27761; Wed, 30 Apr 2014 03:53:46 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h= user-agent:in-reply-to:content-disposition:content-type :content-type:mime-version:references:message-id:subject:subject :from:from:date:date:received:received; s=2010; t=1398844425; bh=BtPkhPH1kJuroxgbBQVQB4oXfU9JhdjtlUtZTTofPPg=; b=klrtNztYFg58 6ixKhemKQ1pbCHgHp8jB87TQOyvX6NL6eaPFaAkGRkbPpN+LhScXNwrfll6zsMZ/ FOR9Y9sxkjC6GNDQhDOUjxtCyHwNkZmIBUzqxO7vty06PFmn9ejm6CfVQuEFjXAZ V2yEwPrqYEeWJepDtUKSks+cn56e8XA= Received: from palahniuk.acksyn.org ([127.0.0.1]) by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id hgmjaTrG4snT; Wed, 30 Apr 2014 03:53:45 -0400 (EDT) Received: from localhost (97e50ded.skybroadband.com [151.229.13.237]) by palahniuk.acksyn.org (Postfix) with ESMTPSA id EC19E25E61; Wed, 30 Apr 2014 03:53:44 -0400 (EDT) Date: Wed, 30 Apr 2014 08:53:44 +0100 From: Michele Baldessari To: Mark Goodwin Cc: pcp@oss.sgi.com, "Bryn M. Reeves" Subject: Re: [pcp] sosreport and pcp Message-ID: <20140430075344.GG15985@marquez.int.rhx> X-ASG-Orig-Subj: Re: [pcp] sosreport and pcp References: <20140429141502.GD15985@marquez.int.rhx> <5360403D.6010806@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5360403D.6010806@redhat.com> User-Agent: Mutt/1.5.21 (2012-12-30) X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26] X-Barracuda-Start-Time: 1398844426 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5385 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature Hi Mark, On Wed, Apr 30, 2014 at 10:13:49AM +1000, Mark Goodwin wrote: > On 04/30/2014 12:15 AM, Michele Baldessari wrote: > >Hi all, > > > >I am in the process of adding support for PCP in sosreport[1]. Here is > >... > > > >[1] https://github.com/sosreport/sos > > > > Bryn Reeves already started on a PCP plugin in branch 'bmr-pcp-plugin' > see his commit : > > commit 78d84262bc289ad934ab689ef113b0fc1e86e0f1 > Author: Bryn M. Reeves > Date: Thu Dec 13 16:56:48 2012 +0000 > > Add basic performance co-pilot (pcp) module > > Add a module to collect PCP logs and configuration files from /var > > > Basically it just grabs /var/log/pcp and /var/lib/pcp/config so it's > fairly rudimentary compared to some of the suggestions posted in earlier > responses. Not sure where Bryn has merged this, if anywhere so far. > [cc: bmr@redhat.com] thanks for the info. Bryn and I had briefly chatted about this ;) I've gotten most of it done now with the suggestions from Frank[1] (i.e. worksforme). I will wait for issue #281 [2] to be fixed and then will submit a proper pull request. Thanks and regards, Michele [1] https://github.com/mbaldessari/sosreport/blob/pcp-plugin/sos/plugins/pcp.py [2] https://github.com/sosreport/sos/issues/281 -- Michele Baldessari C2A5 9DA3 9961 4FFB E01B D0BC DDD4 DCCB 7515 5C6D From michele@acksyn.org Wed Apr 30 03:29:00 2014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: pcp@oss.sgi.com Delivered-To: pcp@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 8E4EF7F37 for ; Wed, 30 Apr 2014 03:29:00 -0500 (CDT) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 8C4A5304067 for ; Wed, 30 Apr 2014 01:28:57 -0700 (PDT) X-ASG-Debug-ID: 1398846535-04cb6c1fcc36510001-S8gJnT Received: from palahniuk.acksyn.org (palahniuk.acksyn.org [5.9.7.26]) by cuda.sgi.com with ESMTP id 14pWR9NUgRC1RDDK for ; Wed, 30 Apr 2014 01:28:56 -0700 (PDT) X-Barracuda-Envelope-From: michele@acksyn.org X-Barracuda-Apparent-Source-IP: 5.9.7.26 Received: from localhost (localhost [127.0.0.1]) by palahniuk.acksyn.org (Postfix) with ESMTP id 544AA27764 for ; Wed, 30 Apr 2014 04:28:55 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h= user-agent:in-reply-to:content-disposition:content-type :content-type:mime-version:references:message-id:subject:subject :from:from:date:date:received:received; s=2010; t=1398846533; bh=KdSY/u1YrCeaf+8NMI0/LYFC/GPu39jrPifbcKmwxg4=; b=GoNoPObBZRXM /AiRh+p8RqFc1M+oSzHzK7XOEX8Iavdy4ZIWKnJBDvOfwuWymsQqYDb1Qs6ULszk x2AXxyEZaKgomt6CkMDzYKcp54ZvgO1Nwgfr8/3zbg7guqUaj/mU7SFqPjf9GZbq wnZetTePByl6kbq81rZehzYPiqX9S3A= Received: from palahniuk.acksyn.org ([127.0.0.1]) by localhost (mail.acksyn.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id LLlAHJYsYGxn for ; Wed, 30 Apr 2014 04:28:53 -0400 (EDT) Received: from localhost (97e50ded.skybroadband.com [151.229.13.237]) by palahniuk.acksyn.org (Postfix) with ESMTPSA id 5CD42261A9 for ; Wed, 30 Apr 2014 04:28:53 -0400 (EDT) Date: Wed, 30 Apr 2014 09:28:53 +0100 From: Michele Baldessari To: pcp@oss.sgi.com Subject: Re: [pcp] sosreport and pcp Message-ID: <20140430082852.GH15985@marquez.int.rhx> X-ASG-Orig-Subj: Re: [pcp] sosreport and pcp References: <20140429141502.GD15985@marquez.int.rhx> <53601960.5000503@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53601960.5000503@internode.on.net> User-Agent: Mutt/1.5.21 (2012-12-30) X-Barracuda-Connect: palahniuk.acksyn.org[5.9.7.26] X-Barracuda-Start-Time: 1398846535 X-Barracuda-URL: http://192.48.176.15:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.5386 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature Hi Ken, On Wed, Apr 30, 2014 at 07:28:00AM +1000, Ken McDonell wrote: > On 30/04/14 05:28, Frank Ch. Eigler wrote: > >>[...] > >>I am in the process of adding support for PCP in sosreport[1]. [...] > >>/var/log/pcp, /var/lib/pcp/config, /etc/pcp, > >>/etc/pcp.conf, /etc/pcp.env, /etc/pcp.sh > > > >Note that /etc/pcp.conf is an input to .env / .sh; the latter are just > >non-adustable shell scripts. The variables set inside /etc/pcp.conf > >can redirect the location of many other bits. For example, instead of > >hardcoding the /var/log/pcp directory name, sosreport -might- consider > >getting the PCP_LOG_DIR value out of /etc/pcp.conf. > > Further to Frank's suggestions ... > > 1. /etc/pcp.env is not that useful to collect > 2. use /etc/pcp.conf (or source /etc/pcp.env) in your collector script to > drive the inventory of collection artifacts, e.g. use $PCP_SYSCONF_DIR > instead of /etc/pcp and use $PCP_LOG_DIR instead of /var/log/pcp and use > $PCP_VAR_DIR/config instead of /var/lib/pcp/config > 3. also of value would be the contents of the $PCP_VAR_DIR/pmns directory > 4. /etc/pcp.sh is probably not useful in this context > 5. if you're interested in current state, capturing the output from the > pcp(1) command would be useful > 6. in $PCP_LOG_DIR, always collect the contents of the pmcd subdirectory and > the NOTICES* files Thanks. I've added 5 and 6. > >>[...] I'd imagine that extra carefulness needs to be taken for > >>/var/log/pcp in order to avoid collecting stuff (logger data?) that > >>is bigger than X unless explicitely asked for. I assume they can > >>grow moderately big, although I don't have any real-world data on > >>that. > > > >The bulk archives (*.[0-9]*, .meta, .index files) certainly grow big: > >10-20 MB per day per host, kept by default for 14 days. It can blow > >up multiplicatively for longer-than-default or multiple-host > >logging. OTOH, the files are highly (90%+) compressible, and provide > >a good detailed performance overview of the host(s). > > If you're able to able to use heuristics to refine the selection here (there > are PCP archives below the $PCP_LOG_DIR/pmlogger and $PCP_LOG_DIR/pmmgr > directories), ... I assume that for the sosreport use case $PCP_LOG_DIR/pmmgr is less interesting as it is mainly to collect data from a bunch of hosts? Or is it commonly used to collect from the host as well? > a. pick all the *.log files > b. restrict the subdirectories to names that match hostname(1) (the others > are remote machines, and less useful for sosreport I presume) Good point, it makes little sense to capture anything that does not belong to the host itself for the sosreport case. I will add this. > c. there is basically a currently being written archive and then archives > for the last N days ... the current and yesterday is probably most useful if > space becomes an issue (name matches on a pattern like *DDDDMMYY* will work, > pmdate -1d '%Y%m%d' is helpful here to get yesterday's date pattern, else > used find ... -mtime -1) Good point, I might add something like the following default behaviour: if total size of dir < threshold collect everything otherwise just the last two days > If you need someone to review / check a collection or exercise your changes > just let us know. Thanks, I'll likely get back to you on this offer ;) Kind regards, Michele -- Michele Baldessari C2A5 9DA3 9961 4FFB E01B D0BC DDD4 DCCB 7515 5C6D