From mailman-bounces@oss.sgi.com Sun Nov 1 05:28:11 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=ALL_TRUSTED,BAYES_00, J_CHICKENPOX_25 autolearn=no version=3.3.0-rupdated Received: from oss.sgi.com (localhost [127.0.0.1]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA1B1ar8192911 for ; Sun, 1 Nov 2009 05:01:36 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: oss.sgi.com mailing list memberships reminder From: mailman-owner@oss.sgi.com To: pcp@oss.sgi.com X-No-Archive: yes Message-ID: Date: Sun, 01 Nov 2009 05:00:29 -0600 Precedence: bulk X-BeenThere: mailman@oss.sgi.com X-Mailman-Version: 2.1.11 List-Id: Mailman mail X-List-Administrivia: yes Sender: mailman-bounces@oss.sgi.com Errors-To: mailman-bounces@oss.sgi.com This is a reminder, sent out once a month, about your oss.sgi.com mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, mailman-request@oss.sgi.com) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. If you have questions, problems, comments, etc, send them to mailman-owner@oss.sgi.com. Thanks! Passwords for pcp@oss.sgi.com: List Password // URL ---- -------- pcp-announce@oss.sgi.com xE+IBctR http://oss.sgi.com/mailman/options/pcp-announce/pcp%40oss.sgi.com From kenj@internode.on.net Wed Nov 4 01:22:52 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_55, J_CHICKENPOX_62 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA47Mp0t229487 for ; Wed, 4 Nov 2009 01:22:52 -0600 X-ASG-Debug-ID: 1257319347-7cd703250000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9096C1816AE5 for ; Tue, 3 Nov 2009 23:22:27 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id ICAmuztLHzCuUnxr for ; Tue, 03 Nov 2009 23:22:27 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.145.208]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7791361-1927428 for ; Wed, 04 Nov 2009 17:52:25 +1030 (CDT) X-ASG-Orig-Subj: Derived Metrics - RFC Subject: Derived Metrics - RFC From: Ken McDonell Reply-To: kenj@internode.on.net To: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Wed, 04 Nov 2009 18:22:23 +1100 Message-ID: <1257319343.4706.72.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1257319350 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13684 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I'm returning to one of the PCP "holy grails" ... derived metrics. For those who've not been down this tortured path before, the idea is that one could define one or more derived metrics in terms of an arithmetic expression over existing metrics, and values for the derived metrics would be available just like regular PCP metrics. The most common examples are ... * the requirement to compute something that is "delta(value) / delta(other_value)", e.g. average time per message where both "time" and "number of messages" are counters * aggregation of existing metrics, e.g. "messages" being the sum of synchronous and asynchrous message sent and received, or total packet rate across all gigE interfaces This all belongs per PMAPI client, so it works for performance metrics from pmcd and from archives. Another reason for this being a client-side feature is that it should be available without access to, or the capability to, reconfigure the PMDAs and/or pmcd on the collector system. And finally the stateful computation of delta(value Let me know your thoughts. Cheers, Ken. ) / delta(other_value) can only be sensibly done at the per-process level on the client side. So one could consider a derived metrics module to be an old SysV-style streams module that has a private configuration file and is inserted between a PMAPI client and the source of PCP metrics. I'd like some initial feedback on the following set of initial limitations and assumptions. 1. Only works for platforms with ELF binaries. I'm planning to use $LD_PRELOAD to optionally insert a DSO between the PCP client and libpcp.so to intercept calls and rewrite them as needed. 2. Only guaranteed to work for the synchronous PMAPI variants. So for example I'll make pmLookupName() work, but invest no effort in the asynchronous pair pmRequestNames() and pmReceiveNames(). As an aside does any know of a living user of these asynchronous interface extensions to the original PMAPI? 3. Configuration file pathname comes via a new PCP environment variable, probably $PCP_DERIVED_CONFIG. 4. No recursive definitions. Each derived metric is an expression involving metrics that are NOT derived. 5. Performance is not an issue. Some of the re-writing is not going to be cheap, especially in terms of the demands on *alloc(). Some of this is piggy-backing on knowledge gained from the recent dynamic PMNS changes in terms, although the implementation is disjoint. I have the bones of a proof of concept implementation, so there is a real chance it may happen this time around. Let me know your thoughts. Cheers, Ken. From goodwinos@gmail.com Wed Nov 4 18:38:16 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_55, J_CHICKENPOX_62 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA50cFgE043049 for ; Wed, 4 Nov 2009 18:38:16 -0600 X-ASG-Debug-ID: 1257381508-680500120000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5376ECC0A87 for ; Wed, 4 Nov 2009 16:38:28 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id iOqhFdmpp9ZEzHJL for ; Wed, 04 Nov 2009 16:38:28 -0800 (PST) Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nA50cOqk027525; Wed, 4 Nov 2009 19:38:24 -0500 Received: from [10.64.176.217] (dhcp-176-217.mel.redhat.com [10.64.176.217]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nA50cMw3014891; Wed, 4 Nov 2009 19:38:23 -0500 Message-ID: <4AF21E7E.7050100@gmail.com> Date: Thu, 05 Nov 2009 11:38:22 +1100 From: Mark Goodwin User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: kenj@internode.on.net CC: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] Derived Metrics - RFC Subject: Re: [pcp] Derived Metrics - RFC References: <1257319343.4706.72.camel@bozo.localdomain> In-Reply-To: <1257319343.4706.72.camel@bozo.localdomain> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1257381510 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13751 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi Ken, could you repost this please, there's a "Cheers, Ken" in the middle of the message and another one at the end, so I'm not sure if we got the whole lot. Cheers -- Mark Ken McDonell wrote: > I'm returning to one of the PCP "holy grails" ... derived metrics. > > For those who've not been down this tortured path before, the idea is > that one could define one or more derived metrics in terms of an > arithmetic expression over existing metrics, and values for the derived > metrics would be available just like regular PCP metrics. > > The most common examples are ... > * the requirement to compute something that is "delta(value) / > delta(other_value)", e.g. average time per message where both > "time" and "number of messages" are counters > * aggregation of existing metrics, e.g. "messages" being the sum > of synchronous and asynchrous message sent and received, or > total packet rate across all gigE interfaces > > This all belongs per PMAPI client, so it works for performance metrics > from pmcd and from archives. Another reason for this being a > client-side feature is that it should be available without access to, or > the capability to, reconfigure the PMDAs and/or pmcd on the collector > system. And finally the stateful computation of delta(value > Let me know your thoughts. > > Cheers, Ken. > ) / delta(other_value) can only be sensibly done at the per-process > level on the client side. > > So one could consider a derived metrics module to be an old SysV-style > streams module that has a private configuration file and is inserted > between a PMAPI client and the source of PCP metrics. > > I'd like some initial feedback on the following set of initial > limitations and assumptions. > 1. Only works for platforms with ELF binaries. I'm planning to use > $LD_PRELOAD to optionally insert a DSO between the PCP client > and libpcp.so to intercept calls and rewrite them as needed. > 2. Only guaranteed to work for the synchronous PMAPI variants. So > for example I'll make pmLookupName() work, but invest no effort > in the asynchronous pair pmRequestNames() and pmReceiveNames(). > As an aside does any know of a living user of these asynchronous > interface extensions to the original PMAPI? > 3. Configuration file pathname comes via a new PCP environment > variable, probably $PCP_DERIVED_CONFIG. > 4. No recursive definitions. Each derived metric is an expression > involving metrics that are NOT derived. > 5. Performance is not an issue. Some of the re-writing is not > going to be cheap, especially in terms of the demands on > *alloc(). > > Some of this is piggy-backing on knowledge gained from the recent > dynamic PMNS changes in terms, although the implementation is disjoint. > > I have the bones of a proof of concept implementation, so there is a > real chance it may happen this time around. > > Let me know your thoughts. > > Cheers, Ken. > > _______________________________________________ > pcp mailing list > pcp@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/pcp From kenj@internode.on.net Wed Nov 4 19:28:51 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_62 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA51Spaa047508 for ; Wed, 4 Nov 2009 19:28:51 -0600 X-ASG-Debug-ID: 1257384543-603a03e20000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E0279181F52D for ; Wed, 4 Nov 2009 17:29:04 -0800 (PST) Received: from mail.internode.on.net (bld-mail18.adl2.internode.on.net [150.101.137.103]) by cuda.sgi.com with ESMTP id XnZAY3lEbhvRc7Kp for ; Wed, 04 Nov 2009 17:29:04 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.214.46]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7824135-1927428 for ; Thu, 05 Nov 2009 11:59:01 +1030 (CDT) X-ASG-Orig-Subj: [resend] Derived Metrics - RFC Subject: [resend] Derived Metrics - RFC From: Ken McDonell Reply-To: kenj@internode.on.net To: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Thu, 05 Nov 2009 12:28:58 +1100 Message-ID: <1257384538.4706.76.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail18.adl2.internode.on.net[150.101.137.103] X-Barracuda-Start-Time: 1257384545 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13754 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean [slight formatting botch in the original, so re-sending] I'm returning to one of the PCP "holy grails" ... derived metrics. For those who've not been down this tortured path before, the idea is that one could define one or more derived metrics in terms of an arithmetic expression over existing metrics, and values for the derived metrics would be available just like regular PCP metrics. The most common examples are ... * the requirement to compute something that is "delta(value) / delta(other_value)", e.g. average time per message where both "time" and "number of messages" are counters * aggregation of existing metrics, e.g. "messages" being the sum of synchronous and asynchrous message sent and received, or total packet rate across all gigE interfaces This all belongs per PMAPI client, so it works for performance metrics from pmcd and from archives. Another reason for this being a client-side feature is that it should be available without access to, or the capability to, reconfigure the PMDAs and/or pmcd on the collector system. And finally the stateful computation of delta(value) / delta(other_value) can only be sensibly done at the per-process level on the client side. So one could consider a derived metrics module to be an old SysV-style streams module that has a private configuration file and is inserted between a PMAPI client and the source of PCP metrics. I'd like some initial feedback on the following set of initial limitations and assumptions. 1. Only works for platforms with ELF binaries. I'm planning to use $LD_PRELOAD to optionally insert a DSO between the PCP client and libpcp.so to intercept calls and rewrite them as needed. 2. Only guaranteed to work for the synchronous PMAPI variants. So for example I'll make pmLookupName() work, but invest no effort in the asynchronous pair pmRequestNames() and pmReceiveNames(). As an aside does any know of a living user of these asynchronous interface extensions to the original PMAPI? 3. Configuration file pathname comes via a new PCP environment variable, probably $PCP_DERIVED_CONFIG. 4. No recursive definitions. Each derived metric is an expression involving metrics that are NOT derived. 5. Performance is not an issue. Some of the re-writing is not going to be cheap, especially in terms of the demands on *alloc(). Some of this is piggy-backing on knowledge gained from the recent dynamic PMNS changes in terms, although the implementation is disjoint. I have the bones of a proof of concept implementation, so there is a real chance it may happen this time around. Let me know your thoughts. Cheers, Ken. From nscott@aconex.com Wed Nov 4 21:36:29 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_62 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA53aSxZ054339 for ; Wed, 4 Nov 2009 21:36:29 -0600 X-ASG-Debug-ID: 1257392201-104501830000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 830C6181F77C for ; Wed, 4 Nov 2009 19:36:42 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id M7wgwFs6UpdVStY4 for ; Wed, 04 Nov 2009 19:36:42 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 4C4AB596E38; Thu, 5 Nov 2009 14:36:39 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id zDfghc8P2CEIiHIH; Thu, 05 Nov 2009 14:36:39 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 5B5CBA502AF; Thu, 5 Nov 2009 14:35:04 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 1CF3F4FD86; Thu, 5 Nov 2009 14:36:39 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vunwuXQb-lb7; Thu, 5 Nov 2009 14:36:34 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id B390E4FD82; Thu, 5 Nov 2009 14:36:34 +1100 (EST) Date: Thu, 5 Nov 2009 14:36:34 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com Message-ID: <2126406291.235831257392194606.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1257384538.4706.76.camel@bozo.localdomain> X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257392203 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13760 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean Hi Ken, ----- "Ken McDonell" wrote: > ... And finally the stateful computation of > delta(value) / delta(other_value) can only be sensibly done at the > per-process level on the client side. > > So one could consider a derived metrics module to be an old SysV-style > streams module that has a private configuration file and is inserted > between a PMAPI client and the source of PCP metrics. > > I'd like some initial feedback on the following set of initial > limitations and assumptions. > 1. Only works for platforms with ELF binaries. I'm planning to use > $LD_PRELOAD to optionally insert a DSO between the PCP client > and libpcp.so to intercept calls and rewrite them as needed. Hmm. That seems quite unwieldy (obviously, wont work on Windows too). What's the rationale for that instead of working in libpcp directly? > 2. Only guaranteed to work for the synchronous PMAPI variants. > So for example I'll make pmLookupName() work, but invest no > effort in the asynchronous pair pmRequestNames() and pmReceiveNames(). > As an aside does any know of a living user of these asynchronous > interface extensions to the original PMAPI? (Not that I know of... perhaps these can die. They are not implemented on Win32 & AFAICT cannot be, so I would rather noone started using them). > 3. Configuration file pathname comes via a new PCP environment > variable, probably $PCP_DERIVED_CONFIG. This seems unwieldy too. Would it be feasible to go more for something that can be specified all-at-once, and as a (easily identified) drop-in replacement for a metric name (I assume these will be given "made-up" PMIDs by Lookup Name?)? ... so maybe: $ pmval 'expr: delta(kernel.all.pswitch) * 1000' Or some other not-valid-as-metric-name keyword in place of "expr:" like 'derive:' or 'formula:' or 'infer:'. Or perhaps force them to be brace enclosed, e.g. $ pmval '(delta(kernel.all.pswitch) * 1000 + 42)' Or perhaps require something with an assignment, like the optional pmie syntax - "cputime = delta(simple.time.user) + delta(simple.time.sys)". I have plans to revisit the pmchart metric selection process for gadgets down the track, and it would be good to allow the metric selector dialog to allow derived metric syntax too (even just if its as free form text), whatever that syntax turns out to be... if it can be done "inline" like this, without needing environment variables, temporary files, etc - that would be great. > 4. No recursive definitions. Each derived metric is an > expression involving metrics that are NOT derived. That seems fair enough. cheers. -- Nathan From makc@iinet.net.au Thu Nov 5 07:16:09 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA5DG8aa096569 for ; Thu, 5 Nov 2009 07:16:09 -0600 X-ASG-Debug-ID: 1257426981-08d501530000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from outbound.icp-qv1-irony-out6.iinet.net.au (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D1D631D8EBE6 for ; Thu, 5 Nov 2009 05:16:22 -0800 (PST) Received: from outbound.icp-qv1-irony-out6.iinet.net.au (outbound.icp-qv1-irony-out6.iinet.net.au [203.59.1.109]) by cuda.sgi.com with ESMTP id bSvqQyhlRwsySXpd for ; Thu, 05 Nov 2009 05:16:22 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAGde8krL2Rwk/2dsb2JhbADeSoQ9BA X-IronPort-AV: E=Sophos;i="4.44,686,1249228800"; d="scan'208";a="23717823" Received: from unknown (HELO alma.crabbed.net) ([203.217.28.36]) by outbound.icp-qv1-irony-out6.iinet.net.au with ESMTP; 05 Nov 2009 21:16:19 +0800 Received: by alma.crabbed.net (Postfix, from userid 501) id C364A3C444A; Fri, 6 Nov 2009 00:15:06 +1100 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19186.53210.639017.193346@iinet.net.au> Date: Fri, 6 Nov 2009 00:15:06 +1100 From: Max Matveev To: kenj@internode.on.net Cc: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC In-Reply-To: <1257384538.4706.76.camel@bozo.localdomain> References: <1257384538.4706.76.camel@bozo.localdomain> X-Mailer: VM 7.17 under 21.4 (patch 19) "Constant Variable" XEmacs Lucid X-Barracuda-Connect: outbound.icp-qv1-irony-out6.iinet.net.au[203.59.1.109] X-Barracuda-Start-Time: 1257426983 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, 05 Nov 2009 12:28:58 +1100, Ken McDonell wrote: kenj> The most common examples are ... kenj> * the requirement to compute something that is "delta(value) / kenj> delta(other_value)", e.g. average time per message where both kenj> "time" and "number of messages" are counters kenj> * aggregation of existing metrics, e.g. "messages" being the sum kenj> of synchronous and asynchrous message sent and received, or kenj> total packet rate across all gigE interfaces Do you have any plans to support "avergage of last 10 samples" thing or rates? Or is it just the metrics from a single pmfetch? kenj> As an aside does any know of a living user of these asynchronous kenj> interface extensions to the original PMAPI? Yes, I do - when collecting metrics from multiple hosts I cannot afford for TCP to pick its nose when one of the hosts go down. max From makc@iinet.net.au Thu Nov 5 07:16:58 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA5DGvNM096653 for ; Thu, 5 Nov 2009 07:16:57 -0600 X-ASG-Debug-ID: 1257427031-105f03530000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from outbound.icp-qv1-irony-out4.iinet.net.au (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8D61C14D3A2E for ; Thu, 5 Nov 2009 05:17:11 -0800 (PST) Received: from outbound.icp-qv1-irony-out4.iinet.net.au (outbound.icp-qv1-irony-out4.iinet.net.au [203.59.1.104]) by cuda.sgi.com with ESMTP id PFm4iYQwSpMhDA30 for ; Thu, 05 Nov 2009 05:17:11 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAFdf8krL2Rwk/2dsb2JhbADeToQ9BIFm X-IronPort-AV: E=Sophos;i="4.44,686,1249228800"; d="scan'208";a="479505400" Received: from unknown (HELO alma.crabbed.net) ([203.217.28.36]) by outbound.icp-qv1-irony-out4.iinet.net.au with ESMTP; 05 Nov 2009 21:17:09 +0800 Received: by alma.crabbed.net (Postfix, from userid 501) id EF3D33C444E; Fri, 6 Nov 2009 00:15:56 +1100 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19186.53260.700661.828724@iinet.net.au> Date: Fri, 6 Nov 2009 00:15:56 +1100 From: Max Matveev To: Nathan Scott Cc: kenj@internode.on.net, pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC In-Reply-To: <2126406291.235831257392194606.JavaMail.root@mail-au.aconex.com> References: <1257384538.4706.76.camel@bozo.localdomain> <2126406291.235831257392194606.JavaMail.root@mail-au.aconex.com> X-Mailer: VM 7.17 under 21.4 (patch 19) "Constant Variable" XEmacs Lucid X-Barracuda-Connect: outbound.icp-qv1-irony-out4.iinet.net.au[203.59.1.104] X-Barracuda-Start-Time: 1257427032 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, 5 Nov 2009 14:36:34 +1100 (EST), Nathan Scott wrote: >> 2. Only guaranteed to work for the synchronous PMAPI variants. >> So for example I'll make pmLookupName() work, but invest no >> effort in the asynchronous pair pmRequestNames() and pmReceiveNames(). >> As an aside does any know of a living user of these asynchronous >> interface extensions to the original PMAPI? nscott> (Not that I know of... perhaps these can die. They are not nscott> implemented on Win32 & AFAICT cannot be, so I would rather nscott> noone started using them). What's the problem of implemeting them on Windows? No select/poll? max From kenj@internode.on.net Thu Nov 5 13:38:07 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_62 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA5Jc62C126106 for ; Thu, 5 Nov 2009 13:38:07 -0600 X-ASG-Debug-ID: 1257449899-7472032a0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id F3B3764F08 for ; Thu, 5 Nov 2009 11:38:20 -0800 (PST) Received: from mail.internode.on.net (bld-mail19.adl2.internode.on.net [150.101.137.104]) by cuda.sgi.com with ESMTP id XeIQZfvn6jfl3XxW for ; Thu, 05 Nov 2009 11:38:20 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.214.46]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7798995-1927428 for multiple; Fri, 06 Nov 2009 06:07:33 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC From: Ken McDonell Reply-To: kenj@internode.on.net To: Nathan Scott Cc: pcp@oss.sgi.com In-Reply-To: <2126406291.235831257392194606.JavaMail.root@mail-au.aconex.com> References: <2126406291.235831257392194606.JavaMail.root@mail-au.aconex.com> Content-Type: text/plain; charset="UTF-8" Date: Fri, 06 Nov 2009 06:37:08 +1100 Message-ID: <1257449828.4706.714.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail19.adl2.internode.on.net[150.101.137.104] X-Barracuda-Start-Time: 1257449901 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13815 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Thanks for the feedback Nathan. I'll try responding insitu, but that is going to get pretty messy if we go round this loop once more, so at that point I think we'll need to pull out the remaining contentious points and deal with them without the larger context of the original mail. On Thu, 2009-11-05 at 14:36 +1100, Nathan Scott wrote: > Hi Ken, > > ----- "Ken McDonell" wrote: > > ... And finally the stateful computation of > > delta(value) / delta(other_value) can only be sensibly done at the > > per-process level on the client side. > > > > So one could consider a derived metrics module to be an old SysV-style > > streams module that has a private configuration file and is inserted > > between a PMAPI client and the source of PCP metrics. > > > > I'd like some initial feedback on the following set of initial > > limitations and assumptions. > > 1. Only works for platforms with ELF binaries. I'm planning to use > > $LD_PRELOAD to optionally insert a DSO between the PCP client > > and libpcp.so to intercept calls and rewrite them as needed. > > Hmm. That seems quite unwieldy (obviously, wont work on Windows too). > What's the rationale for that instead of working in libpcp directly? The real rationale is that for GNU/Linux the compiler/linker folk never really understood the deeper issues of ABI compatibility and certainly have not come close to the elegance of the IRIX approach to evolving DSOs with optional symbols. For this reason I'm hesitant to change the libpcp API/ABI. But I'm willing to be persuaded on this point. There is certainly no architectural obstacle to putting the functionality into libpcp rather than as wrappers to the libpcp routines as I was proposing. > > 2. Only guaranteed to work for the synchronous PMAPI variants. > > So for example I'll make pmLookupName() work, but invest no > > effort in the asynchronous pair pmRequestNames() and pmReceiveNames(). > > As an aside does any know of a living user of these asynchronous > > interface extensions to the original PMAPI? > > (Not that I know of... perhaps these can die. They are not implemented on > Win32 & AFAICT cannot be, so I would rather noone started using them). Perhaps we should start a separate thread on this ... I'm with you and have a philosophical objection to trying to provide an async API to a library infrastructure that was never designed to be thread-safe (deliberately, not by omission). > > 3. Configuration file pathname comes via a new PCP environment > > variable, probably $PCP_DERIVED_CONFIG. > > This seems unwieldy too. > > Would it be feasible to go more for something that can be specified > all-at-once, and as a (easily identified) drop-in replacement for a > metric name (I assume these will be given "made-up" PMIDs by Lookup > Name?)? ... so maybe: > > $ pmval 'expr: delta(kernel.all.pswitch) * 1000' > > Or some other not-valid-as-metric-name keyword in place of "expr:" like > 'derive:' or 'formula:' or 'infer:'. Or perhaps force them to be brace > enclosed, e.g. > > $ pmval '(delta(kernel.all.pswitch) * 1000 + 42)' > > Or perhaps require something with an assignment, like the optional pmie > syntax - "cputime = delta(simple.time.user) + delta(simple.time.sys)". My goal here was to NOT require any application changes ... hence the streams model analogy. The examples you give suggest that the applications might be modified to understand about derived metrics. On reflection I think we may be able to do both. As an early initialization (this bit is a little tricky, but I think pmLoadNameSpace or pmNewContext have to be called before anything we care about could be called), I'll check for $PCP_DERIVED_CONFIG being defined in the environment and if it use, register (see below) the derived metrics from that configuration file. As a prop for platforms that don't have environment variables the routine pmLoadDerivedConfig(char *path) may be used to explicitly register derived metrics from a config file ... so this could be used with command line glue to load one (or indeed more) configuration files as and when an application chooses. To register a derived metric, the routine pmRegisterDerived(char *name, char *expr) is used. The name needs to come from the expression definer, because this (a) acts as the handle for subsequent PMAPI calls, and pmLookupName() in particular, and (b) defines where the derived metric should be spliced into the PMNS. By exposing this routine, ... 1. I am committing to the "change libpcp" path, so the earlier comments about ELF and $LD_PRELOAD are not longer relevant 2. pmval could be modified to parse pmval "cputime = simple.time.user + simple.time.sys" ... (the delta() is not needed here, but would be needed in the delta(v1) / delta(v2) class of expressions) and then pmval could call pmRegisterDerived("cputime", "simple.time.user + simple.time.sys") 3. pmchart can go crazy with the definition of derived metrics via the GUI (and possibly saving them for later reloading or auto reloading) > I have plans to revisit the pmchart metric selection process for gadgets > down the track, and it would be good to allow the metric selector dialog > to allow derived metric syntax too (even just if its as free form text), > whatever that syntax turns out to be... if it can be done "inline" like > this, without needing environment variables, temporary files, etc - that > would be great. > > > 4. No recursive definitions. Each derived metric is an > > expression involving metrics that are NOT derived. > > That seems fair enough. > > cheers. Is that closer to meeting your needs? From kenj@internode.on.net Thu Nov 5 13:54:52 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA5JspHY127542 for ; Thu, 5 Nov 2009 13:54:52 -0600 X-ASG-Debug-ID: 1257450904-2b9401b40000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0FA33CC7930 for ; Thu, 5 Nov 2009 11:55:04 -0800 (PST) Received: from mail.internode.on.net (bld-mail18.adl2.internode.on.net [150.101.137.103]) by cuda.sgi.com with ESMTP id JttXVDN18E2p0ohd for ; Thu, 05 Nov 2009 11:55:04 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.214.46]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7868703-1927428 for multiple; Fri, 06 Nov 2009 06:25:03 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC From: Ken McDonell Reply-To: kenj@internode.on.net To: Max Matveev Cc: pcp@oss.sgi.com In-Reply-To: <19186.53210.639017.193346@iinet.net.au> References: <1257384538.4706.76.camel@bozo.localdomain> <19186.53210.639017.193346@iinet.net.au> Content-Type: text/plain; charset="UTF-8" Date: Fri, 06 Nov 2009 06:54:59 +1100 Message-ID: <1257450899.4706.729.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail18.adl2.internode.on.net[150.101.137.103] X-Barracuda-Start-Time: 1257450907 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13816 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, 2009-11-06 at 00:15 +1100, Max Matveev wrote: > On Thu, 05 Nov 2009 12:28:58 +1100, Ken McDonell wrote: > > kenj> The most common examples are ... > kenj> * the requirement to compute something that is "delta(value) / > kenj> delta(other_value)", e.g. average time per message where both > kenj> "time" and "number of messages" are counters > kenj> * aggregation of existing metrics, e.g. "messages" being the sum > kenj> of synchronous and asynchrous message sent and received, or > kenj> total packet rate across all gigE interfaces > Do you have any plans to support "avergage of last 10 samples" thing > or rates? Or is it just the metrics from a single pmfetch? To do the delta(v1) / delta(v2) thing requires some state ... I'd be willing to extend this to include the arithmetic aggregation functions from pmie ... so avg_sample(N, expr) and ditto for max_sample, min_sample and sum_sample ... and similarly avg_inst(expr) and ditto for avg_inst, min_inst and max_inst. But I'd consider these all version 2 deliverables in the expression language syntax. Since a derived metric is defined over metrics from a single source, none of the *_host() aggregation functions make sense in this context. Also I'm not venturing into pattern matching ... so match_inst() and nomatch_inst() won't be starters. And 1st order predicate calculus won't get a look in, so the some_*(), all_*() and N%_*() families of predicates are not going to be invited over from pmie. > kenj> As an aside does any know of a living user of these asynchronous > kenj> interface extensions to the original PMAPI? > Yes, I do - when collecting metrics from multiple hosts I cannot > afford for TCP to pick its nose when one of the hosts go down. That's a fair justification. But would it be reasonable to assume that 99.99% of these cases are related to pmFetch() as this is the only PMAPI routine most monitoring apps call in steady-state after initialization? If so, a compromise may be to dump all the async stuff in places like PMNS manipulation, pmInDom interrogation, help text retrieval, pmStore, pmDesc? So from this current list (and their associated pmReceive duals) ... $ grep pmRequest /usr/include/pcp/pmapi.h extern int pmRequestDesc (int, pmID ); extern int pmRequestFetch(int, int, pmID *); extern int pmRequestInDom (int, pmInDom); extern int pmRequestInDomInst (int, pmInDom, const char *); extern int pmRequestInDomName (int, pmInDom, int); extern int pmRequestInDomText (int, pmID, int); extern int pmRequestNameID (int, pmID); extern int pmRequestNames (int, int, char *[]); extern int pmRequestNamesOfChildern (int, const char *, int ); extern int pmRequestStore (int, const pmResult *); extern int pmRequestText (int, pmID, int); extern int pmRequestTraversePMNS (int, const char *); we could drop all but pmRequestFetch()? From nscott@aconex.com Thu Nov 5 17:20:22 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_63, URIBL_RHS_DOB autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA5NKMhq148271 for ; Thu, 5 Nov 2009 17:20:22 -0600 X-ASG-Debug-ID: 1257463234-424403c30000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 796E4182786E for ; Thu, 5 Nov 2009 15:20:34 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id 8g3C3K80HN5IsKfd for ; Thu, 05 Nov 2009 15:20:34 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 4C5A157D079; Fri, 6 Nov 2009 10:20:32 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id uteJskrCMh4SbpK6; Fri, 06 Nov 2009 10:20:32 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id F174FA50296; Fri, 6 Nov 2009 10:18:56 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 2F02B4FD86; Fri, 6 Nov 2009 10:20:32 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LyQXyFa75hrr; Fri, 6 Nov 2009 10:20:28 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 275E64FD84; Fri, 6 Nov 2009 10:20:28 +1100 (EST) Date: Fri, 6 Nov 2009 10:20:28 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com Message-ID: <1748235671.257251257463228016.JavaMail.root@mail-au.aconex.com> In-Reply-To: <774101855.257191257463092635.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257463237 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_SA210e X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13830 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC5_SA210e Custom Rule SA210e X-Virus-Status: Clean ----- "Ken McDonell" wrote: > > On Thu, 2009-11-05 at 14:36 +1100, Nathan Scott wrote: > > Hi Ken, > > > > ----- "Ken McDonell" wrote: > > Hmm. That seems quite unwieldy (obviously, wont work on Windows > too). > > What's the rationale for that instead of working in libpcp > directly? > > The real rationale is that for GNU/Linux the compiler/linker folk > never > really understood the deeper issues of ABI compatibility and > certainly > have not come close to the elegance of the IRIX approach to evolving > DSOs with optional symbols. We haven't used the existing infrastructure for this kind of thing yet (and really should at some point) - the ld(1) --version-script does allow multiple API versions to be specified in a single .so & we haven't explored using that at all so far. > For this reason I'm hesitant to change the libpcp API/ABI. > > But I'm willing to be persuaded on this point. An example of the above --version-script file syntax: http://git.savannah.gnu.org/cgit/acl.git/tree/exports (that's as used by libacl on Linux - pretty straightforward syntax and I think does some of the things you're after here... maybe, hopefully?). > There is certainly no architectural obstacle to putting the > functionality into libpcp rather than as wrappers to the libpcp > routines as I was proposing. I'd be much happier with that, esp. having the functionality made available transparently & in the same way for all platforms. > > My goal here was to NOT require any application changes ... hence the > streams model analogy. The examples you give suggest that the > applications might be modified to understand about derived metrics. > > On reflection I think we may be able to do both. > Should focus on doing it in the ideal way, and if applications need to change to make use of the new functionality, then so be it. > As an early initialization (this bit is a little tricky, but I think > pmLoadNameSpace or pmNewContext have to be called before anything we > care about could be called), I'll check for $PCP_DERIVED_CONFIG being > defined in the environment and if it use, register (see below) the > derived metrics from that configuration file. As a prop for > platforms > that don't have environment variables the routine > pmLoadDerivedConfig(char *path) > may be used to explicitly register derived metrics from a config > file ... so this could be used with command line glue to load one (or > indeed more) configuration files as and when an application chooses. Its not clear that PCP_DERIVED_METRIC buys us anything? Seems a very painful-to-use interface when compared to just specifying a formula for the new metric "inline" as one would specify any other metric name. > To register a derived metric, the routine > pmRegisterDerived(char *name, char *expr) > is used. The name needs to come from the expression definer, because > this (a) acts as the handle for subsequent PMAPI calls, and > pmLookupName() in particular, and (b) defines where the derived > metric should be spliced into the PMNS. > > By exposing this routine, ... > 1. I am committing to the "change libpcp" path, so the earlier > comments > about ELF and $LD_PRELOAD are not longer relevant > 2. pmval could be modified to parse > pmval "cputime = simple.time.user + simple.time.sys" ... > (the delta() is not needed here, but would be needed in the > delta(v1) / delta(v2) class of expressions) and then pmval could > call > pmRegisterDerived("cputime", "simple.time.user + simple.time.sys") > 3. pmchart can go crazy with the definition of derived metrics via > the > GUI (and possibly saving them for later reloading or auto > reloading) Thats good, but still seems a little clumsy - not clear that routine is needed - could pmLookupName just do it all, if it detects a name assignment, along the lines: char *names = { "new.metric = sample.one + sample.two", "sample.one" }; if (pmLookupName(2, &names, &pmids) < 0) ... (obviously, thats more hard-coded than most tools would use... but any metric "name" containing an assignment is invalid as a traditional name and would be easy to detect in pmLookupName). > Is that closer to meeting your needs? Yep, definately getting there. cheers. -- Nathan From mort@sgi.com Fri Nov 6 08:58:04 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6Ew4R0224986 for ; Fri, 6 Nov 2009 08:58:04 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id A6B4630406B; Fri, 6 Nov 2009 06:58:17 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 6D263A243300; Fri, 6 Nov 2009 08:58:16 -0600 (CST) Date: Fri, 6 Nov 2009 08:58:16 -0600 From: Martin Hicks To: Ken McDonell Cc: pcp@oss.sgi.com Subject: Re: [pcp] QA updates Message-ID: <20091106145816.GF11632@alcatraz.americas.sgi.com> References: <20091029200637.GY10537@alcatraz.americas.sgi.com> <1256851025.6851.33.camel@bozo.localdomain> <20091030155324.GF10537@alcatraz.americas.sgi.com> <1256960623.5160.49.camel@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1256960623.5160.49.camel@localhost> User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, Oct 31, 2009 at 02:43:43PM +1100, Ken McDonell wrote: > Martin, > > OK, for 600 you should be using 600.out.linux.1 ... that was the version > that worked for SLES9 (the last comment I could find in the qa test) and > looks like it is still the expected version. > > Rather than play the glibc version games, it would probably be simpler > to just use 600.out.linux.1 for SLES ... could you try the attached > version of 600 and see if that works for you? Oops. I applied and pushed a similar patch. I'd forgotten that you had provided me with a modified 600. commit 41c25b671acb639b1e6e019c997d0a81d54caede Author: Martin Hicks Date: Fri Nov 6 09:53:04 2009 -0500 600 Always use linux.1 for SuSE systems opensuse11.1 seems to generate the same out put as sles9 systems, so just use linux.1 for all SuSE systems. diff --git a/600 b/600 index 174cde0..3c891cd 100755 --- a/600 +++ b/600 @@ -64,7 +64,14 @@ then ls -l /lib/libc-*.so exit 1 fi - if [ "$a" -lt 2 ] + + # Suse systems seem to work with linux.1 + if [ -f /etc/SuSE-release ] + then + ln $seq.out.linux.1 $seq.out + + # Otherwise use glibc version + elif [ "$a" -lt 2 ] then # a.?.?, a < 2 ln $seq.out.linux.0 $seq.out mh From mort@sgi.com Fri Nov 6 08:58:30 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6EwUmt225034 for ; Fri, 6 Nov 2009 08:58:30 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay1.corp.sgi.com (Postfix) with ESMTP id 0B24A8F80F1; Fri, 6 Nov 2009 06:58:43 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id D4E88A243300; Fri, 6 Nov 2009 08:58:42 -0600 (CST) Date: Fri, 6 Nov 2009 08:58:42 -0600 From: Martin Hicks To: Ken McDonell Cc: pcp@oss.sgi.com Subject: Re: [pcp] QA updates - need some help Message-ID: <20091106145842.GG11632@alcatraz.americas.sgi.com> References: <20091029200637.GY10537@alcatraz.americas.sgi.com> <1256851025.6851.33.camel@bozo.localdomain> <20091030155324.GF10537@alcatraz.americas.sgi.com> <1257029355.5160.53.camel@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1257029355.5160.53.camel@localhost> User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sun, Nov 01, 2009 at 09:49:15AM +1100, Ken McDonell wrote: > I've rewritten 216 (attached). > > I now need to enlist the assistance of the distributed PCP QA pixies > network to give me some feedback. > > If you're able, please run this in a QA environment, As you already had my 216.out, it is no surprise that this worked fine in my environment. thanks mh > > check 216 > > and send me (a) mail if it works indicating the sort of Linux system > you're on, else (b) the same info as (a) for a failure, plus 216.out.bad > and 216.full. > > Thanks. > > On Fri, 2009-10-30 at 10:53 -0500, Martin Hicks wrote: > > On Fri, Oct 30, 2009 at 08:17:04AM +1100, Ken McDonell wrote: > > > Can you send me the 216.full file ... I'll tweak the test accordingly? > > > > > > I think 600 is one of those problematic ones ... send me the output from > > > > > > ls -l 600* > > > > > > and the 600.full file and I'll see if I can sort it out. The logic in > > > 600 for picking which is the expected outcome is very convoluted ... the > > > fact that we have FOUR possible linux output files gives some indication > > > of the mess we're trying to untangle. > > > > http://oss.sgi.com/~mort/216.full > > http://oss.sgi.com/~mort/600.full > From mort@bork.org Fri Nov 6 09:38:49 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_35, J_CHICKENPOX_36 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6FcmUB226696 for ; Fri, 6 Nov 2009 09:38:49 -0600 X-ASG-Debug-ID: 1257521942-17ec03de0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from darwin.bork.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8F08368114 for ; Fri, 6 Nov 2009 07:39:02 -0800 (PST) Received: from darwin.bork.org (darwin.bork.org [65.49.60.145]) by cuda.sgi.com with ESMTP id DTz1BpKqMzA1P0X2 for ; Fri, 06 Nov 2009 07:39:02 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by darwin.bork.org (Postfix) with ESMTP id D3FB51926E for ; Fri, 6 Nov 2009 10:38:42 -0500 (EST) Received: from darwin.bork.org ([127.0.0.1]) by localhost (darwin.bork.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id XZNVXKzMa03R for ; Fri, 6 Nov 2009 10:38:42 -0500 (EST) Received: from laplace.bork.org (laplace.bork.org [IPv6:2001:470:1f11:499:21a:a0ff:feec:57ab]) by darwin.bork.org (Postfix) with ESMTP id 409E019242 for ; Fri, 6 Nov 2009 10:38:42 -0500 (EST) Received: by laplace.bork.org (Postfix, from userid 1000) id 7F3675362F; Fri, 6 Nov 2009 10:39:00 -0500 (EST) Date: Fri, 6 Nov 2009 10:39:00 -0500 From: Martin Hicks To: pcp@oss.sgi.com X-ASG-Orig-Subj: mmv pmda: stuff always fails the first time? Subject: mmv pmda: stuff always fails the first time? Message-ID: <20091106153900.GU5035@laplace.bork.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) X-Barracuda-Connect: darwin.bork.org[65.49.60.145] X-Barracuda-Start-Time: 1257521944 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13888 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I have a super simple perl MMV PMDA that I was using as a demonstration for someone at SGI, who wanted to see a value via PCP and graph it. This was an excuse to look at the MMV pmda for the first time. It seems that the first time I try to store to mmv.reload it fails, and also when I query a value the first time. mort@laplace:~/src/pcpqa.git$ pmstore mmv.reload 1 mmv.reload: pmLookupDesc: Try again. Information not currently available mort@laplace:~/src/pcpqa.git$ pmstore mmv.reload 1 mmv.reload old value=0 new value=1 mort@laplace:~/src/pcpqa.git$ pminfo mmv mmv.reload mmv.mymmv.export_value mort@laplace:~/src/pcpqa.git$ pminfo -f mmv.mymmv mmv.mymmv.export_value: pmLookupDesc: Try again. Information not currently available mort@laplace:~/src/pcpqa.git$ pminfo -f mmv.mymmv mmv.mymmv.export_value value 15 Is this something wrong with the way I wrote the PMDA, or is this a bug in the MMV PMDA? (My silly example is written this way because they wanted to export a value that is printed in stdout of the program, and monitor it in realtime: run with: ./gendata.sh | ./mmvtest.pl mort@laplace:~$ cat ./gendata.sh value=5 myf () { echo "abc" echo "def" echo "new value: $value" value=$(($value + 5)) echo "ghi" echo "jkl" } while true; do myf sleep 10 done mort@laplace:~$ cat ./mmvtest.pl #!/usr/bin/perl use strict; use warnings; use PCP::MMV; my @indoms = (); my @metrics = ( ['export_value', 1, MMV_TYPE_U64, MMV_INDOM_NULL, mmv_units(0,0,0,0,0,0), MMV_SEM_INSTANT, 'A counter of some value', '' ] ); my $handle = mmv_stats_init('mymmv', 0, MMV_FLAG_PROCESS, \@metrics, \@indoms); die ("mmv_stats_init failed\n") unless (defined($handle)); while (my $line = ) { print $line; next if not $line =~ /^new value:/; my $val; ($val = $line) =~ s/^new value: (\d+)/$1/; chomp $val; mmv_stats_set($handle, 'export_value', '', $val); } -- Martin Hicks || mort@bork.org || PGP/GnuPG: 0x4C7F2BEE From mort@sgi.com Fri Nov 6 10:45:53 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6GjrXD229259 for ; Fri, 6 Nov 2009 10:45:53 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id 23D193040A4 for ; Fri, 6 Nov 2009 08:46:06 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 73CBFA243300; Fri, 6 Nov 2009 10:46:05 -0600 (CST) Date: Fri, 6 Nov 2009 10:46:05 -0600 From: Martin Hicks To: pcp@oss.sgi.com Subject: Regression for handling of ipc_prot=notready Message-ID: <20091106164605.GH11632@alcatraz.americas.sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I got a bug report today about a regression in pcp-2.9.2 and later that was introduced by commit 65ef4b08 Despite removing the text ipc protocol, we still have facilities for ipc_prot=notready, as documented in pmcd(1). It looks like the fix is a small revert, although I've also added a comment: diff --git a/src/pmcd/pmdaproc.sh b/src/pmcd/pmdaproc.sh index ddfcfae..883b79d 100644 --- a/src/pmcd/pmdaproc.sh +++ b/src/pmcd/pmdaproc.sh @@ -718,7 +718,9 @@ __choose_ipc() if [ $ipc_type = pipe ] then - type="pipe binary $_dir/$pmda_name" + # This defaults to binary unless the Install file + # specifies ipc_prot=notready -- See pmcd(1) + type="pipe $ipc_prot $_dir/$pmda_name" else while true do mh From pcp-announce-bounces@oss.sgi.com Fri Nov 6 11:28:12 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from oss.sgi.com (localhost [127.0.0.1]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6H9ZQ2230692; Fri, 6 Nov 2009 11:09:35 -0600 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6H9VWU230672 for ; Fri, 6 Nov 2009 11:09:31 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id CBEE030408E for ; Fri, 6 Nov 2009 09:09:44 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 78533A243300; Fri, 6 Nov 2009 11:09:44 -0600 (CST) Date: Fri, 6 Nov 2009 11:09:44 -0600 From: Martin Hicks To: pcp-announce@oss.sgi.com Message-ID: <20091106170944.GJ11632@alcatraz.americas.sgi.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Subject: [pcp-announce] New git trees for Cluster and Infiniband PMDAs X-BeenThere: pcp-announce@oss.sgi.com X-Mailman-Version: 2.1.11 Precedence: list List-Id: pcp announcements List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: pcp-announce-bounces@oss.sgi.com Errors-To: pcp-announce-bounces@oss.sgi.com As part of reorganization for the PCP 3.0.1 release, we have split the Cluster and Infiniband PMDAs into stand-alone trees in order to avoid the built-time and run-time dependencies that these seldom used PMDAs have. The provided build infrastructure is still pretty rudimentary, with only a Makefile and a specfile provided. I still work on getting autoconf and Debian support soon, hopefully. The git development trees for these PMDAs are here: git://oss.sgi.com/pcp/pcp-pmda-cluster.git git://oss.sgi.com/pcp/pcp-pmda-infiniband.git They should also soon show up on gitweb http://oss.sgi.com/cgi-bin/gitweb.cgi mh _______________________________________________ pcp-announce mailing list pcp-announce@oss.sgi.com http://oss.sgi.com/mailman/listinfo/pcp-announce From kenj@internode.on.net Fri Nov 6 14:36:23 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6KaMQ2238048 for ; Fri, 6 Nov 2009 14:36:23 -0600 X-ASG-Debug-ID: 1257539795-75fa01ce0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 865BC69524 for ; Fri, 6 Nov 2009 12:36:35 -0800 (PST) Received: from mail.internode.on.net (bld-mail18.adl2.internode.on.net [150.101.137.103]) by cuda.sgi.com with ESMTP id aRKqX0X8VFVGslbZ for ; Fri, 06 Nov 2009 12:36:35 -0800 (PST) Received: from [124.187.109.48] (unverified [124.187.109.48]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7931081-1927428 for multiple; Sat, 07 Nov 2009 07:06:33 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] QA updates - need some help Subject: Re: [pcp] QA updates - need some help From: Ken McDonell Reply-To: kenj@internode.on.net To: Martin Hicks Cc: pcp@oss.sgi.com In-Reply-To: <20091106145842.GG11632@alcatraz.americas.sgi.com> References: <20091029200637.GY10537@alcatraz.americas.sgi.com> <1256851025.6851.33.camel@bozo.localdomain> <20091030155324.GF10537@alcatraz.americas.sgi.com> <1257029355.5160.53.camel@localhost> <20091106145842.GG11632@alcatraz.americas.sgi.com> Content-Type: text/plain; charset="UTF-8" Date: Sat, 07 Nov 2009 07:36:33 +1100 Message-ID: <1257539793.2905.10.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail18.adl2.internode.on.net[150.101.137.103] X-Barracuda-Start-Time: 1257539798 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0010 1.0000 -2.0143 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13908 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Actually it is a surprise (sort of), because I changed the whole algorithm used in the test ... anyway that's good so I'll commit my new 216 and push it back to the oss tree. Thanks Martin. On Fri, 2009-11-06 at 08:58 -0600, Martin Hicks wrote: > On Sun, Nov 01, 2009 at 09:49:15AM +1100, Ken McDonell wrote: > > I've rewritten 216 (attached). > > > > I now need to enlist the assistance of the distributed PCP QA pixies > > network to give me some feedback. > > > > If you're able, please run this in a QA environment, > > As you already had my 216.out, it is no surprise that this worked fine > in my environment. > > thanks > mh > > > > > check 216 > > > > and send me (a) mail if it works indicating the sort of Linux system > > you're on, else (b) the same info as (a) for a failure, plus 216.out.bad > > and 216.full. > > > > Thanks. > > > > On Fri, 2009-10-30 at 10:53 -0500, Martin Hicks wrote: > > > On Fri, Oct 30, 2009 at 08:17:04AM +1100, Ken McDonell wrote: > > > > Can you send me the 216.full file ... I'll tweak the test accordingly? > > > > > > > > I think 600 is one of those problematic ones ... send me the output from > > > > > > > > ls -l 600* > > > > > > > > and the 600.full file and I'll see if I can sort it out. The logic in > > > > 600 for picking which is the expected outcome is very convoluted ... the > > > > fact that we have FOUR possible linux output files gives some indication > > > > of the mess we're trying to untangle. > > > > > > http://oss.sgi.com/~mort/216.full > > > http://oss.sgi.com/~mort/600.full > > > > From kenj@internode.on.net Fri Nov 6 14:38:15 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6KcE3q238140 for ; Fri, 6 Nov 2009 14:38:14 -0600 X-ASG-Debug-ID: 1257539907-14bc01eb0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 580EECD5CBC for ; Fri, 6 Nov 2009 12:38:28 -0800 (PST) Received: from mail.internode.on.net (bld-mail19.adl2.internode.on.net [150.101.137.104]) by cuda.sgi.com with ESMTP id ZDpYtLGWMKMF82tX for ; Fri, 06 Nov 2009 12:38:28 -0800 (PST) Received: from [124.187.109.48] (unverified [124.187.109.48]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7860760-1927428 for multiple; Sat, 07 Nov 2009 07:08:26 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] QA updates Subject: Re: [pcp] QA updates From: Ken McDonell Reply-To: kenj@internode.on.net To: Martin Hicks Cc: pcp@oss.sgi.com In-Reply-To: <20091106145816.GF11632@alcatraz.americas.sgi.com> References: <20091029200637.GY10537@alcatraz.americas.sgi.com> <1256851025.6851.33.camel@bozo.localdomain> <20091030155324.GF10537@alcatraz.americas.sgi.com> <1256960623.5160.49.camel@localhost> <20091106145816.GF11632@alcatraz.americas.sgi.com> Content-Type: text/plain; charset="UTF-8" Date: Sat, 07 Nov 2009 07:38:25 +1100 Message-ID: <1257539905.2905.11.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail19.adl2.internode.on.net[150.101.137.104] X-Barracuda-Start-Time: 1257539910 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13909 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Not a problem Martin, I'll drop my local changes and pick up your commit at the next sync. On Fri, 2009-11-06 at 08:58 -0600, Martin Hicks wrote: > On Sat, Oct 31, 2009 at 02:43:43PM +1100, Ken McDonell wrote: > > Martin, > > > > OK, for 600 you should be using 600.out.linux.1 ... that was the version > > that worked for SLES9 (the last comment I could find in the qa test) and > > looks like it is still the expected version. > > > > Rather than play the glibc version games, it would probably be simpler > > to just use 600.out.linux.1 for SLES ... could you try the attached > > version of 600 and see if that works for you? > > Oops. I applied and pushed a similar patch. I'd forgotten that you had > provided me with a modified 600. > > commit 41c25b671acb639b1e6e019c997d0a81d54caede > Author: Martin Hicks > Date: Fri Nov 6 09:53:04 2009 -0500 > > 600 Always use linux.1 for SuSE systems > > opensuse11.1 seems to generate the same out put as sles9 systems, > so just use linux.1 for all SuSE systems. > > diff --git a/600 b/600 > index 174cde0..3c891cd 100755 > --- a/600 > +++ b/600 > @@ -64,7 +64,14 @@ then > ls -l /lib/libc-*.so > exit 1 > fi > - if [ "$a" -lt 2 ] > + > + # Suse systems seem to work with linux.1 > + if [ -f /etc/SuSE-release ] > + then > + ln $seq.out.linux.1 $seq.out > + > + # Otherwise use glibc version > + elif [ "$a" -lt 2 ] > then > # a.?.?, a < 2 > ln $seq.out.linux.0 $seq.out > > > mh > From kenj@internode.on.net Fri Nov 6 14:53:28 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,J_BACKHAIR_44 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6KrShn238659 for ; Fri, 6 Nov 2009 14:53:28 -0600 X-ASG-Debug-ID: 1257540822-761c02610000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 27441696ED for ; Fri, 6 Nov 2009 12:53:43 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id R2JemMuxz2y60IWW for ; Fri, 06 Nov 2009 12:53:43 -0800 (PST) Received: from [124.187.109.48] (unverified [124.187.109.48]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 7942863-1927428 for multiple; Sat, 07 Nov 2009 07:23:41 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] Regression for handling of ipc_prot=notready Subject: Re: [pcp] Regression for handling of ipc_prot=notready From: Ken McDonell Reply-To: kenj@internode.on.net To: Martin Hicks Cc: pcp@oss.sgi.com In-Reply-To: <20091106164605.GH11632@alcatraz.americas.sgi.com> References: <20091106164605.GH11632@alcatraz.americas.sgi.com> Content-Type: text/plain; charset="UTF-8" Date: Sat, 07 Nov 2009 07:53:38 +1100 Message-ID: <1257540818.2905.18.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1257540824 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13910 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I'm not sure when this bit of "notready" fluff got added to pmcd's config file (the pmcd<->PMDA protocol support has been there for a long time, since the first Oracle PMDA was written). However from reading the code, it looks like for a PMDA connected via a pipe, the supported protocols in the pmcd config file are "binary" or "binary notready". So the Install file should set ipc_prot to "binary notready" (I think the comment is a little misleading, that's all). On Fri, 2009-11-06 at 10:46 -0600, Martin Hicks wrote: > I got a bug report today about a regression in pcp-2.9.2 and later that > was introduced by commit 65ef4b08 > > Despite removing the text ipc protocol, we still have facilities for > ipc_prot=notready, as documented in pmcd(1). It looks like the fix is > a small revert, although I've also added a comment: > > diff --git a/src/pmcd/pmdaproc.sh b/src/pmcd/pmdaproc.sh > index ddfcfae..883b79d 100644 > --- a/src/pmcd/pmdaproc.sh > +++ b/src/pmcd/pmdaproc.sh > @@ -718,7 +718,9 @@ __choose_ipc() > > if [ $ipc_type = pipe ] > then > - type="pipe binary $_dir/$pmda_name" > + # This defaults to binary unless the Install file > + # specifies ipc_prot=notready -- See pmcd(1) > + type="pipe $ipc_prot $_dir/$pmda_name" > else > while true > do > > mh > > _______________________________________________ > pcp mailing list > pcp@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/pcp From nscott@aconex.com Fri Nov 6 16:02:38 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6M2cNb241736 for ; Fri, 6 Nov 2009 16:02:38 -0600 X-ASG-Debug-ID: 1257544972-184e019c0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DEE441830340 for ; Fri, 6 Nov 2009 14:02:52 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id Yb9yE6mEPIFKsYTb for ; Fri, 06 Nov 2009 14:02:52 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 9A3B285F519; Sat, 7 Nov 2009 09:02:50 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id XAD5ei8VVl7FLAMy; Sat, 07 Nov 2009 09:02:50 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id BA841A50110; Sat, 7 Nov 2009 09:01:14 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 71EA04FD82; Sat, 7 Nov 2009 09:02:50 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yXpItoWmQNpM; Sat, 7 Nov 2009 09:02:45 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id B9C15C7C01; Sat, 7 Nov 2009 09:02:45 +1100 (EST) Date: Sat, 7 Nov 2009 09:02:45 +1100 (EST) From: Nathan Scott To: Martin Hicks Cc: pcp@oss.sgi.com Message-ID: <1495558594.275861257544965678.JavaMail.root@mail-au.aconex.com> In-Reply-To: <20091106153900.GU5035@laplace.bork.org> X-ASG-Orig-Subj: Re: [pcp] mmv pmda: stuff always fails the first time? Subject: Re: [pcp] mmv pmda: stuff always fails the first time? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [211.28.150.226] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - SAF3 (Mac)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257544973 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13913 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Martin Hicks" wrote: > ... > Is this something wrong with the way I wrote the PMDA, or is this a > bug in the MMV PMDA? Its a bug that cannot really be fixed in the current MMV design (to do with the way pmdammv and pmcd communicate the namespace changes) ... MMV needs to switch to using the all-singing, all- dancing dynamic namespace infrastructure to really fix this. cheers. -- Nathan From nscott@aconex.com Fri Nov 6 16:03:45 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA6M3ivk241791 for ; Fri, 6 Nov 2009 16:03:45 -0600 X-ASG-Debug-ID: 1257545039-571101ab0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id AEDD6697A4 for ; Fri, 6 Nov 2009 14:04:00 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id TotCFMf1zL3lk04z for ; Fri, 06 Nov 2009 14:04:00 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id AC63D85F523; Sat, 7 Nov 2009 09:03:58 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id a31ZclmAvlEFAuGF; Sat, 07 Nov 2009 09:03:58 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id CCFB2A50110; Sat, 7 Nov 2009 09:02:22 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 860724FD82; Sat, 7 Nov 2009 09:03:58 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mLQW-1u9ruVj; Sat, 7 Nov 2009 09:03:53 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id CE513C7C01; Sat, 7 Nov 2009 09:03:53 +1100 (EST) Date: Sat, 7 Nov 2009 09:03:53 +1100 (EST) From: Nathan Scott To: Martin Hicks Cc: pcp@oss.sgi.com Message-ID: <452443250.275901257545033825.JavaMail.root@mail-au.aconex.com> In-Reply-To: <20091106164605.GH11632@alcatraz.americas.sgi.com> X-ASG-Orig-Subj: Re: [pcp] Regression for handling of ipc_prot=notready Subject: Re: [pcp] Regression for handling of ipc_prot=notready MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [211.28.150.226] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - SAF3 (Mac)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257545040 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13913 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Martin Hicks" wrote: > I got a bug report today about a regression in pcp-2.9.2 and later > that > was introduced by commit 65ef4b08 > > Despite removing the text ipc protocol, we still have facilities for > ipc_prot=notready, as documented in pmcd(1). It looks like the fix > is > a small revert, although I've also added a comment: Whoops - thanks! Didn't realise a notready value could appear there too. cheers. -- Nathan From kenj@internode.on.net Sat Nov 7 00:27:59 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_63, URIBL_RHS_DOB autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA76Rwkp007849 for ; Sat, 7 Nov 2009 00:27:59 -0600 X-ASG-Debug-ID: 1257575291-762f03ca0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DAAC5CDB2EF for ; Fri, 6 Nov 2009 22:28:12 -0800 (PST) Received: from mail.internode.on.net (bld-mail14.adl6.internode.on.net [150.101.137.99]) by cuda.sgi.com with ESMTP id 5TCaX8QW1HUliYCI for ; Fri, 06 Nov 2009 22:28:12 -0800 (PST) Received: from [124.187.94.161] (unverified [124.187.94.161]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8058031-1927428 for multiple; Sat, 07 Nov 2009 16:58:10 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC From: Ken McDonell Reply-To: kenj@internode.on.net To: Nathan Scott Cc: pcp@oss.sgi.com In-Reply-To: <1748235671.257251257463228016.JavaMail.root@mail-au.aconex.com> References: <1748235671.257251257463228016.JavaMail.root@mail-au.aconex.com> Content-Type: text/plain; charset="UTF-8" Date: Sat, 07 Nov 2009 17:25:53 +1100 Message-ID: <1257575153.2905.52.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail14.adl6.internode.on.net[150.101.137.99] X-Barracuda-Start-Time: 1257575293 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0003 1.0000 -2.0189 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_SA210e X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.13945 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC5_SA210e Custom Rule SA210e X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, 2009-11-06 at 10:20 +1100, Nathan Scott wrote: ... > We haven't used the existing infrastructure for this kind of thing > yet (and really should at some point) - the ld(1) --version-script > does allow multiple API versions to be specified in a single .so & > we haven't explored using that at all so far. > > > For this reason I'm hesitant to change the libpcp API/ABI. > > > > But I'm willing to be persuaded on this point. > > An example of the above --version-script file syntax: > http://git.savannah.gnu.org/cgit/acl.git/tree/exports (that's as > used by libacl on Linux - pretty straightforward syntax and I think > does some of the things you're after here... maybe, hopefully?). Fair point ... I'll do some research. > > There is certainly no architectural obstacle to putting the > > functionality into libpcp rather than as wrappers to the libpcp > > routines as I was proposing. > > I'd be much happier with that, esp. having the functionality made > available transparently & in the same way for all platforms. OK, consider this decided ... the wrapper option is dead. > > > > My goal here was to NOT require any application changes ... hence the > > streams model analogy. The examples you give suggest that the > > applications might be modified to understand about derived metrics. > > > > On reflection I think we may be able to do both. > > > > Should focus on doing it in the ideal way, and if applications need to > change to make use of the new functionality, then so be it. I am not sure there is an "ideal" way. There are pros and cons, but more on this topic below. > > As an early initialization (this bit is a little tricky, but I think > > pmLoadNameSpace or pmNewContext have to be called before anything we > > care about could be called), I'll check for $PCP_DERIVED_CONFIG being > > defined in the environment and if it use, register (see below) the > > derived metrics from that configuration file. As a prop for > > platforms > > that don't have environment variables the routine > > pmLoadDerivedConfig(char *path) > > may be used to explicitly register derived metrics from a config > > file ... so this could be used with command line glue to load one (or > > indeed more) configuration files as and when an application chooses. > > Its not clear that PCP_DERIVED_METRIC buys us anything? Seems a very > painful-to-use interface when compared to just specifying a formula for > the new metric "inline" as one would specify any other metric name. There are problems with the inline method .... 1. I don't want to have to remember the formula for derived.tricky every time I want to use it ... putting the declaration in a file and having derived.tricky just appear in the PMNS would be preferable. This is even more compelling when you consider common derived metrics that one might want to make available for all PCP apps, perhaps across multiple client machines. 2. While inline does look to work for pmval, it is probably the easiest case. I don't think this will work at all (or at best requires a whole lot of additional work that is not needed in the derived metrics just appear in the PMNS) for pmie nor pmlogger nor pmdumptext nor pmlc. 3. I'm not preventing inline syntax when that makes sense, but since the majority of apps are unlikely to want to use this, I'd prefer to push the effort back to the app that wants to use the inline syntax. > > To register a derived metric, the routine > > pmRegisterDerived(char *name, char *expr) > > is used. The name needs to come from the expression definer, because > > this (a) acts as the handle for subsequent PMAPI calls, and > > pmLookupName() in particular, and (b) defines where the derived > > metric should be spliced into the PMNS. > > > > By exposing this routine, ... > > 1. I am committing to the "change libpcp" path, so the earlier > > comments > > about ELF and $LD_PRELOAD are not longer relevant > > 2. pmval could be modified to parse > > pmval "cputime = simple.time.user + simple.time.sys" ... > > (the delta() is not needed here, but would be needed in the > > delta(v1) / delta(v2) class of expressions) and then pmval could > > call > > pmRegisterDerived("cputime", "simple.time.user + simple.time.sys") > > 3. pmchart can go crazy with the definition of derived metrics via > > the > > GUI (and possibly saving them for later reloading or auto > > reloading) > > Thats good, but still seems a little clumsy - not clear that routine > is needed - could pmLookupName just do it all, if it detects a name > assignment, along the lines: > > char *names = { "new.metric = sample.one + sample.two", "sample.one" }; > if (pmLookupName(2, &names, &pmids) < 0) ... But many tools use pmGetChildren and pmTraversePMNS to expose the PMNS, and there is no similar hook that could be used in these case to insert the derived metric definition automatically. > (obviously, thats more hard-coded than most tools would use... but any > metric "name" containing an assignment is invalid as a traditional name > and would be easy to detect in pmLookupName). > > > Is that closer to meeting your needs? > > Yep, definately getting there. There are actually enough problems already in getting this right, so I'm going to try an initial implementation with pmRegisterDerived() and pmLoadDerivedConfig() in the API. I'll skip the $PCP_DERIVED_CONFIG stuff for the moment, and just use -d derived_config_file at the command line for a few apps to get pmLoadDerivedConfig() called see how it all goes. The "problems" encountered so far are ... + cannot use flex/lex or yacc/bison for parsing expressions due to ugly symbol space pollution in libpcp that will likely kill any application that uses flex/lex or yacc/bison (so pmlogger, pmie, pmlc, dbpmda, pmlogextract, hotproc PMDA) ... sigh + need one set of data structures _per_ context, and some syntax and semantic checks must be deferred from parse time to the time pmNewContext has been called successfully + tricky semantics around some aggregate functions + do we need constants in the expressions? if so, they may need to be dimensioned. From makc@iinet.net.au Sun Nov 8 02:30:19 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=AWL,BAYES_00,J_BACKHAIR_44 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA88UI94088787 for ; Sun, 8 Nov 2009 02:30:18 -0600 X-ASG-Debug-ID: 1257669024-433100740000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from outbound.icp-qv1-irony-out3.iinet.net.au (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 33EA36C65D for ; Sun, 8 Nov 2009 00:30:24 -0800 (PST) Received: from outbound.icp-qv1-irony-out3.iinet.net.au (outbound.icp-qv1-irony-out3.iinet.net.au [203.59.1.148]) by cuda.sgi.com with ESMTP id 9Tu4W3TYh86q4jji for ; Sun, 08 Nov 2009 00:30:24 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEABoQ9krL2Rwk/2dsb2JhbADZIYQ+BA X-IronPort-AV: E=Sophos;i="4.44,703,1249228800"; d="scan'208";a="519793339" Received: from unknown (HELO alma.crabbed.net) ([203.217.28.36]) by outbound.icp-qv1-irony-out3.iinet.net.au with ESMTP; 08 Nov 2009 16:30:22 +0800 Received: by alma.crabbed.net (Postfix, from userid 501) id 2F6A23C4726; Sun, 8 Nov 2009 19:29:03 +1100 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19190.33102.849090.560720@iinet.net.au> Date: Sun, 8 Nov 2009 19:29:02 +1100 From: Max Matveev To: kenj@internode.on.net Cc: Martin Hicks , pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] Regression for handling of ipc_prot=notready Subject: Re: [pcp] Regression for handling of ipc_prot=notready In-Reply-To: <1257540818.2905.18.camel@localhost> References: <20091106164605.GH11632@alcatraz.americas.sgi.com> <1257540818.2905.18.camel@localhost> X-Mailer: VM 7.17 under 21.4 (patch 19) "Constant Variable" XEmacs Lucid X-Barracuda-Connect: outbound.icp-qv1-irony-out3.iinet.net.au[203.59.1.148] X-Barracuda-Start-Time: 1257669026 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, 07 Nov 2009 07:53:38 +1100, Ken McDonell wrote: kenj> I'm not sure when this bit of "notready" fluff got added to pmcd's kenj> config file (the pmcd<->PMDA protocol support has been there for a long kenj> time, since the first Oracle PMDA was written). It was added at the same time then async support was added - we simply could not find a better way to deal with races between HUTA pmda starting up and pmcd wanting to talk to it. kenj> However from reading the code, it looks like for a PMDA connected via a kenj> pipe, the supported protocols in the pmcd config file are "binary" or kenj> "binary notready". This is correct. max From makc@iinet.net.au Sun Nov 8 02:42:57 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA88gvwp089250 for ; Sun, 8 Nov 2009 02:42:57 -0600 X-ASG-Debug-ID: 1257669792-3e9601500000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from outbound.icp-qv1-irony-out3.iinet.net.au (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0E63A6C680 for ; Sun, 8 Nov 2009 00:43:12 -0800 (PST) Received: from outbound.icp-qv1-irony-out3.iinet.net.au (outbound.icp-qv1-irony-out3.iinet.net.au [203.59.1.148]) by cuda.sgi.com with ESMTP id OYOwfnMAh86J3myv for ; Sun, 08 Nov 2009 00:43:12 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAJ8T9krL2Rwk/2dsb2JhbADZIIQ+BA X-IronPort-AV: E=Sophos;i="4.44,703,1249228800"; d="scan'208";a="519798740" Received: from unknown (HELO alma.crabbed.net) ([203.217.28.36]) by outbound.icp-qv1-irony-out3.iinet.net.au with ESMTP; 08 Nov 2009 16:43:11 +0800 Received: by alma.crabbed.net (Postfix, from userid 501) id 343C83C4735; Sun, 8 Nov 2009 19:41:52 +1100 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19190.33871.906994.253229@iinet.net.au> Date: Sun, 8 Nov 2009 19:41:51 +1100 From: Max Matveev To: kenj@internode.on.net Cc: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC In-Reply-To: <1257450899.4706.729.camel@bozo.localdomain> References: <1257384538.4706.76.camel@bozo.localdomain> <19186.53210.639017.193346@iinet.net.au> <1257450899.4706.729.camel@bozo.localdomain> X-Mailer: VM 7.17 under 21.4 (patch 19) "Constant Variable" XEmacs Lucid X-Barracuda-Connect: outbound.icp-qv1-irony-out3.iinet.net.au[203.59.1.148] X-Barracuda-Start-Time: 1257669794 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, 06 Nov 2009 06:54:59 +1100, Ken McDonell wrote: kenj> On Fri, 2009-11-06 at 00:15 +1100, Max Matveev wrote: >> On Thu, 05 Nov 2009 12:28:58 +1100, Ken McDonell wrote: kenj> As an aside does any know of a living user of these asynchronous kenj> interface extensions to the original PMAPI? >> Yes, I do - when collecting metrics from multiple hosts I cannot >> afford for TCP to pick its nose when one of the hosts go down. kenj> That's a fair justification. But would it be reasonable to assume that kenj> 99.99% of these cases are related to pmFetch() as this is the only PMAPI kenj> routine most monitoring apps call in steady-state after initialization? There is occasional indom interrogation when new instances appear (it happens much more often now due to silly implementation). The others are used too when extra sources come online and after the reconnect to the pmcd on a host which went AWOL. Reconnect itself is by far the most convoluted part with multistep procedure just to establish the connection between the client and the pmcd. kenj> If so, a compromise may be to dump all the async stuff in places like kenj> PMNS manipulation, pmInDom interrogation, help text retrieval, pmStore, kenj> pmDesc? I think the only viable dumpee in this list is text unless Nathan has plans to switch kmchart to this interface - text is only useful for humans and humans are not going to sit inside the automated tools which monitor multiple hosts. max From mort@sgi.com Mon Nov 9 09:14:16 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA9FEGOD202502 for ; Mon, 9 Nov 2009 09:14:16 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id C41C5304097; Mon, 9 Nov 2009 07:14:30 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 94F92A243300; Mon, 9 Nov 2009 09:14:30 -0600 (CST) Date: Mon, 9 Nov 2009 09:14:30 -0600 From: Martin Hicks To: Max Matveev Cc: kenj@internode.on.net, pcp@oss.sgi.com Subject: Re: [pcp] Regression for handling of ipc_prot=notready Message-ID: <20091109151430.GN11632@alcatraz.americas.sgi.com> References: <20091106164605.GH11632@alcatraz.americas.sgi.com> <1257540818.2905.18.camel@localhost> <19190.33102.849090.560720@iinet.net.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <19190.33102.849090.560720@iinet.net.au> User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sun, Nov 08, 2009 at 07:29:02PM +1100, Max Matveev wrote: > On Sat, 07 Nov 2009 07:53:38 +1100, Ken McDonell wrote: > > kenj> However from reading the code, it looks like for a PMDA connected via a > kenj> pipe, the supported protocols in the pmcd config file are "binary" or > kenj> "binary notready". > This is correct. Yep, confirmed. I'll fix my comment and push out my fix. mh From mort@sgi.com Mon Nov 9 10:29:51 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA9GToxU208201 for ; Mon, 9 Nov 2009 10:29:50 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay1.corp.sgi.com (Postfix) with ESMTP id 96B3F8F806F for ; Mon, 9 Nov 2009 08:30:05 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 56FF8A243300; Mon, 9 Nov 2009 10:30:05 -0600 (CST) Date: Mon, 9 Nov 2009 10:30:05 -0600 From: Martin Hicks To: pcp Subject: pmdumptext and x11 deps Message-ID: <20091109163005.GO11632@alcatraz.americas.sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I was annoyed for a while that pmdumptext was moved to pcp-gui. I thought that the primary reason was to get rid of the C++ code from the base package. I didn't realize that we also had Qt dependencies in pmdumptext :( I had vague hopes that we could make a pcp-gui-nox11 package or something. pmdumptext is a very useful tool, and behaves much better than pmval in terms of returning exit status. oh well...just a rant. :) I did notice that we no longer support having pmtime within pmdumptext: diff --git a/man/man1/pmdumptext.1 b/man/man1/pmdumptext.1 index 5fffaa3..4d23557 100644 --- a/man/man1/pmdumptext.1 +++ b/man/man1/pmdumptext.1 @@ -194,10 +194,6 @@ are bytes. .IP \f3\-G\f1 Output the values using the shortest of a scientific format or a decimal notation. -.IP \f3\-g\f1 -Run in graphical user interface (GUI) mode, with -.B pmtime -being used for VCR-alike time control functionality. .IP \f3\-h\f1 Fetch performance metrics from .BR pmcd (1) Was this something that was temporarily removed, or did someone intend to deprecate this feature? mh From nscott@aconex.com Mon Nov 9 16:13:31 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA9MDVKY226781 for ; Mon, 9 Nov 2009 16:13:31 -0600 X-ASG-Debug-ID: 1257804826-6bee03470000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C0B8571522 for ; Mon, 9 Nov 2009 14:13:47 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id 5id1ybvcD2jzmnCi for ; Mon, 09 Nov 2009 14:13:47 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 327718678F2; Tue, 10 Nov 2009 09:13:44 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id eS2AGSXLnMQUG4xI; Tue, 10 Nov 2009 09:13:44 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id C6EEFA5028C; Tue, 10 Nov 2009 09:12:06 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 1B66F4FD85; Tue, 10 Nov 2009 09:13:44 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OT4-j6-URQrI; Tue, 10 Nov 2009 09:13:39 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 644634FD82; Tue, 10 Nov 2009 09:13:39 +1100 (EST) Date: Tue, 10 Nov 2009 09:13:39 +1100 (EST) From: Nathan Scott To: Martin Hicks Cc: pcp Message-ID: <1807693608.309471257804819150.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1826168590.308611257804415542.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] pmdumptext and x11 deps Subject: Re: [pcp] pmdumptext and x11 deps MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257804828 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0005 1.0000 -2.0176 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14192 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Martin Hicks" wrote: > I was annoyed for a while that pmdumptext was moved to pcp-gui. I > thought that the primary reason was to get rid of the C++ code from > the > base package. I didn't realize that we also had Qt dependencies in > pmdumptext :( > > I had vague hopes that we could make a pcp-gui-nox11 package or > something. pmdumptext is a very useful tool, and behaves much better > than pmval in terms of returning exit status. Fix pmval? It should return a useful status I think. > oh well...just a rant. :) Get over it. :) I don't think its worth the pain of yet more packages (its possible though) - it all gets too complex & confusing after awhile, with a package structure thats split up to the micro-component level. > I did notice that we no longer support having pmtime within > pmdumptext: Its more that it hasn't been added yet. It is planned. I had also been thinking about introducing QtScript to pmdumptext (and pmchart of course) to do derived metrics a different way, just for the Qt based tools ... but not clear where that will stand with Kens derived metrics stuff - may still be warranted, but maybe not. cheers. -- Nathan From nscott@aconex.com Mon Nov 9 17:03:56 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nA9N3uUN229554 for ; Mon, 9 Nov 2009 17:03:56 -0600 X-ASG-Debug-ID: 1257807851-747900f80000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 69614CEC3DB for ; Mon, 9 Nov 2009 15:04:12 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id LqLD2FOcGHtafYea for ; Mon, 09 Nov 2009 15:04:12 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id C4ADF867BED; Tue, 10 Nov 2009 10:04:10 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id Fj8CBb0onS44vvOz; Tue, 10 Nov 2009 10:04:10 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 5AA3EA502B5; Tue, 10 Nov 2009 10:02:33 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id A670F4FD85; Tue, 10 Nov 2009 10:04:10 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 23VfJYmf1sPd; Tue, 10 Nov 2009 10:04:06 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 7DE274FD82; Tue, 10 Nov 2009 10:04:06 +1100 (EST) Date: Tue, 10 Nov 2009 10:04:06 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com Message-ID: <505173339.311931257807846107.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1797621252.311531257807335696.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] [resend] Derived Metrics - RFC Subject: Re: [pcp] [resend] Derived Metrics - RFC MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257807853 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14195 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Ken McDonell" wrote: > > I am not sure there is an "ideal" way. There are pros and cons, but > more on this topic below. > ... *nod* > > There are problems with the inline method .... > *nod* > There are actually enough problems already in getting this right, so > I'm > going to try an initial implementation with pmRegisterDerived() and > pmLoadDerivedConfig() in the API. I'll skip the $PCP_DERIVED_CONFIG > stuff for the moment, and just use -d derived_config_file at the > command > line for a few apps to get pmLoadDerivedConfig() called see how it > all goes. Good plan. > The "problems" encountered so far are ... > > + cannot use flex/lex or yacc/bison for parsing expressions due to > ugly > symbol space pollution in libpcp that will likely kill any > application > that uses flex/lex or yacc/bison (so pmlogger, pmie, pmlc, dbpmda, > pmlogextract, hotproc PMDA) ... sigh > + need one set of data structures _per_ context, and some syntax and > semantic checks must be deferred from parse time to the time > pmNewContext has been called successfully > + tricky semantics around some aggregate functions > + do we need constants in the expressions? if so, they may need to > be dimensioned. One other thing I should've mentioned, tho its only peripherally related. I have been considering QtScript (http://en.wikipedia.org/wiki/QtScript) in the next iteration of pcp-gui, as this gives those tools a high level language for implementing the derived metrics stuff for the Qt metrics class tools (pmdumptext, pmchart, ... pmgadgets/pmview later) amongst a number of other things (gui scripting). I'm just mentioning this so you can keep it in the back of your mind while doing this... (perhaps what we want here is an API for plugging in metric evaluators, rather than a new mini-language - thinking of your first point above). Not 100% clear on that though. cheers. -- Nathan From nscott@aconex.com Mon Nov 9 23:51:04 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_33, J_CHICKENPOX_43,J_CHICKENPOX_55,J_CHICKENPOX_61,J_CHICKENPOX_62, J_CHICKENPOX_64,J_CHICKENPOX_66,J_CHICKENPOX_71,J_CHICKENPOX_72, J_CHICKENPOX_74,J_CHICKENPOX_84 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAA5p3rS249540 for ; Mon, 9 Nov 2009 23:51:04 -0600 X-ASG-Debug-ID: 1257832277-169200b90000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 081DF721A6 for ; Mon, 9 Nov 2009 21:51:17 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id EeTbAoXYVZwdzwfg for ; Mon, 09 Nov 2009 21:51:17 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id BF8F95B39CE; Tue, 10 Nov 2009 16:51:15 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id 1MW8H0Ef1POQbfjJ; Tue, 10 Nov 2009 16:51:15 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 34388A502B5; Tue, 10 Nov 2009 16:49:38 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id A5E874FD86; Tue, 10 Nov 2009 16:51:15 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KQ8nyfzd73p4; Tue, 10 Nov 2009 16:51:12 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id C9FB04FD82; Tue, 10 Nov 2009 16:51:12 +1100 (EST) Date: Tue, 10 Nov 2009 16:51:12 +1100 (EST) From: Nathan Scott To: Martin Hicks , kenj Cc: pcp@oss.sgi.com Message-ID: <2102239881.326981257832272734.JavaMail.root@mail-au.aconex.com> In-Reply-To: <541918287.326961257832248647.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Experimental patch (was Re: [pcp] mmv pmda: stuff always fails the first time?) Subject: Experimental patch (was Re: [pcp] mmv pmda: stuff always fails the first time?) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_22612_922264040.1257832272733" X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257832281 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14222 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ------=_Part_22612_922264040.1257832272733 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- "Nathan Scott" wrote: > ----- "Martin Hicks" wrote: > > ... > > Is this something wrong with the way I wrote the PMDA, or is this a > > bug in the MMV PMDA? > > Its a bug that cannot really be fixed in the current MMV design > (to do with the way pmdammv and pmcd communicate the namespace > changes) ... MMV needs to switch to using the all-singing, all- > dancing dynamic namespace infrastructure to really fix this. Here's a patch which converts pmdammv to use Kens new "dynamic namespace" interfaces. It seems to function correctly when I throw queries at it via dbpmda ... but no testing beyond that has been done just yet. Lemme know how it goes. It also implements a DSO MMV, which I also ran under dbpmda successfully. Both these binaries should no longer have the annoying-spurious-errors feature. cheers. -- Nathan ------=_Part_22612_922264040.1257832272733 Content-Type: text/x-patch; name=mmv.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=mmv.patch GNUmakefile | 3 Install | 2 mmv.c | 388 ++++++++++++++++++++++++++++++++++++++---------------------- 3 files changed, 254 insertions(+), 139 deletions(-) diff --git a/src/pmdas/mmv/GNUmakefile b/src/pmdas/mmv/GNUmakefile index 6205a8a..63f778a 100644 --- a/src/pmdas/mmv/GNUmakefile +++ b/src/pmdas/mmv/GNUmakefile @@ -21,7 +21,6 @@ DOMAIN = MMV CMDTARGET = pmda$(IAM)$(EXECSUFFIX) LIBTARGET = pmda_$(IAM).$(DSOSUFFIX) TARGETS = $(CMDTARGET) $(LIBTARGET) mmvdump$(EXECSUFFIX) -INSTTARGETS = $(CMDTARGET) mmvdump$(EXECSUFFIX) CFILES = mmv.c LSRCFILES = pmns Install Remove README help root mmvdump.c @@ -40,7 +39,7 @@ include $(BUILDRULES) install: default $(INSTALL) -m 755 -d $(PMDADIR) - $(INSTALL) -m 755 $(INSTTARGETS) Install Remove $(PMDADIR) + $(INSTALL) -m 755 $(TARGETS) Install Remove $(PMDADIR) $(INSTALL) -m 644 README root help pmns domain.h $(PMDADIR) $(CMDTARGET): $(OBJECTS) diff --git a/src/pmdas/mmv/Install b/src/pmdas/mmv/Install index d663f1b..e4cbaa8 100755 --- a/src/pmdas/mmv/Install +++ b/src/pmdas/mmv/Install @@ -18,7 +18,7 @@ . $PCP_SHARE_DIR/lib/pmdaproc.sh iam=mmv -dso_opt=false +dso_opt=true socket_opt=true socket_inet_def=2081 pmda_interface=3 diff --git a/src/pmdas/mmv/mmv.c b/src/pmdas/mmv/mmv.c index ef30ef0..e53e613 100644 --- a/src/pmdas/mmv/mmv.c +++ b/src/pmdas/mmv/mmv.c @@ -30,7 +30,8 @@ #include "./domain.h" #include -static pmdaInterface dispatch; +#define NONLEAF(node) ((node)->pmid == PM_ID_NULL) + static char mypath[MAXPATHLEN]; static int isDSO = 1; @@ -40,6 +41,7 @@ static pmdaIndom * indoms; static int incnt; static int reload; +static __pmnsTree *pmns; static time_t statsdir_ts; /* last statsdir timestamp */ static char * prefix = "mmv"; @@ -67,100 +69,6 @@ typedef struct { static stats_t * slist; static int scnt; -static int -update_namespace(void) -{ - char script[3*MAXPATHLEN]; - int sep = __pmPathSeparator(); - - snprintf(script, sizeof(script), - "%s%c" "lib" "%c" "ReplacePmnsSubtree %s %s%c" "%s.new", - pmGetConfig("PCP_SHARE_DIR"), sep, sep, - prefix, pmnsdir, sep, prefix); - if (system(script) == -1) { - __pmNotifyErr (LOG_ERR, "%s: cannot exec %s", pmProgname, script); - return 1; - } - - return 0; -} - -static void -write_pmnspath(__pmnsNode *base, FILE *f) -{ - if (base && base->parent) { - write_pmnspath(base->parent, f); - fprintf(f, "%s.", base->name); - } -} - -static void -write_pmnsnode(__pmnsNode *base, FILE *f) -{ - __pmnsNode *np; - - /* Print out full path to this part of the tree */ - write_pmnspath(base->parent, f); - fprintf(f, "%s {\n", base->name); - - /* Print out nodes at this level of the tree */ - for (np = base->first; np != NULL; np = np->next) { - if (np->pmid == PM_ID_NULL) - fprintf(f, "\t%s\n", np->name); - else - fprintf(f, "\t%s\t\t%u:%u:%u\n", np->name, - pmid_domain(np->pmid), - pmid_cluster(np->pmid), - pmid_item(np->pmid)); - } - fprintf(f, "}\n\n"); - - /* Print out all the children of this subtree */ - for (np = base->first; np != NULL; np = np->next) - if (np->pmid == PM_ID_NULL) - write_pmnsnode(np, f); -} - -static void -write_pmnsfile(__pmnsTree *pmns) -{ - char tmppath[MAXPATHLEN]; - char path[MAXPATHLEN]; - char *fname = tmppath; - FILE *f = NULL; - - putenv("TMPDIR="); /* temp file must be in pmnsdir, for rename */ - -#if HAVE_MKSTEMP - sprintf(tmppath, "%s%c%s-XXXXXX", pmnsdir, __pmPathSeparator(), prefix); - int fd = mkstemp(tmppath); - if (fd != -1) - f = fdopen(fd, "w"); -#else - fname = tempnam(pmnsdir, prefix); - if (fname != NULL) { - strncpy(tmppath, fname, sizeof(tmppath)); - free(fname); - fname = tmppath; - f = fopen(fname, "w"); - } -#endif - - if (f == NULL) - __pmNotifyErr(LOG_ERR, "%s: failed to generate temporary file %s: %s", - pmProgname, fname, strerror(errno)); - else { - __pmnsNode *node; - for (node = pmns->root->first; node != NULL; node = node->next) - write_pmnsnode(node, f); - fclose(f); - sprintf(path, "%s%c" "%s.new", pmnsdir, __pmPathSeparator(), prefix); - if (rename2(fname, path) < 0) - __pmNotifyErr(LOG_ERR, "%s: cannot rename %s to %s - %s", - pmProgname, fname, path, strerror (errno)); - } -} - /* * Choose an unused cluster ID while honouring specific requests. * If a specific (non-zero) cluster is requested we always use it. @@ -193,24 +101,70 @@ choose_cluster(int requested, const char *path) return requested; } +/* + * Fixup the parent pointers of the tree. + * Fill in the hash table with nodes from the tree. + * Hashing is done on pmid. + */ +static void +mmv_reindex_hash(__pmnsTree *tree, __pmnsNode *root) +{ + __pmnsNode *np; + + for (np = root->first; np != NULL; np = np->next) { + np->parent = root; + if (np->pmid != PM_ID_NULL) { + int i = np->pmid % tree->htabsize; + np->hash = tree->htab[i]; + tree->htab[i] = np; + } + mmv_reindex_hash(tree, np); + } +} + +/* + * "Make the average hash list no longer than 5, and the number + * of hash table entries not a multiple of 2, 3 or 5." + * [From __pmFixPMNSHashTab; without mark_all, dinks with pmids] + */ +static void +mmv_rebuild_hash(__pmnsTree *tree, int numpmid) +{ + int htabsize = numpmid / 5; + + if (htabsize % 2 == 0) htabsize++; + if (htabsize % 3 == 0) htabsize += 2; + if (htabsize % 5 == 0) htabsize += 2; + tree->htabsize = htabsize; + tree->htab = (__pmnsNode **)calloc(htabsize, sizeof(__pmnsNode *)); + if (tree->htab == NULL) + __pmNotifyErr(LOG_ERR, "%s: out of memory in pmns rebuild - %s", + pmProgname, strerror(errno)); + else + mmv_reindex_hash(tree, tree->root); +} + static void -map_stats(void) +map_stats(pmdaExt *pmda) { - __pmnsTree *pmns; struct dirent ** files; char name_reload[64]; int need_reload = 0; int i, sts, num; + if (pmns) + __pmFreePMNS(pmns); + if ((sts = __pmNewPMNS(&pmns)) < 0) { __pmNotifyErr(LOG_ERR, "%s: failed to create new pmns: %s\n", pmProgname, pmErrStr(sts)); + pmns = NULL; return; } mcnt = 1; snprintf(name_reload, sizeof(name_reload), "%s.reload", prefix); - __pmAddPMNSNode(pmns, pmid_build(dispatch.domain, 0, 0), name_reload); + __pmAddPMNSNode(pmns, pmid_build(pmda->e_domain, 0, 0), name_reload); if (indoms != NULL) { for (i = 0; i < incnt; i++) @@ -359,7 +313,7 @@ map_stats(void) metrics[mcnt].m_user = ml + k; metrics[mcnt].m_desc.pmid = pmid_build( - dispatch.domain, s->cluster, ml[k].item); + pmda->e_domain, s->cluster, ml[k].item); if (ml[k].type == MMV_TYPE_ELAPSED) { pmUnits unit = PMDA_PMUNITS(0,1,0,0,PM_TIME_USEC,0); @@ -378,18 +332,20 @@ map_stats(void) metrics[mcnt].m_desc.indom = (!ml[k].indom || ml[k].indom == PM_INDOM_NULL) ? PM_INDOM_NULL : - pmInDom_build(dispatch.domain, + pmInDom_build(pmda->e_domain, (s->cluster << 11) | ml[k].indom); strcat(name, ml[k].name); __pmAddPMNSNode(pmns, pmid_build( - dispatch.domain, s->cluster, ml[k].item), + pmda->e_domain, s->cluster, ml[k].item), name); mcnt++; } } else { __pmNotifyErr(LOG_ERR, "%s: cannot grow metric list", pmProgname); + if (isDSO) + return; exit(1); } break; @@ -405,7 +361,7 @@ map_stats(void) for (k = 0; k < toc[j].count; k++) { ip = &indoms[incnt + k]; - ip->it_indom = pmInDom_build(dispatch.domain, + ip->it_indom = pmInDom_build(pmda->e_domain, (slist[i].cluster << 11) | id[k].serial); ip->it_numinst = id[k].count; ip->it_set = (pmdaInstid *) @@ -422,6 +378,8 @@ map_stats(void) __pmNotifyErr(LOG_ERR, "%s: cannot get memory for instance list", pmProgname); + if (isDSO) + return; exit(1); } } @@ -429,6 +387,8 @@ map_stats(void) } else { __pmNotifyErr(LOG_ERR, "%s: cannot grow indom list", pmProgname); + if (isDSO) + return; exit(1); } break; @@ -445,9 +405,7 @@ map_stats(void) } } - write_pmnsfile(pmns); - __pmFreePMNS(pmns); - + mmv_rebuild_hash(pmns, mcnt); /* for reverse (pmid->name) lookups */ reload = need_reload; } @@ -549,8 +507,8 @@ mmv_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom) return 0; } -static int -mmv_reload_maybe(void) +static void +mmv_reload_maybe(pmdaExt *pmda) { int i; struct stat s; @@ -572,14 +530,8 @@ mmv_reload_maybe(void) } if (need_reload) { - /* something changed - reload */ - pmdaExt * pmda = dispatch.version.two.ext; /* we know it is V.2 */ - - /* Note: this line means we cannot run as shared library */ - __pmSendError(pmda->e_outfd, PDU_BINARY, PM_ERR_PMDANOTREADY); - __pmNotifyErr(LOG_INFO, "%s: reloading", pmProgname); - map_stats(); + map_stats(pmda); pmda->e_indoms = indoms; pmda->e_nindoms = incnt; @@ -590,19 +542,14 @@ mmv_reload_maybe(void) __pmNotifyErr(LOG_INFO, "%s: %d metrics and %d indoms after reload", pmProgname, mcnt, incnt); - - reload = update_namespace(); } - - return need_reload; } /* Intercept request for descriptor and check if we'd have to reload */ static int mmv_desc(pmID pmid, pmDesc *desc, pmdaExt *ep) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); return pmdaDesc(pmid, desc, ep); } @@ -612,9 +559,8 @@ mmv_text(int ident, int type, char **buffer, pmdaExt *ep) if (type & PM_TEXT_INDOM) return PM_ERR_TEXT; - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; - else if (pmid_cluster(ident) == 0) + mmv_reload_maybe(ep); + if (pmid_cluster(ident) == 0) return pmdaText(ident, type, buffer, ep); else { mmv_disk_metric_t * m; @@ -643,16 +589,14 @@ static int mmv_instance(pmInDom indom, int inst, char *name, __pmInResult **result, pmdaExt *ep) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); return pmdaInstance(indom, inst, name, result, ep); } static int mmv_fetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(pmda); return pmdaFetch(numpmid, pmidlist, resp, pmda); } @@ -661,8 +605,7 @@ mmv_store(pmResult *result, pmdaExt *ep) { int i, m; - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); for (i = 0; i < result->numpmid; i++) { pmValueSet * vsp = result->vset[i]; @@ -692,6 +635,175 @@ mmv_store(pmResult *result, pmdaExt *ep) return 0; } +static __pmnsNode * +mmv_lookup_node(__pmnsNode *node, char *name) +{ + while (node != NULL) { + size_t length = strlen(node->name); + if (strncmp(name, node->name, length) == 0) { + if (name[length] == '\0') + return node; + if (name[length] == '.' && NONLEAF(node)) + return mmv_lookup_node(node->first, name + length + 1); + } + node = node->next; + } + return NULL; +} + +static int +mmv_pmid(char *name, pmID *pmid, pmdaExt *pmda) +{ + __pmnsNode *node; + + mmv_reload_maybe(pmda); + if ((node = mmv_lookup_node(pmns->root->first, name)) == NULL) + return PM_ERR_NAME; + if (NONLEAF(node)) + return PM_ERR_NAME; + *pmid = node->pmid; + return 0; +} + +static char * +mmv_absolute_name(__pmnsNode *node, char *buffer) +{ + if (node && node->parent) { + buffer = mmv_absolute_name(node->parent, buffer); + strcpy(buffer, node->name); + buffer += strlen(node->name); + *buffer++ = '.'; + } + return buffer; +} + +static int +mmv_name(pmID pmid, char ***nameset, pmdaExt *pmda) +{ + __pmnsNode *hashchain, *node, *parent; + int nmatch = 0, length = 0; + char *p, **list; + + mmv_reload_maybe(pmda); + if (!pmns) + return PM_ERR_PMID; + + hashchain = pmns->htab[pmid % pmns->htabsize]; + for (node = hashchain; node != NULL; node = node->hash) { + if (node->pmid == pmid) { + for (parent = node; parent->parent; parent = parent->parent) + length += strlen(parent->name) + 1; + nmatch++; + } + } + + if (nmatch == 0) + return PM_ERR_PMID; + + length += nmatch * sizeof(char *); /* pointers to names */ + + if ((list = (char **)malloc(length)) == NULL) + return -errno; + + p = (char *)&list[nmatch]; + nmatch = 0; + for (node = hashchain; node != NULL; node = node->hash) { + if (node->pmid == pmid) { + list[nmatch++] = p; + p = mmv_absolute_name(node, p); + *(p-1) = '\0'; /* overwrite final '.' */ + } + } + + *nameset = list; + return nmatch; +} + +static int +mmv_children_relative(__pmnsNode *base, char ***offspring, int **status) +{ + __pmnsNode *node; + char **list, *p; + int *leaf, length = 0, nmatch = 0; + + for (node = base; node != NULL; node = node->next, nmatch++) + length += strlen(node->name) + 1; + length += nmatch * sizeof(char *); /* pointers to names */ + if ((list = (char **)malloc(length)) == NULL) + return -errno; + if ((leaf = (int *)malloc(nmatch * sizeof(int*))) == NULL) { + free(list); + return -errno; + } + p = (char *)&list[nmatch]; + nmatch = 0; + for (node = base; node != NULL; node = node->next, nmatch++) { + leaf[nmatch] = NONLEAF(node) ? PMNS_NONLEAF_STATUS : PMNS_LEAF_STATUS; + list[nmatch] = p; + strcpy(p, node->name); + p += strlen(node->name); + *p++ = '\0'; + } + + *offspring = list; + *status = leaf; + return nmatch; +} + +static int +mmv_children_absolute(__pmnsNode *base, char ***offspring, int **status) +{ + __pmnsNode *node, *parent; + char **list, *p; + int *leaf, length = 0, nmetrics = 0; + + /* walk to every leaf & then add its (absolute name) length */ + for (node = base; node != NULL; node = node->next) { + if (NONLEAF(node)) + continue; + for (parent = node; parent->parent; parent = parent->parent) + length += strlen(parent->name) + 1; + nmetrics++; + } + length += nmetrics * sizeof(char *); /* pointers to names */ + if ((list = (char **)malloc(length)) == NULL) + return -errno; + if ((leaf = (int *)malloc(nmetrics * sizeof(int*))) == NULL) { + free(list); + return -errno; + } + p = (char *)&list[nmetrics]; + nmetrics = 0; + for (node = base; node != NULL; node = node->next, nmetrics++) { + if (NONLEAF(node)) + continue; + leaf[nmetrics] = PMNS_LEAF_STATUS; + list[nmetrics] = p; + p = mmv_absolute_name(node, p); + *(p-1) = '\0'; /* overwrite final '.' */ + } + + *offspring = list; + *status = leaf; + return nmetrics; +} + +static int +mmv_children(char *name, int traverse, char ***offspring, int **status, pmdaExt *pmda) +{ + __pmnsNode *node; + + mmv_reload_maybe(pmda); + if (!pmns) + return PM_ERR_NAME; + + if ((node = mmv_lookup_node(pmns->root->first, name)) == NULL) + return PM_ERR_NAME; + + if (traverse == 0) + return mmv_children_relative(node, offspring, status); + return mmv_children_absolute(node, offspring, status); +} void mmv_init(pmdaInterface *dp) @@ -701,7 +813,7 @@ mmv_init(pmdaInterface *dp) if (isDSO) { snprintf(mypath, sizeof(mypath), "%s%c" "mmv" "%c" "help", pmGetConfig("PCP_PMDAS_DIR"), sep, sep); - pmdaDSO(dp, PMDA_INTERFACE_3, "MMV DSO", mypath); + pmdaDSO(dp, PMDA_INTERFACE_4, "MMV DSO", mypath); } pcptmpdir = pmGetConfig("PCP_TMP_DIR"); @@ -729,11 +841,14 @@ mmv_init(pmdaInterface *dp) exit(0); } - dp->version.two.fetch = mmv_fetch; - dp->version.two.store = mmv_store; - dp->version.two.desc = mmv_desc; - dp->version.two.text = mmv_text; - dp->version.two.instance = mmv_instance; + dp->version.four.fetch = mmv_fetch; + dp->version.four.store = mmv_store; + dp->version.four.desc = mmv_desc; + dp->version.four.text = mmv_text; + dp->version.four.instance = mmv_instance; + dp->version.four.pmid = mmv_pmid; + dp->version.four.name = mmv_name; + dp->version.four.children = mmv_children; pmdaSetFetchCallBack(dp, mmv_fetchCallBack); @@ -762,6 +877,7 @@ main(int argc, char **argv) int err = 0; int sep = __pmPathSeparator(); char logfile[32]; + pmdaInterface dispatch = { 0 }; isDSO = 0; __pmSetProgname(argv[0]); @@ -770,7 +886,7 @@ main(int argc, char **argv) snprintf(mypath, sizeof(mypath), "%s%c" "%s%c" "help", pmGetConfig("PCP_PMDAS_DIR"), sep, prefix, sep); snprintf(logfile, sizeof(logfile), "%s.log", prefix); - pmdaDaemon(&dispatch, PMDA_INTERFACE_3, pmProgname, MMV, logfile, mypath); + pmdaDaemon(&dispatch, PMDA_INTERFACE_4, pmProgname, MMV, logfile, mypath); if ((pmdaGetOpt(argc, argv, "D:d:l:?", &dispatch, &err) != EOF) || err || argc != optind) ------=_Part_22612_922264040.1257832272733-- From mort@sgi.com Tue Nov 10 12:33:10 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_73, URIBL_BLACK autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAAIX9KD043616 for ; Tue, 10 Nov 2009 12:33:09 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id 1E0DC3040CE for ; Tue, 10 Nov 2009 10:33:25 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id F1F96A243300; Tue, 10 Nov 2009 12:33:24 -0600 (CST) Date: Tue, 10 Nov 2009 12:33:24 -0600 From: Martin Hicks To: pcp@oss.sgi.com Subject: pcp updates (mort-dev) Message-ID: <20091110183324.GE5895@alcatraz.americas.sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Changes committed to git://oss.sgi.com/mort/pcp.git VERSION.pcp | 4 ++-- configure.in | 3 ++- debian/changelog | 6 ++++++ src/pmcd/pmdaproc.sh | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-) commit b3549934521ea99482467345388e7000235eae9d Author: Martin Hicks Date: Tue Nov 10 13:31:21 2009 -0500 Fix PCP_BINADM_DIR on ia64/linux It should be /usr/lib/pcp/bin. Nothing should be in /usr/lib64 on ia64. commit 94cca2c82e3c34062936dde7c64626eff2b76e55 Author: Martin Hicks Date: Mon Nov 9 10:42:01 2009 -0500 Bump version to 3.0.2 (development) commit a7234db427c90847e58dfaa4e0217b5f6962449d Author: Martin Hicks Date: Mon Nov 9 10:16:04 2009 -0500 Fix regression of "binary notready" IPC protocol This ability was accidentally removed when the text protocol was removed. From goodwinos@gmail.com Tue Nov 10 16:57:15 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAAMvEoN065999 for ; Tue, 10 Nov 2009 16:57:14 -0600 X-ASG-Debug-ID: 1257893851-3e0b010b0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 823E6CFFE49; Tue, 10 Nov 2009 14:57:31 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Hq0aJSmUT8E6FyAC; Tue, 10 Nov 2009 14:57:31 -0800 (PST) Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nAAMvUsM028427 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 10 Nov 2009 17:57:31 -0500 Received: from [10.64.176.217] (dhcp-176-217.mel.redhat.com [10.64.176.217]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nAAMvTlu025640; Tue, 10 Nov 2009 17:57:30 -0500 Message-ID: <4AF9EFD9.1020802@gmail.com> Date: Wed, 11 Nov 2009 09:57:29 +1100 From: Mark Goodwin User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Martin Hicks CC: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] pcp updates (mort-dev) Subject: Re: [pcp] pcp updates (mort-dev) References: <20091110183324.GE5895@alcatraz.americas.sgi.com> In-Reply-To: <20091110183324.GE5895@alcatraz.americas.sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1257893852 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14283 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Martin Hicks wrote: ... > commit b3549934521ea99482467345388e7000235eae9d > Author: Martin Hicks > Date: Tue Nov 10 13:31:21 2009 -0500 > > Fix PCP_BINADM_DIR on ia64/linux > > It should be /usr/lib/pcp/bin. Nothing should be in > /usr/lib64 on ia64. > Hi Martin, just wondering what the story behind this change is? The original change was to conform with the Fedora rules (and avoid rpmlint complaints). This has something to do with archs that support multilib. BTW, I don't have access to any ia64 h/w any more .. Cheers -- Mark From mort@sgi.com Tue Nov 10 17:12:34 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAANCY7Y067895 for ; Tue, 10 Nov 2009 17:12:34 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay3.corp.sgi.com (Postfix) with ESMTP id 88A0EAC00B; Tue, 10 Nov 2009 15:12:49 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id F27C7A243300; Tue, 10 Nov 2009 17:12:48 -0600 (CST) Date: Tue, 10 Nov 2009 17:12:48 -0600 From: Martin Hicks To: Mark Goodwin Cc: pcp@oss.sgi.com Subject: Re: [pcp] pcp updates (mort-dev) Message-ID: <20091110231248.GK5895@alcatraz.americas.sgi.com> References: <20091110183324.GE5895@alcatraz.americas.sgi.com> <4AF9EFD9.1020802@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4AF9EFD9.1020802@gmail.com> User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Nov 11, 2009 at 09:57:29AM +1100, Mark Goodwin wrote: > Martin Hicks wrote: > ... >> commit b3549934521ea99482467345388e7000235eae9d >> Author: Martin Hicks >> Date: Tue Nov 10 13:31:21 2009 -0500 >> >> Fix PCP_BINADM_DIR on ia64/linux >> It should be /usr/lib/pcp/bin. Nothing should be in >> /usr/lib64 on ia64. >> > > Hi Martin, just wondering what the story behind this change is? > > The original change was to conform with the Fedora rules > (and avoid rpmlint complaints). This has something to do with > archs that support multilib. BTW, I don't have access to any ia64 > h/w any more .. I just thought this was wrong. We have PCP_LIB_DIR as /usr/lib on ia64, but PCP_BINADM_DIR was /usr/lib64/pcp/bin There is nothing else in /usr/lib64 on an ia64 machine, so I assumed this must be wrong. If this patch is wrong then please enlighten me :) The test looked a lot like an x86 vs. x86_64 arch test that went awry and also caught ia64. mh From goodwinos@gmail.com Tue Nov 10 17:24:33 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAANOW0r068682 for ; Tue, 10 Nov 2009 17:24:32 -0600 X-ASG-Debug-ID: 1257895489-508101840000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5BA87D033F9; Tue, 10 Nov 2009 15:24:49 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id AGXcIQP7c4qeHnTN; Tue, 10 Nov 2009 15:24:49 -0800 (PST) Received: from int-mx03.intmail.prod.int.phx2.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nAANOneq022529 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 10 Nov 2009 18:24:49 -0500 Received: from [10.64.176.217] (dhcp-176-217.mel.redhat.com [10.64.176.217]) by int-mx03.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nAANOmqV001557; Tue, 10 Nov 2009 18:24:48 -0500 Message-ID: <4AF9F63F.50907@gmail.com> Date: Wed, 11 Nov 2009 10:24:47 +1100 From: Mark Goodwin User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Martin Hicks CC: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] pcp updates (mort-dev) Subject: Re: [pcp] pcp updates (mort-dev) References: <20091110183324.GE5895@alcatraz.americas.sgi.com> <4AF9EFD9.1020802@gmail.com> <20091110231248.GK5895@alcatraz.americas.sgi.com> In-Reply-To: <20091110231248.GK5895@alcatraz.americas.sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.16 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1257895490 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14285 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Martin Hicks wrote: > I just thought this was wrong. We have PCP_LIB_DIR as /usr/lib on ia64, > but PCP_BINADM_DIR was /usr/lib64/pcp/bin > > There is nothing else in /usr/lib64 on an ia64 machine, so I assumed > this must be wrong. If this patch is wrong then please enlighten me :) > The test looked a lot like an x86 vs. x86_64 arch test that went awry > and also caught ia64. > if there is nothing else in /usr/lib64, then you're probably right (I don't have ia64 h/w to investigate). On x86_64, we have /usr/lib64 containing 64bit binaries and /usr/lib containing 32bit binaries. Looks like on ia64 the native binaries are in /usr/lib, with (presumably) the ia32 compat binaries in /usr/lib32? In any case, I guess ia64 isn't a true multilib arch since ia32 is effectively emulated. - Mark From trev@sgi.com Tue Nov 10 22:13:22 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [150.166.39.100]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAB4DLRN088443 for ; Tue, 10 Nov 2009 22:13:21 -0600 Received: from [10.220.3.214] ([10.220.3.214]) by cthulhu.engr.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id nAB4Dcbk032218; Tue, 10 Nov 2009 20:13:38 -0800 Message-ID: <4AFA39D4.2090509@sgi.com> Date: Tue, 10 Nov 2009 20:13:08 -0800 From: Trevor Hurst User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Mark Goodwin CC: Martin Hicks , pcp@oss.sgi.com Subject: Re: [pcp] pcp updates (mort-dev) References: <20091110183324.GE5895@alcatraz.americas.sgi.com> <4AF9EFD9.1020802@gmail.com> <20091110231248.GK5895@alcatraz.americas.sgi.com> <4AF9F63F.50907@gmail.com> In-Reply-To: <4AF9F63F.50907@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Mark Goodwin wrote: > Martin Hicks wrote: >> I just thought this was wrong. We have PCP_LIB_DIR as /usr/lib on ia64, >> but PCP_BINADM_DIR was /usr/lib64/pcp/bin >> >> There is nothing else in /usr/lib64 on an ia64 machine, so I assumed >> this must be wrong. If this patch is wrong then please enlighten me :) >> The test looked a lot like an x86 vs. x86_64 arch test that went awry >> and also caught ia64. >> > > if there is nothing else in /usr/lib64, then you're probably right > (I don't have ia64 h/w to investigate). > > On x86_64, we have /usr/lib64 containing 64bit binaries > and /usr/lib containing 32bit binaries. Looks like on ia64 > the native binaries are in /usr/lib, with (presumably) the > ia32 compat binaries in /usr/lib32? > > In any case, I guess ia64 isn't a true multilib arch since > ia32 is effectively emulated. > > - Mark I think a lot of apps on Itanium use the emulated, is this the same thing with PCP? I ran into this issue when we first started migrating apps to Itanium with our A350 product line. -- Trev From makc@iinet.net.au Wed Nov 11 05:13:30 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nABBDTeM126613 for ; Wed, 11 Nov 2009 05:13:30 -0600 X-ASG-Debug-ID: 1257938025-09b9026a0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from outbound.icp-qv1-irony-out2.iinet.net.au (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 069001BA924C for ; Wed, 11 Nov 2009 03:13:46 -0800 (PST) Received: from outbound.icp-qv1-irony-out2.iinet.net.au (outbound.icp-qv1-irony-out2.iinet.net.au [203.59.1.107]) by cuda.sgi.com with ESMTP id bImEcngYRlbXtq1r for ; Wed, 11 Nov 2009 03:13:46 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAIIr+krL2Rwk/2dsb2JhbADcLYQ8BA X-IronPort-AV: E=Sophos;i="4.44,722,1249228800"; d="scan'208";a="579114788" Received: from unknown (HELO alma.crabbed.net) ([203.217.28.36]) by outbound.icp-qv1-irony-out2.iinet.net.au with ESMTP; 11 Nov 2009 19:13:44 +0800 Received: by alma.crabbed.net (Postfix, from userid 501) id DAA483C4D08; Wed, 11 Nov 2009 22:12:17 +1100 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19194.39953.833739.476968@iinet.net.au> Date: Wed, 11 Nov 2009 22:12:17 +1100 From: Max Matveev To: Mark Goodwin Cc: Martin Hicks , pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] pcp updates (mort-dev) Subject: Re: [pcp] pcp updates (mort-dev) In-Reply-To: <4AF9EFD9.1020802@gmail.com> References: <20091110183324.GE5895@alcatraz.americas.sgi.com> <4AF9EFD9.1020802@gmail.com> X-Mailer: VM 7.17 under 21.4 (patch 19) "Constant Variable" XEmacs Lucid X-Barracuda-Connect: outbound.icp-qv1-irony-out2.iinet.net.au[203.59.1.107] X-Barracuda-Start-Time: 1257938028 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0255 1.0000 -1.8559 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.86 X-Barracuda-Spam-Status: No, SCORE=-1.86 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14328 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 11 Nov 2009 09:57:29 +1100, Mark Goodwin wrote: goodwinos> The original change was to conform with the Fedora rules goodwinos> (and avoid rpmlint complaints). This has something to do with goodwinos> archs that support multilib. BTW, I don't have access to any ia64 goodwinos> h/w any more .. Does Fedora have a concept of libexec? Because this is what those binaries are, they're not libraries, why are we pushing them in /usr/lib? max From goodwinos@gmail.com Wed Nov 11 05:58:59 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nABBwwFq130755 for ; Wed, 11 Nov 2009 05:58:59 -0600 X-ASG-Debug-ID: 1257940755-256001550000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id EF79979012; Wed, 11 Nov 2009 03:59:15 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id UmRZ1F6FamfwAh3I; Wed, 11 Nov 2009 03:59:15 -0800 (PST) Received: from int-mx05.intmail.prod.int.phx2.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.18]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nABBxCB6031708 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 11 Nov 2009 06:59:12 -0500 Received: from [10.11.10.9] (vpn-10-9.rdu.redhat.com [10.11.10.9]) by int-mx05.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nABBx9qj007328; Wed, 11 Nov 2009 06:59:10 -0500 Message-ID: <4AFAA70B.7020302@gmail.com> Date: Wed, 11 Nov 2009 22:59:07 +1100 From: Mark Goodwin User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Max Matveev CC: Martin Hicks , pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] pcp updates (mort-dev) Subject: Re: [pcp] pcp updates (mort-dev) References: <20091110183324.GE5895@alcatraz.americas.sgi.com> <4AF9EFD9.1020802@gmail.com> <19194.39953.833739.476968@iinet.net.au> In-Reply-To: <19194.39953.833739.476968@iinet.net.au> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.18 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1257940755 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0002 1.0000 -2.0200 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14332 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Max Matveev wrote: > On Wed, 11 Nov 2009 09:57:29 +1100, Mark Goodwin wrote: > > goodwinos> The original change was to conform with the Fedora rules > goodwinos> (and avoid rpmlint complaints). This has something to do with > goodwinos> archs that support multilib. BTW, I don't have access to any ia64 > goodwinos> h/w any more .. > > Does Fedora have a concept of libexec? Because this is what those > binaries are, they're not libraries, why are we pushing them in > /usr/lib? Both /usr/lib*/PACKAGE and /usr/libexec/PACKAGE are in wide-spread use (poke around on a Fedora system and you'll see) and there are RPM macros for both, e.g. %{libexecdir}. My Fedora reviewer suggested to use /usr/lib in preference, so I went with that. FWIW, the FSH does not include any provision for libexec. The Fedora packaging guidelines in this respect are documented here: https://fedoraproject.org/wiki/Packaging/Guidelines#Libexecdir FSH is here: http://www.pathname.com/fhs It'd be easy enough to move PCP_BINADM_DIR to %{libexecdir}/pcp if we want to - that may provide more consistency for the ia64 platform too. I don't particularly mind either way. Cheers - Mark From nscott@aconex.com Wed Nov 11 16:27:43 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nABMRgkN187115 for ; Wed, 11 Nov 2009 16:27:42 -0600 X-ASG-Debug-ID: 1257978479-3dfc00c50000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6BC1F185F485 for ; Wed, 11 Nov 2009 14:28:00 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id VXnxkNsvwoXS9xXf for ; Wed, 11 Nov 2009 14:28:00 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id AF66C86BAF5; Thu, 12 Nov 2009 09:27:58 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id FdEgCvQPDWFFtheC; Thu, 12 Nov 2009 09:27:58 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 3C699A50280; Thu, 12 Nov 2009 09:26:20 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 96B81C7BB4; Thu, 12 Nov 2009 09:27:58 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7jAq6FPvH8Vs; Thu, 12 Nov 2009 09:27:53 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id D4724C7BAA; Thu, 12 Nov 2009 09:27:53 +1100 (EST) Date: Thu, 12 Nov 2009 09:27:53 +1100 (EST) From: Nathan Scott To: Mark Goodwin Cc: pcp@oss.sgi.com, Max Matveev Message-ID: <1166010705.357711257978473129.JavaMail.root@mail-au.aconex.com> In-Reply-To: <4AFAA70B.7020302@gmail.com> X-ASG-Orig-Subj: Re: [pcp] pcp updates (mort-dev) Subject: Re: [pcp] pcp updates (mort-dev) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1257978481 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14373 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Mark Goodwin" wrote: > > It'd be easy enough to move PCP_BINADM_DIR to %{libexecdir}/pcp > if we want to - that may provide more consistency for the ia64 > platform too. I don't particularly mind either way. I'm in two minds - having a "variable" path (lib vs lib64) for things like {pmlogger,pmie}_{daily,check} is unfortunate -they often run from cron and sourcing /etc/pcp.conf isn't so simple there, but changing it again would be painful too. Long-term its probably best to switch to libexec though (d'oh). cheers. -- Nathan From nscott@aconex.com Thu Nov 12 20:00:32 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_33, J_CHICKENPOX_43,J_CHICKENPOX_55,J_CHICKENPOX_61,J_CHICKENPOX_62, J_CHICKENPOX_64,J_CHICKENPOX_66,J_CHICKENPOX_71,J_CHICKENPOX_72, J_CHICKENPOX_74,J_CHICKENPOX_84 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAD20Vsq049375 for ; Thu, 12 Nov 2009 20:00:32 -0600 X-ASG-Debug-ID: 1258077647-5e3500030000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 3A4C7D202FF for ; Thu, 12 Nov 2009 18:00:48 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id izqUADUHmly11KZE for ; Thu, 12 Nov 2009 18:00:48 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 4D94D5C37E7; Fri, 13 Nov 2009 13:00:46 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id akDXe1NYDYRqpB4C; Fri, 13 Nov 2009 13:00:46 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 1268EA5028C; Fri, 13 Nov 2009 12:59:07 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 0E0E54FD82; Fri, 13 Nov 2009 13:00:46 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IeozD4TLZFgI; Fri, 13 Nov 2009 13:00:42 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 073544FD85; Fri, 13 Nov 2009 13:00:42 +1100 (EST) Date: Fri, 13 Nov 2009 13:00:41 +1100 (EST) From: Nathan Scott To: Martin Hicks , kenj Cc: pcp@oss.sgi.com Message-ID: <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_26400_1511119290.1258077641851" X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258077650 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14475 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ------=_Part_26400_1511119290.1258077641851 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit This version fixes a problem in mmv_children(), adds the new syntax into the pmns file, and in general works Much More Betterer when run from pmcd (rather than dbpmda). cheers. ----- "Nathan Scott" wrote: > ----- "Nathan Scott" wrote: > > ----- "Martin Hicks" wrote: > > > ... > > > Is this something wrong with the way I wrote the PMDA, or is this > a > > > bug in the MMV PMDA? > > > > Its a bug that cannot really be fixed in the current MMV design > > (to do with the way pmdammv and pmcd communicate the namespace > > changes) ... MMV needs to switch to using the all-singing, all- > > dancing dynamic namespace infrastructure to really fix this. > > Here's a patch which converts pmdammv to use Kens new "dynamic > namespace" interfaces. It seems to function correctly when I > throw queries at it via dbpmda ... but no testing beyond that > has been done just yet. > > Lemme know how it goes. It also implements a DSO MMV, which I > also ran under dbpmda successfully. Both these binaries should > no longer have the annoying-spurious-errors feature. -- Nathan ------=_Part_26400_1511119290.1258077641851 Content-Type: text/x-patch; name=mmv.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=mmv.patch GNUmakefile | 3 Install | 2 mmv.c | 421 ++++++++++++++++++++++++++++++++++++++++-------------------- pmns | 7 4 files changed, 289 insertions(+), 144 deletions(-) diff --git a/src/pmdas/mmv/GNUmakefile b/src/pmdas/mmv/GNUmakefile index 6205a8a..63f778a 100644 --- a/src/pmdas/mmv/GNUmakefile +++ b/src/pmdas/mmv/GNUmakefile @@ -21,7 +21,6 @@ DOMAIN = MMV CMDTARGET = pmda$(IAM)$(EXECSUFFIX) LIBTARGET = pmda_$(IAM).$(DSOSUFFIX) TARGETS = $(CMDTARGET) $(LIBTARGET) mmvdump$(EXECSUFFIX) -INSTTARGETS = $(CMDTARGET) mmvdump$(EXECSUFFIX) CFILES = mmv.c LSRCFILES = pmns Install Remove README help root mmvdump.c @@ -40,7 +39,7 @@ include $(BUILDRULES) install: default $(INSTALL) -m 755 -d $(PMDADIR) - $(INSTALL) -m 755 $(INSTTARGETS) Install Remove $(PMDADIR) + $(INSTALL) -m 755 $(TARGETS) Install Remove $(PMDADIR) $(INSTALL) -m 644 README root help pmns domain.h $(PMDADIR) $(CMDTARGET): $(OBJECTS) diff --git a/src/pmdas/mmv/Install b/src/pmdas/mmv/Install index d663f1b..e4cbaa8 100755 --- a/src/pmdas/mmv/Install +++ b/src/pmdas/mmv/Install @@ -18,7 +18,7 @@ . $PCP_SHARE_DIR/lib/pmdaproc.sh iam=mmv -dso_opt=false +dso_opt=true socket_opt=true socket_inet_def=2081 pmda_interface=3 diff --git a/src/pmdas/mmv/mmv.c b/src/pmdas/mmv/mmv.c index ef30ef0..68c020d 100644 --- a/src/pmdas/mmv/mmv.c +++ b/src/pmdas/mmv/mmv.c @@ -30,7 +30,8 @@ #include "./domain.h" #include -static pmdaInterface dispatch; +#define NONLEAF(node) ((node)->pmid == PM_ID_NULL) + static char mypath[MAXPATHLEN]; static int isDSO = 1; @@ -40,6 +41,7 @@ static pmdaIndom * indoms; static int incnt; static int reload; +static __pmnsTree *pmns; static time_t statsdir_ts; /* last statsdir timestamp */ static char * prefix = "mmv"; @@ -67,100 +69,6 @@ typedef struct { static stats_t * slist; static int scnt; -static int -update_namespace(void) -{ - char script[3*MAXPATHLEN]; - int sep = __pmPathSeparator(); - - snprintf(script, sizeof(script), - "%s%c" "lib" "%c" "ReplacePmnsSubtree %s %s%c" "%s.new", - pmGetConfig("PCP_SHARE_DIR"), sep, sep, - prefix, pmnsdir, sep, prefix); - if (system(script) == -1) { - __pmNotifyErr (LOG_ERR, "%s: cannot exec %s", pmProgname, script); - return 1; - } - - return 0; -} - -static void -write_pmnspath(__pmnsNode *base, FILE *f) -{ - if (base && base->parent) { - write_pmnspath(base->parent, f); - fprintf(f, "%s.", base->name); - } -} - -static void -write_pmnsnode(__pmnsNode *base, FILE *f) -{ - __pmnsNode *np; - - /* Print out full path to this part of the tree */ - write_pmnspath(base->parent, f); - fprintf(f, "%s {\n", base->name); - - /* Print out nodes at this level of the tree */ - for (np = base->first; np != NULL; np = np->next) { - if (np->pmid == PM_ID_NULL) - fprintf(f, "\t%s\n", np->name); - else - fprintf(f, "\t%s\t\t%u:%u:%u\n", np->name, - pmid_domain(np->pmid), - pmid_cluster(np->pmid), - pmid_item(np->pmid)); - } - fprintf(f, "}\n\n"); - - /* Print out all the children of this subtree */ - for (np = base->first; np != NULL; np = np->next) - if (np->pmid == PM_ID_NULL) - write_pmnsnode(np, f); -} - -static void -write_pmnsfile(__pmnsTree *pmns) -{ - char tmppath[MAXPATHLEN]; - char path[MAXPATHLEN]; - char *fname = tmppath; - FILE *f = NULL; - - putenv("TMPDIR="); /* temp file must be in pmnsdir, for rename */ - -#if HAVE_MKSTEMP - sprintf(tmppath, "%s%c%s-XXXXXX", pmnsdir, __pmPathSeparator(), prefix); - int fd = mkstemp(tmppath); - if (fd != -1) - f = fdopen(fd, "w"); -#else - fname = tempnam(pmnsdir, prefix); - if (fname != NULL) { - strncpy(tmppath, fname, sizeof(tmppath)); - free(fname); - fname = tmppath; - f = fopen(fname, "w"); - } -#endif - - if (f == NULL) - __pmNotifyErr(LOG_ERR, "%s: failed to generate temporary file %s: %s", - pmProgname, fname, strerror(errno)); - else { - __pmnsNode *node; - for (node = pmns->root->first; node != NULL; node = node->next) - write_pmnsnode(node, f); - fclose(f); - sprintf(path, "%s%c" "%s.new", pmnsdir, __pmPathSeparator(), prefix); - if (rename2(fname, path) < 0) - __pmNotifyErr(LOG_ERR, "%s: cannot rename %s to %s - %s", - pmProgname, fname, path, strerror (errno)); - } -} - /* * Choose an unused cluster ID while honouring specific requests. * If a specific (non-zero) cluster is requested we always use it. @@ -193,24 +101,70 @@ choose_cluster(int requested, const char *path) return requested; } +/* + * Fixup the parent pointers of the tree. + * Fill in the hash table with nodes from the tree. + * Hashing is done on pmid. + */ +static void +mmv_reindex_hash(__pmnsTree *tree, __pmnsNode *root) +{ + __pmnsNode *np; + + for (np = root->first; np != NULL; np = np->next) { + np->parent = root; + if (np->pmid != PM_ID_NULL) { + int i = np->pmid % tree->htabsize; + np->hash = tree->htab[i]; + tree->htab[i] = np; + } + mmv_reindex_hash(tree, np); + } +} + +/* + * "Make the average hash list no longer than 5, and the number + * of hash table entries not a multiple of 2, 3 or 5." + * [From __pmFixPMNSHashTab; without mark_all, dinks with pmids] + */ static void -map_stats(void) +mmv_rebuild_hash(__pmnsTree *tree, int numpmid) +{ + int htabsize = numpmid / 5; + + if (htabsize % 2 == 0) htabsize++; + if (htabsize % 3 == 0) htabsize += 2; + if (htabsize % 5 == 0) htabsize += 2; + tree->htabsize = htabsize; + tree->htab = (__pmnsNode **)calloc(htabsize, sizeof(__pmnsNode *)); + if (tree->htab == NULL) + __pmNotifyErr(LOG_ERR, "%s: out of memory in pmns rebuild - %s", + pmProgname, strerror(errno)); + else + mmv_reindex_hash(tree, tree->root); +} + +static void +map_stats(pmdaExt *pmda) { - __pmnsTree *pmns; struct dirent ** files; char name_reload[64]; int need_reload = 0; int i, sts, num; + if (pmns) + __pmFreePMNS(pmns); + if ((sts = __pmNewPMNS(&pmns)) < 0) { __pmNotifyErr(LOG_ERR, "%s: failed to create new pmns: %s\n", pmProgname, pmErrStr(sts)); + pmns = NULL; return; } mcnt = 1; snprintf(name_reload, sizeof(name_reload), "%s.reload", prefix); - __pmAddPMNSNode(pmns, pmid_build(dispatch.domain, 0, 0), name_reload); + __pmAddPMNSNode(pmns, pmid_build(pmda->e_domain, 0, 0), name_reload); if (indoms != NULL) { for (i = 0; i < incnt; i++) @@ -359,7 +313,7 @@ map_stats(void) metrics[mcnt].m_user = ml + k; metrics[mcnt].m_desc.pmid = pmid_build( - dispatch.domain, s->cluster, ml[k].item); + pmda->e_domain, s->cluster, ml[k].item); if (ml[k].type == MMV_TYPE_ELAPSED) { pmUnits unit = PMDA_PMUNITS(0,1,0,0,PM_TIME_USEC,0); @@ -378,18 +332,20 @@ map_stats(void) metrics[mcnt].m_desc.indom = (!ml[k].indom || ml[k].indom == PM_INDOM_NULL) ? PM_INDOM_NULL : - pmInDom_build(dispatch.domain, + pmInDom_build(pmda->e_domain, (s->cluster << 11) | ml[k].indom); strcat(name, ml[k].name); __pmAddPMNSNode(pmns, pmid_build( - dispatch.domain, s->cluster, ml[k].item), + pmda->e_domain, s->cluster, ml[k].item), name); mcnt++; } } else { __pmNotifyErr(LOG_ERR, "%s: cannot grow metric list", pmProgname); + if (isDSO) + return; exit(1); } break; @@ -405,7 +361,7 @@ map_stats(void) for (k = 0; k < toc[j].count; k++) { ip = &indoms[incnt + k]; - ip->it_indom = pmInDom_build(dispatch.domain, + ip->it_indom = pmInDom_build(pmda->e_domain, (slist[i].cluster << 11) | id[k].serial); ip->it_numinst = id[k].count; ip->it_set = (pmdaInstid *) @@ -422,6 +378,8 @@ map_stats(void) __pmNotifyErr(LOG_ERR, "%s: cannot get memory for instance list", pmProgname); + if (isDSO) + return; exit(1); } } @@ -429,6 +387,8 @@ map_stats(void) } else { __pmNotifyErr(LOG_ERR, "%s: cannot grow indom list", pmProgname); + if (isDSO) + return; exit(1); } break; @@ -445,9 +405,7 @@ map_stats(void) } } - write_pmnsfile(pmns); - __pmFreePMNS(pmns); - + mmv_rebuild_hash(pmns, mcnt); /* for reverse (pmid->name) lookups */ reload = need_reload; } @@ -549,8 +507,8 @@ mmv_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom) return 0; } -static int -mmv_reload_maybe(void) +static void +mmv_reload_maybe(pmdaExt *pmda) { int i; struct stat s; @@ -572,14 +530,8 @@ mmv_reload_maybe(void) } if (need_reload) { - /* something changed - reload */ - pmdaExt * pmda = dispatch.version.two.ext; /* we know it is V.2 */ - - /* Note: this line means we cannot run as shared library */ - __pmSendError(pmda->e_outfd, PDU_BINARY, PM_ERR_PMDANOTREADY); - __pmNotifyErr(LOG_INFO, "%s: reloading", pmProgname); - map_stats(); + map_stats(pmda); pmda->e_indoms = indoms; pmda->e_nindoms = incnt; @@ -590,19 +542,14 @@ mmv_reload_maybe(void) __pmNotifyErr(LOG_INFO, "%s: %d metrics and %d indoms after reload", pmProgname, mcnt, incnt); - - reload = update_namespace(); } - - return need_reload; } /* Intercept request for descriptor and check if we'd have to reload */ static int mmv_desc(pmID pmid, pmDesc *desc, pmdaExt *ep) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); return pmdaDesc(pmid, desc, ep); } @@ -612,9 +559,8 @@ mmv_text(int ident, int type, char **buffer, pmdaExt *ep) if (type & PM_TEXT_INDOM) return PM_ERR_TEXT; - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; - else if (pmid_cluster(ident) == 0) + mmv_reload_maybe(ep); + if (pmid_cluster(ident) == 0) return pmdaText(ident, type, buffer, ep); else { mmv_disk_metric_t * m; @@ -643,16 +589,14 @@ static int mmv_instance(pmInDom indom, int inst, char *name, __pmInResult **result, pmdaExt *ep) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); return pmdaInstance(indom, inst, name, result, ep); } static int mmv_fetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(pmda); return pmdaFetch(numpmid, pmidlist, resp, pmda); } @@ -661,8 +605,7 @@ mmv_store(pmResult *result, pmdaExt *ep) { int i, m; - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); for (i = 0; i < result->numpmid; i++) { pmValueSet * vsp = result->vset[i]; @@ -692,6 +635,208 @@ mmv_store(pmResult *result, pmdaExt *ep) return 0; } +static __pmnsNode * +mmv_lookup_node(__pmnsNode *node, char *name) +{ + while (node != NULL) { + size_t length = strlen(node->name); + if (strncmp(name, node->name, length) == 0) { + if (name[length] == '\0') + return node; + if (name[length] == '.' && NONLEAF(node)) + return mmv_lookup_node(node->first, name + length + 1); + } + node = node->next; + } + return NULL; +} + +static int +mmv_pmid(char *name, pmID *pmid, pmdaExt *pmda) +{ + __pmnsNode *node; + + mmv_reload_maybe(pmda); + if ((node = mmv_lookup_node(pmns->root->first, name)) == NULL) + return PM_ERR_NAME; + if (NONLEAF(node)) + return PM_ERR_NAME; + *pmid = node->pmid; + return 0; +} + +static char * +mmv_absolute_name(__pmnsNode *node, char *buffer) +{ + if (node && node->parent) { + buffer = mmv_absolute_name(node->parent, buffer); + strcpy(buffer, node->name); + buffer += strlen(node->name); + *buffer++ = '.'; + } + return buffer; +} + +static int +mmv_name(pmID pmid, char ***nameset, pmdaExt *pmda) +{ + __pmnsNode *hashchain, *node, *parent; + int nmatch = 0, length = 0; + char *p, **list; + + mmv_reload_maybe(pmda); + if (!pmns) + return PM_ERR_PMID; + + hashchain = pmns->htab[pmid % pmns->htabsize]; + for (node = hashchain; node != NULL; node = node->hash) { + if (node->pmid == pmid) { + for (parent = node; parent->parent; parent = parent->parent) + length += strlen(parent->name) + 1; + nmatch++; + } + } + + if (nmatch == 0) + return PM_ERR_PMID; + + length += nmatch * sizeof(char *); /* pointers to names */ + + if ((list = (char **)malloc(length)) == NULL) + return -errno; + + p = (char *)&list[nmatch]; + nmatch = 0; + for (node = hashchain; node != NULL; node = node->hash) { + if (node->pmid == pmid) { + list[nmatch++] = p; + p = mmv_absolute_name(node, p); + *(p-1) = '\0'; /* overwrite final '.' */ + } + } + + *nameset = list; + return nmatch; +} + +static int +mmv_children_relative(__pmnsNode *base, char ***offspring, int **status) +{ + __pmnsNode *node; + char **list, *p; + int *leaf, length = 0, nmatch = 0; + + for (node = base; node != NULL; node = node->next, nmatch++) + length += strlen(node->name) + 1; + length += nmatch * sizeof(char *); /* pointers to names */ + if ((list = (char **)malloc(length)) == NULL) + return -errno; + if ((leaf = (int *)malloc(nmatch * sizeof(int*))) == NULL) { + free(list); + return -errno; + } + p = (char *)&list[nmatch]; + nmatch = 0; + for (node = base; node != NULL; node = node->next, nmatch++) { + leaf[nmatch] = NONLEAF(node) ? PMNS_NONLEAF_STATUS : PMNS_LEAF_STATUS; + list[nmatch] = p; + strcpy(p, node->name); + p += strlen(node->name); + *p++ = '\0'; + } + + *offspring = list; + *status = leaf; + return nmatch; +} + +static void +mmv_children_getsize(__pmnsNode *base, int *length, int *nmetrics) +{ + __pmnsNode *node, *parent; + + /* walk to every leaf & then add its (absolute name) length */ + for (node = base; node != NULL; node = node->next) { + if (NONLEAF(node)) { + mmv_children_getsize(node->first, length, nmetrics); + continue; + } + for (parent = node; parent->parent; parent = parent->parent) + *length += strlen(parent->name) + 1; + (*nmetrics)++; + } +} + +/* + * Fill the pmdaChildren buffers - names and leaf status. Called recursively + * to descend down to all leaf nodes. Final parameter is current offset into + * the name list buffer, and its also returned at the end of each call - used + * to keep track of where the next name should start in (list) output buffer. + */ +static char * +mmv_children_getlist(__pmnsNode *base, int *nmetrics, char *p, char **list, int *leaf) +{ + __pmnsNode *node; + int count = *nmetrics; + char *start = p; + + for (node = base; node != NULL; node = node->next) { + if (NONLEAF(node)) { + p = mmv_children_getlist(node->first, &count, p, list, leaf); + continue; + } + leaf[count] = PMNS_LEAF_STATUS; + list[count] = start; + p = mmv_absolute_name(node, p); + *(p-1) = '\0'; /* overwrite final '.' */ + start = p; + count++; + } + *nmetrics = count; + return p; +} + +static int +mmv_children_absolute(__pmnsNode *node, char ***offspring, int **status) +{ + char *p, **list; + int *leaf, length = 0, nmetrics = 0; + + mmv_children_getsize(node, &length, &nmetrics); + + length += nmetrics * sizeof(char *); /* pointers to names */ + if ((list = (char **)malloc(length)) == NULL) + return -errno; + if ((leaf = (int *)malloc(nmetrics * sizeof(int*))) == NULL) { + free(list); + return -errno; + } + + p = (char *)&list[nmetrics]; + nmetrics = 0; /* start at the start */ + mmv_children_getlist(node, &nmetrics, p, list, leaf); + + *offspring = list; + *status = leaf; + return nmetrics; +} + +static int +mmv_children(char *name, int traverse, char ***offspring, int **status, pmdaExt *pmda) +{ + __pmnsNode *node; + + mmv_reload_maybe(pmda); + if (!pmns) + return PM_ERR_NAME; + + if ((node = mmv_lookup_node(pmns->root->first, name)) == NULL) + return PM_ERR_NAME; + + if (traverse == 0) + return mmv_children_relative(node, offspring, status); + return mmv_children_absolute(node, offspring, status); +} void mmv_init(pmdaInterface *dp) @@ -701,7 +846,7 @@ mmv_init(pmdaInterface *dp) if (isDSO) { snprintf(mypath, sizeof(mypath), "%s%c" "mmv" "%c" "help", pmGetConfig("PCP_PMDAS_DIR"), sep, sep); - pmdaDSO(dp, PMDA_INTERFACE_3, "MMV DSO", mypath); + pmdaDSO(dp, PMDA_INTERFACE_4, "MMV DSO", mypath); } pcptmpdir = pmGetConfig("PCP_TMP_DIR"); @@ -729,11 +874,14 @@ mmv_init(pmdaInterface *dp) exit(0); } - dp->version.two.fetch = mmv_fetch; - dp->version.two.store = mmv_store; - dp->version.two.desc = mmv_desc; - dp->version.two.text = mmv_text; - dp->version.two.instance = mmv_instance; + dp->version.four.fetch = mmv_fetch; + dp->version.four.store = mmv_store; + dp->version.four.desc = mmv_desc; + dp->version.four.text = mmv_text; + dp->version.four.instance = mmv_instance; + dp->version.four.pmid = mmv_pmid; + dp->version.four.name = mmv_name; + dp->version.four.children = mmv_children; pmdaSetFetchCallBack(dp, mmv_fetchCallBack); @@ -762,6 +910,7 @@ main(int argc, char **argv) int err = 0; int sep = __pmPathSeparator(); char logfile[32]; + pmdaInterface dispatch = { 0 }; isDSO = 0; __pmSetProgname(argv[0]); @@ -770,7 +919,7 @@ main(int argc, char **argv) snprintf(mypath, sizeof(mypath), "%s%c" "%s%c" "help", pmGetConfig("PCP_PMDAS_DIR"), sep, prefix, sep); snprintf(logfile, sizeof(logfile), "%s.log", prefix); - pmdaDaemon(&dispatch, PMDA_INTERFACE_3, pmProgname, MMV, logfile, mypath); + pmdaDaemon(&dispatch, PMDA_INTERFACE_4, pmProgname, MMV, logfile, mypath); if ((pmdaGetOpt(argc, argv, "D:d:l:?", &dispatch, &err) != EOF) || err || argc != optind) diff --git a/src/pmdas/mmv/pmns b/src/pmdas/mmv/pmns index 38241f0..4c3cb82 100644 --- a/src/pmdas/mmv/pmns +++ b/src/pmdas/mmv/pmns @@ -1,8 +1,5 @@ /* - * Metrics for MMV PMDA - * + * Metrics for MMV PMDA - all dynamic! */ -mmv { - reload MMV:0:0 -} +mmv MMV:*:* ------=_Part_26400_1511119290.1258077641851-- From mort@sgi.com Fri Nov 13 07:50:15 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nADDoEsl109277 for ; Fri, 13 Nov 2009 07:50:14 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id BBF753040F3; Fri, 13 Nov 2009 05:50:30 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 4610AA243300; Fri, 13 Nov 2009 07:50:30 -0600 (CST) Date: Fri, 13 Nov 2009 07:50:30 -0600 From: Martin Hicks To: Nathan Scott Cc: kenj , pcp@oss.sgi.com Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Message-ID: <20091113135030.GJ5895@alcatraz.americas.sgi.com> References: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Nov 13, 2009 at 01:00:41PM +1100, Nathan Scott wrote: > This version fixes a problem in mmv_children(), adds the > new syntax into the pmns file, and in general works Much > More Betterer when run from pmcd (rather than dbpmda). > I didn't have any time to debug, but running my same simple test of `./gendata.sh | perl ./mmvtest.pl` 0x00007f7d01541210 in strlen () from /lib/libc.so.6 (gdb) bt #0 0x00007f7d01541210 in strlen () from /lib/libc.so.6 #1 0x00007f7d01828ffe in __pmSendNameList () from /usr/lib/libpcp.so.3 #2 0x00007f7d01a5904e in __pmdaMainPDU () from /usr/lib/libpcp_pmda.so.3 #3 0x00007f7d01a590d0 in pmdaMain () from /usr/lib/libpcp_pmda.so.3 #4 0x0000000000404057 in main (argc=3, argv=0x7fff3971a178) at mmv.c:898 (gdb) > cheers. > > ----- "Nathan Scott" wrote: > > ----- "Nathan Scott" wrote: > > > ----- "Martin Hicks" wrote: > > > > ... > > > > Is this something wrong with the way I wrote the PMDA, or is this > > a > > > > bug in the MMV PMDA? > > > > > > Its a bug that cannot really be fixed in the current MMV design > > > (to do with the way pmdammv and pmcd communicate the namespace > > > changes) ... MMV needs to switch to using the all-singing, all- > > > dancing dynamic namespace infrastructure to really fix this. > > > > Here's a patch which converts pmdammv to use Kens new "dynamic > > namespace" interfaces. It seems to function correctly when I > > throw queries at it via dbpmda ... but no testing beyond that > > has been done just yet. > > > > Lemme know how it goes. It also implements a DSO MMV, which I > > also ran under dbpmda successfully. Both these binaries should > > no longer have the annoying-spurious-errors feature. > > -- > Nathan > _______________________________________________ > pcp mailing list > pcp@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/pcp From mort@sgi.com Fri Nov 13 07:57:28 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nADDvS6b109937 for ; Fri, 13 Nov 2009 07:57:28 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay2.corp.sgi.com (Postfix) with ESMTP id A5771304059 for ; Fri, 13 Nov 2009 05:57:48 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 8485CA243300; Fri, 13 Nov 2009 07:57:48 -0600 (CST) Date: Fri, 13 Nov 2009 07:57:48 -0600 From: Martin Hicks To: pcp@oss.sgi.com Subject: perl PMDA RPM is gone? Message-ID: <20091113135748.GL5895@alcatraz.americas.sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean What happened to perl-PCP-PMDA for pcp-3.0? mh From kenj@internode.on.net Fri Nov 13 18:35:26 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAE0ZQKM146859 for ; Fri, 13 Nov 2009 18:35:26 -0600 X-ASG-Debug-ID: 1258158943-255403df0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 052661878DB1 for ; Fri, 13 Nov 2009 16:35:43 -0800 (PST) Received: from mail.internode.on.net (bld-mail19.adl2.internode.on.net [150.101.137.104]) by cuda.sgi.com with ESMTP id VYd7wbu9TyTZdtoD for ; Fri, 13 Nov 2009 16:35:43 -0800 (PST) Received: from [124.187.75.41] (unverified [124.187.75.41]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8266430-1927428 for multiple; Sat, 14 Nov 2009 11:05:41 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) From: Ken McDonell Reply-To: kenj@internode.on.net To: Martin Hicks Cc: Nathan Scott , pcp@oss.sgi.com In-Reply-To: <20091113135030.GJ5895@alcatraz.americas.sgi.com> References: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> <20091113135030.GJ5895@alcatraz.americas.sgi.com> Content-Type: text/plain; charset="UTF-8" Date: Sat, 14 Nov 2009 11:35:43 +1100 Message-ID: <1258158943.3041.8.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail19.adl2.internode.on.net[150.101.137.104] X-Barracuda-Start-Time: 1258158945 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14560 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Martin, Any chance you can post gendata.sh and mmvtest.pl (or send me copies) so I can investigate? ... this is certainly in a place where the dynamic pmns changes were made. On Fri, 2009-11-13 at 07:50 -0600, Martin Hicks wrote: > On Fri, Nov 13, 2009 at 01:00:41PM +1100, Nathan Scott wrote: > > This version fixes a problem in mmv_children(), adds the > > new syntax into the pmns file, and in general works Much > > More Betterer when run from pmcd (rather than dbpmda). > > > > I didn't have any time to debug, but running my same simple test of > `./gendata.sh | perl ./mmvtest.pl` > > 0x00007f7d01541210 in strlen () from /lib/libc.so.6 > (gdb) bt > #0 0x00007f7d01541210 in strlen () from /lib/libc.so.6 > #1 0x00007f7d01828ffe in __pmSendNameList () from /usr/lib/libpcp.so.3 > #2 0x00007f7d01a5904e in __pmdaMainPDU () from /usr/lib/libpcp_pmda.so.3 > #3 0x00007f7d01a590d0 in pmdaMain () from /usr/lib/libpcp_pmda.so.3 > #4 0x0000000000404057 in main (argc=3, argv=0x7fff3971a178) at mmv.c:898 > (gdb) > > > cheers. > > > > ----- "Nathan Scott" wrote: > > > ----- "Nathan Scott" wrote: > > > > ----- "Martin Hicks" wrote: > > > > > ... > > > > > Is this something wrong with the way I wrote the PMDA, or is this > > > a > > > > > bug in the MMV PMDA? > > > > > > > > Its a bug that cannot really be fixed in the current MMV design > > > > (to do with the way pmdammv and pmcd communicate the namespace > > > > changes) ... MMV needs to switch to using the all-singing, all- > > > > dancing dynamic namespace infrastructure to really fix this. > > > > > > Here's a patch which converts pmdammv to use Kens new "dynamic > > > namespace" interfaces. It seems to function correctly when I > > > throw queries at it via dbpmda ... but no testing beyond that > > > has been done just yet. > > > > > > Lemme know how it goes. It also implements a DSO MMV, which I > > > also ran under dbpmda successfully. Both these binaries should > > > no longer have the annoying-spurious-errors feature. > > > > -- > > Nathan > > > > _______________________________________________ > > pcp mailing list > > pcp@oss.sgi.com > > http://oss.sgi.com/mailman/listinfo/pcp > From kenj@internode.on.net Sat Nov 14 13:46:03 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_36 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAEJk2I3208137 for ; Sat, 14 Nov 2009 13:46:02 -0600 X-ASG-Debug-ID: 1258227980-4ff4039e0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BF0D38A1F2 for ; Sat, 14 Nov 2009 11:46:20 -0800 (PST) Received: from mail.internode.on.net (bld-mail16.adl2.internode.on.net [150.101.137.101]) by cuda.sgi.com with ESMTP id xTXUud1QVuMSSlbY for ; Sat, 14 Nov 2009 11:46:20 -0800 (PST) Received: from [124.177.136.14] (unverified [124.177.136.14]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8358181-1927428 for multiple; Sun, 15 Nov 2009 06:16:18 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) From: Ken McDonell Reply-To: kenj@internode.on.net To: Martin Hicks Cc: pcp@oss.sgi.com In-Reply-To: <20091114162915.GI5035@laplace.bork.org> References: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> <20091113135030.GJ5895@alcatraz.americas.sgi.com> <1258158943.3041.8.camel@localhost> <20091114162915.GI5035@laplace.bork.org> Content-Type: text/plain; charset="UTF-8" Date: Sun, 15 Nov 2009 06:46:25 +1100 Message-ID: <1258227985.3041.19.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail16.adl2.internode.on.net[150.101.137.101] X-Barracuda-Start-Time: 1258227981 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14627 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Thanks Martin. May I have your permission to include these code snippets into the QA suite ... clearly there is a coverage issue here. For me, the code works and the metric appears and is being updated as expected. But there is a problem ... mmv.mymmv.export_value appears TWICE in the PMNS with the same PMID ... $ pminfo -m mmv mmv.mymmv.export_value PMID: 70.1.1 mmv.mymmv.export_value PMID: 70.1.1 (and note mmv.reload is no longer in the PMNS) I've seen the same thing with QA 647 where after the mmv PMDA is installed, this metric appears (I have an independent pmdaproc.sh fix to make the warning message go away from the Check line) ... Check mmv metrics have appeared ... 1 metrics and 1 values $ pminfo -fm mmv mmv.reload PMID: 70.0.0 value 0 but then as soon as the mmv PMDA has a data file to work on, mmv.reload vanishes from the PMNS and mmv.test.strings appears twice in the PMNS. $ pminfo -fm mmv mmv.test.strings PMID: 70.1.6 inst [0 or "zero"] value "00oo00" inst [1 or "hero"] value "" mmv.test.string PMID: 70.1.5 value "g'day world" mmv.test.interval PMID: 70.1.4 inst [0 or "bird"] value 0 inst [1 or "tree"] value 0 inst [2 or "eggs"] value 28 mmv.test.indom PMID: 70.1.3 inst [0 or "zero"] value 43 inst [1 or "hero"] value 0 mmv.test.discrete PMID: 70.1.2 value 42 mmv.test.counter PMID: 70.1.1 value 41 mmv.test.strings PMID: 70.1.6 inst [0 or "zero"] value "00oo00" inst [1 or "hero"] value "" Hopefully this is enough evidence for Nathan to go ... ahhhaa ... but if that does not happen over the next day or so, I'll go digging some more. On Sat, 2009-11-14 at 11:29 -0500, Martin Hicks wrote: > On Sat, Nov 14, 2009 at 11:35:43AM +1100, Ken McDonell wrote: > > Martin, > > > > Any chance you can post gendata.sh and mmvtest.pl (or send me copies) so > > I can investigate? ... this is certainly in a place where the dynamic > > pmns changes were made. > > Attached. It was my first, quick attempt to get something working, and > an example for someone who wanted to extract a metric out of another > commands stdout to export to PCP. > > mh > From kenj@internode.on.net Sun Nov 15 14:20:28 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_36 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAFKKR6w045084 for ; Sun, 15 Nov 2009 14:20:28 -0600 X-ASG-Debug-ID: 1258316445-4f4500440000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DA77DD3BF9C for ; Sun, 15 Nov 2009 12:20:45 -0800 (PST) Received: from mail.internode.on.net (bld-mail14.adl6.internode.on.net [150.101.137.99]) by cuda.sgi.com with ESMTP id SxKO86CwWuB1tNui for ; Sun, 15 Nov 2009 12:20:45 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.162.158]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8517963-1927428 for multiple; Mon, 16 Nov 2009 06:50:43 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) From: Ken McDonell Reply-To: kenj@internode.on.net To: Martin Hicks Cc: pcp@oss.sgi.com In-Reply-To: <1258227985.3041.19.camel@localhost> References: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> <20091113135030.GJ5895@alcatraz.americas.sgi.com> <1258158943.3041.8.camel@localhost> <20091114162915.GI5035@laplace.bork.org> <1258227985.3041.19.camel@localhost> Content-Type: text/plain; charset="UTF-8" Date: Mon, 16 Nov 2009 07:20:29 +1100 Message-ID: <1258316429.21790.5.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail14.adl6.internode.on.net[150.101.137.99] X-Barracuda-Start-Time: 1258316447 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14715 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Found this one. In mmv.c ... mmv_children_getlist(__pmnsNode *base, int *nmetrics, char *p, char **list, int *leaf) { __pmnsNode *node; int count = *nmetrics; char *start = p; for (node = base; node != NULL; node = node->next) { if (NONLEAF(node)) { p = mmv_children_getlist(node->first, &count, p, list, leaf); start = p; <------ THIS WAS MISSING --------- continue; } leaf[count] = PMNS_LEAF_STATUS; list[count] = start; p = mmv_absolute_name(node, p); *(p-1) = '\0'; /* overwrite final '.' */ start = p; count++; } *nmetrics = count; return p; } This makes Martin's test work, and QA 647 works (modulo the order of the metrics is no longer the same and my pmdaproc.sh fix changes the (previously wrong) Install message for the mmv PMDA. I'm not sure this will fix the failure Martin saw, as this seems to involve not consuming enough of the allocated buffer, rather than over-running the buffer. But it is a start ... 8^)> On Sun, 2009-11-15 at 06:46 +1100, Ken McDonell wrote: > Thanks Martin. > > May I have your permission to include these code snippets into the QA > suite ... clearly there is a coverage issue here. > > For me, the code works and the metric appears and is being updated as > expected. > > But there is a problem ... mmv.mymmv.export_value appears TWICE in the > PMNS with the same PMID ... > $ pminfo -m mmv > mmv.mymmv.export_value PMID: 70.1.1 > mmv.mymmv.export_value PMID: 70.1.1 > (and note mmv.reload is no longer in the PMNS) > > I've seen the same thing with QA 647 where after the mmv PMDA is > installed, this metric appears (I have an independent pmdaproc.sh fix to > make the warning message go away from the Check line) ... > > Check mmv metrics have appeared ... 1 metrics and 1 values > $ pminfo -fm mmv > > mmv.reload PMID: 70.0.0 > value 0 > > but then as soon as the mmv PMDA has a data file to work on, mmv.reload > vanishes from the PMNS and mmv.test.strings appears twice in the PMNS. > > $ pminfo -fm mmv > > mmv.test.strings PMID: 70.1.6 > inst [0 or "zero"] value "00oo00" > inst [1 or "hero"] value "" > > mmv.test.string PMID: 70.1.5 > value "g'day world" > > mmv.test.interval PMID: 70.1.4 > inst [0 or "bird"] value 0 > inst [1 or "tree"] value 0 > inst [2 or "eggs"] value 28 > > mmv.test.indom PMID: 70.1.3 > inst [0 or "zero"] value 43 > inst [1 or "hero"] value 0 > > mmv.test.discrete PMID: 70.1.2 > value 42 > > mmv.test.counter PMID: 70.1.1 > value 41 > > mmv.test.strings PMID: 70.1.6 > inst [0 or "zero"] value "00oo00" > inst [1 or "hero"] value "" > > Hopefully this is enough evidence for Nathan to go ... ahhhaa ... but if > that does not happen over the next day or so, I'll go digging some more. > > On Sat, 2009-11-14 at 11:29 -0500, Martin Hicks wrote: > > On Sat, Nov 14, 2009 at 11:35:43AM +1100, Ken McDonell wrote: > > > Martin, > > > > > > Any chance you can post gendata.sh and mmvtest.pl (or send me copies) so > > > I can investigate? ... this is certainly in a place where the dynamic > > > pmns changes were made. > > > > Attached. It was my first, quick attempt to get something working, and > > an example for someone who wanted to extract a metric out of another > > commands stdout to export to PCP. > > > > mh > > > > > _______________________________________________ > pcp mailing list > pcp@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/pcp From kenj@internode.on.net Sun Nov 15 14:45:17 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_34, J_CHICKENPOX_36 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAFKjG5w048839 for ; Sun, 15 Nov 2009 14:45:16 -0600 X-ASG-Debug-ID: 1258317935-3c1903190000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 39F7AD3BF78 for ; Sun, 15 Nov 2009 12:45:36 -0800 (PST) Received: from mail.internode.on.net (bld-mail16.adl2.internode.on.net [150.101.137.101]) by cuda.sgi.com with ESMTP id tW4dgXcyS8XLax8H for ; Sun, 15 Nov 2009 12:45:36 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.162.158]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8393928-1927428 for multiple; Mon, 16 Nov 2009 07:15:34 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) From: Ken McDonell Reply-To: kenj@internode.on.net To: Nathan Scott Cc: pcp@oss.sgi.com In-Reply-To: <1258316429.21790.5.camel@bozo.localdomain> References: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> <20091113135030.GJ5895@alcatraz.americas.sgi.com> <1258158943.3041.8.camel@localhost> <20091114162915.GI5035@laplace.bork.org> <1258227985.3041.19.camel@localhost> <1258316429.21790.5.camel@bozo.localdomain> Content-Type: text/plain; charset="UTF-8" Date: Mon, 16 Nov 2009 07:45:20 +1100 Message-ID: <1258317920.21790.10.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail16.adl2.internode.on.net[150.101.137.101] X-Barracuda-Start-Time: 1258317937 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14717 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean There is another more fundamental problem with the PMNS traversal in the proposed mmv changes. I haven't found the fix for this one yet. This is correct ... $ pminfo mmv mmv.test.strings mmv.test.string mmv.test.interval mmv.test.indom mmv.test.discrete mmv.test.counter mmv.reload $ pminfo mmv.reload mmv.reload But these are all wrong (seems like the traversal is not stopping properly and traversing all of the tree from the target node on, even when the target node is a leaf, and in the last case when the target node is not the last subtree). $ pminfo mmv.test.interval mmv.test.interval mmv.test.indom mmv.test.discrete mmv.test.counter $ pminfo mmv.test.strings mmv.test.strings mmv.test.string mmv.test.interval mmv.test.indom mmv.test.discrete mmv.test.counter $ pminfo mmv.test mmv.test.strings mmv.test.string mmv.test.interval mmv.test.indom mmv.test.discrete mmv.test.counter mmv.reload From nscott@aconex.com Sun Nov 15 17:05:29 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAFN5Tkv056984 for ; Sun, 15 Nov 2009 17:05:29 -0600 X-ASG-Debug-ID: 1258326347-303c00ba0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 672651879982 for ; Sun, 15 Nov 2009 15:05:48 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id fdiGoz9CnCPNIgt2 for ; Sun, 15 Nov 2009 15:05:48 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 41C375CB561; Mon, 16 Nov 2009 10:05:46 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id JSCCoGcDzuLU6AEB; Mon, 16 Nov 2009 10:05:46 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 8223FA5028C; Mon, 16 Nov 2009 10:04:05 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 0D3994FD85; Mon, 16 Nov 2009 10:05:46 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bXkT8R4qeoyW; Mon, 16 Nov 2009 10:05:41 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 49D1F4FD82; Mon, 16 Nov 2009 10:05:41 +1100 (EST) Date: Mon, 16 Nov 2009 10:05:41 +1100 (EST) From: Nathan Scott To: Martin Hicks Cc: pcp@oss.sgi.com Message-ID: <726479492.413231258326341013.JavaMail.root@mail-au.aconex.com> In-Reply-To: <20091113135748.GL5895@alcatraz.americas.sgi.com> X-ASG-Orig-Subj: Re: [pcp] perl PMDA RPM is gone? Subject: Re: [pcp] perl PMDA RPM is gone? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258326350 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14726 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Martin Hicks" wrote: > What happened to perl-PCP-PMDA for pcp-3.0? It is there (on oss, download dir, alongside the pcp rpms of same binary flavour). cheers. -- Nathan From nscott@aconex.com Sun Nov 15 17:12:10 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAFNCAZa057288 for ; Sun, 15 Nov 2009 17:12:10 -0600 X-ASG-Debug-ID: 1258326749-025a02fa0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E321618799FF for ; Sun, 15 Nov 2009 15:12:30 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id A1yZvTIbHWXNmHBL for ; Sun, 15 Nov 2009 15:12:30 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 8251077C4E1; Mon, 16 Nov 2009 10:12:28 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id W3ercW2d4spLXaZO; Mon, 16 Nov 2009 10:12:28 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id BFCF9A5028C; Mon, 16 Nov 2009 10:10:47 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 457CE4FD86; Mon, 16 Nov 2009 10:12:28 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9PN8Cn50uOyr; Mon, 16 Nov 2009 10:12:24 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 12F9E4FD84; Mon, 16 Nov 2009 10:12:24 +1100 (EST) Date: Mon, 16 Nov 2009 10:12:23 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com, Martin Hicks Message-ID: <1524061135.413471258326743950.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1004417490.413401258326684184.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_28150_1598801916.1258326743948" X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258326750 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14726 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ------=_Part_28150_1598801916.1258326743948 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- "Ken McDonell" wrote: > ... > I've seen the same thing with QA 647 where after the mmv PMDA is > installed, this metric appears (I have an independent pmdaproc.sh fix > to make the warning message go away from the Check line) ... Ah, I was looking at a similar thing on the train this morning ... can you send yours? This patch made the problem go away (now using the PMNS PDUs with pmcd, instead of an explicit namespace file to pminfo, in the default case). The root (of ./Install giving bogus warnings) seems to be this: $ pminfo -n /var/lib/pcp/pmns/root -f mmv mmv: pmLookupDesc: Unknown or illegal metric identifier So, I used the attached patch & the correct values are reported by Install. What was your approach? cheers. -- Nathan ------=_Part_28150_1598801916.1258326743948 Content-Type: text/x-patch; name=pmdaproc.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=pmdaproc.patch diff --git a/src/pmcd/pmdaproc.sh b/src/pmcd/pmdaproc.sh index cf3856a..058c5bb 100644 --- a/src/pmcd/pmdaproc.sh +++ b/src/pmcd/pmdaproc.sh @@ -1100,12 +1100,14 @@ _install() # if $do_check then + __ns="" + [ "$NAMESPACE" != "$PMNSDIR/root" ] && __ns="-n $NAMESPACE" [ "$check_delay" -gt 5 ] && echo "Wait $check_delay seconds for the $iam agent to initialize ..." sleep $check_delay for __n in $pmns_name do $PCP_ECHO_PROG $PCP_ECHO_N "Check $__n metrics have appeared ... ""$PCP_ECHO_C" - pminfo -n $NAMESPACE -f $__n | __filter $__n + pminfo -f $__ns $__n | __filter $__n done fi else @@ -1181,10 +1183,12 @@ _remove() if $do_check then + __ns="" + [ "$NAMESPACE" != "$PMNSDIR/root" ] && __ns="-n $NAMESPACE" for __n in $pmns_name do $PCP_ECHO_PROG $PCP_ECHO_N "Check $__n metrics have gone away ... ""$PCP_ECHO_C" - if pminfo -n $NAMESPACE -f $__n >$tmp 2>&1 + if pminfo -f $__ns $__n >$tmp 2>&1 then echo "Arrgh, something has gone wrong!" cat $tmp ------=_Part_28150_1598801916.1258326743948-- From goodwinos@gmail.com Sun Nov 15 17:22:06 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAFNM532057706 for ; Sun, 15 Nov 2009 17:22:06 -0600 X-ASG-Debug-ID: 1258327345-4d8c00930000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 65382BEC1B7; Sun, 15 Nov 2009 15:22:25 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id VEbDwKWTRts1LB2r; Sun, 15 Nov 2009 15:22:25 -0800 (PST) Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nAFNMOsX024868 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 15 Nov 2009 18:22:24 -0500 Received: from [10.64.176.217] (dhcp-176-217.mel.redhat.com [10.64.176.217]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nAFNMN9F015001; Sun, 15 Nov 2009 18:22:23 -0500 Message-ID: <4B008D2E.9020208@gmail.com> Date: Mon, 16 Nov 2009 10:22:22 +1100 From: Mark Goodwin User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Martin Hicks CC: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] perl PMDA RPM is gone? Subject: Re: [pcp] perl PMDA RPM is gone? References: <20091113135748.GL5895@alcatraz.americas.sgi.com> In-Reply-To: <20091113135748.GL5895@alcatraz.americas.sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1258327346 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14727 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Martin Hicks wrote: > What happened to perl-PCP-PMDA for pcp-3.0? > Martin, not sure what you mean. The RPMs seem to be there afaict, e.g. ftp://oss.sgi.com/projects/pcp/download/rpm/Latest/fc10/x86_64/perl-PCP-PMDA-3.0.1-2.fc10.x86_64.rpm The perl RPMs are built from the common RPM build now (that's a change from pcp2.x). -- Mark From nscott@aconex.com Sun Nov 15 19:46:26 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_33, J_CHICKENPOX_43,J_CHICKENPOX_55,J_CHICKENPOX_61,J_CHICKENPOX_62, J_CHICKENPOX_64,J_CHICKENPOX_66,J_CHICKENPOX_71,J_CHICKENPOX_72, J_CHICKENPOX_74,J_CHICKENPOX_84,LOCAL_GNU_PATCH autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAG1kPUe067871 for ; Sun, 15 Nov 2009 19:46:26 -0600 X-ASG-Debug-ID: 1258336004-352701040000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 014F714DD57D for ; Sun, 15 Nov 2009 17:46:44 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id g1tD1APC9lfHbTR6 for ; Sun, 15 Nov 2009 17:46:44 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 4C83377CFF9; Mon, 16 Nov 2009 12:46:43 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id p0DsanfoZ0iqLzCr; Mon, 16 Nov 2009 12:46:43 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 85009A5028C; Mon, 16 Nov 2009 12:45:02 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 215004FD86; Mon, 16 Nov 2009 12:46:43 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qvxhrbe7P8FU; Mon, 16 Nov 2009 12:46:39 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id F03334FD85; Mon, 16 Nov 2009 12:46:38 +1100 (EST) Date: Mon, 16 Nov 2009 12:46:38 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com Message-ID: <214880482.427361258335998646.JavaMail.root@mail-au.aconex.com> In-Reply-To: <393806685.427341258335989316.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_29250_2105714606.1258335998645" X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258336006 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14737 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ------=_Part_29250_2105714606.1258335998645 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- "Ken McDonell" wrote: > There is another more fundamental problem with the PMNS traversal in > the proposed mmv changes. I haven't found the fix for this one yet. Attached version fixes it. I didn't find a way to get dbpmda to call the children() method with "traverse" set - is there a way? Also the dbpmda help for pmid, name, and children methods doesn't seem to give useful info... dbpmda> children mmv Metric: mmv non-leaf test leaf reload dbpmda> help children ^ at or near here Error: syntax error Type 'help' for a list of commands. dbpmda> help pmid ^ at or near here Error: syntax error Type 'help' for a list of commands. dbpmda> help open open dso dsoname init_routine [ domain# ] open pipe execname [ arg ... ] Open a PMDA as either a DSO or a daemon (connected with a pipe). The 'dsoname' and 'execname' fields are the path to the PMDA shared object file or executable. The arguments to this command are similar to a line in the pmcd.conf file. dbpmda> Looks like we're missing some cases in dbpmda util.c dohelp() routine? cheers. -- Nathan ------=_Part_29250_2105714606.1258335998645 Content-Type: text/x-patch; name=mmv.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=mmv.patch diff --git a/src/pmdas/mmv/GNUmakefile b/src/pmdas/mmv/GNUmakefile index 6205a8a..63f778a 100644 --- a/src/pmdas/mmv/GNUmakefile +++ b/src/pmdas/mmv/GNUmakefile @@ -21,7 +21,6 @@ DOMAIN = MMV CMDTARGET = pmda$(IAM)$(EXECSUFFIX) LIBTARGET = pmda_$(IAM).$(DSOSUFFIX) TARGETS = $(CMDTARGET) $(LIBTARGET) mmvdump$(EXECSUFFIX) -INSTTARGETS = $(CMDTARGET) mmvdump$(EXECSUFFIX) CFILES = mmv.c LSRCFILES = pmns Install Remove README help root mmvdump.c @@ -40,7 +39,7 @@ include $(BUILDRULES) install: default $(INSTALL) -m 755 -d $(PMDADIR) - $(INSTALL) -m 755 $(INSTTARGETS) Install Remove $(PMDADIR) + $(INSTALL) -m 755 $(TARGETS) Install Remove $(PMDADIR) $(INSTALL) -m 644 README root help pmns domain.h $(PMDADIR) $(CMDTARGET): $(OBJECTS) diff --git a/src/pmdas/mmv/Install b/src/pmdas/mmv/Install index d663f1b..e4cbaa8 100755 --- a/src/pmdas/mmv/Install +++ b/src/pmdas/mmv/Install @@ -18,7 +18,7 @@ . $PCP_SHARE_DIR/lib/pmdaproc.sh iam=mmv -dso_opt=false +dso_opt=true socket_opt=true socket_inet_def=2081 pmda_interface=3 diff --git a/src/pmdas/mmv/mmv.c b/src/pmdas/mmv/mmv.c index ef30ef0..d5fd366 100644 --- a/src/pmdas/mmv/mmv.c +++ b/src/pmdas/mmv/mmv.c @@ -30,7 +30,8 @@ #include "./domain.h" #include -static pmdaInterface dispatch; +#define NONLEAF(node) ((node)->pmid == PM_ID_NULL) + static char mypath[MAXPATHLEN]; static int isDSO = 1; @@ -40,6 +41,7 @@ static pmdaIndom * indoms; static int incnt; static int reload; +static __pmnsTree *pmns; static time_t statsdir_ts; /* last statsdir timestamp */ static char * prefix = "mmv"; @@ -67,100 +69,6 @@ typedef struct { static stats_t * slist; static int scnt; -static int -update_namespace(void) -{ - char script[3*MAXPATHLEN]; - int sep = __pmPathSeparator(); - - snprintf(script, sizeof(script), - "%s%c" "lib" "%c" "ReplacePmnsSubtree %s %s%c" "%s.new", - pmGetConfig("PCP_SHARE_DIR"), sep, sep, - prefix, pmnsdir, sep, prefix); - if (system(script) == -1) { - __pmNotifyErr (LOG_ERR, "%s: cannot exec %s", pmProgname, script); - return 1; - } - - return 0; -} - -static void -write_pmnspath(__pmnsNode *base, FILE *f) -{ - if (base && base->parent) { - write_pmnspath(base->parent, f); - fprintf(f, "%s.", base->name); - } -} - -static void -write_pmnsnode(__pmnsNode *base, FILE *f) -{ - __pmnsNode *np; - - /* Print out full path to this part of the tree */ - write_pmnspath(base->parent, f); - fprintf(f, "%s {\n", base->name); - - /* Print out nodes at this level of the tree */ - for (np = base->first; np != NULL; np = np->next) { - if (np->pmid == PM_ID_NULL) - fprintf(f, "\t%s\n", np->name); - else - fprintf(f, "\t%s\t\t%u:%u:%u\n", np->name, - pmid_domain(np->pmid), - pmid_cluster(np->pmid), - pmid_item(np->pmid)); - } - fprintf(f, "}\n\n"); - - /* Print out all the children of this subtree */ - for (np = base->first; np != NULL; np = np->next) - if (np->pmid == PM_ID_NULL) - write_pmnsnode(np, f); -} - -static void -write_pmnsfile(__pmnsTree *pmns) -{ - char tmppath[MAXPATHLEN]; - char path[MAXPATHLEN]; - char *fname = tmppath; - FILE *f = NULL; - - putenv("TMPDIR="); /* temp file must be in pmnsdir, for rename */ - -#if HAVE_MKSTEMP - sprintf(tmppath, "%s%c%s-XXXXXX", pmnsdir, __pmPathSeparator(), prefix); - int fd = mkstemp(tmppath); - if (fd != -1) - f = fdopen(fd, "w"); -#else - fname = tempnam(pmnsdir, prefix); - if (fname != NULL) { - strncpy(tmppath, fname, sizeof(tmppath)); - free(fname); - fname = tmppath; - f = fopen(fname, "w"); - } -#endif - - if (f == NULL) - __pmNotifyErr(LOG_ERR, "%s: failed to generate temporary file %s: %s", - pmProgname, fname, strerror(errno)); - else { - __pmnsNode *node; - for (node = pmns->root->first; node != NULL; node = node->next) - write_pmnsnode(node, f); - fclose(f); - sprintf(path, "%s%c" "%s.new", pmnsdir, __pmPathSeparator(), prefix); - if (rename2(fname, path) < 0) - __pmNotifyErr(LOG_ERR, "%s: cannot rename %s to %s - %s", - pmProgname, fname, path, strerror (errno)); - } -} - /* * Choose an unused cluster ID while honouring specific requests. * If a specific (non-zero) cluster is requested we always use it. @@ -193,24 +101,70 @@ choose_cluster(int requested, const char *path) return requested; } +/* + * Fixup the parent pointers of the tree. + * Fill in the hash table with nodes from the tree. + * Hashing is done on pmid. + */ +static void +mmv_reindex_hash(__pmnsTree *tree, __pmnsNode *root) +{ + __pmnsNode *np; + + for (np = root->first; np != NULL; np = np->next) { + np->parent = root; + if (np->pmid != PM_ID_NULL) { + int i = np->pmid % tree->htabsize; + np->hash = tree->htab[i]; + tree->htab[i] = np; + } + mmv_reindex_hash(tree, np); + } +} + +/* + * "Make the average hash list no longer than 5, and the number + * of hash table entries not a multiple of 2, 3 or 5." + * [From __pmFixPMNSHashTab; without mark_all, dinks with pmids] + */ +static void +mmv_rebuild_hash(__pmnsTree *tree, int numpmid) +{ + int htabsize = numpmid / 5; + + if (htabsize % 2 == 0) htabsize++; + if (htabsize % 3 == 0) htabsize += 2; + if (htabsize % 5 == 0) htabsize += 2; + tree->htabsize = htabsize; + tree->htab = (__pmnsNode **)calloc(htabsize, sizeof(__pmnsNode *)); + if (tree->htab == NULL) + __pmNotifyErr(LOG_ERR, "%s: out of memory in pmns rebuild - %s", + pmProgname, strerror(errno)); + else + mmv_reindex_hash(tree, tree->root); +} + static void -map_stats(void) +map_stats(pmdaExt *pmda) { - __pmnsTree *pmns; struct dirent ** files; char name_reload[64]; int need_reload = 0; int i, sts, num; + if (pmns) + __pmFreePMNS(pmns); + if ((sts = __pmNewPMNS(&pmns)) < 0) { __pmNotifyErr(LOG_ERR, "%s: failed to create new pmns: %s\n", pmProgname, pmErrStr(sts)); + pmns = NULL; return; } mcnt = 1; snprintf(name_reload, sizeof(name_reload), "%s.reload", prefix); - __pmAddPMNSNode(pmns, pmid_build(dispatch.domain, 0, 0), name_reload); + __pmAddPMNSNode(pmns, pmid_build(pmda->e_domain, 0, 0), name_reload); if (indoms != NULL) { for (i = 0; i < incnt; i++) @@ -359,7 +313,7 @@ map_stats(void) metrics[mcnt].m_user = ml + k; metrics[mcnt].m_desc.pmid = pmid_build( - dispatch.domain, s->cluster, ml[k].item); + pmda->e_domain, s->cluster, ml[k].item); if (ml[k].type == MMV_TYPE_ELAPSED) { pmUnits unit = PMDA_PMUNITS(0,1,0,0,PM_TIME_USEC,0); @@ -378,18 +332,20 @@ map_stats(void) metrics[mcnt].m_desc.indom = (!ml[k].indom || ml[k].indom == PM_INDOM_NULL) ? PM_INDOM_NULL : - pmInDom_build(dispatch.domain, + pmInDom_build(pmda->e_domain, (s->cluster << 11) | ml[k].indom); strcat(name, ml[k].name); __pmAddPMNSNode(pmns, pmid_build( - dispatch.domain, s->cluster, ml[k].item), + pmda->e_domain, s->cluster, ml[k].item), name); mcnt++; } } else { __pmNotifyErr(LOG_ERR, "%s: cannot grow metric list", pmProgname); + if (isDSO) + return; exit(1); } break; @@ -405,7 +361,7 @@ map_stats(void) for (k = 0; k < toc[j].count; k++) { ip = &indoms[incnt + k]; - ip->it_indom = pmInDom_build(dispatch.domain, + ip->it_indom = pmInDom_build(pmda->e_domain, (slist[i].cluster << 11) | id[k].serial); ip->it_numinst = id[k].count; ip->it_set = (pmdaInstid *) @@ -422,6 +378,8 @@ map_stats(void) __pmNotifyErr(LOG_ERR, "%s: cannot get memory for instance list", pmProgname); + if (isDSO) + return; exit(1); } } @@ -429,6 +387,8 @@ map_stats(void) } else { __pmNotifyErr(LOG_ERR, "%s: cannot grow indom list", pmProgname); + if (isDSO) + return; exit(1); } break; @@ -445,9 +405,7 @@ map_stats(void) } } - write_pmnsfile(pmns); - __pmFreePMNS(pmns); - + mmv_rebuild_hash(pmns, mcnt); /* for reverse (pmid->name) lookups */ reload = need_reload; } @@ -549,8 +507,8 @@ mmv_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom) return 0; } -static int -mmv_reload_maybe(void) +static void +mmv_reload_maybe(pmdaExt *pmda) { int i; struct stat s; @@ -572,14 +530,8 @@ mmv_reload_maybe(void) } if (need_reload) { - /* something changed - reload */ - pmdaExt * pmda = dispatch.version.two.ext; /* we know it is V.2 */ - - /* Note: this line means we cannot run as shared library */ - __pmSendError(pmda->e_outfd, PDU_BINARY, PM_ERR_PMDANOTREADY); - __pmNotifyErr(LOG_INFO, "%s: reloading", pmProgname); - map_stats(); + map_stats(pmda); pmda->e_indoms = indoms; pmda->e_nindoms = incnt; @@ -590,19 +542,14 @@ mmv_reload_maybe(void) __pmNotifyErr(LOG_INFO, "%s: %d metrics and %d indoms after reload", pmProgname, mcnt, incnt); - - reload = update_namespace(); } - - return need_reload; } /* Intercept request for descriptor and check if we'd have to reload */ static int mmv_desc(pmID pmid, pmDesc *desc, pmdaExt *ep) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); return pmdaDesc(pmid, desc, ep); } @@ -612,9 +559,8 @@ mmv_text(int ident, int type, char **buffer, pmdaExt *ep) if (type & PM_TEXT_INDOM) return PM_ERR_TEXT; - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; - else if (pmid_cluster(ident) == 0) + mmv_reload_maybe(ep); + if (pmid_cluster(ident) == 0) return pmdaText(ident, type, buffer, ep); else { mmv_disk_metric_t * m; @@ -643,16 +589,14 @@ static int mmv_instance(pmInDom indom, int inst, char *name, __pmInResult **result, pmdaExt *ep) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); return pmdaInstance(indom, inst, name, result, ep); } static int mmv_fetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda) { - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(pmda); return pmdaFetch(numpmid, pmidlist, resp, pmda); } @@ -661,8 +605,7 @@ mmv_store(pmResult *result, pmdaExt *ep) { int i, m; - if (mmv_reload_maybe()) - return PM_ERR_PMDAREADY; + mmv_reload_maybe(ep); for (i = 0; i < result->numpmid; i++) { pmValueSet * vsp = result->vset[i]; @@ -692,6 +635,219 @@ mmv_store(pmResult *result, pmdaExt *ep) return 0; } +static __pmnsNode * +mmv_lookup_node(__pmnsNode *node, char *name) +{ + while (node != NULL) { + size_t length = strlen(node->name); + if (strncmp(name, node->name, length) == 0) { + if (name[length] == '\0') + return node; + if (name[length] == '.' && NONLEAF(node)) + return mmv_lookup_node(node->first, name + length + 1); + } + node = node->next; + } + return NULL; +} + +static int +mmv_pmid(char *name, pmID *pmid, pmdaExt *pmda) +{ + __pmnsNode *node; + + mmv_reload_maybe(pmda); + if ((node = mmv_lookup_node(pmns->root->first, name)) == NULL) + return PM_ERR_NAME; + if (NONLEAF(node)) + return PM_ERR_NAME; + *pmid = node->pmid; + return 0; +} + +static char * +mmv_absolute_name(__pmnsNode *node, char *buffer) +{ + if (node && node->parent) { + buffer = mmv_absolute_name(node->parent, buffer); + strcpy(buffer, node->name); + buffer += strlen(node->name); + *buffer++ = '.'; + } + return buffer; +} + +static int +mmv_name(pmID pmid, char ***nameset, pmdaExt *pmda) +{ + __pmnsNode *hashchain, *node, *parent; + int nmatch = 0, length = 0; + char *p, **list; + + mmv_reload_maybe(pmda); + if (!pmns) + return PM_ERR_PMID; + + hashchain = pmns->htab[pmid % pmns->htabsize]; + for (node = hashchain; node != NULL; node = node->hash) { + if (node->pmid == pmid) { + for (parent = node; parent->parent; parent = parent->parent) + length += strlen(parent->name) + 1; + nmatch++; + } + } + + if (nmatch == 0) + return PM_ERR_PMID; + + length += nmatch * sizeof(char *); /* pointers to names */ + + if ((list = (char **)malloc(length)) == NULL) + return -errno; + + p = (char *)&list[nmatch]; + nmatch = 0; + for (node = hashchain; node != NULL; node = node->hash) { + if (node->pmid == pmid) { + list[nmatch++] = p; + p = mmv_absolute_name(node, p); + *(p-1) = '\0'; /* overwrite final '.' */ + } + } + + *nameset = list; + return nmatch; +} + +static int +mmv_children_relative(__pmnsNode *base, char ***offspring, int **status) +{ + __pmnsNode *node; + char **list, *p; + int *leaf, length = 0, nmatch = 0; + + for (node = base; node != NULL; node = node->next, nmatch++) + length += strlen(node->name) + 1; + length += nmatch * sizeof(char *); /* pointers to names */ + if ((list = (char **)malloc(length)) == NULL) + return -errno; + if ((leaf = (int *)malloc(nmatch * sizeof(int*))) == NULL) { + free(list); + return -errno; + } + p = (char *)&list[nmatch]; + nmatch = 0; + for (node = base; node != NULL; node = node->next, nmatch++) { + leaf[nmatch] = NONLEAF(node) ? PMNS_NONLEAF_STATUS : PMNS_LEAF_STATUS; + list[nmatch] = p; + strcpy(p, node->name); + p += strlen(node->name); + *p++ = '\0'; + } + + *offspring = list; + *status = leaf; + return nmatch; +} + +static void +mmv_children_getsize(__pmnsNode *base, int kids, int *length, int *nmetrics) +{ + __pmnsNode *node, *parent; + + /* walk to every leaf & then add its (absolute name) length */ + for (node = base; node != NULL; node = node->next) { + if (NONLEAF(node)) { + mmv_children_getsize(node->first, 1, length, nmetrics); + continue; + } + for (parent = node; parent->parent; parent = parent->parent) + *length += strlen(parent->name) + 1; + (*nmetrics)++; + if (!kids) + break; + } +} + +/* + * Fill the pmdaChildren buffers - names and leaf status. Called recursively + * to descend down to all leaf nodes. Offset parameter is the current offset + * into the name list buffer, and its also returned at the end of each call - + * it keeps track of where the next name is to start in (list) output buffer. + */ +static char * +mmv_children_getlist(__pmnsNode *base, int kids, int *nmetrics, char *p, char **list, int *leaf) +{ + __pmnsNode *node; + int count = *nmetrics; + char *start = p; + + for (node = base; node != NULL; node = node->next) { + if (NONLEAF(node)) { + p = mmv_children_getlist(node->first, 1, &count, p, list, leaf); + start = p; + continue; + } + leaf[count] = PMNS_LEAF_STATUS; + list[count] = start; + p = mmv_absolute_name(node, p); + *(p-1) = '\0'; /* overwrite final '.' */ + start = p; + count++; + if (!kids) + break; + } + *nmetrics = count; + return p; +} + +static int +mmv_children_absolute(__pmnsNode *node, char ***offspring, int **status) +{ + char *p, **list; + int *leaf, descend = 0, length = 0, nmetrics = 0; + + if (NONLEAF(node)) { + node = node->first; + descend = 1; + } + mmv_children_getsize(node, descend, &length, &nmetrics); + + length += nmetrics * sizeof(char *); /* pointers to names */ + if ((list = (char **)malloc(length)) == NULL) + return -errno; + if ((leaf = (int *)malloc(nmetrics * sizeof(int*))) == NULL) { + free(list); + return -errno; + } + + p = (char *)&list[nmetrics]; + nmetrics = 0; /* start at the start */ + mmv_children_getlist(node, descend, &nmetrics, p, list, leaf); + + *offspring = list; + *status = leaf; + return nmetrics; +} + +static int +mmv_children(char *name, int traverse, char ***offspring, int **status, pmdaExt *pmda) +{ + __pmnsNode *node; + + mmv_reload_maybe(pmda); + if (!pmns) + return PM_ERR_NAME; + + __pmNotifyErr(LOG_INFO, "%s: children of %s (traverse=%d)", pmProgname, name, traverse); + + if ((node = mmv_lookup_node(pmns->root->first, name)) == NULL) + return PM_ERR_NAME; + + if (traverse == 0) + return mmv_children_relative(node->first, offspring, status); + return mmv_children_absolute(node, offspring, status); +} void mmv_init(pmdaInterface *dp) @@ -701,7 +857,7 @@ mmv_init(pmdaInterface *dp) if (isDSO) { snprintf(mypath, sizeof(mypath), "%s%c" "mmv" "%c" "help", pmGetConfig("PCP_PMDAS_DIR"), sep, sep); - pmdaDSO(dp, PMDA_INTERFACE_3, "MMV DSO", mypath); + pmdaDSO(dp, PMDA_INTERFACE_4, "mmv", mypath); } pcptmpdir = pmGetConfig("PCP_TMP_DIR"); @@ -729,11 +885,14 @@ mmv_init(pmdaInterface *dp) exit(0); } - dp->version.two.fetch = mmv_fetch; - dp->version.two.store = mmv_store; - dp->version.two.desc = mmv_desc; - dp->version.two.text = mmv_text; - dp->version.two.instance = mmv_instance; + dp->version.four.fetch = mmv_fetch; + dp->version.four.store = mmv_store; + dp->version.four.desc = mmv_desc; + dp->version.four.text = mmv_text; + dp->version.four.instance = mmv_instance; + dp->version.four.pmid = mmv_pmid; + dp->version.four.name = mmv_name; + dp->version.four.children = mmv_children; pmdaSetFetchCallBack(dp, mmv_fetchCallBack); @@ -762,6 +921,8 @@ main(int argc, char **argv) int err = 0; int sep = __pmPathSeparator(); char logfile[32]; + char p[] = "mmv", *prefix = p; + pmdaInterface dispatch = { 0 }; isDSO = 0; __pmSetProgname(argv[0]); @@ -770,7 +931,7 @@ main(int argc, char **argv) snprintf(mypath, sizeof(mypath), "%s%c" "%s%c" "help", pmGetConfig("PCP_PMDAS_DIR"), sep, prefix, sep); snprintf(logfile, sizeof(logfile), "%s.log", prefix); - pmdaDaemon(&dispatch, PMDA_INTERFACE_3, pmProgname, MMV, logfile, mypath); + pmdaDaemon(&dispatch, PMDA_INTERFACE_4, pmProgname, MMV, logfile, mypath); if ((pmdaGetOpt(argc, argv, "D:d:l:?", &dispatch, &err) != EOF) || err || argc != optind) diff --git a/src/pmdas/mmv/pmns b/src/pmdas/mmv/pmns index 38241f0..4c3cb82 100644 --- a/src/pmdas/mmv/pmns +++ b/src/pmdas/mmv/pmns @@ -1,8 +1,5 @@ /* - * Metrics for MMV PMDA - * + * Metrics for MMV PMDA - all dynamic! */ -mmv { - reload MMV:0:0 -} +mmv MMV:*:* ------=_Part_29250_2105714606.1258335998645-- From nscott@aconex.com Sun Nov 15 20:53:40 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAG2rdAC072855 for ; Sun, 15 Nov 2009 20:53:40 -0600 X-ASG-Debug-ID: 1258340039-566d020a0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5E849D3C0B2 for ; Sun, 15 Nov 2009 18:53:59 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id dSwuDj1cDR8t8XW4 for ; Sun, 15 Nov 2009 18:53:59 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 799595CC1F9; Mon, 16 Nov 2009 13:53:58 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id gOUCBWkjISDp5OGZ; Mon, 16 Nov 2009 13:53:58 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id B9598A5028C; Mon, 16 Nov 2009 13:52:17 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 5B2774FD84; Mon, 16 Nov 2009 13:53:58 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id czU9AR1ump9s; Mon, 16 Nov 2009 13:53:54 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 3A3664FD82; Mon, 16 Nov 2009 13:53:54 +1100 (EST) Date: Mon, 16 Nov 2009 13:53:54 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com Message-ID: <490350578.430211258340033987.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1524061135.413471258326743950.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258340040 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14741 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Nathan Scott" wrote: > ----- "Ken McDonell" wrote: > > ... > > I've seen the same thing with QA 647 where after the mmv PMDA is > > installed, this metric appears (I have an independent pmdaproc.sh > fix > > to make the warning message go away from the Check line) ... > > Ah, I was looking at a similar thing on the train this morning ... > can you send yours? This patch made the problem go away > ... I've pushed my current set of changes into my development tree on oss, want to resolve this one before I tackle QA ... lemme know once you've decided on an approach here (think this is a dynamic-pmns-wide kind of issue, isn't it? i.e. not just affecting MMV?). cheers. -- Nathan From nscott@aconex.com Sun Nov 15 20:54:25 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAG2sPY4072911 for ; Sun, 15 Nov 2009 20:54:25 -0600 X-ASG-Debug-ID: 1258340084-486c02c30000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 84F2218812D6 for ; Sun, 15 Nov 2009 18:54:45 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id KOJ2gHcxIMeKsKHD for ; Sun, 15 Nov 2009 18:54:45 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id ED0E15CC475 for ; Mon, 16 Nov 2009 13:54:43 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id rUAQ7biGzWygraQw for ; Mon, 16 Nov 2009 13:54:43 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 3956BA5028C for ; Mon, 16 Nov 2009 13:53:03 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id D1B0F4FD84 for ; Mon, 16 Nov 2009 13:54:43 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3-rT+c2ndehA for ; Mon, 16 Nov 2009 13:54:39 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id C5CA14FD82 for ; Mon, 16 Nov 2009 13:54:39 +1100 (EST) Date: Mon, 16 Nov 2009 13:54:39 +1100 (EST) From: Nathan Scott To: pcp@oss.sgi.com Message-ID: <1959840420.430251258340079623.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: pcp updates Subject: pcp updates MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258340086 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14742 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean Changes committed to git://oss.sgi.com/nathans/pcp.git build/slack/.gitignore | 2 debian/libpcp-pmda3-dev.install | 3 man/man3/GNUmakefile | 1 man/man3/pmdatrace.3 | 14 - src/include/impl.h | 4 src/pmdas/mmv/GNUmakefile | 3 src/pmdas/mmv/Install | 2 src/pmdas/mmv/mmv.c | 430 +++++++++++++++++++++++++++------------- src/pmdas/mmv/pmns | 7 9 files changed, 313 insertions(+), 153 deletions(-) commit 5f8018e729e73056de784bd3d8c8568a8a8cf371 Author: Nathan Scott Date: Mon Nov 16 13:44:04 2009 +1100 Mark some other generated files for ignoring by git commit 9569a358e6025d4588676ca70596551c2f2e95de Author: Nathan Scott Date: Mon Nov 16 13:43:31 2009 +1100 Convert pmdammv to using the new dynamic namespace interfaces. This change convert pmdammv to use the libpcp_pmda dynamic namespace interfaces instead of the traditional pmcd namespace file updating mechanism. For a PMDA like MMV, the old way results in spurious errors (EAGAIN) whenever a metadata change occurs, due to the nature of the exchange with pmcd resulting in the namespace being updated. This is inappropriate for a production PMDA, and is now resolved. As another benefit of these changes, we are now able to provide a shared library variant of pmdammv for all platforms - this patch also includes that change. commit bc6b4c52222a1fcad1fe52acf7c6f64fa9ff81f6 Author: Nathan Scott Date: Mon Nov 16 13:34:59 2009 +1100 Add dynamic pmns man pages into the generated packages (binary and source). commit d3668810fbd149511b689d030c2e89370c38ad79 Author: Nathan Scott Date: Mon Nov 16 13:34:13 2009 +1100 Fix spelling mistake in impl.h commit 6c86132244d4db76c04d777e1e95242c628b4d88 Author: Nathan Scott Date: Mon Nov 16 13:33:53 2009 +1100 Fix a table-width error being reported by man with --warnings and 80 column width (pmdatrace.3). From mort@bork.org Mon Nov 16 07:49:29 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAGDnTfq153978 for ; Mon, 16 Nov 2009 07:49:29 -0600 X-ASG-Debug-ID: 1258379389-2e56030f0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from darwin.bork.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 77C3414DDFF7 for ; Mon, 16 Nov 2009 05:49:49 -0800 (PST) Received: from darwin.bork.org (darwin.bork.org [65.49.60.145]) by cuda.sgi.com with ESMTP id mp2f5ucMXD5xgOHI for ; Mon, 16 Nov 2009 05:49:49 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by darwin.bork.org (Postfix) with ESMTP id 243BA19349; Mon, 16 Nov 2009 08:49:34 -0500 (EST) Received: from darwin.bork.org ([127.0.0.1]) by localhost (darwin.bork.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id n2Gu7i0RTlBQ; Mon, 16 Nov 2009 08:49:34 -0500 (EST) Received: from laplace.bork.org (laplace.bork.org [IPv6:2001:470:1f11:499:21a:a0ff:feec:57ab]) by darwin.bork.org (Postfix) with ESMTP id ECEF81923A; Mon, 16 Nov 2009 08:49:33 -0500 (EST) Received: by laplace.bork.org (Postfix, from userid 1000) id 1F54953623; Mon, 16 Nov 2009 08:49:48 -0500 (EST) Date: Mon, 16 Nov 2009 08:49:48 -0500 From: Martin Hicks To: Ken McDonell Cc: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Message-ID: <20091116134948.GM5035@laplace.bork.org> References: <1370327498.387811258077480321.JavaMail.root@mail-au.aconex.com> <58136227.387901258077641856.JavaMail.root@mail-au.aconex.com> <20091113135030.GJ5895@alcatraz.americas.sgi.com> <1258158943.3041.8.camel@localhost> <20091114162915.GI5035@laplace.bork.org> <1258227985.3041.19.camel@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1258227985.3041.19.camel@localhost> User-Agent: Mutt/1.5.20 (2009-06-14) X-Barracuda-Connect: darwin.bork.org[65.49.60.145] X-Barracuda-Start-Time: 1258379390 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14773 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sun, Nov 15, 2009 at 06:46:25AM +1100, Ken McDonell wrote: > Thanks Martin. > > May I have your permission to include these code snippets into the QA > suite ... clearly there is a coverage issue here. Yes, please use it if it helps. mh -- Martin Hicks || mort@bork.org || PGP/GnuPG: 0x4C7F2BEE From mort@sgi.com Tue Nov 17 09:04:12 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAHF4CES041016 for ; Tue, 17 Nov 2009 09:04:12 -0600 Received: from attica.americas.sgi.com (attica.americas.sgi.com [128.162.236.44]) by relay3.corp.sgi.com (Postfix) with ESMTP id 78945AC011; Tue, 17 Nov 2009 07:04:31 -0800 (PST) Received: by attica.americas.sgi.com (Postfix, from userid 44682) id 01B4DA243300; Tue, 17 Nov 2009 09:04:30 -0600 (CST) Date: Tue, 17 Nov 2009 09:04:30 -0600 From: Martin Hicks To: Mark Goodwin Cc: pcp@oss.sgi.com Subject: Re: [pcp] perl PMDA RPM is gone? Message-ID: <20091117150430.GO5895@alcatraz.americas.sgi.com> References: <20091113135748.GL5895@alcatraz.americas.sgi.com> <4B008D2E.9020208@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B008D2E.9020208@gmail.com> User-Agent: Mutt/1.5.17 (2007-11-01) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 16, 2009 at 10:22:22AM +1100, Mark Goodwin wrote: > Martin Hicks wrote: >> What happened to perl-PCP-PMDA for pcp-3.0? >> > > Martin, not sure what you mean. The RPMs seem to be there afaict, e.g. > ftp://oss.sgi.com/projects/pcp/download/rpm/Latest/fc10/x86_64/perl-PCP-PMDA-3.0.1-2.fc10.x86_64.rpm > > The perl RPMs are built from the common RPM build now > (that's a change from pcp2.x). Oops. It was the LogSummary RPM that I don't see anymore. mh From nscott@aconex.com Tue Nov 17 16:29:56 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_47 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAHMTu1Y081086 for ; Tue, 17 Nov 2009 16:29:56 -0600 X-ASG-Debug-ID: 1258497016-7eff03e10000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 28ACC1892567 for ; Tue, 17 Nov 2009 14:30:16 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id 1IpKlVHb2SDwCLZ4 for ; Tue, 17 Nov 2009 14:30:16 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 1455887408B; Wed, 18 Nov 2009 09:30:14 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id 3sAFA4KHBhOBhGwQ; Wed, 18 Nov 2009 09:30:14 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 5C417A502B2; Wed, 18 Nov 2009 09:28:32 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id E672B4FD82; Wed, 18 Nov 2009 09:30:13 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id np1xl-fh+JBl; Wed, 18 Nov 2009 09:30:09 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 6190D4FD84; Wed, 18 Nov 2009 09:30:09 +1100 (EST) Date: Wed, 18 Nov 2009 09:30:09 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com Message-ID: <71314918.481951258497009329.JavaMail.root@mail-au.aconex.com> In-Reply-To: <2023990958.481921258496982041.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_33053_1171300881.1258497009328" X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258497018 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14889 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ------=_Part_33053_1171300881.1258497009328 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- "Nathan Scott" wrote: > ----- "Nathan Scott" wrote: > > ----- "Ken McDonell" wrote: > > > ... > > > I've seen the same thing with QA 647 where after the mmv PMDA is > > > installed, this metric appears (I have an independent pmdaproc.sh > > fix > > > to make the warning message go away from the Check line) ... > > > > Ah, I was looking at a similar thing on the train this morning ... > > can you send yours? This patch made the problem go away > > ... > > I've pushed my current set of changes into my development tree on oss, > want to resolve this one before I tackle QA ... lemme know once you've > decided on an approach here (think this is a dynamic-pmns-wide kind > of issue, isn't it? i.e. not just affecting MMV?). I've got to set this stuff aside for a little while - attached patch is where I've got up to on this side of things. Haven't attempted to completely remove -N from Install/Remove at this stage, as per our IRC discussion. cheers. -- Nathan ------=_Part_33053_1171300881.1258497009328 Content-Type: text/x-patch; name=interim.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=interim.patch I think the right thing is ... 1. remove -n namespace .... it is broken for a pmcd install and makes no sense for a client install 2. where pminfo is called, never use -h localhost 3. where pminfo is called, never use -n namespace I also added a -V option so you can see what comes back from the pminfo call to validate a PMDA install How does that seem? For the dbpmda issues ... that sounds good to me - more invasive than I'd wanted to attempt myself, but correct a. traverse is the command to do the recursive descent, not children wasnt there a -N option to the Install scripts ... for the old, pre-pcp-2.0 days ... guess that could all be removed too (might break here) b. I have some work to do to fix up the help text ... 8^)> ah, ok ... traverse, missed it - thanks i'll leave b in your capable hands :) Yep -N can go to for Install/Remove. kenj, theres several pmcd_wait -h localhost too ... thoughts? hmm, not to mention setup_localhost() ... you sure you want -h localhost gone? i assume there was some reason for adding it originally...? The only rationale I can think of for -h localhost in anything to do with pmcd setup and control would be a hangover from the days of pcp 1.x when this was _sometimes_ needed to force a pmcd context to be establised. I think those days are long gone. setup_localhost() is cool ... these are args to ping(1). pmdaproc.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/pmcd/pmdaproc.sh b/src/pmcd/pmdaproc.sh index cf3856a..d131493 100644 --- a/src/pmcd/pmdaproc.sh +++ b/src/pmcd/pmdaproc.sh @@ -324,7 +324,7 @@ END { exit status }' # signal pmcd if it is running # - if pminfo -h localhost -v pmcd.version >/dev/null 2>&1 + if pminfo -v pmcd.version >/dev/null 2>&1 then pmsignal -a -s HUP pmcd >/dev/null 2>&1 # allow signal processing to be done before checking status @@ -410,7 +410,7 @@ $1=="'$myname'" && $2=="'$mydomain'" { next } # signal pmcd if it is running, else start it # - if pminfo -h localhost -v pmcd.version >/dev/null 2>&1 + if pminfo -v pmcd.version >/dev/null 2>&1 then pmsignal -a -s HUP pmcd >/dev/null 2>&1 # allow signal processing to be done before checking status @@ -909,7 +909,7 @@ _install() for __n in $pmns_name do - if pminfo -h localhost -n $NAMESPACE $__n >/dev/null 2>&1 + if pminfo $__n >/dev/null 2>&1 then cd $PMNSDIR if pmnsdel -n $PMNSROOT $__n >$tmp 2>&1 @@ -1105,7 +1105,7 @@ _install() for __n in $pmns_name do $PCP_ECHO_PROG $PCP_ECHO_N "Check $__n metrics have appeared ... ""$PCP_ECHO_C" - pminfo -n $NAMESPACE -f $__n | __filter $__n + pminfo -f $__n | __filter $__n done fi else @@ -1184,7 +1184,7 @@ _remove() for __n in $pmns_name do $PCP_ECHO_PROG $PCP_ECHO_N "Check $__n metrics have gone away ... ""$PCP_ECHO_C" - if pminfo -n $NAMESPACE -f $__n >$tmp 2>&1 + if pminfo -f $__n >$tmp 2>&1 then echo "Arrgh, something has gone wrong!" cat $tmp ------=_Part_33053_1171300881.1258497009328-- From kenj@internode.on.net Tue Nov 17 17:23:34 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAHNNYKF087591 for ; Tue, 17 Nov 2009 17:23:34 -0600 X-ASG-Debug-ID: 1258500233-7ede022b0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9D27A18928A3 for ; Tue, 17 Nov 2009 15:23:54 -0800 (PST) Received: from mail.internode.on.net (bld-mail14.adl6.internode.on.net [150.101.137.99]) by cuda.sgi.com with ESMTP id jzmfkOXoo9otxV3g for ; Tue, 17 Nov 2009 15:23:54 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.162.158]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8658673-1927428 for multiple; Wed, 18 Nov 2009 09:53:52 +1030 (CDT) X-ASG-Orig-Subj: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) Subject: Re: [pcp] Experimental patch (was Re: mmv pmda: stuff always fails the first time?) From: Ken McDonell Reply-To: kenj@internode.on.net To: Nathan Scott Cc: pcp@oss.sgi.com In-Reply-To: <71314918.481951258497009329.JavaMail.root@mail-au.aconex.com> References: <71314918.481951258497009329.JavaMail.root@mail-au.aconex.com> Content-Type: text/plain; charset="UTF-8" Date: Wed, 18 Nov 2009 10:23:36 +1100 Message-ID: <1258500216.10829.3.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail14.adl6.internode.on.net[150.101.137.99] X-Barracuda-Start-Time: 1258500235 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14893 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Removing -N is a bad idea ... there are all sorts of QA dependencies with the "broken" PMDA there that I don't want to try and unscramble. Similarly removing -n for Install/Remove is not that simple. So I've converged on part of my changes and part of Nathan's idea of hiding -n namespace to pminfo unless -n appears in the Install/Remove command line. I'm close to getting this all passing QA before I post my commits back to my oss tree ... but Ubuntu 9.10 still has tool chain strangeness and I've encountered a couple of day one bugs (one in libpcp and one in the txmon PMDA, if you can believe that). So I still have a bit more work needed on this front before it is clear ... however the pmdaproc.sh changes look to be OK now. On Wed, 2009-11-18 at 09:30 +1100, Nathan Scott wrote: > ----- "Nathan Scott" wrote: > > ----- "Nathan Scott" wrote: > > > ----- "Ken McDonell" wrote: > > > > ... > > > > I've seen the same thing with QA 647 where after the mmv PMDA is > > > > installed, this metric appears (I have an independent pmdaproc.sh > > > fix > > > > to make the warning message go away from the Check line) ... > > > > > > Ah, I was looking at a similar thing on the train this morning ... > > > can you send yours? This patch made the problem go away > > > ... > > > > I've pushed my current set of changes into my development tree on oss, > > want to resolve this one before I tackle QA ... lemme know once you've > > decided on an approach here (think this is a dynamic-pmns-wide kind > > of issue, isn't it? i.e. not just affecting MMV?). > > I've got to set this stuff aside for a little while - attached patch > is where I've got up to on this side of things. Haven't attempted to > completely remove -N from Install/Remove at this stage, as per our IRC > discussion. > > cheers. > From goodwinos@gmail.com Tue Nov 17 18:29:35 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAI0TZbS094757 for ; Tue, 17 Nov 2009 18:29:35 -0600 X-ASG-Debug-ID: 1258504195-4a7500e70000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2DAE8D56379; Tue, 17 Nov 2009 16:29:55 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id sjc7g7c3UjJ8dPmr; Tue, 17 Nov 2009 16:29:55 -0800 (PST) Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nAI0TsTv029771 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 17 Nov 2009 19:29:55 -0500 Received: from [10.64.176.217] (dhcp-176-217.mel.redhat.com [10.64.176.217]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nAI0Th70023552; Tue, 17 Nov 2009 19:29:53 -0500 Message-ID: <4B033FF7.6040801@gmail.com> Date: Wed, 18 Nov 2009 11:29:43 +1100 From: Mark Goodwin User-Agent: Thunderbird 2.0.0.23 (X11/20090825) MIME-Version: 1.0 To: Martin Hicks CC: pcp@oss.sgi.com X-ASG-Orig-Subj: Re: [pcp] perl PMDA RPM is gone? Subject: Re: [pcp] perl PMDA RPM is gone? References: <20091113135748.GL5895@alcatraz.americas.sgi.com> <4B008D2E.9020208@gmail.com> <20091117150430.GO5895@alcatraz.americas.sgi.com> In-Reply-To: <20091117150430.GO5895@alcatraz.americas.sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1258504197 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14896 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Martin Hicks wrote: > On Mon, Nov 16, 2009 at 10:22:22AM +1100, Mark Goodwin wrote: >> Martin Hicks wrote: >>> What happened to perl-PCP-PMDA for pcp-3.0? >>> >> Martin, not sure what you mean. The RPMs seem to be there afaict, e.g. >> ftp://oss.sgi.com/projects/pcp/download/rpm/Latest/fc10/x86_64/perl-PCP-PMDA-3.0.1-2.fc10.x86_64.rpm >> >> The perl RPMs are built from the common RPM build now >> (that's a change from pcp2.x). > > Oops. It was the LogSummary RPM that I don't see anymore. > ah yes that's not being packaged - it has no DSO component and the perl build/packaging just didn't work - ran out of time trying to workaround it. So we dropped it from the 3.0 RPM packaging. As Nathan said at the time, it's still in the src tarball if anyone wants to use it. Longer term, it woul dbe better to update the cpan build infrastructure to one of the modern day MakeMaker replacements, but haven't had time to pursue that yet. Cheers -- Mark From nscott@aconex.com Tue Nov 17 21:42:09 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAI3g87a112288 for ; Tue, 17 Nov 2009 21:42:08 -0600 X-ASG-Debug-ID: 1258515747-6c8d02be0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D59C2957DA for ; Tue, 17 Nov 2009 19:42:28 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id ugDIui2fb6Yye3eH for ; Tue, 17 Nov 2009 19:42:28 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 13C075D8834; Wed, 18 Nov 2009 14:42:26 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id XZ7s3Pra3ddWABI0; Wed, 18 Nov 2009 14:42:26 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 47C3FA50289; Wed, 18 Nov 2009 14:40:44 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 001334FD86; Wed, 18 Nov 2009 14:42:25 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bEE9jFpr55LY; Wed, 18 Nov 2009 14:42:21 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 468654FD82; Wed, 18 Nov 2009 14:42:21 +1100 (EST) Date: Wed, 18 Nov 2009 14:42:21 +1100 (EST) From: Nathan Scott To: kenj Cc: pcp Message-ID: <851688687.495311258515741111.JavaMail.root@mail-au.aconex.com> In-Reply-To: <967560114.495241258515608818.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: pmlc handshake timeout needed? Subject: pmlc handshake timeout needed? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258515749 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14908 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean Hi Ken, I have a script which checks pmloggers are logging what they are meant to be, restarting them if need be more often than the daily rotation - it seems to have tickled a bug somewhere. We found many occurences of pmlc had been kicked off and just hanging (see trace below) - they were trying to connect to a pmlogger that wasn't running (anymore? not clear what happened to that pmlogger at this stage). But, the hang ended up with many pmlc processes and the caller script was unable to exit as a result. It looks to stuck in a recv(2) call - in a __pmGetPDU with explicit timout-never. Is that something we should change to some other timeout? Would seem to have at least made this problem fail cleanly with an error message. Thoughts? thanks! (ps: XXX below was the correct local host name) $ ps -ef | grep 27258 root 468 31900 0 Oct28 ? 00:00:00 pmlc 27258 root 631 32142 0 Oct28 ? 00:00:00 pmlc 27258 root 1289 31177 0 Oct28 ? 00:00:00 pmlc 27258 root 1376 581 0 Oct28 ? 00:00:00 pmlc 27258 root 1649 31564 0 Oct28 ? 00:00:00 pmlc 27258 root 4217 1814 0 Oct28 ? 00:00:00 pmlc 27258 nathans 4541 11169 0 13:51 pts/3 00:00:00 grep 27258 root 6599 5311 0 Oct28 ? 00:00:00 pmlc 27258 root 6841 5771 0 Oct28 ? 00:00:00 pmlc 27258 root 7183 6353 0 Oct28 ? 00:00:00 pmlc 27258 root 7335 6469 0 Oct28 ? 00:00:00 pmlc 27258 root 8120 5233 0 Oct28 ? 00:00:00 pmlc 27258 root 8383 5955 0 Oct28 ? 00:00:00 pmlc 27258 root 9149 7973 0 Oct28 ? 00:00:00 pmlc 27258 root 12779 11747 0 Oct28 ? 00:00:00 pmlc 27258 root 12995 11900 0 Oct28 ? 00:00:00 pmlc 27258 root 13203 12006 0 Oct28 ? 00:00:00 pmlc 27258 root 13380 12594 0 Oct28 ? 00:00:00 pmlc 27258 root 14376 11624 0 Oct28 ? 00:00:00 pmlc 27258 root 14683 12099 0 Oct28 ? 00:00:00 pmlc 27258 root 19325 16550 0 Oct28 ? 00:00:00 pmlc 27258 root 19328 18309 0 Oct28 ? 00:00:00 pmlc 27258 root 19725 18968 0 Oct28 ? 00:00:00 pmlc 27258 root 19920 18892 0 Oct28 ? 00:00:00 pmlc 27258 root 20021 19106 0 Oct28 ? 00:00:00 pmlc 27258 root 20653 18011 0 Oct28 ? 00:00:00 pmlc 27258 root 23943 22759 0 Oct28 ? 00:00:00 pmlc 27258 root 25983 24895 0 Oct28 ? 00:00:00 pmlc 27258 root 26325 25283 0 Oct28 ? 00:00:00 pmlc 27258 root 26780 25975 0 Oct28 ? 00:00:00 pmlc 27258 root 27321 26331 0 Oct28 ? 00:00:00 pmlc 27258 root 27417 24641 0 Oct28 ? 00:00:00 pmlc 27258 root 29521 25079 0 Oct28 ? 00:00:00 pmlc 27258 root 30008 29164 0 Oct28 ? 00:00:00 pmlc 27258 root 32610 31367 0 Oct28 ? 00:00:00 pmlc 27258 $ gdb -p 32610 ... Attaching to process 32610 Reading symbols from /usr/bin/pmlc...(no debugging symbols found)...done. Using host libthread_db library "/lib64/libthread_db.so.1". Reading symbols from /usr/lib64/libpcp.so.3...done. Loaded symbols for /usr/lib64/libpcp.so.3 Reading symbols from /lib64/libc.so.6...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libdl.so.2...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /lib64/libnss_dns.so.2...done. Loaded symbols for /lib64/libnss_dns.so.2 Reading symbols from /lib64/libresolv.so.2...done. Loaded symbols for /lib64/libresolv.so.2 0x0000003dba2d25a5 in recv () from /lib64/libc.so.6 (gdb) bt #0 0x0000003dba2d25a5 in recv () from /lib64/libc.so.6 #1 0x000000307da12171 in pduread (fd=3, buf=0x14ba6000 "", len=12, mode=-1, timeout=0) at pdu.c:175 #2 0x000000307da127f9 in __pmGetPDU (fd=3, mode=0, timeout=0, result=0x7fff91e49388) at pdu.c:331 #3 0x000000307da34790 in __pmConnectLogger (hostname=0x7fff91e49480 "[XXX]", pid=0x60c500, port=0x60c504) at logconnect.c:124 #4 0x00000000004041f8 in ConnectLogger () #5 0x0000000000402559 in main () (gdb) -- Nathan From kenj@internode.on.net Wed Nov 18 14:19:59 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAIKJwxx193274 for ; Wed, 18 Nov 2009 14:19:59 -0600 X-ASG-Debug-ID: 1258575618-433101d70000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id AE9D3D59D13 for ; Wed, 18 Nov 2009 12:20:18 -0800 (PST) Received: from mail.internode.on.net (bld-mail19.adl2.internode.on.net [150.101.137.104]) by cuda.sgi.com with ESMTP id ea653rKSBjVwXd7z for ; Wed, 18 Nov 2009 12:20:18 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.162.158]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8520430-1927428 for ; Thu, 19 Nov 2009 06:50:17 +1030 (CDT) X-ASG-Orig-Subj: pcp updates Subject: pcp updates From: Ken McDonell Reply-To: kenj@internode.on.net To: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Thu, 19 Nov 2009 07:20:00 +1100 Message-ID: <1258575600.10829.20.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail19.adl2.internode.on.net[150.101.137.104] X-Barracuda-Start-Time: 1258575620 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14969 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean There are some minor fixes to some very old and some new problems here. Probably most significant is making the bogus warnings go away from the mmv PMDA's Install script. Changes committed to git://oss.sgi.com/kenj/pcp.git dev INSTALL | 2 +- src/libpcp/src/pmns.c | 7 ++++--- src/libpcp/src/units.c | 2 +- src/pmcd/pmdaproc.sh | 28 ++++++++++++++++++++-------- src/pmcd/rc_pcp | 2 +- src/pmcd/src/dopdus.c | 2 +- src/pmdas/txmon/txmon.c | 22 +++++++++++++++++++++- 7 files changed, 49 insertions(+), 16 deletions(-) commit e490d2cae97f60200ca3891a73cd7f05e7b0dc17 Author: Ken McDonell Date: Wed Nov 18 07:24:24 2009 +1100 txmon.c Bad call to pmdaInit() 'Tis unbelievable that this bug has persisted for so long ... we were trampling over random memory and only by good luck did QA 324 fail in a way that allowed this to be found and fixed. commit 9990999a7211af1274a7eedb151fa3499e543ac8 Author: Ken McDonell Date: Wed Nov 18 06:17:01 2009 +1100 pmdaproc.sh Cleanup Significant change is to _not_ use -n option to pminfo unless -n was specified in the command line. This is needed to avoid warnings for PMDAs that install dynamic metrics into the PMNS (like the new mmv). Also dropped -h localhost (this is the default). And added -V option to allow more verbose output (at this stage the full output from pminfo after a PMDA install with metrics that have errors or warnings). commit 5ae6e920fc48dd83615dbfa86851cad1988345bd Author: Ken McDonell Date: Wed Nov 18 06:14:57 2009 +1100 units.c Fix day 1 error Wrong variable used and uninitialized value was being used ... previous code was 100% wrong, but had low probability of being encountered and low impact. commit a200cebdc1c0a0a17760dc31d279cb4447fd3d8d Author: Ken McDonell Date: Wed Nov 18 06:13:03 2009 +1100 pmns.c Rework last PMNS_LEAF_* changes to make them correct this time! commit c4491479f4c151b3ce82549f09c9e1e18aec4890 Author: Ken McDonell Date: Wed Nov 18 06:12:15 2009 +1100 INSTALL Toolchain problem persists with Ubuntu 9.10 commit 86795a1e10627dd3ee3c598c756027652c849630 Author: Ken McDonell Date: Wed Nov 18 06:10:31 2009 +1100 rc_pcp Drop -h localhost from pmcd_wait call Cosmetic cleanup ... pmcd_wait and pmcd_wait -h localhost behave exactly the same. commit 8d031568beb88dfd4d5add78a663db03b05c7e8e Author: Ken McDonell Date: Tue Nov 10 13:42:56 2009 +1100 dopdus.c Fix typo in comment commit 3baf50585bc1c5a29e92b052298e93afe7aefe03 Author: Ken McDonell Date: Tue Nov 10 13:41:49 2009 +1100 pmns.c Use PMNS_LEAF_STATUS and PMNS_NONLEAF_STATUS instead of 0 and 1 Makes the code a little more readable. From nscott@aconex.com Wed Nov 18 17:02:48 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_34 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAIN2l9P207373 for ; Wed, 18 Nov 2009 17:02:47 -0600 X-ASG-Debug-ID: 1258585388-71bc00a90000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 08700CCE2C6 for ; Wed, 18 Nov 2009 15:03:08 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id HCHYuBg3SpnFVbVF for ; Wed, 18 Nov 2009 15:03:08 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 8B1B25DC00C; Thu, 19 Nov 2009 10:03:07 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id 76w37eLBfsKixmCX; Thu, 19 Nov 2009 10:03:07 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 5207CA503DB; Thu, 19 Nov 2009 10:01:25 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 6BE53C7A17; Thu, 19 Nov 2009 10:03:07 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ECE0WKfgDRdg; Thu, 19 Nov 2009 10:03:02 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id AEC42C7A12; Thu, 19 Nov 2009 10:03:02 +1100 (EST) Date: Thu, 19 Nov 2009 10:03:02 +1100 (EST) From: Nathan Scott To: kenj Cc: pcp Message-ID: <1271437514.516191258585382571.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1215282402.515991258585256224.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: Re: pmlc handshake timeout needed? Subject: Re: pmlc handshake timeout needed? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258585390 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14979 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean Quick followup - you asked yesterday where the PIDs used here were coming from (first arg to pmlc in the ps output earlier) ... they are delivered via this shell function: get_running_loggers() { # Gets the list of pid+host pairs for all running pmlogger processes pminfo -f pmcd.pmlogger.pmcd_host \ | perl -ne 'if (/^ +inst \[(\d+).*value "(.*)"/) { print "$1 $2\n" }' } cheers. -- Nathan From nscott@aconex.com Wed Nov 18 21:02:55 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAJ32svP225969 for ; Wed, 18 Nov 2009 21:02:55 -0600 X-ASG-Debug-ID: 1258599795-624d00d40000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E9EA018A1555 for ; Wed, 18 Nov 2009 19:03:15 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id kJAzYiDRBMTO6dzi for ; Wed, 18 Nov 2009 19:03:15 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 64F3F5DD825 for ; Thu, 19 Nov 2009 14:03:13 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id xvhHICw4CYmEHTJv for ; Thu, 19 Nov 2009 14:03:13 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id F325FA50295 for ; Thu, 19 Nov 2009 14:01:30 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 36A5A4FD84 for ; Thu, 19 Nov 2009 14:03:13 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NbJKVwu0sjjR for ; Thu, 19 Nov 2009 14:03:09 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 5D6004FD82 for ; Thu, 19 Nov 2009 14:03:09 +1100 (EST) Date: Thu, 19 Nov 2009 14:03:09 +1100 (EST) From: Nathan Scott To: pcp@oss.sgi.com Message-ID: <698943612.530721258599789224.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: pcp updates Subject: pcp updates MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258599796 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.14994 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Status: Clean [also merged Kens -dev tree into mine] Changes committed to git://oss.sgi.com/nathans/pcp.git src/cpan/PMDA/Changes | 9 + src/cpan/PMDA/PMDA.pm | 4 src/cpan/PMDA/PMDA.xs | 24 +- src/cpan/PMDA/local.c | 1 src/pmdas/GNUmakefile | 2 src/pmdas/bonding/.gitignore | 1 src/pmdas/dtsrun/.gitignore | 2 src/pmdas/dtsrun/GNUmakefile | 49 +++++ src/pmdas/dtsrun/Install | 41 ++++ src/pmdas/dtsrun/Remove | 25 +++ src/pmdas/dtsrun/pmdadtsrun.pl | 340 +++++++++++++++++++++++++++++++++++++++++ src/pmdas/mmv/mmvdump.c | 65 +++++++ src/pmdas/netfilter/.gitignore | 1 src/pmdas/news/.gitignore | 1 src/pmns/stdpmid.pcp | 1 15 files changed, 550 insertions(+), 16 deletions(-) commit 680957224e17d4d4b7e14d26de1fd436527ca22a Author: Nathan Scott Date: Thu Nov 19 13:47:32 2009 +1100 Add decoded metric type/semantics info into mmvdump. commit 950850239d5a7c6d680383421067b0ce9c7dd6bf Author: Nathan Scott Date: Thu Nov 19 13:25:26 2009 +1100 Include the dtsrun (Data Transformation Services) PMDA. This instrumentation is extracted from the dtsrun executable which performs database transformations for SQL Server. It includes status and timing information for overall packages and individual steps within each package. commit 047b4f3401fcf95f35742a72560a331d792fe2a5 Author: Nathan Scott Date: Thu Nov 19 13:16:09 2009 +1100 Improvements and fixes to the PCP::PMDA perl module (1.08). Correct the export name used for reverse mapping instance name to instance identifiers. Add a fast path in here as well which first attempts direct mapping before iterative lookup. Finally, start any file tail monitors at the current end- of-file, not the start, as we only want data from the time the PMDA starts, onward (also avoids potentially expensive scan of the entire log file up to that point). commit 4959e511010e8e7ed47fdc6589616b565ee85f36 Author: Nathan Scott Date: Thu Nov 19 13:07:28 2009 +1100 Add a few missing gitignore lines in various Perl PMDAs. From nscott@aconex.com Thu Nov 19 21:06:04 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAK36350100526 for ; Thu, 19 Nov 2009 21:06:03 -0600 X-ASG-Debug-ID: 1258686384-0f81005e0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 573371343948 for ; Thu, 19 Nov 2009 19:06:25 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id XG3h0lAK1EozXVev for ; Thu, 19 Nov 2009 19:06:25 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 7E3C1875774 for ; Fri, 20 Nov 2009 14:06:23 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id HHp7zw7EFJW8TypE for ; Fri, 20 Nov 2009 14:06:23 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id AAD4CA50283 for ; Fri, 20 Nov 2009 14:04:40 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 7463F4FD84 for ; Fri, 20 Nov 2009 14:06:23 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YDK+6iUfetUf for ; Fri, 20 Nov 2009 14:06:19 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 6E6524FD85 for ; Fri, 20 Nov 2009 14:06:19 +1100 (EST) Date: Fri, 20 Nov 2009 14:06:19 +1100 (EST) From: Nathan Scott To: pcp@oss.sgi.com Message-ID: <2122901728.556391258686379381.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: pcpqa updates Subject: pcpqa updates MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258686386 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15080 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean Changes committed to git://oss.sgi.com/pcp/pcpqa.git 646 | 1 - 646.out | 14 +++++++------- 2 files changed, 7 insertions(+), 8 deletions(-) commit 19541a73ab7373d52edd7ac40883dae23403b3d0 Author: Nathan Scott Date: Fri Nov 20 13:25:13 2009 +1100 Update 646 using mmvdump to reflect format tweaks. Ditch SOME_PID filter. From kenj@internode.on.net Sat Nov 21 04:34:57 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nALAYuUZ016715 for ; Sat, 21 Nov 2009 04:34:57 -0600 X-ASG-Debug-ID: 1258799718-42db00d20000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9DCB41BB54F3 for ; Sat, 21 Nov 2009 02:35:18 -0800 (PST) Received: from mail.internode.on.net (bld-mail13.adl6.internode.on.net [150.101.137.98]) by cuda.sgi.com with ESMTP id E5PITv5RNdAfcF4j for ; Sat, 21 Nov 2009 02:35:18 -0800 (PST) Received: from [192.168.5.2] (unverified [58.167.49.202]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8827685-1927428 for ; Sat, 21 Nov 2009 21:05:16 +1030 (CDT) X-ASG-Orig-Subj: pcp updates Subject: pcp updates From: Ken McDonell Reply-To: kenj@internode.on.net To: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Sat, 21 Nov 2009 21:35:15 +1100 Message-ID: <1258799715.28154.1.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail13.adl6.internode.on.net[150.101.137.98] X-Barracuda-Start-Time: 1258799720 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4635 1.0000 0.0000 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15193 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Fixes a problem Nathan observed. Changes committed to git://oss.sgi.com/kenj/pcp.git dev src/dbpmda/src/gram.y | 46 ++++++++++++----- src/dbpmda/src/util.c | 128 ++++++++++++++++++++++++++++++-------------------- 2 files changed, 109 insertions(+), 65 deletions(-) commit 985d91be524ea80443534e0da088f5f7f5137119 Author: Ken McDonell Date: Sat Nov 21 09:09:48 2009 +1100 dbpmda Add missing help Grammar and verbage support for extended help for additional commands added in the dynamic PMNS changes. From kenj@internode.on.net Mon Nov 23 00:29:20 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_34 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAN6TKh6234610 for ; Mon, 23 Nov 2009 00:29:20 -0600 X-ASG-Debug-ID: 1258957783-155002620000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5A6B81C01D9F for ; Sun, 22 Nov 2009 22:29:43 -0800 (PST) Received: from mail.internode.on.net (bld-mail16.adl2.internode.on.net [150.101.137.101]) by cuda.sgi.com with ESMTP id hTHSd1gYY8hdNmBA for ; Sun, 22 Nov 2009 22:29:43 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.137.163]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8808708-1927428 for multiple; Mon, 23 Nov 2009 16:59:42 +1030 (CDT) X-ASG-Orig-Subj: Re: pmlc handshake timeout needed? Subject: Re: pmlc handshake timeout needed? From: Ken McDonell Reply-To: kenj@internode.on.net To: Nathan Scott Cc: pcp In-Reply-To: <1271437514.516191258585382571.JavaMail.root@mail-au.aconex.com> References: <1271437514.516191258585382571.JavaMail.root@mail-au.aconex.com> Content-Type: text/plain; charset="UTF-8" Date: Mon, 23 Nov 2009 17:29:40 +1100 Message-ID: <1258957780.3402.9.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail16.adl2.internode.on.net[150.101.137.101] X-Barracuda-Start-Time: 1258957785 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0104 1.0000 -1.9533 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.95 X-Barracuda-Spam-Status: No, SCORE=-1.95 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15353 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean OK, that means /var/tmp/pmlogger is probably OK, and the pmlogger process still existed (else it would not accept the connection from pmlc and you'd never get to hang pmlc). I thought it might be possible to get away with a new -w timeout option to pmlc, but they connection to the pmlogger involves a handshake protocol and one PDU from pmlogger to pmlc ... and this is buried in libpcp with no way to get a configurable timeout into the __pmGetPDU call. So I think this means we need another environment variable, and I suggest $PMLOGGER_REQUEST_TIMEOUT. Any objections or better suggestions? On Thu, 2009-11-19 at 10:03 +1100, Nathan Scott wrote: > Quick followup - you asked yesterday where the PIDs used here were > coming from (first arg to pmlc in the ps output earlier) ... they > are delivered via this shell function: > > get_running_loggers() > { > # Gets the list of pid+host pairs for all running pmlogger processes > pminfo -f pmcd.pmlogger.pmcd_host \ > | perl -ne 'if (/^ +inst \[(\d+).*value "(.*)"/) { print "$1 $2\n" }' > } > > cheers. > From nscott@aconex.com Mon Nov 23 00:40:51 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAN6epw7235462 for ; Mon, 23 Nov 2009 00:40:51 -0600 X-ASG-Debug-ID: 1258958473-20e101c50000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5E149A57A8 for ; Sun, 22 Nov 2009 22:41:13 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id JeV39hkrD75PDMur for ; Sun, 22 Nov 2009 22:41:13 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 6215B6B7D; Mon, 23 Nov 2009 17:41:11 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id cV4KZ1IcLB9cJ5j8; Mon, 23 Nov 2009 17:41:11 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id C2AAFA502B6; Mon, 23 Nov 2009 17:39:26 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 3D62A4FD82; Mon, 23 Nov 2009 17:41:11 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yGI8ht1Q9REb; Mon, 23 Nov 2009 17:41:06 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 7D0954FD84; Mon, 23 Nov 2009 17:41:06 +1100 (EST) Date: Mon, 23 Nov 2009 17:41:06 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp Message-ID: <2146329038.611811258958466424.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1258957780.3402.9.camel@bozo.localdomain> X-ASG-Orig-Subj: Re: pmlc handshake timeout needed? Subject: Re: pmlc handshake timeout needed? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1258958476 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0734 1.0000 -1.5541 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.55 X-Barracuda-Spam-Status: No, SCORE=-1.55 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15353 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Ken McDonell" wrote: > OK, that means /var/tmp/pmlogger is probably OK, and the pmlogger > process still existed (else it would not accept the connection from > pmlc and you'd never get to hang pmlc). > > I thought it might be possible to get away with a new -w timeout > option > to pmlc, but they connection to the pmlogger involves a handshake > protocol and one PDU from pmlogger to pmlc ... and this is buried in > libpcp with no way to get a configurable timeout into the __pmGetPDU > call. > > So I think this means we need another environment variable, and I > suggest $PMLOGGER_REQUEST_TIMEOUT. > > Any objections or better suggestions? That all sounds OK to me. cheers. -- Nathan From nscott@aconex.com Mon Nov 23 17:43:46 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nANNhkbw062123 for ; Mon, 23 Nov 2009 17:43:46 -0600 X-ASG-Debug-ID: 1259019849-387b02960000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6D014AABA3 for ; Mon, 23 Nov 2009 15:44:10 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id OD27sey9HIVYuTeq for ; Mon, 23 Nov 2009 15:44:10 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 2CB50784722 for ; Tue, 24 Nov 2009 10:44:08 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id 5lYHnCUHDRAzckwY for ; Tue, 24 Nov 2009 10:44:08 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 456DEA5017F for ; Tue, 24 Nov 2009 10:42:23 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 1F5084FD85 for ; Tue, 24 Nov 2009 10:44:08 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1vkWt5ntrg-3 for ; Tue, 24 Nov 2009 10:44:04 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 184134FD82 for ; Tue, 24 Nov 2009 10:44:04 +1100 (EST) Date: Tue, 24 Nov 2009 10:44:04 +1100 (EST) From: Nathan Scott To: pcp Message-ID: <270111066.630181259019844010.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: pcp updates Subject: pcp updates MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1259019851 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0070 1.0000 -1.9754 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.98 X-Barracuda-Spam-Status: No, SCORE=-1.98 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15409 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean Changes committed to git://oss.sgi.com/nathans/pcp.git src/dbpmda/src/gram.y | 46 ++++++++++++----- src/dbpmda/src/util.c | 128 ++++++++++++++++++++++++++++++-------------------- src/pmstat/pmstat.c | 26 +++++++--- 3 files changed, 127 insertions(+), 73 deletions(-) commit 34a1ac4e453a1d1bafff91ca6c328a8c78c53f65 Author: Nathan Scott Date: Tue Nov 24 10:35:05 2009 +1100 Improve pmstat handling of a terminal resize, wrt header reporting. commit 71a549645740cce27dc20092b8c6dec47ad423aa Merge: 356fb7a 985d91b Author: Ken McDonell Date: Mon Nov 23 11:34:21 2009 +1100 Merge branch 'dev' of git://oss.sgi.com/kenj/pcp into dev commit 985d91be524ea80443534e0da088f5f7f5137119 Author: Ken McDonell Date: Sat Nov 21 09:09:48 2009 +1100 dbpmda Add missing help Grammar and verbage support for extended help for additional commands added in the dynamic PMNS changes. From daw@sgi.com Tue Nov 24 13:37:41 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAOJbeWY160559 for ; Tue, 24 Nov 2009 13:37:41 -0600 Received: from zion.americas.sgi.com (zion.americas.sgi.com [128.162.236.17]) by relay3.corp.sgi.com (Postfix) with ESMTP id 69754AC009 for ; Tue, 24 Nov 2009 11:38:04 -0800 (PST) Received: (from daw@localhost) by zion.americas.sgi.com (8.12.9/ASC-news-1.4) id nAOJc4Ik28108285 for pcp@oss.sgi.com; Tue, 24 Nov 2009 13:38:04 -0600 (CST) Message-Id: <200911241938.nAOJc4Ik28108285@zion.americas.sgi.com> Subject: Re: pcp Digest, Vol 13, Issue 23 To: pcp@oss.sgi.com Date: Tue, 24 Nov 2009 13:38:03 -0600 (CST) In-Reply-To: from "pcp-request@oss.sgi.com" at Nov 24, 2009 12:00:01 PM From: David Wright X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello, Todays pcp-glider is hanging my vista system when moving/resizing the window. Dave Wright From kenj@internode.on.net Wed Nov 25 16:34:20 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAPMYKNk012729 for ; Wed, 25 Nov 2009 16:34:20 -0600 X-ASG-Debug-ID: 1259188484-01fb03a50000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B79BFD8336C for ; Wed, 25 Nov 2009 14:34:44 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id cotpNDdvuX1XJ4L9 for ; Wed, 25 Nov 2009 14:34:44 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.137.163]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8999680-1927428 for ; Thu, 26 Nov 2009 09:04:43 +1030 (CDT) X-ASG-Orig-Subj: pcpqa updates Subject: pcpqa updates From: Ken McDonell Reply-To: kenj@internode.on.net To: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Thu, 26 Nov 2009 09:34:39 +1100 Message-ID: <1259188479.25811.1.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1259188486 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15583 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Some tidying up ... 230 is the most recent "newie" for the pmlc <--> pmlogger timeout changes. Changes committed to git://oss.sgi.com/pcp/pcpqa.git 216 | 314 +++++++++++++++++++++++------------------------------ 219 | 61 ++++++++++ 219.out | 4 225 | 69 +++++++++++ 225.out | 56 +++++++++ 226 | 44 +++++++ 226.out | 226 ++++++++++++++++++++++++++++++++++++++ 230 | 240 ++++++++++++++++++++++++++++++++++++++++ 230.out | 112 ++++++++++++++++++ 288.out | 216 ++++++++++++++++++------------------ 310 | 2 group | 10 + new | 14 +- owner | 6 - src-oss/pducheck.c | 5 15 files changed, 1091 insertions(+), 288 deletions(-) commit f3c90aa2cbedb54b01d33034b58948e43cbe0f8f Merge: 19541a7 e4a02d9 Author: Ken McDonell Date: Thu Nov 26 08:55:42 2009 +1100 Merge branch 'master' of /home/kenj/src/pcpqa into gpush_staging commit e4a02d99b80109099d77c2312751db325837bda2 Author: Ken McDonell Date: Thu Nov 26 08:50:05 2009 +1100 288.out remade after -z change commit 3bdc2ed3f593d5766a6caf02cda4368e56d859ee Author: Ken McDonell Date: Thu Nov 26 07:13:27 2009 +1100 pducheck.c Fix long-standing logic error (wrong variable use) commit 2f2cb2d33be104a9af964220b39ff0261ebb9f1f Author: Ken McDonell Date: Thu Nov 26 07:11:41 2009 +1100 sync up group and owner with latest additions, tweak new script commit f912f2444214b2d59db4d82d4544b7c262f701f8 Author: Ken McDonell Date: Thu Nov 26 07:10:58 2009 +1100 230 Exercise pmlc <--> pmlogger timeouts commit e2799bb693a91a5f6661abe908466b3d87df4040 Author: Ken McDonell Date: Thu Nov 26 07:10:08 2009 +1100 226 Check dbpmda help text is complete commit a3391f1b016cb542f37714bcc93e17f3d8402c51 Author: Ken McDonell Date: Thu Nov 26 07:09:40 2009 +1100 225 basic testing for derived metrics commit c480587c6d2de8b6984b4959b9bc4e9cd49dc0d4 Author: Ken McDonell Date: Tue Nov 10 14:24:37 2009 +1100 219 Check consistency between pmdbg -l and Precursor to adding DBG_TRACE_DERIVE for derived metrics. commit 020c2b6785eaf28237b466467cca96318a822809 Merge: 08be4e3 41c25b6 Author: Ken McDonell Date: Sat Nov 7 07:40:27 2009 +1100 Merge branch 'master' of git://oss.sgi.com/pcp/pcpqa commit 08be4e397be5705f75490a8091e5fa4243e177c6 Author: Ken McDonell Date: Sat Nov 7 07:38:53 2009 +1100 216 rewrite Changed the logic for matching PCP network.* metrics to netstat output on Linux. Also minor cosmetic change to copyright in 310. From kenj@internode.on.net Wed Nov 25 17:28:42 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAPNSgJn016285 for ; Wed, 25 Nov 2009 17:28:42 -0600 X-ASG-Debug-ID: 1259191746-049b01320000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B245A18E5736 for ; Wed, 25 Nov 2009 15:29:07 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id oLnwM6DFQwAFrIQ7 for ; Wed, 25 Nov 2009 15:29:07 -0800 (PST) Received: from [192.168.1.100] (unverified [118.209.137.163]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 8876825-1927428 for ; Thu, 26 Nov 2009 09:59:05 +1030 (CDT) X-ASG-Orig-Subj: pcp updates Subject: pcp updates From: Ken McDonell Reply-To: kenj@internode.on.net To: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Thu, 26 Nov 2009 10:29:02 +1100 Message-ID: <1259191742.25811.2.camel@bozo.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1259191748 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15588 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Changes committed to git://oss.sgi.com/kenj/pcp.git dev man/man1/pcpintro.1 | 33 ++++++++++++++++++++--- src/include/impl.h | 5 ++- src/libpcp/src/logconnect.c | 48 ++++++++++++++++++++++++++++++++-- src/libpcp/src/logcontrol.c | 6 ++-- src/pmlc/actions.c | 62 +++++++++++++++++++++++++++++++++++++++----- src/pmlc/gram.y | 8 +++++ src/pmlc/lex.l | 1 src/pmlc/pmlc.c | 8 +++++ src/pmlc/pmlc.h | 1 src/pmlogger/dopdu.c | 42 +++++++++++++++++++++++++---- src/pmlogger/logger.h | 5 +++ src/pmlogger/pmlogger.c | 2 + 12 files changed, 198 insertions(+), 23 deletions(-) commit 9cc29e2a97ba110d295044c263bbdfa6556ce79d Author: Ken McDonell Date: Thu Nov 26 10:23:17 2009 +1100 Add PMLOGGER_REQUEST_TIMEOUT support Setting $PMLOGGER_REQUEST_TIMEOUT in the environment defines a number of seconds for the timeout to be applied to interactions between pmlc and pmlogger ... specifically, pmlc will timeout in a __pmGetPDU call if the pmlogger is not responding. Added __pmLoggerTimeout() to libpcp as checks need to be made in libpcp (logconnect.c and logcontrol.c) and in pmlc (mostly in actions.c) Requested by Nathan after strange failure of pmlogger monitoring scripts in their production environment when pmlc connected to a morbid pmlogger but the pmlc hung trying to get information from pmlogger. From nscott@aconex.com Thu Nov 26 03:33:53 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAQ9Xr2i047433 for ; Thu, 26 Nov 2009 03:33:53 -0600 X-ASG-Debug-ID: 1259228058-23ff01e60000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 39D48D8457D for ; Thu, 26 Nov 2009 01:34:18 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id ROKcDdb2C9eefFv8 for ; Thu, 26 Nov 2009 01:34:18 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 1FE10209CA; Thu, 26 Nov 2009 20:34:16 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id x3G26DAJ030jWFuS; Thu, 26 Nov 2009 20:34:16 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id C9CEBA50283; Thu, 26 Nov 2009 20:32:29 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id E9C034FD85; Thu, 26 Nov 2009 20:34:15 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p4Snrl119BRM; Thu, 26 Nov 2009 20:34:11 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id 401234FD82; Thu, 26 Nov 2009 20:34:11 +1100 (EST) Date: Thu, 26 Nov 2009 20:34:11 +1100 (EST) From: Nathan Scott To: David Wright Cc: pcp@oss.sgi.com Message-ID: <870525181.711261259228051154.JavaMail.root@mail-au.aconex.com> In-Reply-To: <200911241938.nAOJc4Ik28108285@zion.americas.sgi.com> X-ASG-Orig-Subj: Re: Hang on Windows Vista Subject: Re: Hang on Windows Vista MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [211.28.150.226] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - SAF3 (Mac)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1259228060 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0001 1.0000 -2.0204 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15623 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "David Wright" wrote: > Hello, > > Todays pcp-glider is hanging my vista system when moving/resizing > the window. Can you describe "hanging" in a bit more detail? Does the machine respond to ping? By pcp-glider, did you specifically mean pmchart? I have several users in close proximity to me using this pmchart on Vista with good success for several weeks, so keen to hear details. If you monitor the hung machine from another host on the network (I assume & hope you are on a network at this point :) ... is it CPU/ Disk/Net saturated, which process(es) consuming resources, etc - if you can uncover any of this info it would be a big help to further understanding this problem. cheers. -- Nathan From nscott@aconex.com Sun Nov 29 18:15:30 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_33 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAU0FUko116141 for ; Sun, 29 Nov 2009 18:15:30 -0600 X-ASG-Debug-ID: 1259540157-4a3002cc0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A03E818F586E for ; Sun, 29 Nov 2009 16:15:57 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id IxIbik1ORp7IXRiM for ; Sun, 29 Nov 2009 16:15:57 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id CA9B278C8DC for ; Mon, 30 Nov 2009 11:15:53 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id 5opoWRT3ybcz4Itf for ; Mon, 30 Nov 2009 11:15:53 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id A214FA50264 for ; Mon, 30 Nov 2009 11:14:05 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id A0A914FD82 for ; Mon, 30 Nov 2009 11:15:53 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Sc9D+VJgQGNF for ; Mon, 30 Nov 2009 11:15:50 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id CAC8F4FD85 for ; Mon, 30 Nov 2009 11:15:50 +1100 (EST) Date: Mon, 30 Nov 2009 11:15:50 +1100 (EST) From: Nathan Scott To: pcp@oss.sgi.com Message-ID: <252236315.760621259540150654.JavaMail.root@mail-au.aconex.com> X-ASG-Orig-Subj: pcp updates Subject: pcp updates MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1259540159 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15925 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Status: Clean Changes committed to git://oss.sgi.com/nathans/pcp.git install-sh | 2 +- src/include/builddefs.in | 2 +- src/pmdas/cisco/Samples | 12 ++++-------- src/pmdas/cisco/Tested | 35 +++++++++++------------------------ src/pmdas/cisco/cisco.h | 4 ++-- src/pmdas/cisco/probe.c | 5 +++-- src/pmdas/cisco/telnet.c | 24 ++++++++++++++++-------- 7 files changed, 38 insertions(+), 46 deletions(-) commit 239ecbcbf295ed13f0c4b265d628821959018e57 Author: Nathan Scott Date: Mon Nov 30 11:06:34 2009 +1100 Two changes to get PCP up and running on 64 bit Mac OS 10.6. Generate universal binary version of all libraries (so, both 32 and 64 bit code included in the one binary). The chmod in install-sh has been switched to usr:grp form as the usr.grp support is no longer there in this version. Hmm, dodgey. commit 38b581eda888ee046e0e9a67bf79a644606ee892 Author: Nathan Scott Date: Mon Nov 30 11:03:40 2009 +1100 Make pmdacisco work for our local office router (2960 model). There's a problem in the parsing code where both a username and password are required. The username extraction ends with the Password: prompt, & consumes this token - the password extraction starts by looking for that token - so it hangs indefinately in the interface probe stage. Addressed this by passing a pointer to the "Password:" string, if it was observed, so that the next stage can (optionally) start with that token. Otherwise everything is done exactly as before. Cleansed explicit hostnames from the pmdacisco Samples/Tested files also. From kenj@internode.on.net Sun Nov 29 20:41:46 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAU2fjsO171735 for ; Sun, 29 Nov 2009 20:41:46 -0600 X-ASG-Debug-ID: 1259548932-0a4d023b0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 864E818F5C02 for ; Sun, 29 Nov 2009 18:42:12 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id 3z4AWyuga53WDkvg for ; Sun, 29 Nov 2009 18:42:12 -0800 (PST) Received: from [124.182.113.194] (unverified [124.182.113.194]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 9212881-1927428 for multiple; Mon, 30 Nov 2009 13:12:11 +1030 (CDT) X-ASG-Orig-Subj: mmv and qa/648 questions Subject: mmv and qa/648 questions From: Ken McDonell Reply-To: kenj@internode.on.net To: Max Matveev , Nathan Scott Cc: pcp@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Mon, 30 Nov 2009 10:58:11 +1100 Message-ID: <1259539091.12861.4.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1259548934 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15935 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Guys, I've been looking at the mmv pmda and QA coverage. What is the expected behaviour when more than one mmv stats file mentions the _same_ cluster? If it is OK, the code is wrong as it keeps registering metrics with the same pmids as existing metrics, but only the first registered is fetchable. Also forming the union of the instance domains is not happening either. If it is not OK, then the PMDA does not detect this properly, and mmv_instances.c in the QA suite needs to be changed. I'll chase it down, if you can give me an indication of what was supposed to happen. I think for the Aconex-style semantics, forming the union of the mmap'd files over a common cluster is probably correct, but I don't think was within the scope of the original mmv implementation. From nscott@aconex.com Sun Nov 29 20:52:28 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAU2qS0j172514 for ; Sun, 29 Nov 2009 20:52:28 -0600 X-ASG-Debug-ID: 1259549575-122a03b70000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from postoffice2.aconex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2A44D12D7196 for ; Sun, 29 Nov 2009 18:52:55 -0800 (PST) Received: from postoffice2.aconex.com (mail.aconex.com [203.89.202.182]) by cuda.sgi.com with ESMTP id r8qEnhcRnoKyGRBL for ; Sun, 29 Nov 2009 18:52:55 -0800 (PST) Received: from postoffice.aconex.com (localhost [127.0.0.1]) by postoffice2.aconex.com (Spam & Virus Firewall) with ESMTP id 724922EBD3; Mon, 30 Nov 2009 13:52:54 +1100 (EST) Received: from postoffice.aconex.com (postoffice.yarra.acx [192.168.102.1]) by postoffice2.aconex.com with ESMTP id yGNK5BPeqTWG238S; Mon, 30 Nov 2009 13:52:54 +1100 (EST) Received: from gatekeeper.aconex.com (gatekeeper.yarra.acx [192.168.102.10]) by postoffice.aconex.com (Postfix) with ESMTP id 478D0A5017F; Mon, 30 Nov 2009 13:51:06 +1100 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by gatekeeper.aconex.com (Postfix) with ESMTP id 631214FD84; Mon, 30 Nov 2009 13:52:54 +1100 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at gatekeeper.yarra.acx Received: from gatekeeper.aconex.com ([127.0.0.1]) by localhost (gatekeeper.aconex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xV4DqBabW2ab; Mon, 30 Nov 2009 13:52:49 +1100 (EST) Received: from mail-au.aconex.com (mail-au.aconex.com [192.168.102.12]) by gatekeeper.aconex.com (Postfix) with ESMTP id AB08D4FD82; Mon, 30 Nov 2009 13:52:49 +1100 (EST) Date: Mon, 30 Nov 2009 13:52:49 +1100 (EST) From: Nathan Scott To: kenj@internode.on.net Cc: pcp@oss.sgi.com, Max Matveev Message-ID: <1774404665.766611259549569388.JavaMail.root@mail-au.aconex.com> In-Reply-To: <1259539091.12861.4.camel@localhost> X-ASG-Orig-Subj: Re: mmv and qa/648 questions Subject: Re: mmv and qa/648 questions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [203.89.192.141] X-Mailer: Zimbra 5.0.18_GA_3011.RHEL5_64 (ZimbraWebClient - [unknown] (Linux)/5.0.18_GA_3011.RHEL5_64) X-Barracuda-Connect: mail.aconex.com[203.89.202.182] X-Barracuda-Start-Time: 1259549577 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15936 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean ----- "Ken McDonell" wrote: > Guys, > > I've been looking at the mmv pmda and QA coverage. > > What is the expected behaviour when more than one mmv stats file > mentions the _same_ cluster? > > ... > I'll chase it down, if you can give me an indication of what was > supposed to happen. > > I think for the Aconex-style semantics, forming the union of the > mmap'd files over a common cluster is probably correct, but I don't > think was within the scope of the original mmv implementation. You're on the money there. It was indeed not in the scope of the original PMDA, and it is planned to be used by the revised Aconex PMDA. I just need to get back to it, to fix it up (it worked at one point, when that QA test was written, but IIRC something was found wrong with it - so I commented the test out till I could get time for a more detailed look). I'd love to see it working ... ! On a related note, there was some discussion the other day about making pmdammv a default install like pmcd and kernel PMDAs - any thoughts there? Idea was to make everything work out-of-the-box with MMV (since it should work on all platforms & since we can now do a DSO MMV, there wouldn't be any extra daemon running, etc). cheers. -- Nathan From makc@gmx.co.uk Mon Nov 30 16:12:55 2009 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id nAUMCsWc066067 for ; Mon, 30 Nov 2009 16:12:55 -0600 X-ASG-Debug-ID: 1259619202-764b014b0000-S8gJnT X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from outbound.icp-qv1-irony-out6.iinet.net.au (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1CBAF12D8FF5 for ; Mon, 30 Nov 2009 14:13:22 -0800 (PST) Received: from outbound.icp-qv1-irony-out6.iinet.net.au (outbound.icp-qv1-irony-out6.iinet.net.au [203.59.1.109]) by cuda.sgi.com with ESMTP id Pr2x2PF5HGOxwAnS for ; Mon, 30 Nov 2009 14:13:22 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkAjAP7RE0s6aAAC/2dsb2JhbACHKIs7hXaECrwShDEE X-IronPort-AV: E=Sophos;i="4.47,316,1257091200"; d="scan'208";a="31208619" Received: from unknown (HELO [58.104.0.2]) ([58.104.0.2]) by outbound.icp-qv1-irony-out6.iinet.net.au with ESMTP/TLS/AES128-SHA; 01 Dec 2009 06:13:17 +0800 References: <1259539091.12861.4.camel@localhost> Message-Id: From: Max To: "kenj@internode.on.net" In-Reply-To: <1259539091.12861.4.camel@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit X-Mailer: iPhone Mail (7A400) Mime-Version: 1.0 (iPhone Mail 7A400) X-ASG-Orig-Subj: Re: [pcp] mmv and qa/648 questions Subject: Re: [pcp] mmv and qa/648 questions Date: Tue, 1 Dec 2009 09:13:11 +1100 Cc: Max Matveev , Nathan Scott , "pcp@oss.sgi.com" X-Barracuda-Connect: outbound.icp-qv1-irony-out6.iinet.net.au[203.59.1.109] X-Barracuda-Start-Time: 1259619204 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.15972 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 30/11/2009, at 10:58, Ken McDonell wrote: > What is the expected behaviour when more than one mmv stats file > mentions the _same_ cluster? > > If it is OK, the code is wrong as it keeps registering metrics with > the > same pmids as existing metrics, but only the first registered is > fetchable. Also forming the union of the instance domains is not > happening either. I did not consider this scenario in the original design - which ever it does is a fluke. > I think for the Aconex-style semantics, forming the union of the > mmap'd > files over a common cluster is probably correct, but I don't think was > within the scope of the original mmv implementation. > It makes sense to me. max