From kenmcd@melbourne.sgi.com Fri Nov 8 02:54:47 2002 Received: with ECARTIS (v1.0.0; list pcp); Fri, 08 Nov 2002 02:54:53 -0800 (PST) Received: from deliverator.sgi.com (deliverator.SGI.COM [204.94.214.10] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gA8AsluR025771 for ; Fri, 8 Nov 2002 02:54:47 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by deliverator.sgi.com (980327.SGI.8.8.8-aspam/980304.SGI-aspam: SGI does not authorize the use of its proprietary systems or networks for unsolicited or bulk email from the Internet.) via SMTP id CAA00449 for ; Fri, 8 Nov 2002 02:56:00 -0800 (PST) mail_from (kenmcd@melbourne.sgi.com) From: kenmcd@melbourne.sgi.com Received: from kenj-ppp-a.melbourne.sgi.com (ppp-kenmcd.melbourne.sgi.com [134.14.52.219]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id VAA08312 for ; Fri, 8 Nov 2002 21:54:26 +1100 Date: Fri, 8 Nov 2002 21:57:45 +1100 (EST) Reply-To: kenmcd@melbourne.sgi.com To: pcp@oss.sgi.com Subject: PCP QA Open Source Release V1.0 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 45 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: kenmcd@melbourne.sgi.com Precedence: bulk X-list: pcp The PCP QA suite has been released to open source under the terms and conditions of the GPL. If you are a PCP developer, the QA suite can be downloaded from ftp://oss.sgi.com/projects/pcp/download/pcp-qa-1.0.tar.gz The README from the distribution is copied below. Please send comments, corrections and suggestions to pcp@oss.sgi.com. -------- Open Source Release of the PCP QA Suite - Version 1.0 ===================================================== Preliminaries ------------- PCP needs to be installed on the local host, with pmcd operational. Basic getting started --------------------- Unpack the gzipped tarball into a directory. There is some local configuration needed ... check the file "common.config" ... this script uses heuristics to set a number of interesting variables, specifically: $PCPQA_CLOSE_X_SERVER The $DISPLAY setting for an X server that is willing to accept connections from X clients running on the local machine. Not used in the Open Source PCP QA tests. $PCPQA_FAR_PMCD The hostname for a host running pmcd, but the host is preferably a long way away (over a WAN) for timing test. Test 131 is the only one that uses this variable. $PCPQA_HYPHEN_HOST The hostname for a host running pmcd, with a hyphen (-) in the hostname. Test 116 is the only Open Source PCP QA test that uses this variable. Next, mk.qa_hosts is a script that includes heuristics for selecting and sorting the list of potential remote PCP QA hosts (qa_hosts.master). Refer to the comments in qa_hosts.master, and make appropriate changes. Once you've modified common.config and qa_hosts.master, then run "chk.setup" to validate the settings. The PCP QA tests are designed to be run by a non-root user. Where root privileges are needed, e.g. to stop or start pmcd, install/remove PMDAs, etc. the "sudo" application is used. WARNING: sudo is a GIANT security hole, so do not run the PCP QA on on a machine with sensitivity to security and hacking issues. To get sudo set up, $ ./check 0 # remake sudo as suggested $ ./check 0 The first time you run "check" (see below) it will descend into the src-oss directory (see below) and make all of the QA test programs and dynamic PCP archives, so some patience may be required. If test 000 fails, it may be that you have locally developed PMDAs or optional PMDAs installed. Edit common.filter, and modify the _filter_top_pmns() procedure to strip the top-level name components for any new metric names (there are lots of examples already there) ... if these are distributed PMDAs, you should send patches back to pcp@oss.sgi.com. Doing the Real Work ------------------- check ... This script runs tests and verifies the output (in general, test NNN is expected to produce the output found in the file NNN.out ... failures leave the current output in NNN.out.bad) The command line options to check are: NNN run test NNN (leading zeros will be added as necessary to the test sequence number, so 00N and N are equivalent) NNN- all tests >= NNN NNN-MMM all tests in the range NNN ... MMM -l diffs in line mode (the default is to use xdiff or similar) -n show me, do not run any tests -q quick mode, by-pass the initial setup integrity checks (recommended that you do not use this the first time, nor if the last run test failed) -g xxx include tests from a named group (xxx) ... refer to the "groups" file -x xxx exclude tests from a named group (xxx) ... refer to the "groups" file If none of the NNN variants or -g is specified, then the default is to run all tests. remake NNN This script creates a new NNN.out file. Since the NNN.out files are precious, and reflect the state of the qualified and expected output, they should typically not be changed unless some change has been made to the NNN script or the filters it uses. new Make sure "group" and "owner" are writeable, then run "new" to create the skeletal framework of a new test. It is strongly suggested that you base your test on an existing test ... pay particular attention to making the output deterministic so the test uses the "not run" protocols (see 009 and check for examples) to avoid running the test (and hence failing) if an optional application, feature or platform is not available, and uses appropriate filters (see common.filter for lots of useful filters already packaged as shell procedures). show-me ... Report differences between the NNN.out and NNN.out.bad files. By default, uses all of the NNN.out.bad files in the current directory, but can also specify test numbers or ranges of test numbers on the command line. Other options may be used to fetch good and bad output files from various exotic remote locations (refer to the script). Make in the src-oss Directory ----------------------------- The src-oss directory contains a number of test applications that are designed to exercise some of the more exotic corners of the PCP functionality. In making these applications, you may see this ... Error: trace_dev.h and ../../src/include/trace_dev.h are different! make: [trace_dev.h] Error 1 (ignored) this is caused by the source for the pcp_trace library being out of sync with the src-oss applications. If this happens, please ... 1. cd src-oss 2. diff -u trace_dev.h ../../src/include/trace_dev.h and mail the differences to pcp@oss.sgi.com so we can refine the Makefiles to avoid cosmetic differences 3. mv trace_dev.h trace_dev.h.orig cp ../../src/include/trace_dev.h trace_dev.h 4. make cisco PMDA Issues ----------------- The original PCP QA required the cisco PMDA to be installed as a pre-condition to running any tests. The issue here is that not everyone has access to a Cisco and/or the necessary permissions to telnet to a Cisco router and extract per-interface traffic metrics. The current dependencies on the installation of the cisco PMDA are as follows: test 159 Exercises the installation and removal of the cisco PMDA. test 212 Is a regression test for a problem in an earlier version of the cisco PMDA. there are other references to parts of the cisco PMDA configuration files in other places, but none of these assume the PMDA is actually installed and running. If you are unable to install the cisco PMDA, then the workarounds are: 1. don't run tests 159 and 212, or 2. edit the "owner" file to comment out the lines for tests 159 and 212, or 3. alway run tests using -x pmda.cisco with the "check" script (this will exclude all of the pmda.cisco group, which is somewhat larger than just tests 159 and 212). 008 Issues ---------- Test 008 depends on the local disk configuration, so you need to make your own 008.out file (or rather a variant that 008 will link to 008.out when the test is run). Refer to the 008 script, but here is the basic recipe: $ touch touch 008.out.`hostname` $ remake 008 $ mv 008.out 008.out.`hostname` Fixes ----- If you find something that does not work, and fix it, or create additional QA tests, please send the details to pcp@oss.sgi.com. From todd.c.davis@intel.com Fri Nov 8 12:37:58 2002 Received: with ECARTIS (v1.0.0; list pcp); Fri, 08 Nov 2002 12:38:05 -0800 (PST) Received: from momus.sc.intel.com (momus.sc.intel.com [143.183.152.8]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gA8KbwuR018872 for ; Fri, 8 Nov 2002 12:37:58 -0800 Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by momus.sc.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.48 2002/10/16 23:47:34 dmccart Exp $) with SMTP id gA8Kco018389 for ; Fri, 8 Nov 2002 20:39:00 GMT Received: from FMSMSX018.fm.intel.com ([132.233.42.197]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002110812385815680 for ; Fri, 08 Nov 2002 12:38:58 -0800 Received: by fmsmsx018.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 8 Nov 2002 12:38:35 -0800 Message-ID: <29AD895CE780D511A8870002A50A666D04F9089E@hdsmsx106.hd.intel.com> From: "Davis, Todd C" To: pcp@oss.sgi.com Subject: RedHat 8.0 build Date: Fri, 8 Nov 2002 12:45:37 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 46 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: todd.c.davis@intel.com Precedence: bulk X-list: pcp I had to make the following changes to build pcp-2.2.2-11 RPMs in RedHat 8.0: In file ./configure.in add lines dnl check if the rpmbuild program is available test -z "$RPMBUILD" && AC_PATH_PROG(RPMBUILD, rpmbuild, /usr/bin/rpmbuild) rpmbuild=$RPMBUILD AC_SUBST(rpmbuild) before the line dnl check if the rpm program is available In file ./src/include/builddefs.in add line RPMBUILD = @rpmbuild@ In file ./build/rpm/GNUmakefile change $(RPM) -ba --rcfile rpm-$(RPM_VERSION).rc \ to $(RPMBUILD) -ba --rcfile rpm-$(RPM_VERSION).rc \ RedHat has discontinued the rpm -ba option so the rpmbuild utility must be used. I believe these changes will work for previous versions of RedHat since rmp -ba was using rpmbuild. Todd C. Davis These are my opinions and absolutely not official opinions of Intel Corp. Telco Systems Development Intel Corporation, Columbia Design Center CBA-2, Suite 100 250 Berry Hill Road Columbia, SC 29210 (803) 461-6108 fax:: (803) 461-6292 mailto:todd.c.davis@intel.com From markgw@sgi.com Sun Nov 10 18:28:02 2002 Received: with ECARTIS (v1.0.0; list pcp); Sun, 10 Nov 2002 18:28:04 -0800 (PST) Received: from deliverator.sgi.com (deliverator.SGI.COM [204.94.214.10] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAB2S2uR013762 for ; Sun, 10 Nov 2002 18:28:02 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by deliverator.sgi.com (980327.SGI.8.8.8-aspam/980304.SGI-aspam: SGI does not authorize the use of its proprietary systems or networks for unsolicited or bulk email from the Internet.) via SMTP id SAA05268 for ; Sun, 10 Nov 2002 18:29:24 -0800 (PST) mail_from (markgw@sgi.com) Received: from sherman.melbourne.sgi.com (sherman.melbourne.sgi.com [134.14.55.232]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id NAA27301; Mon, 11 Nov 2002 13:28:06 +1100 Date: Mon, 11 Nov 2002 13:28:05 +1100 (EST) From: Mark Goodwin X-X-Sender: markgw@sherman.melbourne.sgi.com To: "Davis, Todd C" cc: pcp@oss.sgi.com Subject: Re: RedHat 8.0 build In-Reply-To: <29AD895CE780D511A8870002A50A666D04F9089E@hdsmsx106.hd.intel.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 47 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: markgw@sgi.com Precedence: bulk X-list: pcp On Fri, 8 Nov 2002, Davis, Todd C wrote: > I had to make the following changes to build pcp-2.2.2-11 RPMs in RedHat > 8.0: > ... thanks, these changes graciously accepted. > RedHat has discontinued the rpm -ba option so the rpmbuild utility must be > used. I believe these changes will work for previous versions of RedHat > since rmp -ba was using rpmbuild. > yes this seems to be OK. I tested all the way back to rpm v3.0.5 on RH6.2. -- Mark From markgw@sgi.com Mon Nov 11 22:20:48 2002 Received: with ECARTIS (v1.0.0; list pcp); Mon, 11 Nov 2002 22:20:50 -0800 (PST) Received: from deliverator.sgi.com (deliverator.SGI.COM [204.94.214.10] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAC6KmuR025629 for ; Mon, 11 Nov 2002 22:20:48 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by deliverator.sgi.com (980327.SGI.8.8.8-aspam/980304.SGI-aspam: SGI does not authorize the use of its proprietary systems or networks for unsolicited or bulk email from the Internet.) via SMTP id WAA00256 for ; Mon, 11 Nov 2002 22:22:20 -0800 (PST) mail_from (markgw@sgi.com) Received: from sherman.melbourne.sgi.com (sherman.melbourne.sgi.com [134.14.55.232]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id RAA10327 for ; Tue, 12 Nov 2002 17:21:04 +1100 Date: Tue, 12 Nov 2002 17:21:04 +1100 (EST) From: Mark Goodwin X-X-Sender: markgw@sherman.melbourne.sgi.com To: pcp@oss.sgi.com Subject: [announce] PCP 2.3.0-pre available for testing Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 48 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: markgw@sgi.com Precedence: bulk X-list: pcp SGI is pleased to announce a pre-release version 2.3.0 of Performance Co-Pilot (PCP) open source is now available for download from ftp://oss.sgi.com/projects/pcp/download/dev/pcp-2.3.0.src.tar.gz For those who contributed, please test and review. This is a pre-release version for testing, source code only. The final 2.3.0 release will be next week - please see the CHANGELOG file in the top-level of the src. Unpack the tarball and build RPMs : # tar xvzf pcp-2.3.0.src.tar.gz # cd pcp-2.3.0 # ./Makepkgs thanks -- Mark Goodwin SGI Engineering. From todd.c.davis@intel.com Tue Nov 12 07:10:26 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 12 Nov 2002 07:10:28 -0800 (PST) Received: from momus.sc.intel.com (momus.sc.intel.com [143.183.152.8]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gACFALuR021315 for ; Tue, 12 Nov 2002 07:10:21 -0800 Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128]) by momus.sc.intel.com (8.11.6/8.11.6/d: solo.mc,v 1.48 2002/10/16 23:47:34 dmccart Exp $) with SMTP id gACFBUn07696 for ; Tue, 12 Nov 2002 15:11:40 GMT Received: from FMSMSX018.fm.intel.com ([132.233.42.197]) by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002111207114723187 ; Tue, 12 Nov 2002 07:11:47 -0800 Received: by fmsmsx018.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 12 Nov 2002 07:11:20 -0800 Message-ID: <29AD895CE780D511A8870002A50A666D04F908A4@hdsmsx106.hd.intel.com> From: "Davis, Todd C" To: "'Mark Goodwin'" , pcp@oss.sgi.com Subject: RE: [announce] PCP 2.3.0-pre available for testing Date: Tue, 12 Nov 2002 07:18:25 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 49 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: todd.c.davis@intel.com Precedence: bulk X-list: pcp The fix for man2html and khelpcenter to the man pages that Ken McDonell developed is not included in pcp-2.3.0-5. Ken changed .ie \(.g .TH PMINFO 1 "SGI" "Performance Co-Pilot" .el \{\ To .ie \(.g \{\ .\" ... groff ... assume Linux (hack for khelpcenter, man2html, etc.) .TH PMINFO 1 "SGI" "Performance Co-Pilot" \} .el \{\ .\" ... not groff, so likely to be IRIX Todd C. Davis These are my opinions and absolutely not official opinions of Intel Corp. -----Original Message----- From: Mark Goodwin [mailto:markgw@sgi.com] Sent: Tuesday, November 12, 2002 1:21 AM To: pcp@oss.sgi.com Subject: [announce] PCP 2.3.0-pre available for testing SGI is pleased to announce a pre-release version 2.3.0 of Performance Co-Pilot (PCP) open source is now available for download from ftp://oss.sgi.com/projects/pcp/download/dev/pcp-2.3.0.src.tar.gz For those who contributed, please test and review. This is a pre-release version for testing, source code only. The final 2.3.0 release will be next week - please see the CHANGELOG file in the top-level of the src. Unpack the tarball and build RPMs : # tar xvzf pcp-2.3.0.src.tar.gz # cd pcp-2.3.0 # ./Makepkgs thanks -- Mark Goodwin SGI Engineering. From kenmcd@melbourne.sgi.com Tue Nov 12 15:31:43 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 12 Nov 2002 15:31:45 -0800 (PST) Received: from deliverator.sgi.com (deliverator.SGI.COM [204.94.214.10] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gACNVguR021370 for ; Tue, 12 Nov 2002 15:31:43 -0800 Received: from rattle.melbourne.sgi.com (rattle.melbourne.sgi.com [134.14.55.145]) by deliverator.sgi.com (980327.SGI.8.8.8-aspam/980304.SGI-aspam: SGI does not authorize the use of its proprietary systems or networks for unsolicited or bulk email from the Internet.) via ESMTP id PAA06695 for ; Tue, 12 Nov 2002 15:33:17 -0800 (PST) mail_from (kenmcd@melbourne.sgi.com) Received: from localhost (kenmcd@localhost) by rattle.melbourne.sgi.com (SGI-8.9.3/8.9.3) with ESMTP id KAA54712; Wed, 13 Nov 2002 10:31:59 +1100 (EST) X-Authentication-Warning: rattle.melbourne.sgi.com: kenmcd owned process doing -bs Date: Wed, 13 Nov 2002 10:31:58 +1100 From: Ken McDonell To: "Davis, Todd C" cc: "'Mark Goodwin'" , Subject: RE: [announce] PCP 2.3.0-pre available for testing In-Reply-To: <29AD895CE780D511A8870002A50A666D04F908A4@hdsmsx106.hd.intel.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 50 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: kenmcd@melbourne.sgi.com Precedence: bulk X-list: pcp On Tue, 12 Nov 2002, Davis, Todd C wrote: > The fix for man2html and khelpcenter to the man pages that Ken McDonell > developed is not included in pcp-2.3.0-5. > > Ken changed > > .ie \(.g .TH PMINFO 1 "SGI" "Performance Co-Pilot" > .el \{\ > > To > > .ie \(.g \{\ > .\" ... groff ... assume Linux (hack for khelpcenter, man2html, etc.) > .TH PMINFO 1 "SGI" "Performance Co-Pilot" > \} > .el \{\ > .\" ... not groff, so likely to be IRIX OK, by all of the rules of troff (since 1975 if I recall correctly) these two are forms have _exactly_ the same semantics. If khelpcenter, man2html, etc. can process the second form, but not the first they are even more broken than I had first suspected. The verbose variant will appear in the next packaging from Mark. From markgw@sgi.com Tue Nov 12 20:03:49 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 12 Nov 2002 20:03:51 -0800 (PST) Received: from rj.sgi.com (rj.SGI.COM [192.82.208.96]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAD43nuR009546 for ; Tue, 12 Nov 2002 20:03:49 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by rj.sgi.com (8.12.2/8.12.2/linux-outbound_gateway-1.2) with SMTP id gAD25MG8021823 for ; Tue, 12 Nov 2002 18:05:23 -0800 Received: from sherman.melbourne.sgi.com (sherman.melbourne.sgi.com [134.14.55.232]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id PAA20788; Wed, 13 Nov 2002 15:03:59 +1100 Date: Wed, 13 Nov 2002 15:03:59 +1100 (EST) From: Mark Goodwin X-X-Sender: markgw@sherman.melbourne.sgi.com To: "Davis, Todd C" cc: "'Mark Goodwin'" , Subject: RE: [announce] PCP 2.3.0-pre available for testing In-Reply-To: <29AD895CE780D511A8870002A50A666D04F908A4@hdsmsx106.hd.intel.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 51 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: markgw@sgi.com Precedence: bulk X-list: pcp On Tue, 12 Nov 2002, Davis, Todd C wrote: > The fix for man2html and khelpcenter to the man pages that Ken McDonell > developed is not included in pcp-2.3.0-5. A new src tarball is now available (including Ken's fix for the man pages) from ftp://oss.sgi.com/projects/pcp/download/dev/pcp-2.3.0.src.tar.gz Please test, review and provide feed-back if you can before the main release next week. Unpack the tarball and build RPMs : # tar xvzf pcp-2.3.0.src.tar.gz # cd pcp-2.3.0 # ./Makepkgs thanks -- Mark Goodwin SGI Engineering. From todd.c.davis@intel.com Wed Nov 13 07:44:48 2002 Received: with ECARTIS (v1.0.0; list pcp); Wed, 13 Nov 2002 07:44:53 -0800 (PST) Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gADFiluR026680 for ; Wed, 13 Nov 2002 07:44:48 -0800 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gADFg5C07556 for ; Wed, 13 Nov 2002 15:42:05 GMT Received: from fmsmsxvs041.fm.intel.com (fmsmsxvs041.fm.intel.com [132.233.42.126]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gADFgJm27633 for ; Wed, 13 Nov 2002 15:42:19 GMT Received: from fmsmsx28.fm.intel.com ([132.233.42.28]) by fmsmsxvs041.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002111307445411460 ; Wed, 13 Nov 2002 07:44:54 -0800 Received: by fmsmsx28.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Wed, 13 Nov 2002 07:46:25 -0800 Message-ID: <29AD895CE780D511A8870002A50A666D04F908A8@hdsmsx106.hd.intel.com> From: "Davis, Todd C" To: "'Mark Goodwin'" , "Davis, Todd C" Cc: pcp@oss.sgi.com Subject: RE: [announce] PCP 2.3.0-pre available for testing Date: Wed, 13 Nov 2002 07:53:26 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 52 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: todd.c.davis@intel.com Precedence: bulk X-list: pcp These man pages work well in khelpcenter. Todd C. Davis These are my opinions and absolutely not official opinions of Intel Corp. -----Original Message----- From: Mark Goodwin [mailto:markgw@sgi.com] Sent: Tuesday, November 12, 2002 11:04 PM To: Davis, Todd C Cc: 'Mark Goodwin'; pcp@oss.sgi.com Subject: RE: [announce] PCP 2.3.0-pre available for testing On Tue, 12 Nov 2002, Davis, Todd C wrote: > The fix for man2html and khelpcenter to the man pages that Ken McDonell > developed is not included in pcp-2.3.0-5. A new src tarball is now available (including Ken's fix for the man pages) from ftp://oss.sgi.com/projects/pcp/download/dev/pcp-2.3.0.src.tar.gz Please test, review and provide feed-back if you can before the main release next week. Unpack the tarball and build RPMs : # tar xvzf pcp-2.3.0.src.tar.gz # cd pcp-2.3.0 # ./Makepkgs thanks -- Mark Goodwin SGI Engineering. From mmlnx@us.ibm.com Wed Nov 13 17:41:01 2002 Received: with ECARTIS (v1.0.0; list pcp); Wed, 13 Nov 2002 17:41:03 -0800 (PST) Received: from e33.co.us.ibm.com (e33.co.us.ibm.com [32.97.110.131]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAE1f1uR020123 for ; Wed, 13 Nov 2002 17:41:01 -0800 Received: from westrelay01.boulder.ibm.com (westrelay01.boulder.ibm.com [9.17.194.22]) by e33.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id gAE1gfSe022420 for ; Wed, 13 Nov 2002 20:42:41 -0500 Received: from us.ibm.com (dyn9-47-18-222.beaverton.ibm.com [9.47.18.222]) by westrelay01.boulder.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gAE1geQU260386 for ; Wed, 13 Nov 2002 18:42:41 -0700 Message-ID: <3DD2FF8E.7080506@us.ibm.com> Date: Wed, 13 Nov 2002 17:42:38 -0800 From: Mike Mason User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4.1) Gecko/20020314 Netscape6/6.2.2 X-Accept-Language: en-us MIME-Version: 1.0 To: pcp@oss.sgi.com Subject: rpmbuild problem References: <29AD895CE780D511A8870002A50A666D04F9089E@hdsmsx106.hd.intel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 53 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: mmlnx@us.ibm.com Precedence: bulk X-list: pcp I tried building pcp-2.3.0 on the United Linux 1.0 release candidate and found there is no rpmbuild, only rpm. Here's a patch that modifies Todd's changes so rpmbuild is used if available, otherwise rpm is used. Mike diff -Naur pcp-2.3.0.2/build/rpm/GNUmakefile pcp-2.3.0.1/build/rpm/GNUmakefile --- pcp-2.3.0.2/build/rpm/GNUmakefile 2002-11-12 19:41:39.000000000 -0800 +++ pcp-2.3.0.1/build/rpm/GNUmakefile 2002-11-13 17:00:08.000000000 -0800 @@ -49,7 +49,7 @@ # Blow that spec away after rpm has finished its work - it IS # going to make sure it'll be rebuild next time we come here pack_pcp : rpm-$(RPM_VERSION).rc $(SPEC) - $(RPMBUILD) -ba --rcfile rpm-$(RPM_VERSION).rc \ + $(RPMPROG) -ba --rcfile rpm-$(RPM_VERSION).rc \ --target $(TARGET_CPU)-$(TARGET_VENDOR)-$(TARGET_OS) \ $(SPEC) diff -Naur pcp-2.3.0.2/configure.in pcp-2.3.0.1/configure.in --- pcp-2.3.0.2/configure.in 2002-11-12 19:39:45.000000000 -0800 +++ pcp-2.3.0.1/configure.in 2002-11-13 17:03:23.000000000 -0800 @@ -202,6 +202,15 @@ rpm=$RPM AC_SUBST(rpm) +dnl if rpmbuild exists, use it, otherwise use rpm +if test -x $RPMBUILD +then + rpmprog=$RPMBUILD +else + rpmprog=$RPM +fi +AC_SUBST(rpmprog) + dnl .. and what version is rpm if test -x $RPM then diff -Naur pcp-2.3.0.2/src/include/builddefs.in pcp-2.3.0.1/src/include/builddefs.in --- pcp-2.3.0.2/src/include/builddefs.in 2002-11-12 19:39:47.000000000 -0800 +++ pcp-2.3.0.1/src/include/builddefs.in 2002-11-13 17:04:47.000000000 -0800 @@ -111,6 +111,7 @@ BZIP2 = @bzip2@ TAR = @tar@ RPMBUILD= @rpmbuild@ +RPMPROG = @rpmprog@ RPM = @rpm@ RPM_VERSION = @rpm_version@ GENDIST = @gendist@ -- Mike Mason IBM Linux Technology Center, RAS Group Beaverton, OR, USA mmlnx@us.ibm.com (503) 578-4123 From markgw@sgi.com Wed Nov 13 18:02:02 2002 Received: with ECARTIS (v1.0.0; list pcp); Wed, 13 Nov 2002 18:02:04 -0800 (PST) Received: from zok.sgi.com (zok.SGI.COM [204.94.215.101]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAE222uR021181 for ; Wed, 13 Nov 2002 18:02:02 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by zok.sgi.com (8.12.2/8.12.2/linux-outbound_gateway-1.2) with SMTP id gAE173Kp007235 for ; Wed, 13 Nov 2002 17:07:04 -0800 Received: from sherman.melbourne.sgi.com (sherman.melbourne.sgi.com [134.14.55.232]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id NAA01381; Thu, 14 Nov 2002 13:02:18 +1100 Date: Thu, 14 Nov 2002 13:02:18 +1100 (EST) From: Mark Goodwin X-X-Sender: markgw@sherman.melbourne.sgi.com To: Mike Mason cc: pcp@oss.sgi.com Subject: Re: rpmbuild problem In-Reply-To: <3DD2FF8E.7080506@us.ibm.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 54 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: markgw@sgi.com Precedence: bulk X-list: pcp On Wed, 13 Nov 2002, Mike Mason wrote: > I tried building pcp-2.3.0 on the United Linux 1.0 release candidate > and found there is no rpmbuild, only rpm. Here's a patch that > modifies Todd's changes so rpmbuild is used if available, otherwise > rpm is used. thanks Mike, I'll roll that in. Any other problems show up? I changed your ksyms stuff around a fair bit: added the _end symbol check, etc. Thanks -- Mark From mmlnx@us.ibm.com Thu Nov 14 08:19:30 2002 Received: with ECARTIS (v1.0.0; list pcp); Thu, 14 Nov 2002 08:19:32 -0800 (PST) Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAEGJUuR029203 for ; Thu, 14 Nov 2002 08:19:30 -0800 Received: from westrelay01.boulder.ibm.com (westrelay01.boulder.ibm.com [9.17.194.22]) by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id gAEGLDN7015826; Thu, 14 Nov 2002 11:21:13 -0500 Received: from us.ibm.com (sig-9-65-58-63.mts.ibm.com [9.65.58.63]) by westrelay01.boulder.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gAEGJtCT111716; Thu, 14 Nov 2002 09:19:55 -0700 Message-ID: <3DD3CD2C.7060104@us.ibm.com> Date: Thu, 14 Nov 2002 08:19:56 -0800 From: Mike Mason User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mark Goodwin CC: pcp@oss.sgi.com Subject: Re: rpmbuild problem References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 55 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: mmlnx@us.ibm.com Precedence: bulk X-list: pcp Yes, in fact, I am seeing another problem. pmcd sometimes core dumps when I start it on UL. I haven't had time to look into it yet, but I'll try to today. Mark Goodwin wrote: > On Wed, 13 Nov 2002, Mike Mason wrote: > > >>I tried building pcp-2.3.0 on the United Linux 1.0 release candidate >>and found there is no rpmbuild, only rpm. Here's a patch that >>modifies Todd's changes so rpmbuild is used if available, otherwise >>rpm is used. > > > thanks Mike, I'll roll that in. Any other problems show up? I changed > your ksyms stuff around a fair bit: added the _end symbol check, etc. > > Thanks > -- Mark > > -- Mike Mason IBM Linux Technology Center, RAS Group Beaverton, OR, USA mmlnx@us.ibm.com (503) 578-4123 From todd.c.davis@intel.com Thu Nov 14 14:44:10 2002 Received: with ECARTIS (v1.0.0; list pcp); Thu, 14 Nov 2002 14:44:14 -0800 (PST) Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAEMiAuR005886 for ; Thu, 14 Nov 2002 14:44:10 -0800 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gAEMfVB29718 for ; Thu, 14 Nov 2002 22:41:31 GMT Received: from fmsmsxv040-1.fm.intel.com (fmsmsxvs040.fm.intel.com [132.233.42.124]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gAEMmRb03484 for ; Thu, 14 Nov 2002 22:48:27 GMT Received: from FMSMSX016.fm.intel.com ([132.233.42.195]) by fmsmsxv040-1.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002111414461800778 ; Thu, 14 Nov 2002 14:46:18 -0800 Received: by fmsmsx016.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 14 Nov 2002 14:45:48 -0800 Message-ID: <29AD895CE780D511A8870002A50A666D04F908AA@hdsmsx106.hd.intel.com> From: "Davis, Todd C" To: "'Mike Mason'" , Mark Goodwin Cc: pcp@oss.sgi.com Subject: RE: rpmbuild problem Date: Thu, 14 Nov 2002 14:52:49 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 56 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: todd.c.davis@intel.com Precedence: bulk X-list: pcp pmcd dumps core everytime on TurboLinux 8.0. "Unexpected signal 11 ..." Todd C. Davis These are my opinions and absolutely not official opinions of Intel Corp. -----Original Message----- From: Mike Mason [mailto:mmlnx@us.ibm.com] Sent: Thursday, November 14, 2002 11:20 AM To: Mark Goodwin Cc: pcp@oss.sgi.com Subject: Re: rpmbuild problem Yes, in fact, I am seeing another problem. pmcd sometimes core dumps when I start it on UL. I haven't had time to look into it yet, but I'll try to today. Mark Goodwin wrote: > On Wed, 13 Nov 2002, Mike Mason wrote: > > >>I tried building pcp-2.3.0 on the United Linux 1.0 release candidate >>and found there is no rpmbuild, only rpm. Here's a patch that >>modifies Todd's changes so rpmbuild is used if available, otherwise >>rpm is used. > > > thanks Mike, I'll roll that in. Any other problems show up? I changed > your ksyms stuff around a fair bit: added the _end symbol check, etc. > > Thanks > -- Mark > > -- Mike Mason IBM Linux Technology Center, RAS Group Beaverton, OR, USA mmlnx@us.ibm.com (503) 578-4123 From markgw@sgi.com Thu Nov 14 14:55:11 2002 Received: with ECARTIS (v1.0.0; list pcp); Thu, 14 Nov 2002 14:55:15 -0800 (PST) Received: from tolkor.sgi.com (tolkor.sgi.com [198.149.18.6]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAEMt5uR006658 for ; Thu, 14 Nov 2002 14:55:06 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by tolkor.sgi.com (8.12.2/8.12.2/linux-outbound_gateway-1.2) with SMTP id gAEN0Xkq004568 for ; Thu, 14 Nov 2002 17:00:34 -0600 Received: from sherman.melbourne.sgi.com (sherman.melbourne.sgi.com [134.14.55.232]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id JAA01104; Fri, 15 Nov 2002 09:55:24 +1100 Date: Fri, 15 Nov 2002 09:55:24 +1100 (EST) From: Mark Goodwin X-X-Sender: markgw@sherman.melbourne.sgi.com To: "Davis, Todd C" cc: "'Mike Mason'" , Subject: RE: rpmbuild problem In-Reply-To: <29AD895CE780D511A8870002A50A666D04F908AA@hdsmsx106.hd.intel.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 57 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: markgw@sgi.com Precedence: bulk X-list: pcp On Thu, 14 Nov 2002, Davis, Todd C wrote: > pmcd dumps core everytime on TurboLinux 8.0. "Unexpected signal 11 ..." I don't have turbo installed any where handy. Can you send me a stack traceback or can I access a turbo machine the net anywhere? You can start pmcd manually: # /usr/share/pcp/bin/pmcd -f or even better, in a gdb session. thanks -- Mark From todd.c.davis@intel.com Thu Nov 14 14:56:16 2002 Received: with ECARTIS (v1.0.0; list pcp); Thu, 14 Nov 2002 14:56:17 -0800 (PST) Received: from caduceus.fm.intel.com (fmr02.intel.com [192.55.52.25]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAEMuGuR006671 for ; Thu, 14 Nov 2002 14:56:16 -0800 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by caduceus.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gAEMrbi08164 for ; Thu, 14 Nov 2002 22:53:37 GMT Received: from fmsmsxvs041.fm.intel.com (fmsmsxvs041.fm.intel.com [132.233.42.126]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gAEMrr623260 for ; Thu, 14 Nov 2002 22:53:53 GMT Received: from FMSMSX017.fm.intel.com ([132.233.42.196]) by fmsmsxvs041.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002111414562809838 ; Thu, 14 Nov 2002 14:56:28 -0800 Received: by fmsmsx017.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 14 Nov 2002 14:57:59 -0800 Message-ID: <29AD895CE780D511A8870002A50A666D04F908AB@hdsmsx106.hd.intel.com> From: "Davis, Todd C" To: "'Mike Mason'" , Mark Goodwin Cc: pcp@oss.sgi.com Subject: RE: rpmbuild problem Date: Thu, 14 Nov 2002 15:04:57 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 58 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: todd.c.davis@intel.com Precedence: bulk X-list: pcp Here's the stack if this helps anyone: (gdb) bt #0 0x4009e881 in __kill () at __kill:-1 #1 0x4009e555 in raise () from /lib/libc.so.6 #2 0x4009fced in abort () from /lib/libc.so.6 #3 0x0804cfbc in SigBad () #4 #5 0x400eeb55 in strncpy () from /lib/libc.so.6 #6 0x401b346e in read_ksyms (end_addr=0xbfffe7a0) at ksym.c:258 #7 0x401b3e8c in read_ksym_sources () at ksym.c:479 #8 0x401a91a2 in linux_init (dp=0x8064488) at pmda.c:4005 #9 0x08051a5e in PrintAgentInfo () #10 0x08051ce3 in PrintAgentInfo () #11 0x08052039 in ParseInitAgents () #12 0x0804d2ba in main () #13 0x4008d3d8 in __libc_start_main () from /lib/libc.so.6 (gdb) Todd C. Davis These are my opinions and absolutely not official opinions of Intel Corp. -----Original Message----- From: Davis, Todd C [mailto:todd.c.davis@intel.com] Sent: Thursday, November 14, 2002 5:53 PM To: 'Mike Mason'; Mark Goodwin Cc: pcp@oss.sgi.com Subject: RE: rpmbuild problem pmcd dumps core everytime on TurboLinux 8.0. "Unexpected signal 11 ..." Todd C. Davis These are my opinions and absolutely not official opinions of Intel Corp. -----Original Message----- From: Mike Mason [mailto:mmlnx@us.ibm.com] Sent: Thursday, November 14, 2002 11:20 AM To: Mark Goodwin Cc: pcp@oss.sgi.com Subject: Re: rpmbuild problem Yes, in fact, I am seeing another problem. pmcd sometimes core dumps when I start it on UL. I haven't had time to look into it yet, but I'll try to today. Mark Goodwin wrote: > On Wed, 13 Nov 2002, Mike Mason wrote: > > >>I tried building pcp-2.3.0 on the United Linux 1.0 release candidate >>and found there is no rpmbuild, only rpm. Here's a patch that >>modifies Todd's changes so rpmbuild is used if available, otherwise >>rpm is used. > > > thanks Mike, I'll roll that in. Any other problems show up? I changed > your ksyms stuff around a fair bit: added the _end symbol check, etc. > > Thanks > -- Mark > > -- Mike Mason IBM Linux Technology Center, RAS Group Beaverton, OR, USA mmlnx@us.ibm.com (503) 578-4123 From acahalan@saturn.cs.uml.edu Tue Nov 26 02:26:21 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 26 Nov 2002 02:26:26 -0800 (PST) Received: from saturn.cs.uml.edu (saturn.cs.uml.edu [129.63.8.2]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAQAQJuR025696 for ; Tue, 26 Nov 2002 02:26:20 -0800 Received: (from acahalan@localhost) by saturn.cs.uml.edu (8.11.6/8.11.6) id gAQASqG293056; Tue, 26 Nov 2002 05:28:52 -0500 (EST) Date: Tue, 26 Nov 2002 05:28:52 -0500 (EST) Message-Id: <200211261028.gAQASqG293056@saturn.cs.uml.edu> From: "Albert D. Cahalan" To: pcp@oss.sgi.com Cc: markgw@sgi.com, todd.c.davis@intel.com, mmlnx@us.ibm.com, kenmcd@melbourne.sgi.com Subject: Re: PCP patch for top/libgtop conversion X-archive-position: 59 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: acahalan@cs.uml.edu Precedence: bulk X-list: pcp Interesting thread I found on a web archive... I'm making some libproc changes, so it's not a bad time to consider what the API should look like. Please Cc: me on any response. > The copyout problem ... I think there remains a systemic Linux issue > with any /proc "file" that is longer than your page size du jour ... > if the underlying data is subject to change, there is no way a user > space app (cat(1) is the simplest example) is guaranteed to see a > consistent view ... you may miss whole lines, you may see some lines > more than once. > > If this assertion is not correct, I'd like to understand why. You're 1/3 right. The app won't get a perfect snapshot of the data, but it won't get missing or duplicate lines either. Reading X bytes moves the file pointer by Y and Y>=X. For example, the file pointer may be encoded as: 5 bits character-within-a-field (the least significant bits) 3 bits field number 9 bits line number Plus any part of the data (one field, one line, etc.) could be kept for the reader so that 9999 changing to 10000 doesn't ever look like 99000 to the reader. > The changes to top and libgtop to use PCP are essentially done, but I > don't yet have a way to release them. We go through a formal process at > IBM to get permission to contribute to outside projects. I don't have > permission to contribute to procps or libgtop. I'll work on that. Be glad you have such a process. I don't have much of an idea what your project is doing, but it seems that you pre-parse /proc files and then ship them over the net to a hacked-up libproc. I wonder what you expect to do about machine differences: page size, clock tick rate, word size... Also, do you cache stuff in a local daemon? Do you have an async interface? I'm interested in suggestions for a public libproc API. Right now the interface is totally volatile. Some part of that needs to get cleaned up for non-procps use. Let me know if there's an existing API that you'd like to have. Warning: the kernel interface is more volatile than it looks; you should consider using libproc on the data-collection side once libproc exports something you can rely on. For those that don't know, procps-3.1.1 is out now. http://procps.sf.net/ From kaos@sgi.com Tue Nov 26 14:22:06 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 26 Nov 2002 14:22:10 -0800 (PST) Received: from mail.ocs.com.au (mail.ocs.com.au [203.34.97.2]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAQMM3uR003856 for ; Tue, 26 Nov 2002 14:22:04 -0800 Received: (qmail 19640 invoked from network); 26 Nov 2002 22:24:37 -0000 Received: from ocs3.intra.ocs.com.au (192.168.255.3) by mail.ocs.com.au with SMTP; 26 Nov 2002 22:24:37 -0000 Received: by ocs3.intra.ocs.com.au (Postfix, from userid 16331) id C9D073000B8; Wed, 27 Nov 2002 09:24:32 +1100 (EST) Received: from ocs3.intra.ocs.com.au (localhost [127.0.0.1]) by ocs3.intra.ocs.com.au (Postfix) with ESMTP id A19D985; Wed, 27 Nov 2002 09:24:32 +1100 (EST) X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 From: Keith Owens To: "Albert D. Cahalan" Cc: pcp@oss.sgi.com, todd.c.davis@intel.com, mmlnx@us.ibm.com Subject: Re: PCP patch for top/libgtop conversion In-reply-to: Your message of "Tue, 26 Nov 2002 05:28:52 CDT." <200211261028.gAQASqG293056@saturn.cs.uml.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 27 Nov 2002 09:24:27 +1100 Message-ID: <27293.1038349467@ocs3.intra.ocs.com.au> X-archive-position: 60 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: kaos@sgi.com Precedence: bulk X-list: pcp On Tue, 26 Nov 2002 05:28:52 -0500 (EST), "Albert D. Cahalan" wrote: >Interesting thread I found on a web archive... >I'm making some libproc changes, so it's not a >bad time to consider what the API should look like. >Please Cc: me on any response. > >> The copyout problem ... I think there remains a systemic Linux issue >> with any /proc "file" that is longer than your page size du jour ... >> if the underlying data is subject to change, there is no way a user >> space app (cat(1) is the simplest example) is guaranteed to see a >> consistent view ... you may miss whole lines, you may see some lines >> more than once. >> >> If this assertion is not correct, I'd like to understand why. > >You're 1/3 right. The app won't get a perfect snapshot of the >data, but it won't get missing or duplicate lines either. >Reading X bytes moves the file pointer by Y and Y>=X. >For example, the file pointer may be encoded as: > >5 bits character-within-a-field (the least significant bits) >3 bits field number >9 bits line number > >Plus any part of the data (one field, one line, etc.) could >be kept for the reader so that 9999 changing to 10000 doesn't >ever look like 99000 to the reader. I disagree, any /proc file that reflects a kernel list structure and requires more than one read to get the data from /proc is subject to missing and/or duplicate entries. The problem is the lack of space in the "file pointer". On 64 bit machines, there is not enough room to store an actual kernel address in the file pointer. Even on 32 bit machines, there is no easy way to validate any kernel address passed out to user space and back in again. Even worse, you cannot guarantee that the list entry address stored in the "file pointer" is not deleted between reads. And it is guaranteed that storing addresses will result in a file pointer that does not go in ascending order. Instead of using addresses, people store an index in the file pointer and use the index to reposition in the kernel list for the next read. To guarantee no duplicate or missing lines, the second and subsequent reads must pick up where the previous read left off. What happens when an entry is deleted from the list between read calls? The next read will try to restablish position by counting through the list until it reaches the index position. But the list now contains one less entry, if the deletion is prior to the index entry then one extra entry will be skipped during repositioning and will be missing from the /proc output. What happens when an entry is added to the list between read calls? If the underlying kernel structure supports insertions anywhere in the list then the repositioning count can see the extra list entry and will reposition one entry short of where it should be. This results in duplicate lines in the /proc output. Unless all deletes and inserts are at the end of the list, repositioning a /proc file via a counter will always be dubious. From acahalan@saturn.cs.uml.edu Tue Nov 26 15:25:57 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 26 Nov 2002 15:25:59 -0800 (PST) Received: from saturn.cs.uml.edu (saturn.cs.uml.edu [129.63.8.2]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAQNPtuR004291 for ; Tue, 26 Nov 2002 15:25:56 -0800 Received: (from acahalan@localhost) by saturn.cs.uml.edu (8.11.6/8.11.6) id gAQNSIu358662; Tue, 26 Nov 2002 18:28:18 -0500 (EST) From: "Albert D. Cahalan" Message-Id: <200211262328.gAQNSIu358662@saturn.cs.uml.edu> Subject: Re: PCP patch for top/libgtop conversion To: kaos@sgi.com (Keith Owens) Date: Tue, 26 Nov 2002 18:28:17 -0500 (EST) Cc: acahalan@cs.uml.edu (Albert D. Cahalan), pcp@oss.sgi.com, todd.c.davis@intel.com, mmlnx@us.ibm.com In-Reply-To: <27293.1038349467@ocs3.intra.ocs.com.au> from "Keith Owens" at Nov 27, 2002 09:24:27 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 61 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: acahalan@cs.uml.edu Precedence: bulk X-list: pcp Keith Owens writes: > "Albert D. Cahalan" wrote: >> Interesting thread I found on a web archive... >> I'm making some libproc changes, so it's not a >> bad time to consider what the API should look like. >> Please Cc: me on any response. >> >>> The copyout problem ... I think there remains a systemic Linux issue >>> with any /proc "file" that is longer than your page size du jour ... >>> if the underlying data is subject to change, there is no way a user >>> space app (cat(1) is the simplest example) is guaranteed to see a >>> consistent view ... you may miss whole lines, you may see some lines >>> more than once. >>> >>> If this assertion is not correct, I'd like to understand why. >> >> You're 1/3 right. The app won't get a perfect snapshot of the >> data, but it won't get missing or duplicate lines either. >> Reading X bytes moves the file pointer by Y and Y>=X. >> For example, the file pointer may be encoded as: >> >> 5 bits character-within-a-field (the least significant bits) >> 3 bits field number >> 9 bits line number >> >> Plus any part of the data (one field, one line, etc.) could >> be kept for the reader so that 9999 changing to 10000 doesn't >> ever look like 99000 to the reader. > > I disagree, any /proc file that reflects a kernel list structure and > requires more than one read to get the data from /proc is subject to > missing and/or duplicate entries. > > The problem is the lack of space in the "file pointer". On 64 bit The kernel has a per-fd struct that can hold a bit more data. Results certainly may depend on implementation details of the proc file being read. It's been a damn long time since procps was commonly hurt by this problem when reading files. I never seek to any position except the beginning, so the kernel can cache all sorts of per-fd stuff to help out. Don't worry. Reading the /proc directory itself is another matter. Due to glibc defining an oversize directory entry structure, simply reading the /proc directory will cause seeks. It's been quite some time since I've heard complaints about this one though, so maybe it has been fixed somehow. (new system call maybe?) In any case, it's not a common problem. From kaos@sgi.com Tue Nov 26 17:05:08 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 26 Nov 2002 17:05:12 -0800 (PST) Received: from rj.sgi.com (rj.SGI.COM [192.82.208.96]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAR158uR006617 for ; Tue, 26 Nov 2002 17:05:08 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by rj.sgi.com (8.12.2/8.12.2/linux-outbound_gateway-1.2) with SMTP id gAQN7gG8011376 for ; Tue, 26 Nov 2002 15:07:43 -0800 Received: from kao2.melbourne.sgi.com (kao2.melbourne.sgi.com [134.14.55.180]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id MAA16025 for ; Wed, 27 Nov 2002 12:06:23 +1100 From: kaos@sgi.com Received: by kao2.melbourne.sgi.com (Postfix, from userid 16331) id B70D23000B8; Wed, 27 Nov 2002 12:06:22 +1100 (EST) Received: from kao2.melbourne.sgi.com (localhost [127.0.0.1]) by kao2.melbourne.sgi.com (Postfix) with ESMTP id 9B4F285 for ; Wed, 27 Nov 2002 12:06:22 +1100 (EST) X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 Subject: Re: PCP patch for top/libgtop conversion To: pcp@oss.sgi.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 27 Nov 2002 12:06:17 +1100 Message-ID: <30541.1038359177@kao2.melbourne.sgi.com> X-archive-position: 62 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: kaos@sgi.com Precedence: bulk X-list: pcp Albert D. Cahalan wrote: >Keith Owens wrote: >> I disagree, any /proc file that reflects a kernel list structure and >> requires more than one read to get the data from /proc is subject to >> missing and/or duplicate entries. >> >> The problem is the lack of space in the "file pointer". On 64 bit > > The kernel has a per-fd struct that can hold a bit more data. > Results certainly may depend on implementation details of the > proc file being read. It's been a damn long time since procps > was commonly hurt by this problem when reading files. Most /proc code that generates more than a page of data uses seq_file. That implementation saves a position index which is passed to the seq_operations->start() routine on the next read. Every such routine in 2.4.19 uses the index as a counter into a list, i.e. all the seq_file code is vulnerable to this race. You will only see the race when the kernel list is modifified between two reads from user space _and_ that modification is earlier in the list than the current "position" in the /proc file. Even then, the missing or duplicate entry is transient, close and reopen the /proc file and the entries will be normal (assuming there are no more kernel changes). The race is a low probability and only appears as noise in the /proc output, but the race is there. From kaos@sgi.com Tue Nov 26 17:09:39 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 26 Nov 2002 17:09:41 -0800 (PST) Received: from tolkor.sgi.com ([198.149.18.6]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAR19XuR006696 for ; Tue, 26 Nov 2002 17:09:33 -0800 Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by tolkor.sgi.com (8.12.2/8.12.2/linux-outbound_gateway-1.2) with SMTP id gAR1Gbkq020661 for ; Tue, 26 Nov 2002 19:16:38 -0600 Received: from kao2.melbourne.sgi.com (kao2.melbourne.sgi.com [134.14.55.180]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id MAA16054; Wed, 27 Nov 2002 12:10:42 +1100 Received: by kao2.melbourne.sgi.com (Postfix, from userid 16331) id 5FA863000B8; Wed, 27 Nov 2002 12:10:42 +1100 (EST) Received: from kao2.melbourne.sgi.com (localhost [127.0.0.1]) by kao2.melbourne.sgi.com (Postfix) with ESMTP id 478A885; Wed, 27 Nov 2002 12:10:42 +1100 (EST) X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 From: Keith Owens To: "Albert D. Cahalan" Cc: pcp@oss.sgi.com, todd.c.davis@intel.com, mmlnx@us.ibm.com Subject: Re: PCP patch for top/libgtop conversion In-reply-to: Your message of "Tue, 26 Nov 2002 20:00:35 CDT." <200211270100.gAR10ZU443901@saturn.cs.uml.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 27 Nov 2002 12:10:37 +1100 Message-ID: <30577.1038359437@kao2.melbourne.sgi.com> X-archive-position: 63 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: kaos@sgi.com Precedence: bulk X-list: pcp On Tue, 26 Nov 2002 20:00:35 -0500 (EST), "Albert D. Cahalan" wrote: >Keith Owens wrote: >> Most /proc code that generates more than a page of data uses seq_file. >> That implementation saves a position index which is passed to the >> seq_operations->start() routine on the next read. Every such routine >> in 2.4.19 uses the index as a counter into a list, i.e. all the >> seq_file code is vulnerable to this race. > >It would be dumb for /proc/*/maps to do this. Each line has >an address that may be used to find back the position. > >For /proc/*/cmdline and /proc/*/environ as well, there is >an address that should be used. > >The stat, statm, and status files are tiny. You are concentrating on just the files used by procps, but PCP has to worry about other /proc files. Small files whose data fits in a single page are not a problem, larger /proc files have races. From acahalan@saturn.cs.uml.edu Tue Nov 26 17:15:02 2002 Received: with ECARTIS (v1.0.0; list pcp); Tue, 26 Nov 2002 17:15:03 -0800 (PST) Received: from saturn.cs.uml.edu (saturn.cs.uml.edu [129.63.8.2]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gAR1F1uR006841 for ; Tue, 26 Nov 2002 17:15:01 -0800 Received: (from acahalan@localhost) by saturn.cs.uml.edu (8.11.6/8.11.6) id gAR1HZ7444665; Tue, 26 Nov 2002 20:17:35 -0500 (EST) From: "Albert D. Cahalan" Message-Id: <200211270117.gAR1HZ7444665@saturn.cs.uml.edu> Subject: Re: PCP patch for top/libgtop conversion To: kaos@sgi.com (Keith Owens) Date: Tue, 26 Nov 2002 20:17:35 -0500 (EST) Cc: acahalan@cs.uml.edu (Albert D. Cahalan), pcp@oss.sgi.com, todd.c.davis@intel.com, mmlnx@us.ibm.com In-Reply-To: <30577.1038359437@kao2.melbourne.sgi.com> from "Keith Owens" at Nov 27, 2002 12:10:37 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 64 X-ecartis-version: Ecartis v1.0.0 Sender: pcp-bounce@oss.sgi.com Errors-to: pcp-bounce@oss.sgi.com X-original-sender: acahalan@cs.uml.edu Precedence: bulk X-list: pcp >>> Most /proc code that generates more than a page of data uses seq_file. >>> That implementation saves a position index which is passed to the >>> seq_operations->start() routine on the next read. Every such routine >>> in 2.4.19 uses the index as a counter into a list, i.e. all the >>> seq_file code is vulnerable to this race. >> >> It would be dumb for /proc/*/maps to do this. Each line has >> an address that may be used to find back the position. ... > You are concentrating on just the files used by procps, but PCP has to > worry about other /proc files. Small files whose data fits in a single > page are not a problem, larger /proc files have races. It was /proc/*/maps that was originally mentioned.