pcp
[Top] [All Lists]

Re: Development trees - kmchart and PCP

To: Michael Newton <kimbrr@xxxxxxx>
Subject: Re: Development trees - kmchart and PCP
From: Nathan Scott <nscott@xxxxxxxxxx>
Date: Fri, 15 Jun 2007 16:04:13 +1000
Cc: pcp@xxxxxxxxxxx
In-reply-to: <1181886404.3758.243.camel@edge.yarra.acx>
Organization: Aconex
References: <1181886404.3758.243.camel@edge.yarra.acx>
Reply-to: nscott@xxxxxxxxxx
Sender: pcp-bounce@xxxxxxxxxxx
Hi Michael,

On Fri, 2007-06-15 at 15:46 +1000, Nathan Scott wrote:
> 
> git://oss.sgi.com:8090/nathans/pcp.git
>     Tree has two branches - "master" which is unmodified PCP code from
>     the 'official' SGI PCP tarballs, and "nathans" which is all of the
>     fixes and features from my working tree.  This includes the kmtime
>     extensions to pmval, and all the fixes that aren't yet in mainline
>     PCP releases.

This tree will be a much saner way for you to pick up PCP patches
from me (and others here) ... I'm planning to just send a note when
I update this tree, instead of patch-bombing you as I've done in
the past, so you can pull from it at your leisure.

Below is the current set of patches in the "nathans" branch.  The git
docs describe how to extract a patch from git in a form that should
be easy to drop into your ptools tree(s) - see my example at the end.

cheers.

--
Nathan


15:53 nathans@edge ~ 1> cd /source/git/pcp
15:54 nathans@edge /source/git/pcp 2> git branch
* master
  nathans
  origin
15:54 nathans@edge /source/git/pcp 3> git log
commit 0d122added4cb56af881cc943bf2d2f26c160d1e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:47:08 2007 +1000

    Initial commit of SGI 2.7.1-1 PCP source.
15:54 nathans@edge /source/git/pcp 4> git checkout nathans
15:54 nathans@edge /source/git/pcp 5> git branch
  master
* nathans
  origin
15:54 nathans@edge /source/git/pcp 6> git log
commit 65946bf7339338963e0b468dd1608c2d479a8d75
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Append current date string to build number

commit 57be02a88757b81d2b488da1b9dc553dd6f6668e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Update pmie start and stop scripts

    Incorporate bugfixes and platform independence changes from the
    pmlogger_check script into pmie_check.

commit 56dd951709324ae0786ef35d7c16321185120a44
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    pmlogsummary report sum option

    Allow pmloggsummary to report the sum() of metrics/instances,
    in addition to all of the other statistics reported.

commit 9635c7f5f4eec7f3694f82bb3d399ab0be8eff24
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix Windows uname metrics

    Somehow, this chunk of an earlier Windows PMDA patch was missed,
    and theres no code behind the pmda.* metrics there at the moment.

commit a3a12b249675f6ad9efcb0e57e083d2ca7fd62f9
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix PCP start script regressions

    Partially revert a change to the pcp start script, so that it will
    attempt to shutdown running pmcd processes under all circumstances.
    Also ensure the $PCP_RUN_DIR exists, else pmcd fails to start from
    this script on Windows at least.

commit 8e7f8da4e316087772db5d45edd122353ce7289a
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    pmval kmtime support

    Introduce libkmtime, which provides a kmtime client interface for
    C/C++ programs using a similar protocol to that used by pmtime.
    This library contains no GUI code (adds no additional dependencies
    to PCP - used POSIX calls only).  It provides time sychronisation
    for PCP tools that support it.  Support has been added to pmval
    and this co-exists with its existing pmtime support.

    The kmtime tool is entirely separate, and is packaged and released
    separately to PCP itself (it does depend on certain GUI libraries,
    of course).  An updated version of Ken's kmchart utility, which now
    uses kmtime, is also being made generally available.

commit 462af4362ec3df582960daba675408acec838304
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Generate LSM file

    The version number update was missed in the old LSM file.  This
    patch makes the LSM (Linux Software Map) file configure-generated
    so that its stamped with the correct version number, date, etc,
    without any manual intervention.  Also updated PCP to use version
    4 of of the LSM format (incl. differently named file).

commit c643892bb0f8be3d64a39e13933dcf24d14197d4
Author: David Chatterton <dchatterton@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Upgrade Windows compiler version

    Switch to newer compiler version for native Windows components.

commit aabb75f1f9a5a735c5675697ba7a81cf9cbdfc34
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Revert syslog for Cygwin

    Cygwin build fails with this recent addition, so reverting it.
    Possibly a missing (not yet installed?) header on the machine
    where the problem that lead to this change was observed?

commit af6398aef276f250782017e3eb5b110b7219a42f
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Windows uuencoded binaries

    Add in new uuencoded shim and show-all-counters binary files,
    to make merging updates simpler (ascii text is patch-able).

commit 1500bd088898317c42eede0f0748f1fd09989c69
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Additional Linux SNMP metrics

    Support for additional network.udp and network.udplite metrics,
    which are exported through the /proc/net/snmp kernel interface.

commit bb8a668bcb72283a994b85316e3de96d2f07a2ed
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix Linux vmstat nr_slab metrics

    Update the Linux PMDA vmstat metrics, after changes and additions
    to these counters in 2.6.18 virtual memory subsystem.

commit cf618856627c2354566f2c7d50daf3feecc98034
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Make pcp status command report build version

    Update the pcp(1) command to report build version.  Add in the
    missing help text for pmcd.build as well.

commit 329aada114b0dd0cc5798ecaf23cb027b2447daa
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Windows split_io metrics

    Add support for the per-device and aggregate Windows "Split IO"
    performance counters.

commit a058a8446b208a30583873689414b9666ade354a
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix pmdapmcd empty pmie instance

    Fix the handling of the pmie instance domain, such that when pmie
    instances are stopped, the pmcd PMDA instance domain is correctly
    updated in all cases.

commit 30be08b73af760067f13c928dbfd88e8a4c99caf
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    pmdamailq filename regex

    Allow the mailq agent to be used for MTAs other than sendmail,
    by providing a way for regular expressions to be used to match
    mail message files instead of hard-coding the sendmail naming
    convention.

commit b6d58c31f4845963fccf14616ef9ed6e59a5e25b
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix Windows filesys.used metric

    Fix an error in the Windows PMDA filesystem freespace calculation.

commit ecab101bea0fc58ff43103d0e32c38dd34254202
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Quiet libpcp_pmda with appversion error code

    PM_ERR_APPVERSION is a reasonable value for a PMDA fetch callback
    to return - this change stops libpcp_pmda from filling such agents
    logs with warnings when this is used, unless in debug mode.

commit 8bf3c03805c97b3db0faf93c693af1a6f65b141e
Author: David Chatterton <dchatterton@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Windows DDK unresolved symbol fix

    Fix compilation of parts of the Windows PMDA with newer versions
    of the Windows C compiler (and DDK).

commit f40b51dbeb9e93dcd82aab1d861f1683fded86fe
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Windows TCP metrics

    Add support for the network.tcp metric hierarchy to the Windows
    agent (naming convention matches those used on other platforms).

commit fa5665062ea327e84453d78b97f10cf598d5a7f3
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix Linux context switch metric value

    Fix a cut-and-paste error in the value reported by the Linux PMDA
    for context switches.

commit 8d5c06982c523c25d5339d2f22fe190798c7c60e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix filesys.free calculation to use b_avail

    Most filesystems reserve an amount of space that can only be
    allocated by the root user, once out-of-space conditions are
    reached.  This is reflected by differences in the statfs(2)
    b_avail and b_free values.

    This patch makes PCP filesystem metric values match up with
    reality according to df(1).

commit c8d743924d0ee89f6566c5335efe09b70bfa101e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix objstyle magic file checks

    Add another place to search for magic files, which fixes some
    build issues on certain recent Linux distributions.  Also fix
    a shell error from some versions of sh(1) where whitespace is
    required before a tests closing square bracket.

commit 95395026d5e47cb23d138d5cd25bd32a07b9a7a5
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    pmie log file rotation

    Allow pmie logs to be rotated as part of the regular daily PCP
    house-keeping scripts (introducing a pmie_daily to do this).

    The pmie "print" action, when used in conjunction with other
    actions for regular alarming, can provide an invaluable audit
    trail of historical rule firings.  However, in production one
    cannot have unbounded log growth - so, this implements rotation,
    compression, and culling in a similar way to the pmlogger log
    management.

    At the same time I fixed several small issues that were found
    while reviewing the pmlogger daily archive maintenance scripts.
    I also changed the default compression program to be bzip2(1),
    from compress (which doesn't seem to exist on any PCP platforms
    other than IRIX).

commit 6d489840bede03c31ecb571c5593afe2bfab8d8a
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix pmie %sample calculations

    Resolve a rounding problem in the pmie percent-{sample,host,inst}
    expressions.  The percent calculations could result in values which
    should have been zero, but were very small positive/negative values.
    End result was rules not evaluating to true when they should have.

commit c323229fe46a2d550f420664b0d9082f87e7140c
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Convert instant to discrete in numerous metrics

    When using the pmlogger log-once directive for discrete metrics, it
    helps alot if they actually are exported as discrete metrics.  When
    they are not, the client tools are unable to operate correctly on
    the daily archives produced by pmlogger_daily, as interpolation is
    attempted (but cannot be done with just the one value).

    This fixes a fistful of kernel metrics for Linux, Mac and Windows
    that exhibited this problem.

commit ef7ac2472b53f760e8cc5710b5d9b2e16b238480
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Replace sginap with nanosleep

    Remove use of sginap from several tools.  The sginap interface is
    terrible for our needs in PCP - its argument is a number-of-jiffies
    which means all tools using it have to multiply their time-to-sleep
    by CLK_TCK at each call site.  The non-IRIX implementations all then
    promptly divide by CLK_TCK to get back to the parameter which they
    pass to the underlying sleep routine.

    Worse, since CLK_TCK is actually a large number, when using a pmie
    rule with a relatively large delta (1 hour), the multiplication also
    overflows, and incorrect values are reported.

    This patch changes pmie and pmval (which were the tools I was using
    at the time) to use the POSIX nanosleep(2) interface.  Probably this
    patch should go further and remove each and every reference to
sginap
    from the PCP source, but I've not done that at this stage.  That may
    also result in some autoconf cleanup, to rid us of all the usleep /
    sleep / what-have-you platform-specific checks.  AFAICT, nanosleep
    seems to exist on every supported platform.

commit 28a73b5100b09f75d45b28b2369d21cc040ce427
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix pmie constant folding

    Fix a bug when pmie evaluates rules that make multiple references
    to a single constant, where it was incorrectly folding a constant
    after the first use.  Corruption of pmie's address space results,
    when subsequent rules make use of the folded constant expression,
    which has been observed as causing SIGSEGVs, infinite loops, and
    other random wierdness.

commit c82841863880e9d6149c91368f97f855f060900d
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix Linux proc metric units

    Update the metric units for several Linux PMDA counter metrics in
the
    proc hierarchy.

commit 1deedfaafd53cc021e882bad23b996eeef4e04d5
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:32:06 2007 +1000

    Fix pmie stomp action in archive mode

    Ensure related functions are kept together in pmie, this convention
    was accidentally dropped earlier.  Also ensure we don't act on stomp
    actions in archive mode (as is done for other action types).

commit 0d122added4cb56af881cc943bf2d2f26c160d1e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Thu Jun 14 10:47:08 2007 +1000

    Initial commit of SGI 2.7.1-1 PCP source.
15:54 nathans@edge /source/git/pcp 7> 
15:55 nathans@edge /source/git/pcp 7> git diff
1deedfaafd53cc021e882bad23b996eeef4e04d5..c82841863880e9d6149c91368f97f855f060900d
diff --git a/src/pmdas/linux/pmda.c b/src/pmdas/linux/pmda.c
index 61ad5fd..227cb1d 100644
--- a/src/pmdas/linux/pmda.c
+++ b/src/pmdas/linux/pmda.c
@@ -1163,22 +1163,22 @@ static pmdaMetric metrictab[] = {
 /* proc.psinfo.minflt */
   { NULL,
     { PMDA_PMID(CLUSTER_PID_STAT,9), PM_TYPE_U32, PROC_INDOM,
PM_SEM_COUNTER,
-    PMDA_PMUNITS(0,0,0,0,0,0) } },
+    PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) } },

 /* proc.psinfo.cmin_flt */
   { NULL,
     { PMDA_PMID(CLUSTER_PID_STAT,10), PM_TYPE_U32, PROC_INDOM,
PM_SEM_COUNTER,
-    PMDA_PMUNITS(0,0,0,0,0,0) } },
+    PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) } },

 /* proc.psinfo.maj_flt */
   { NULL,
     { PMDA_PMID(CLUSTER_PID_STAT,11), PM_TYPE_U32, PROC_INDOM,
PM_SEM_COUNTER,
-    PMDA_PMUNITS(0,0,0,0,0,0) } },
+    PMDA_PMUNITS(0,0,1,0,0,PM_COUNT_ONE) } },

15:55 nathans@edge /source/git/pcp 8> 



<Prev in Thread] Current Thread [Next in Thread>