pcp
[Top] [All Lists]

pcp updates: merges (kenj,fche,lberk,suruchi) + small freebsd build, pmd

To: PCP <pcp@xxxxxxxxxxx>
Subject: pcp updates: merges (kenj,fche,lberk,suruchi) + small freebsd build, pmdaroot tweaks
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Thu, 8 Oct 2015 23:28:07 -0400 (EDT)
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <820503962.51633523.1444361128229.JavaMail.zimbra@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: 8ta/J9f7OirNOo/VwBuiBFsQ4bkglw==
Thread-topic: pcp updates: merges (kenj,fche,lberk,suruchi) + small freebsd build, pmdaroot tweaks
Changes committed to git://git.pcp.io/pcp.git master

Ken McDonell (9):
      qa/519: tweak filtering after recent pmie change
      qa/622: updated output after pmie report format change
      qa/583: tweak filtering after recent pmie change
      qa/294: track recent pmie changes
      src/pmie/show.c: small changes afer FQDN changes
      pmie man page: minor wordsmithing around (new) %c format item
      qa/admin/check-vm: report correct Debian package for JSON Perl module
      linux pmda: add Intel copyright to proc_partions.c
      qa/869: (new) check NVME disk names for archive from linux pmda

Frank Ch. Eigler (8):
      pmmgr: subtarget-containers support
      pmmgr: subtarget-containers qa/668
      pmmgr: lengthen subtarget-containers qa/668
      pmmgr subtarget-containers: disable by default
      pmmgr container pmie qa: track expected number of pmie.log files
      PR1105: fqdn -h support for pmie
      pmie-fqdn qa/983: pmie -h test
      pmie-fqdn: support "%c" substitution in action strings

Nathan Scott (4):
      pmmgr PR1105: enable pmie for subtarget-containers
      pmdaroot: add a metric to aid mapping containers to their cgroups
      build: FreeBSD build fix for 10.2 with dtrace probes auto-enabled
      pmmgr: remove unhelpful comments about libpcp and containers

Lukas Berk (1):
      Modify qa/812 to use hostzone options (fixes buildbot timezone based 
failures)

Suruchi Kadu (1):
      linux pmda: add support for NVME devices


 man/man1/pmie.1                   |   27 
 man/man1/pmmgr.1                  |   24 
 qa/158.out.2                      |    2 
 qa/294.out                        |   32 
 qa/318.out                        |    2 
 qa/321.out                        |    2 
 qa/356.out                        |    4 
 qa/378.out                        |   14 
 qa/433.out                        |    2 
 qa/479.out.2                      |    4 
 qa/501-linux.out                  |  114 -
 qa/514.out.3                      |    2 
 qa/519                            |   16 
 qa/549.out                        |    2 
 qa/556.out                        |    2 
 qa/557.out                        |    4 
 qa/558.out.irix                   |   11 
 qa/558.out.linux                  |   11 
 qa/581.out                        |   16 
 qa/583                            |   26 
 qa/583.out                        |    6 
 qa/622.out                        |    2 
 qa/668                            |  412 ++++-
 qa/668.out                        |   22 
 qa/733.out.2                      |    2 
 qa/761.out                        |   18 
 qa/764.out                        |    4 
 qa/777.out                        |    2 
 qa/787.out                        |    2 
 qa/812                            |   24 
 qa/812.out                        |  893 +++++------
 qa/869                            |   26 
 qa/869.out                        | 2902 ++++++++++++++++++++++++++++++++++++++
 qa/922.out                        |   10 
 qa/951.out                        |    4 
 qa/983                            |  112 +
 qa/983.out                        |   49 
 qa/admin/check-vm                 |    2 
 qa/archives/simple-nvme.0         |binary
 qa/archives/simple-nvme.index     |binary
 qa/archives/simple-nvme.meta      |binary
 qa/common.filter                  |    1 
 qa/group                          |    4 
 src/libpcp_pmcd/src/GNUmakefile   |    7 
 src/pmdas/linux/proc_partitions.c |   16 
 src/pmdas/root/help               |    1 
 src/pmdas/root/root.c             |    5 
 src/pmdas/root/root.h             |    1 
 src/pmdas/root/root_root          |    1 
 src/pmie/src/dstruct.c            |   16 
 src/pmie/src/dstruct.h            |   13 
 src/pmie/src/eval.c               |   11 
 src/pmie/src/eval.h               |    3 
 src/pmie/src/fetch.sk             |    7 
 src/pmie/src/pmie.c               |   41 
 src/pmie/src/pragmatics.c         |  164 --
 src/pmie/src/pragmatics.h         |    5 
 src/pmie/src/show.c               |   83 -
 src/pmie/src/symbol.c             |    5 
 src/pmie/src/symbol.h             |    5 
 src/pmie/src/syntax.c             |   35 
 src/pmmgr/config/GNUmakefile      |   12 
 src/pmmgr/pmmgr.cxx               |  219 ++
 src/pmmgr/pmmgr.h                 |    2 
 64 files changed, 4590 insertions(+), 876 deletions(-)


commit 7c679d92cd3da8e166c58b0b807bc526035e1f9b
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri Oct 9 14:16:10 2015 +1100

    pmmgr: remove unhelpful comments about libpcp and containers
    
    If ever we support nested containers, its going to be pmdaroot
    that needs to handle that, not libpcp.  And in general, if you
    are adding comments to code: please don't express a personal /
    sarcastic opinion, and don't say "we should document X" (then
    proceed to not do so).  When someone does do these things, the
    comments will be out of date and need fixing too, but are quite
    likely to be forgotten since they're in unrelated code.

commit c67c67ef3f48211bc7d79d0d5def284fc5db00fd
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri Oct 9 13:53:02 2015 +1100

    build: FreeBSD build fix for 10.2 with dtrace probes auto-enabled
    
    It seems FreeBSD needs to use the Solaris mode of running dtrace
    helper tool - otherwise build failure results.

commit d3a01547e60855882efe8f11483fdcc089f370cf
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri Oct 9 13:50:57 2015 +1100

    pmdaroot: add a metric to aid mapping containers to their cgroups
    
    Client tools need a way to map from the instances of the cgroup.*
    metrics back to the containers for which those cgroups were built.

commit faf4c1b47912849e5a5b8233da58b9c2be760143
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Fri Oct 9 13:23:15 2015 +1100

    qa/869: (new) check NVME disk names for archive from linux pmda

commit edd49eb87fe107f11c2fe3a91e50ea75628f4848
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Fri Oct 9 13:20:20 2015 +1100

    linux pmda: add Intel copyright to proc_partions.c
    
    As per email from Doug Nelson (doug.nelson@xxxxxxxxx) ...
    
        The lawyer said we could use the same copyright line that everybody
        else was using, so here's a new patch.

commit 452ec707498fb828bc4c049cc578e52e70856808
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Fri Oct 9 10:17:22 2015 +1100

    qa/admin/check-vm: report correct Debian package for JSON Perl module

commit fc8f9b86e39bf29ebc073c427d2a8495ec8e1ebf
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Oct 8 20:05:07 2015 +1100

    pmie man page: minor wordsmithing around (new) %c format item

commit 3f9ebb0358790ac256ed18aeb8932979eed3b274
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Oct 8 20:03:51 2015 +1100

    src/pmie/show.c: small changes afer FQDN changes
    
    In showSyn() should report m->hname not m->hconn for CND_FETCH
    or CND_DELAY nodes in the expression tree to match the user's
    input rule (remove question in comment).
    
    Change type of findFormat() from int to char and add the default
    case for %? (action is to keep searching, which was the implied
    but is now explicit).
    
    In showSatisfyingValue() revert
        length = concat("\n<tab>   ", length, &string);
    to
        length = concat("\n<space>   ", length, &string);

commit d654b30fec41a1861d689c3c23ba38ff21b1f70c
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Oct 8 20:02:18 2015 +1100

    qa/294: track recent pmie changes
    
    The number of context connections and disconnections is different
    because there are now fewer pmNewContext() calls on the default
    context path.

commit 062fda7abed17e2699ac0f8b763426ab5acccb4e
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Oct 8 20:01:07 2015 +1100

    qa/583: tweak filtering after recent pmie change
    
    The hostname in the output lines are sensitive to DNS configuration on
    the local and remote hosts where pmcd is running ... they may or may
    not contain a FQDN ... adjust filter to accommodate both possibilities.

commit d19a4625acb8ae9f5b84529af2b7621d8535bca6
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Oct 8 10:53:16 2015 +1100

    qa/622: updated output after pmie report format change

commit 8214db66045487fdb39eedb7e4b925e6f1be74ea
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date:   Thu Oct 8 10:49:33 2015 +1100

    qa/519: tweak filtering after recent pmie change
    
    The hostname in the output lines are sensitive to DNS configuration
    on the remote hosts where pmcd is running ... they may or may not
    contain a FQDN ... adjust filter to accommodate both possibilities.

commit ec5e592e7c514a542602fcf53042613fcb046536
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sun Aug 23 11:55:33 2015 -0400

    pmie-fqdn: support "%c" substitution in action strings
    
    Add this peer to "%h" to expose the recently reified
    connection-string value.

commit 48ddab879fc5dd3c2350de16dc3288dadc9fac4a
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sun Aug 23 11:34:14 2015 -0400

    pmie-fqdn qa/983: pmie -h test
    
    New test for pmie fqdn functionality, to assert that it will only
    connect to a remote host via the -h $CONNSTRING rather than "host
    name".  Done by suppressing normal pmie; starting at some arbitrary
    high address, and for bonus marks also starting a socat relay proxy.
    A brief pmie job against these (and a control scenario) is run.

commit 547cb50dd0b739988c81858e9a5346e8caed31c2
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Fri Mar 7 21:35:36 2014 -0500

    PR1105: fqdn -h support for pmie
    
    pmie's use of host names as interchangeable with pmNewContext
    connection strings was deeply embedded.  The new fields in structs
    'metric' and 'host' track a connection string alongside the host name.
    The newContext() function is now solely responsible for looking up
    host names from archive or real-time pmcd connections, and updating
    the pmie mmv data with it.  Some other functions that used to take an
    ambiguous name parameter now document which, or else take both.
    
    pmie diagnostic messages try to more consistently print one or the
    other, requiring many trivial qa/FOO.out updates.
    
    In the man page, pmie's -H (don't do dns lookup) became undocumented,
    since it has long been unimplemented.  A little bit of clarification
    regarding host names vs connection strings was added.
    
    Some const qualifiers to char*'s have been added,
    out of a sense of unchanging constervativeness.

commit 30e4fe45b73715a6f7ede02fbaf6ff592fe97410
Author: Suruchi Kadu <suruchi.a.kadu@xxxxxxxxx>
Date:   Thu Oct 8 07:03:11 2015 +1100

    linux pmda: add support for NVME devices
    
    Patch from Intel.
    Handle NVME device partition names like nvme0n1p1 using the
    same recipe as the other SSD ones, "ram...", "mmcblk...", etc.

commit 5328a95b3f9dae7aa952f791736b044a565ea013
Author: Lukas Berk <lberk@xxxxxxxxxx>
Date:   Wed Oct 7 13:16:13 2015 -0400

    Modify qa/812 to use hostzone options (fixes buildbot timezone based 
failures)
    
    Buildbot tests were failing 812 due to having several hour differences
    in the dump/extract output.  Appending '-z' output fixes this, along
    with updating the output for the testcase.

commit 1f09b70af7d856eebd6d54451fbb211906a97fe2
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Oct 6 18:43:12 2015 -0400

    pmmgr container pmie qa: track expected number of pmie.log files

commit 93a76c70157a501f7d21c08766931b42b05484c1
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 6 17:09:50 2015 -0400

    pmmgr PR1105: enable pmie for subtarget-containers

commit be079febc8bfc6744e440f40be074272636c9eaf
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sun Apr 5 17:17:34 2015 -0400

    pmmgr subtarget-containers: disable by default
    
    Investigation of PR1106 indicates that pmcd is too fragile to use with
    containers safely, e.g., a pmda can get stuck within a container
    namespace, rendering further normal use of that pmda useless.  Until
    this is fixed, let's not supply /etc/pcp/pmmgr/subtarget-containers.

commit 5410ba6d2b4fd2f1d09048e3f3f4704c85d450b2
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Apr 1 21:05:21 2015 -0400

    pmmgr: lengthen subtarget-containers qa/668
    
    Further testing on a slower virtual machine shows situations where
    pmlogconf & docker run dramatically slower than on full workstations,
    whether due to bare-metalness, number of processors, presence of
    selinux bugs & troubleshooting thereof, etc.  This test case is
    made more conservative in terms of timing (3ish minutes) and more
    defensive of iffy docker situations.

commit f1b8bb9794b07c967925038c6c68e9463b2a931f
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Mar 31 21:15:23 2015 -0400

    pmmgr: subtarget-containers qa/668
    
    A relatively short (60-second) test that fires up a pmmgr only in
    order to attach pmloggers to a few *real live* docker containers.
    (That means the qa-invoking user probably needs to be in group
    'docker'; something checked by the qa script.)  The docker container
    being used is a tiny "busybox sleep XX" kind, so there should be
    only minimal disk/network consumption.
    
    The tests do not assert anything about the contents of the pcp
    archives resulting from the per-container attachment.  In my tests,
    they got apprx. zero actual content, other than proper pmcd.hostname.
    There ought to be a pmlogconf ditty for the container's own cgroup
    stats, and disk/network stuff should eventually be correct.

commit 4e0017d4ab85626fe22d0107cd0a8b75cbcf5953
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Mar 31 18:35:23 2015 -0400

    pmmgr: subtarget-containers support
    
    Add support for a config option "subtarget-containers", which
    implicitly forks further pmlogger (but not pmie: PR1105) instances for
    all running containers on an identified target host.  The host's
    host-id string gets a "--CONTAINER" code attached, and the connection
    hostspec gets a "container=XXXXX" appended with a ? or & suffix-prefix.

commit 583b350d98246f9082253f04b76b682985c0de55
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sun Apr 5 17:17:34 2015 -0400

    pmmgr subtarget-containers: disable by default
    
    Investigation of PR1106 indicates that pmcd is too fragile to use with
    containers safely, e.g., a pmda can get stuck within a container
    namespace, rendering further normal use of that pmda useless.  Until
    this is fixed, let's not supply /etc/pcp/pmmgr/subtarget-containers.

commit 35c490f3f6ca7c800a09d1903bfc2a9e03a83352
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Apr 1 21:05:21 2015 -0400

    pmmgr: lengthen subtarget-containers qa/668
    
    Further testing on a slower virtual machine shows situations where
    pmlogconf & docker run dramatically slower than on full workstations,
    whether due to bare-metalness, number of processors, presence of
    selinux bugs & troubleshooting thereof, etc.  This test case is
    made more conservative in terms of timing (3ish minutes) and more
    defensive of iffy docker situations.

commit d42afe606b02e2fa570b34a0313add8aa555619a
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Mar 31 21:15:23 2015 -0400

    pmmgr: subtarget-containers qa/668
    
    A relatively short (60-second) test that fires up a pmmgr only in
    order to attach pmloggers to a few *real live* docker containers.
    (That means the qa-invoking user probably needs to be in group
    'docker'; something checked by the qa script.)  The docker container
    being used is a tiny "busybox sleep XX" kind, so there should be
    only minimal disk/network consumption.
    
    The tests do not assert anything about the contents of the pcp
    archives resulting from the per-container attachment.  In my tests,
    they got apprx. zero actual content, other than proper pmcd.hostname.
    There ought to be a pmlogconf ditty for the container's own cgroup
    stats, and disk/network stuff should eventually be correct.

commit 8864f4e2d9824933d8d598e8361869d58815a2b8
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Mar 31 18:35:23 2015 -0400

    pmmgr: subtarget-containers support
    
    Add support for a config option "subtarget-containers", which
    implicitly forks further pmlogger (but not pmie: PR1105) instances for
    all running containers on an identified target host.  The host's
    host-id string gets a "--CONTAINER" code attached, and the connection
    hostspec gets a "container=XXXXX" appended with a ? or & suffix-prefix.

<Prev in Thread] Current Thread [Next in Thread>