pcp
[Top] [All Lists]

pcp updates: hostnames, qa

To: pcp@xxxxxxxxxxx
Subject: pcp updates: hostnames, qa
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue, 29 Oct 2013 04:01:43 -0400 (EDT)
Delivered-to: pcp@xxxxxxxxxxx
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: 1jXn7Rgiw1+gWOmr/oswA8sGPlGlPw==
Thread-topic: pcp updates: hostnames, qa
Changes committed to git://oss.sgi.com/pcp/pcp.git dev

 build/GNUmakefile              |    6 +
 build/rpm/fedora.spec          |    4 
 build/rpm/pcp.spec.in          |    8 -
 man/man1/pcp.1                 |    2 
 man/man1/pcpintro.1            |    2 
 man/man1/pmcd.1                |    7 -
 man/man1/pmhostname.1          |    8 +
 man/man1/pmlogconf.1           |    2 
 man/man1/pmtop.1               |    2 
 man/man1/pmwebd.1              |   25 +++---
 qa/.gitignore                  |    1 
 qa/044                         |    4 
 qa/115                         |    2 
 qa/151                         |    2 
 qa/204                         |    7 -
 qa/230                         |    2 
 qa/230.out                     |    2 
 qa/280                         |    2 
 qa/294                         |   17 ++--
 qa/294.out.1                   |   14 +--
 qa/294.out.2                   |   49 ++++++-----
 qa/294.out.3                   |   10 +-
 qa/294.out.4                   |   33 +++++--
 qa/295                         |   11 +-
 qa/295.out                     |   17 ----
 qa/295.out.1                   |   17 ++++
 qa/295.out.2                   |   17 ++++
 qa/297                         |    5 -
 qa/297.out                     |    6 -
 qa/338                         |    2 
 qa/365                         |   24 ++++-
 qa/430                         |    1 
 qa/456                         |   10 --
 qa/503                         |    6 +
 qa/504                         |    4 
 qa/529                         |    2 
 qa/530                         |    2 
 qa/532                         |    2 
 qa/581                         |    3 
 qa/645                         |    1 
 qa/651                         |    1 
 src/pcp/pcp.sh                 |    2 
 src/pmdas/pmcd/src/pmcd.c      |    8 +
 src/pmevent/doargs.c           |   36 ++++----
 src/pmevent/pmevent.c          |   10 --
 src/pmevent/pmevent.h          |    2 
 src/pmie/pmie_check.sh         |   98 ++++++++++-------------
 src/pmie/pmie_daily.sh         |   76 ++++++------------
 src/pmie/rc_pmie               |    9 --
 src/pmie/src/dstruct.c         |    8 -
 src/pmie/src/dstruct.h         |    4 
 src/pmie/src/pmie.c            |   88 +++++++++++++--------
 src/pmie/src/pragmatics.c      |   27 ++----
 src/pmie/src/pragmatics.h      |    3 
 src/pmie/src/syntax.c          |    3 
 src/pmlc/actions.c             |    2 
 src/pmlogger/control           |    4 
 src/pmlogger/pmlogger_check.sh |  170 ++++++++++++-----------------------------
 src/pmlogger/pmlogger_daily.sh |  131 ++++++-------------------------
 src/pmlogger/rc_pmlogger       |    4 
 src/pmlogger/src/dopdu.c       |   19 ++--
 src/pmlogger/src/logger.h      |    1 
 src/pmlogger/src/pmlogger.c    |   18 ++--
 63 files changed, 492 insertions(+), 573 deletions(-)

commit d7bb8c1abbe6dead2b94db083d1078dfdbd45b5e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 16:10:24 2013 +1100

    Resolve hostname issues and non-determinism in test qa/365
    
    The advent of commands being run as part of startup using the
    unix: specifier can cause this test to spuriously log messages
    due to the pmcd access control specification, after pmcd is up
    (which is what the test is really exercising - that pmcd comes
    up despite dodgey-ness in the access section).

commit aee9d444dac9dca572878d1527a7fb22ba1f65dc
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:39:54 2013 +1100

    Try harder in pmie to come up with a hostname for local connections
    
    In the default startup case (i.e. no host/archive options) should
    pmie startup with no pmcd running, its handling of hostname lookup
    has been improved so that it falls back to using a local-context.
    This is important because pmie can be very long-lived and we would
    really like it to get a more sensible host name than "?" if at all
    possible.  Once it establishes a context when pmcd eventually comes
    up, it may run for a very long time and cannot nonsense forever -
    not such an unusual scenario.  Possibly pmie should really be doing
    the pmcd.hostname check whenever it reconnects.
    
    While there, remove an unused variable in pmie initArchive() code,
    and make dfltConn local to the argument parsing code instead of a
    global.

commit 53df655c0d41511a467e669f3b1999187fb32f03
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:29:33 2013 +1100

    Update several QA test filters to deal with local: host default

commit b76e063964122238f788312c64f067f7f6248de8
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:27:41 2013 +1100

    Update pmhostname man page, describe it as a command instead of a script

commit 1dfcbcdffcf06e9738b61d4b963da534d340d119
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:26:32 2013 +1100

    Clarify pmcd man page section that discusses it running as root

commit e901fcb9a0ab06243bc5d79f03e7cc7b18815245
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:26:04 2013 +1100

    Use PCP_VAR_DIR in place of hard coded path in pcp.spec.in

commit 720b989f360effb7b750e64e744479155d73214e
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:22:18 2013 +1100

    Ensure build cleans up a temp file properly after a failure
    
    I observed a QA failure because an earlier build had left a
    temporary file around, named with a PID prefix, which just so
    happened to match a subsequent QA test PID.  Failure to unlink
    that file in QA cleanup caused the test to fail.

commit d2271f51782842250b0619fa72ac2920c5ce3d03
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:19:58 2013 +1100

    Update tests qa/29[45] to cope with changed pmie hostname handling
    
    We recently changed pmie to use pmGetContextHostName in order that
    it can better grok the remote hostname.  For hosts, we establish a
    temporary context early on to extract the correct hostname for log
    file reporting, the stats file, and so on.  This causes QA tests to
    fail if they are counting the number of pmcd connects/disconnects,
    as they are now off-by-one.
    
    Further, in the default case (i.e. no host/archive options) should
    pmie startup with no pmcd running, its handling of hostname lookup
    has been improved so that it falls back to using a local-context.
    This is important because pmie can be very long-lived and we would
    really like it to get a more sensible host name than "?" if at all
    possible.  Once its (re)connects a context when pmcd finally comes
    up, it may run for a very long time - not such an unusual scenario.

commit 16fdca8c54784fad8869e374f6f70b544753f731
Merge: 648fe74 49a7ce7
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Tue Oct 29 15:05:06 2013 +1100

    Merge branch 'fche/dev' of ../pcpfans into fche
    
    Conflicts:
        build/rpm/fedora.spec
        src/pmie/pmie_check.sh
        src/pmie/pmie_daily.sh
        src/pmlogger/pmlogger_check.sh
        src/pmlogger/pmlogger_daily.sh

commit 49a7ce795530c50de29f453fd8e9136369e59060
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Thu Oct 17 16:39:07 2013 -0400

    pmwebd: document security benefits of authenticated PMAPI contexts

commit fba80a45eaff5ee2fc590f9f5461cb4756e26537
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 16:55:53 2013 -0400

    pcpqa FQDN cleanup cont'd: test case 503

commit 3470e5ac35a89dcb1763cea4f3430337ba718b16
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 16:17:10 2013 -0400

    pcpqa FQDN cleanup cont'd: 297.* test outputs

commit 89ffc9ec98368013ce37912d1b9e62993c6809b7
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 16:12:14 2013 -0400

    pcpqa FQDN cleanup cont'd: 294.* test outputs

commit 7a7999bfbd21912ae22c14ca7e318748c3d31fe2
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 15:53:33 2013 -0400

    pcpqa FQDN cleanup cont'd

commit b65a57ab0a3591d02ae37764fcab02e99186ac9e
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 15:45:44 2013 -0400

    pcpqa FQDN cleanup: tolerate new relaxed hostnames in qa suite

commit 0cbcc9747f400a9caf40e4be9b1a20b79e4eac62
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 11:05:48 2013 -0400

    *.spec: for pcp-testsuite, put pcpqa homedir under own directory
    
    The default, /home, may not generally be writeable, so put the home
    directory under the testsuite work directory itself.

commit d35e1768c5d8c8af118e8fb871b658dba14cb8d5
Merge: 79044fb 8da1d1a
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 16 09:39:56 2013 -0400

    Merge remote-tracking branch 'pcp/dev' into fche/dev
    
    * pcp/dev:
      Fix a race condition in test qa/749 (pmcd probes)
      Handle absense of server unix domain socket file gracefully
      Address potential lock inversion issues in libpcp derived metrics

commit 79044fbee849ed664c04ab8d013d27d2a9ca3d81
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Oct 15 16:04:00 2013 -0400

    FQDN cleanup: fix merge snafu in pmlogger_check.sh

commit e9706dd290db120195390a2fea772d7fe3121a59
Merge: b34fde8 21be617
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Oct 15 14:46:15 2013 -0400

    Merge remote-tracking branch 'pcp/dev' into fche/dev
    
    * pcp/dev: (68 commits)
      Resolve recent failures observed in test qa/067
      Fix up a recent pmevent argument parsing regression
      Make pmdammv pmcd.conf updater makefile rules now match pmdapmcd
      Tighten notrun check on auth test 823, needs plain mech
      Give stap more time in test 749, 1 sec for startup proved racey
      Resolve daylight-savings related failure in test 527
      Improve test 444 handling of different hostname len
      Make test 438 deterministic on s390, where an extra I/O happens
      Tweak qa/177 log read filter, s390 platform makes one more
      Add Linux disk.partitions metrics to the default logged set
      Add Linux hinv.nlv DM metric, improve disk metrics help text
      Simplify test qa/364 to one output form.
      Tighten test 359 filter for kernels built without schedstats
      Allow default pmdaproc cgroup setting from the command line
      Add support for extracting client PID for unix sockets
      Alloc proc DSO PMDA builds - local context is fine
      Remove indeterminism from test 200 for initial connect
      Update test 652 output to match expected
      pmevent: use local: as default connection
      Start a new seq.full file between each run of test 652
      ...

commit b34fde81c0ec72e81985543499f8e016610188da
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Tue Oct 15 14:39:46 2013 -0400

    FQDN cleanup: pm{ie}_{check,daily}.sh now 100% pmhostname-free
    
    All these rather-similar scripts use the new PM{IE,LOGGER}_PID
    files to identify corresponding server processes for signalling
    or restarting.

commit 6ce75ca0a5eaca920d16e74df1ea0f125729f0f5
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sun Oct 13 16:43:10 2013 -0400

    FQDN cleanup: pm{logger,ie}_check tweaks continued
    
    Eradicating pmhostname-based searches for pmie/pmlogger
    instances, instead using a .pid file within the individual
    log subdirectories, allows much simplification of the code.
    
    Also, reinterpret chkconfig-off state to shut down any
    *_check-controlled pmlogger/pmie instances (not just the
    "primary" logger).

commit 9dd8090e1806f0e485eec24a7c033a6c6a46022a
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sun Oct 13 11:02:30 2013 -0400

    fedora.spec: make /var/log/pcp also pcp:pcp owned, for NOTICES go there

commit 1cc165f7b7f6d4d78e6a525c118ce21c3ed8500f
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sat Oct 12 12:29:37 2013 -0400

    FQDN cleanup: note pmhostname as deprecated in its man page

commit b533c54bb06f1248f7ce2d8740a37e1a9ff0f251
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sat Oct 12 12:15:30 2013 -0400

    FQDN cleanup: pcp.sh shan't call deprecated pmhostname

commit 50c8c8f678301ddf09867cea0a19a19ab534a51e
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sat Oct 12 12:05:56 2013 -0400

    FQDN cleanup: pmie local:ization
    
    Switch pmie sources to default to local: as the connection path.
    Fork the former dfltHost global into dfltHostConn (as connection path
    string) and dfltHostName (as pmContextGetName).

commit f9737888285026b2b9d06d957d773b8ebd853cb8
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Sat Oct 12 11:50:13 2013 -0400

    pmie FQDN cleanup: rc & _check files
    
    Use a logdirectory pmie.pid file to identify pmie instances running
    locally, not fuzzy notion of monitoring-target hostname.

commit 1f7de9967bc4272a7c6e8cdfe5b3215afdeee7ff
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Fri Oct 11 07:46:48 2013 -0400

    no change is good change

commit ad17a7cc26155af098db9e3f2a2dd7fdfc933553
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Fri Oct 11 07:45:11 2013 -0400

    nothing will ever change

commit 8feaca44e23f6fb1d43c8dec20aca7422970808e
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Fri Oct 11 07:40:57 2013 -0400

    nothing ever changes

commit 1f6f2410f4f0f5542bfabedaa2742bc67c3b5d51
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Fri Oct 11 07:33:37 2013 -0400

    nothing ever changes

commit 2c81e04f0263bfcc299715e47b1c4ed721ace74b
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Thu Oct 10 21:10:53 2013 -0400

    pmevent: use local: as default connection
    
    Change the global "host" to be the result of pmGetContextHostName
    after connection.

commit 96f0ade93a1b0ec0722a2bfe09f2ed2dd23300ad
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 9 14:53:40 2013 -0400

    pmlogger FQDN cleanup: use local: throughout, stop DNSification
    
    Remove use of DNS forward/backward mappings in pmlogger configuration
    and reporting, thereby separating access path (-h FOO parameter) vice
    actual target hostname.
    
    * src/pmlogger/control: Note peculiar treatment of LOCALHOSTNAME.
    * src/pmlogger/pmlogger_check.sh: Rework host name lookup.  Force
      primary logger to use implied "-h local:".  Use full hostname(2)
      for PCP_LOG_DIR subdir.
    * src/pmlogger/rc_pmlogger: Drop unused definition of $LOCALHOSTNAME.
    
    * src/pmlogger/src/pmlogger.c: Preserve original pmcd connection path
      in new variable pmcd_host_conn.  Note it in the pmlogger startup
      message.
    * src/pmlogger/src/dopdu.c: Send back pmcd_host_conn in pmlc "fqdn"
      field, now that we send back the proper pmcd.hostname already in
      the archive-header.ill_hostname thence "ls_hostname".

commit 74354ad745af836df85531d525ba17f9a6bfc824
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 9 14:53:18 2013 -0400

    pmcd FQDN cleanup: drop DNSifying pmcd.hostname

commit f3dd0df6ec3d76b71dd5b9064440f2b14a0a2635
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Wed Oct 9 14:04:21 2013 -0400

    man pages: note default of local: vice localhost

<Prev in Thread] Current Thread [Next in Thread>
  • pcp updates: hostnames, qa, Nathan Scott <=