This commit will allow all PMDAs (C, Perl and Python) to get early access to
pmdaConnect().
This should avoid most of the slow start issues where a PMDA is launched but
takes some time to discover what metrics and/or instance domains are available
... if this process takes too long, pmcd will assume the initial credentials
handshake has timed out and the PMDA will not be a happy camper thereafter.
By giving the PMDA developer the option to connect to pmcd sooner, this problem
is largely avoided.
What remains is the slow PMDA issue where a subsequent PDU request from a PMAPI
client is received from pmcd while the PMDA is "busy" (including slow start)
... this requires much more serious surgery to fix.
If you've been seeing PMDAs not starting (perhaps sometimes, not all times),
this change should help ... feedback and confirmation of this hypothesis would
be most valuable.
Changes committed to git://oss.sgi.com/kenj/pcp.git dev
man/man3/pmdaconnect.3 | 40 ++++-
qa/274.out.2 | 6
qa/553 | 1
qa/763 | 84 +++++++++-
qa/763.out | 49 ++++++
qa/779 | 63 ++++++++
qa/779.out | 26 +++
qa/841 | 82 +++++++++-
qa/841.out | 71 +++++++--
qa/972 | 1
qa/admin/pcp-daily | 54 ++++--
qa/group | 5
qa/pmdas/GNUmakefile | 6
qa/pmdas/GNUmakefile.install | 6
qa/pmdas/slow-python/GNUmakefile | 76 ++++-----
qa/pmdas/slow-python/GNUmakefile.install | 14 -
qa/pmdas/slow-python/Install | 76 ++++-----
qa/pmdas/slow-python/Remove | 50 +++---
qa/pmdas/slow-python/pmdaslow.python | 168 ++++++++++-----------
qa/pmdas/slow/GNUmakefile | 83 +++++++---
qa/pmdas/slow/GNUmakefile.install | 13 +
qa/pmdas/slow/Install | 40 ++++-
qa/pmdas/slow/Remove | 25 +++
qa/pmdas/slow/pmdaslow.pl | 75 +++++++++
qa/pmdas/slow_python/GNUmakefile | 40 +++++
qa/pmdas/slow_python/GNUmakefile.install | 5
qa/pmdas/slow_python/Install | 38 ++++
qa/pmdas/slow_python/Remove | 25 +++
qa/pmdas/slow_python/pmdaslow_python.python | 84 ++++++++++
qa/src/.gitignore | 1
qa/src/GNUlocaldefs | 5
qa/src/badpmda.c | 126 ++++++++++++++++
src/include/builddefs.in | 2
src/include/pcp/pmda.h | 7
src/libpcp_pmda/src/open.c | 31 +++
src/perl/PMDA/PMDA.pm | 219 ++++++++++++++++++++++++++++
src/perl/PMDA/PMDA.xs | 34 ++++
src/pmcd/pmdaproc.sh | 80 +++++++---
src/pmdas/kvm/pmdakvm.pl | 1
src/pmdas/postfix/pmdapostfix.pl | 1
src/pmdas/rsyslog/pmdarsyslog.pl | 2
src/pmdas/simple/pmdasimple.perl | 1
src/pmdas/simple/pmdasimple.python | 2
src/pmdas/simple/simple.c | 2
src/pmns/stdpmid.pcp | 6
src/python/pcp/pmda.py | 3
src/python/pmda.c | 60 ++++++-
47 files changed, 1562 insertions(+), 327 deletions(-)
commit 6ebf92133c77d0e74965ff33dfebd8ae6f792cb2
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 09:09:13 2014 +1000
src/perl/PMDA/PMDA.pm - add connect_pmcd() and some basic documentation
The functional change is the Perl support for connect_pmcd().
The cosmetic change is start documentation for the Perl PMDA API ...
this is a WIP.
commit 2ae3d2080695bac68517fe3bf9b5734102776e09
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 08:37:34 2014 +1000
stdpmid.pcp - allocate unique domain number for SLOW_PYTHON
Just causes too many qa issues to have SLOW and SLOW_PYTHON using
the same domain number.
commit ce4c821f37d2c86185dab1fbf184d260cb52c323
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 06:38:27 2014 +1000
qa/slow PMDA (Perl version) - cleanup
Changes to track changes made elsewhere in pmdaproc.sh and qa/763.
commit 0e92676a7d84eec38eee717407d2688f1f5ea891
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 06:37:29 2014 +1000
qa/763 - install Perl version of slow PMDA from qa/pmdas/slow
commit 24e5bf2b0d5caf31ee6117ecac096c9157d29e17
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 06:31:00 2014 +1000
src/pmcd/pmdaproc.sh - allow Install/Remove from any directory
When the Perl and Python PMDA support was added, it introduced
a restriction that these PMDAs can only be installed from a
directory below $PCP_VAR_DIR/pmdas ... this is quite unnecessary.
Remove that restriction.
Also promote $pmda_dir (that was being used locally in pmdaSetup to
the global scope so that an Install/Remove script can call source
pmdaproc.sh and then change $pmda_dir to be some other absolute
pathname before calling pmdaSetup. This would only be useful
if your PMDA files for some bizarre reason live in a different
directory to the one where the Install/Remove scripts are being
run ... but does allow reversion to the previous behaviour on
the off chance that some PMDA was depending upon it.
commit 47cda57a87c0cf4106c6bc4e98464ff2d46a5741
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 06:29:00 2014 +1000
qa/slow PMDA (Python variant) - changes, part 2
Remove the old qa/pmdas/slow-python files from the git tree.
commit 42f1ab0167a048f02709221b3fdcc282825fcc14
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Apr 7 06:23:40 2014 +1000
qa/slow PMDA (Python variant) - changes
To make this work with the Perl version of the PMDA both in the
tree and in the "testsuite" package environmment, needed to make
a number of changes.
1. change "iam" from "slow" (duplicating the Perl version) to
"slow_python"
2. need stdpmid reference for SLOW_PYTHON (use the same domain number
as SLOW)
3. rename slow-python to slow_python in the git tree (needed for
PMDA Insall/Remove scripts to work)
Much of this originally triggered by moving the qa use of the Install
and Remove scripts to be run in qa/pmdas/slow_python instead of
$PCP_VAR_DIR/pmdas/slow which is much tidier.
commit ec05a573ce8362989dab492bf9bdb6cd11950371
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Apr 6 17:53:19 2014 +1000
qa/553 & qa/972 - remove bad $pmda_dir use
Both scripts used to
cd "$pmda_dir"
but pmda_dir was never set.
Dropped the cd, and both still pass.
commit d2b670eb565c2f50a6cb60f70754b62b78972095
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Apr 6 06:38:57 2014 +1000
src/include/builddefs.in - remove diagnostic fluff
Not sure when the appending to /tmp/eek.raw was added, but this is
totally useless ... and fails if /tmp/eek.raw exists and is owned
by someone else (root in my failing case).
Given the name of the file, I am sure I added the diagnostic without
review, so I'll remove it without review.
commit e40149046b5e80ac958d6284ea70cd87afc7b73f
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 18:27:26 2014 +1100
qa/763 - update
Better diags and cleanup ... tracking changes in the dual qa/841 test.
commit 394949d471b907c0fe57ab8bfdc9825559a58b72
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 18:22:00 2014 +1100
qa/841 (new) & slow PMDA (Python version) - slow start PMDA
The Python version of the slow PMDA shows how self.connect_pmcd()
may be used to better handle slow startup for Python PMDAs ... the
use of self.connect_pmcd() is optional to allow the passing and
failing cases to be explored. The PMDA will also do slow fetches,
but the library support for this is not yet done.
qa/841 exercises the Python version of the slow PMDA in the passing
and failing cases. This is the dual of qa/763 that exercise the
Perl version of the slow PMDA.
commit bce4758e26df0090825af111b660d79e2841fafd
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 18:19:43 2014 +1100
slow PMDA (Perl version) - makefile rework
commit 2b57dacef947d94cd5600210f377f43eca14f613
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 18:14:03 2014 +1100
slow PMDA (Perl version) - update copyright
commit ddc6e7ce8c495753843d2a8291c3203ad9aeb4ea
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 07:23:36 2014 +1100
slow start PMDA changes - part 3 (PMDAs written in Python)
Add the new self.connect_pmcd() method that allows Python PMDAs
to make a pmdaConnect() call anytime after the PMDA.__init__()
constructor.
This will help address slow start issues for Python PMDAs.
Change the Python version of the simple PMDA connect_pmcd.
commit 41716278a0f1855e712ed7b6805025c8d9e1c647
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 07:19:14 2014 +1100
qa/763 & slow PMDA - slow starting PMDA
The slow PMDA shows how connect_pmcd() may be used to better
handle slow startup for Perl PMDAs ... the use of connect_pmcd()
is optional to allow tha passing an failing cases to be explored.
The PMDA will also do slow fetches, but the library support for
this is not yet done.
qa/763 exercises the slow PMDA in the passing and failing cases.
commit 3cb41fcd7d425dbb4f5ab7497839e062c2982b26
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Apr 5 07:13:06 2014 +1100
pmdaproc.sh - add $perl_args and $python_args
Hooks to allow Perl and Python PMDAs to receive command line
arguments. These can be setup in the Install script and then
they are saved in pmcd.conf.
commit e709ec43c3f313443d83afb602c5c46efb4f8b69
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Apr 4 07:21:01 2014 +1100
slow start PMDA changes - part 2 (PMDAs written in Perl)
Add the new pmda->connect_pmcd() method that allows Perl PMDAs
to make a pmdaConnect() call anytime after the new() constructor.
This will address slow start issues for many PMDAs.
Change the following (randomly selected) PMDAs to use connect_pmcd
as a proof of concept: kvm, postfix, rsyslog and simple.
commit acef3014542b0cc78f69608ff56e91612242c421
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Apr 4 06:42:51 2014 +1100
qa/src/badpmda.c - fix compilation warning
commit 5bac3133fe63fa44aa07c604ddf75b83b9e8fec8
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Thu Apr 3 16:38:22 2014 +1100
libpcp_pmda - slow start PMDA changes - part 1 (PMDAs written in C)
Allow PMDAs to call pmdaConnect() earlier.
This reduces the risk of a slow starting PMDA being killed with a
timeout in the credentials handshake that pmcd starts as soon as
a daemon PMDA is launched.
There is little code change here, just additional internal state
and checking to make sure the pmdaInterface structure has been
properly setup when pmdaConnect is called.
The rest is QA (qa/799 is new) and more concise wording for the
pmdaConnect(3) man page.
The C version of the simple PMDA has been changed to show how
pmdaConnect() can be called earlier in the PMDA's initialization.
commit c45073faf55275fde086c636f3aa66cb92a71d61
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Thu Apr 3 06:29:24 2014 +1100
qa/admin/pcp-daily - track recent build changes
Use --clean to Makepkgs (to be clear).
Need to look elsewhere for packages for many builds.
|