The containers stuff is getting there but is not integrated
into the build yet - there is more to be done (get cron working
for pmlogger cron scripts, improve /var/run/pcp/* stale pid
handling, and of course QA). Until then I'm not integrating
it into the build. At this stage it may or may not make this
release.
Time to focus on QA ...
Changes committed to git://git.pcp.io/pcp/pcp.git master
Mark Goodwin (16):
Don't delete pcp.spec after pack_pcp, we often need to check it.
New docker files and build infrustructure.
More tweaks to docker files, and add pcp-testsuite container.
Don't fork pmcheck co-processes in a container environment.
Ignore build targets and temp RPMS directory
Add pcp-base container.
Use pcp-base as the base image for all other PCP containers.
rc scripts require /usr/bin/which
Print a bit more detail on why pmlogger thinks there's already a primary
pmlogger running.
Use full path to pmlogger in pmlogger_check.
Use the pcp-test repo from the pcp-base image.
Bind mount /etc/cron.d from the host for pmlogger_{check,daily}.
Set the bash prompt for when the container is interactive.
Deprecate hinv.nlv and hinv.map.lvname and the LV_NAME instance domain.
QA updates after deprecating hinv.nlv and hinv.map.lvname.
More container refinements.
Ken McDonell (8):
qa/260: improve stability
pmnewlog: clean up usage message
pmlogger: revert small part of local context change
qa/948: add -K exerciser and some error cases
pmlogger: add comment explaining -o => opts.Lflag usage
pmlogger.c: initialize linger
qa/948: extend coverage
configure: add /bin/sh as possible pid==1 candidate
Nathan Scott (3):
python: fix bug found by qa/739, unguarded destructor call
qa: add ppc64le to list of known architecture wordsizes
qa: control the timezone used by pmie in test 951
build/containers/.gitignore | 2
build/containers/GNUlocalrules | 21 +++
build/containers/GNUmakefile | 43 +++++-
build/containers/README | 125 ++++++++++++++----
build/containers/pcp-base/Dockerfile | 82 ++++++++++--
build/containers/pcp-base/GNUmakefile | 33 ++++
build/containers/pcp-collector/Dockerfile | 197 ++++++++++++++++++-----------
build/containers/pcp-collector/GNUmakefile | 43 +++++-
build/containers/pcp-monitor/Dockerfile | 107 ++++++++++-----
build/containers/pcp-monitor/GNUmakefile | 43 +++++-
build/containers/pcp-pmcd/Dockerfile | 65 ---------
build/containers/pcp-pmlogger/Dockerfile | 119 ++++++++++++-----
build/containers/pcp-pmlogger/GNUmakefile | 43 +++++-
build/containers/pcp-testsuite/Dockerfile | 117 ++++++++++++-----
build/containers/pcp-testsuite/GNUmakefile | 39 +++++
build/containers/scripts/pcp-docker-run | 31 ++++
build/rpm/GNUmakefile | 2
build/rpm/pcp.spec.in | 2
configure | 3
configure.ac | 3
qa/003 | 1
qa/232 | 3
qa/232.out | 1
qa/260 | 21 ++-
qa/260.out | 16 +-
qa/540 | 80 -----------
qa/540.out | 2
qa/948 | 88 +++++++++++-
qa/948.out | 23 +++
qa/951 | 12 -
qa/951.out | 2
qa/common.check | 2
qa/group | 1
src/pmdas/linux/GNUmakefile | 5
src/pmdas/linux/clusters.h | 2
src/pmdas/linux/devmapper.c | 86 ------------
src/pmdas/linux/devmapper.h | 29 ----
src/pmdas/linux/help | 4
src/pmdas/linux/indom.h | 2
src/pmdas/linux/pmda.c | 44 ------
src/pmdas/linux/root_linux | 2
src/pmlogger/pmlogger_check.sh | 2
src/pmlogger/pmnewlog.sh | 48 +++----
src/pmlogger/rc_pmlogger | 22 ++-
src/pmlogger/src/fetch.c | 2
src/pmlogger/src/pmlogger.c | 8 +
src/pmlogger/src/ports.c | 4
src/python/pcp/pmapi.py | 3
48 files changed, 1012 insertions(+), 623 deletions(-)
Details ...
commit 10e9929a559962323cb02aa1b789c74609d7653d
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Fri Jun 5 18:50:26 2015 +1000
More container refinements.
Inherit bash prompt from pcp-base, using $CP_CONTAINER_IMAGE. More RUN label
refinements, preparation for pmlogger cron scripts and check for builddefs.
modified: build/containers/GNUmakefile
modified: build/containers/pcp-base/Dockerfile
modified: build/containers/pcp-collector/Dockerfile
modified: build/containers/pcp-monitor/Dockerfile
deleted: build/containers/pcp-pmcd/Dockerfile
modified: build/containers/pcp-pmlogger/Dockerfile
modified: build/containers/pcp-testsuite/Dockerfile
commit 950c318ab0bf48e749d4d39d38908999a53473c7
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Fri Jun 5 18:15:47 2015 +1000
QA updates after deprecating hinv.nlv and hinv.map.lvname.
modified: qa/003
modified: qa/232
modified: qa/232.out
deleted: qa/540
deleted: qa/540.out
modified: qa/group
commit 9ab615d4ca7e85501b72587194fbe46c72143aab
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 17:12:59 2015 +1000
configure: add /bin/sh as possible pid==1 candidate
For https://github.com/performancecopilot/pcp/issues/23
commit 79f88b7107d2c397ff2215b052bb671add4fe81b
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 17:12:29 2015 +1000
qa/948: extend coverage
commit 7718db828ee37f4d0592118cf385ff06b33ad459
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 16:40:11 2015 +1000
pmlogger.c: initialize linger
commit ab4eb4adbfec797f51becf3298f5b78f350dcaed
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Fri Jun 5 14:23:37 2015 +1000
Deprecate hinv.nlv and hinv.map.lvname and the LV_NAME instance domain.
The instance domain for hinv.map.lvname is the dm names e.g. dm-1,
which are not persistent. The values are the logical names, which
are persistent. These two metrics are now deprecated in favor of
hinv.map.dmname, which instead uses the persistent names for the
instance domain, and the dm names as the value of the mapping.
Note also - hinv.map.dmname is a superset of the lvm instances -
the dmname instance domain also includes non-lvm devices such as
dm-multipath, dm-cache, etc. as well as all lvm logical devices.
The only known user of the deprecated hinv.map.lvname metric is the
old pmatop command, which itself has been deprecated.
modified: src/pmdas/linux/GNUmakefile
modified: src/pmdas/linux/clusters.h
deleted: src/pmdas/linux/devmapper.c
deleted: src/pmdas/linux/devmapper.h
modified: src/pmdas/linux/help
modified: src/pmdas/linux/indom.h
modified: src/pmdas/linux/pmda.c
modified: src/pmdas/linux/root_linux
commit 894688c329c99d0816e0b7d3f3093ae7e8fd4ce5
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Fri Jun 5 13:44:48 2015 +1000
qa: control the timezone used by pmie in test 951
commit aeebc9fa069d55ee32574e4a2b9e42b2997a606b
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Fri Jun 5 13:37:13 2015 +1000
qa: add ppc64le to list of known architecture wordsizes
commit 55a448b3d9629e22abafb004471133adc0842b74
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Fri Jun 5 13:36:39 2015 +1000
python: fix bug found by qa/739, unguarded destructor call
Need to guard the pmOptions class destructor call so that
it does not to attempt cleanup via objects that have not
been initialised yet. Similar guards to that in pmContext
destructor prevent this failure mode:
[...]pcp/pmapi.py", line 758, in __del__
TypeError: 'NoneType' object is not callable
commit 59a03622722bb9c1f6e013f401663892f27fe096
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 11:11:37 2015 +1000
pmlogger: add comment explaining -o => opts.Lflag usage
commit 104fcd1393569fc52291c3effdc245e7022ea894
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 09:58:49 2015 +1000
qa/948: add -K exerciser and some error cases
commit 47c15dc36ee7bc29dd2cc7db3e9d16d15f0b5ff1
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 09:55:45 2015 +1000
pmlogger: revert small part of local context change
Re-instate first argument to __pmEncodeResult() in myFetch().
commit abacb4863ba5c3a2976ae37b7f9ed2c9e73b36dc
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 09:52:09 2015 +1000
pmnewlog: clean up usage message
- no = for short form options
- whitespace consistency
- short and long form for all relevant options
- add some missing option arguments
e.g. "-n FILE, --namespace=FILE" instead of "--namespace"
commit 124009a0ee2b242dc8f0fadfad69e7c0cc2778f0
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri Jun 5 06:28:57 2015 +1000
qa/260: improve stability
On slow platforms (especially VMs on low powered hosts) the pmie
startup time may be non-zero, especially where complex rules are
involved and derived metrics are in use. This makes the time of
the first fetch variable and since the metrics are counters the
values from the firect fetch are still in use at the reporting after
the second fetch ... this makes the first results unknown (which
is deterministic) but the second results are not stable in the
presence of the rate() function because the values and timestamps
for the firtst fetch are a bit dodgey ... so filter these ones out.
After the second fetch the results are rock solid and stable
across all platforms.
commit 21829d7b2ec95264c22a78a8d8a8e1d59696fa17
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Wed Jun 3 17:33:02 2015 +1000
Set the bash prompt for when the container is interactive.
Set the root bash prompt so you can tell which container you're in after
e.g.
atomic run CONTAINER /bin/bash or docker exec -it CONTAINER /bin/bash
modified: build/containers/pcp-base/Dockerfile
modified: build/containers/pcp-collector/Dockerfile
modified: build/containers/pcp-monitor/Dockerfile
modified: build/containers/pcp-pmlogger/Dockerfile
modified: build/containers/pcp-testsuite/Dockerfile
commit ca9e8350d21a3fa322c2914f07b0c5e2f35c2337
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Wed Jun 3 17:30:21 2015 +1000
Bind mount /etc/cron.d from the host for pmlogger_{check,daily}.
modified: build/containers/pcp-pmlogger/Dockerfile
commit 5138de1a2424d0a9077bc4fe70ee9614edb43ce3
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 20:08:01 2015 +1000
Use the pcp-test repo from the pcp-base image.
Now that the containers are based on the pcp-base image, the
pcp-test yum repo is already set up so we don't need to duplicate
that for the layered containers.
modified: build/containers/pcp-base/Dockerfile
modified: build/containers/pcp-collector/Dockerfile
modified: build/containers/pcp-collector/GNUmakefile
modified: build/containers/pcp-monitor/Dockerfile
modified: build/containers/pcp-monitor/GNUmakefile
modified: build/containers/pcp-pmlogger/Dockerfile
modified: build/containers/pcp-pmlogger/GNUmakefile
modified: build/containers/pcp-testsuite/Dockerfile
modified: build/containers/pcp-testsuite/GNUmakefile
commit f6e3da702b16f00cc467e0cf8799c51aaceb7a27
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 19:56:13 2015 +1000
Use full path to pmlogger in pmlogger_check.
In a container, the PATH has $PCP_SHARE_DIR/lib ahead of $PCP_BINADM_DIR,
so trying to run "pmlogger" in pmlogger_check is going to relaunch the
rc script (recursively). Using the full path to $PCP_BINADM_DIR/pmlogger
in pmlogger_check fixes it.
modified: src/pmlogger/pmlogger_check.sh
commit d34314c53b04bd593d6007cc05104ab613847cb0
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 16:49:01 2015 +1000
Print a bit more detail on why pmlogger thinks there's already a primary
pmlogger running.
modified: src/pmlogger/src/ports.c
commit f54dfd6c570254f62233c129358112eecee4e73d
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 16:35:49 2015 +1000
rc scripts require /usr/bin/which
'which' is not in the fedora base container image.
modified: build/rpm/pcp.spec.in
commit cb9442257ae547daccbc778f2014813dbab2aaf0
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 16:31:51 2015 +1000
Use pcp-base as the base image for all other PCP containers.
modified: build/containers/pcp-base/Dockerfile
modified: build/containers/pcp-collector/Dockerfile
modified: build/containers/pcp-monitor/Dockerfile
modified: build/containers/pcp-pmlogger/Dockerfile
modified: build/containers/pcp-testsuite/Dockerfile
commit 6dc60318ceff2781a39b6f54e0afd17cc274954c
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 14:21:54 2015 +1000
Add pcp-base container.
Add pcp-base container, based on f22/rawhide to avoid the pcp-compat
dependency that drags in everything under the sun. This container is
not ment to be run - it's the base of other layered containers.
modified: build/containers/GNUmakefile
new file: build/containers/pcp-base/Dockerfile
new file: build/containers/pcp-base/GNUmakefile
commit 3652f761ac18dac80aae39b5f82ac091d09f4eda
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 14:10:46 2015 +1000
Ignore build targets and temp RPMS directory
modified: build/containers/.gitignore
commit 9870afc6e8bc13936c535c044591a7512a6a6b55
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue Jun 2 13:52:18 2015 +1000
Don't fork pmcheck co-processes in a container environment.
In a container environment, pmpause is used to prevent the rc
script exiting until a SIGCHLD is received. In the rc script,
we can't run start_pmcheck in the background, otherwise when
each start_pmcheck co-process exits, pmpause would be signalled
to reap the child (start_pmcheck) rather than the grandchild
(pmlogger), and then exit - causing the whole container to exit.
modified: src/pmlogger/rc_pmlogger
commit 19957166fac9ff1a5e3f9ce411c5cf6ad3c794d9
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue May 26 19:14:16 2015 +1000
More tweaks to docker files, and add pcp-testsuite container.
After much mucking around - can't docker-exec into an existing
running container unless it is run with --pid=host. Running a
bash shell via "docker exec -it pcp-collector /usr/bin/bash"
into a running pcp-collector container is the way we configure
optional PMDAs (at least for now, pending more work in this area).
modified: build/containers/GNUmakefile
modified: build/containers/README
modified: build/containers/pcp-collector/Dockerfile
modified: build/containers/pcp-collector/GNUmakefile
modified: build/containers/pcp-monitor/Dockerfile
modified: build/containers/pcp-monitor/GNUmakefile
modified: build/containers/pcp-pmlogger/Dockerfile
modified: build/containers/pcp-pmlogger/GNUmakefile
new file: build/containers/pcp-testsuite/Dockerfile
new file: build/containers/pcp-testsuite/GNUmakefile
commit 674b6751d5b069069a732dd4f1b3d780f5d533ec
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue May 26 13:51:16 2015 +1000
New docker files and build infrustructure.
Rename the pcp-pmcd container to pcp-collector, and add two new containers:
pcp-monitor and pcp-pmlogger. Also adds build infrastructure and a script to
extract the RUN label and then launch with docker-run(1).
For details, see build/containers/README.
new file: build/containers/GNUlocalrules
new file: build/containers/GNUmakefile
new file: build/containers/README
new file: build/containers/pcp-collector/Dockerfile
new file: build/containers/pcp-collector/GNUmakefile
new file: build/containers/pcp-monitor/Dockerfile
new file: build/containers/pcp-monitor/GNUmakefile
new file: build/containers/pcp-pmlogger/Dockerfile
new file: build/containers/pcp-pmlogger/GNUmakefile
new file: build/containers/scripts/pcp-docker-run
deleted: build/containers/pcp-pmcd/Dockerfile
commit 69f1e0ae309ff17c2e8a36393f2bfa02479e6286
Author: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Mon May 11 21:06:30 2015 +1000
Don't delete pcp.spec after pack_pcp, we often need to check it.
modified: build/rpm/GNUmakefile
|