Changes committed to git://git.pcp.io/nathans/pcp.git master
Frank Ch. Eigler (17):
pmFetchGroup API - a simplification of pmLookup*/pmFetch/pmExtract*
pmfg: qa/802
pmfg archive mode improvement: seek to end for lookup retry
pmfg docs: mention PR1129
pmfg: pmstat conversion
pcp fetchgroups: python binding
pmfg python testcase: shrink the lambda
pmfg pmapi: use 'utf-8' rather than "utf-8" in encoding literals
pmfg python testsuite: add some parameter-error cases
pmfg python bindings: type inference & arg reordering
pcp fetchgroup v3: with a more private context
pmfg book additions: add new section on fetchgroup PMAPI, C+Python
man3/pmfetchgroup.3: spell out requirement for unique output pmAtomValues.
pmfg qa: escape the doldrums of the low 800s
pmfg: special handling for intermittently unavailable PM_SEM_DISCRETE
metrics
pmclient: restore old PMAPI version, plus add pmfg C and python clones
fetchgroups: event-field support
Nathan Scott (6):
build: fix install of python pmclient script
docs: fetchgroup documentation updates on review
qa: add tests for pmclient_fg, binary and python variants
libpcp: code consistency cleanups for fetchgroup code
pmclient: small code cleanups on review
pmstat: small code cleanups on review
books/PCP_PG/Revision_History.xml | 9
books/PCP_PG/pcp-programmers-guide.pdf |binary
books/PCP_PG/pcp-programmers-guide.xml |11641 ++++++++++++++++++++-------------
man/man1/pmclient.1 | 28
man/man3/pmapi.3 | 16
man/man3/pmextractvalue.3 | 10
man/man3/pmfetch.3 | 12
man/man3/pmfetchgroup.3 | 1316 ++-
qa/053 | 57
qa/053.out | 25
qa/056 | 85
qa/056.out | 22
qa/073 | 64
qa/073.out | 13
qa/079.out | 10
qa/109 | 49
qa/109.out | 19
qa/293.out | 10
qa/294.out | 2
qa/320.out | 6
qa/803 | 46
qa/803.out | 4
qa/858 | 35
qa/858.out | 12
qa/859 | 23
qa/859.out | 2
qa/919.out | 2
qa/group | 18
qa/src/GNUlocaldefs | 5
qa/src/fetchgroup.c | 294
qa/src/test_pmfg.python | 184
src/include/pcp/pmapi.h | 44
src/libpcp/src/GNUmakefile | 2
src/libpcp/src/check-statics | 1
src/libpcp/src/exports | 14
src/libpcp/src/fetchgroup.c | 2671 ++++++-
src/libpcp/src/hash.c | 2
src/pmclient/.gitignore | 1
src/pmclient/GNUmakefile | 64
src/pmclient/GNUmakefile.install | 23
src/pmclient/README | 18
src/pmclient/pmclient.c | 728 +-
src/pmclient/pmclient_fg.c | 498 +
src/pmclient/pmclient_fg.py | 330
src/pmclient/pmclient_fg.python | 211
src/pmclient/pmnsmap.spec | 22
src/pmmgr/pmmgr.cxx | 182
src/pmstat/pmstat.c | 1253 +--
src/python/pcp/pmapi.py | 416 +
49 files changed, 13623 insertions(+), 6876 deletions(-)
commit 24f3ba9f1d76cfaa247c74ec006adf62d96cb714
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue Jan 19 12:24:27 2016 +1100
pmstat: small code cleanups on review
Simplify the statsrc structure (readable, improves error
handling in getNewContext), usual whitespace and comment
tidy-ups (revert some earlier inconsistencies introduced
there).
Fixed the bug in the dtot calculation that was recorded
in prior commit here (add iowait to idle, steal to sys).
commit 5c97fc090644aba9ff027a1428e1f050e65b1180
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue Jan 19 12:17:12 2016 +1100
pmclient: small code cleanups on review
Add pmclient_fg to gitignore file, update README for the
pmclient tools mentioning the new ones, make whitespace
and comments consistent with rest of pcp code, rename a
local "u" to something more descriptive (lots of pylint
cleaned, actually).
Add tests 073 and 056 to the python group.
commit 5aef83fd2d3b0e3b51c089a324c63a41723119a1
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue Jan 19 11:56:35 2016 +1100
libpcp: code consistency cleanups for fetchgroup code
Nothing earth-shattering here - just keeping libpcp code
consistent in small ways - whitespace, comments, columns
per code line, that sort of thing.
Other cleanups include a start on removal of some of the
less-useful goto use (goto used instead of direct return).
Used max/min constants from config.h instead of some local
re-definitions.
commit f9a68e4a9ccd09101b80b4a74162734f98c7ebc7
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue Jan 19 11:30:41 2016 +1100
qa: add tests for pmclient_fg, binary and python variants
Ensure all of pmclient, pmclient_fg and pmclient_fg.python
have test coverage. In doing so uncovered an issue in the
pmclient makefile, neither binary was being installed into
/usr/bin anymore (install target referencing non-existent
macros).
Fixed up incorrect return code on test 858, causing it to
fail when run from qa/check.
commit f2512f51c599db5972243a503d51ef7fd75858f3
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue Jan 19 09:45:01 2016 +1100
docs: fetchgroup documentation updates on review
Add man entry for pmclient_fg, shared with pmclient(1).
Remainder is all small stuff, mostly formatting or other
corrections to follow PCP style, so new man page renders
looking like the rest. One or two typos fixed and also
removed the now dated no-events-support note.
commit da5dfa76d547772bf4fede174dba7962675cd9a4
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue Jan 19 09:23:32 2016 +1100
build: fix install of python pmclient script
Use the usual renaming approach to avoid rpmbuild failure
from Makepkgs and python pre-built cache/object files.
commit e3e0b564111b6636ce1d81e91820bacee1c3645e
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Sat Jan 2 00:19:10 2016 -0500
fetchgroups: event-field support
Extend the fetchgroup api with an _event variant, which processes
named fields from named event metrics through the extract/convert/cast
pipeline. This lets the following toy program trace systemd/journal
events:
from pcp import pmapi
import cpmapi as c_api
import time
pmfg = pmapi.fetchgroup(c_api.PM_CONTEXT_HOST, "local:")
vvv = pmfg.extend_event("systemd.journal.records",
field="systemd.journal.field.string")
while True:
pmfg.fetch()
for ts, line in vvv():
print ("%s : %s" % (ts, line()))
time.sleep(1.0)
Python & docs & QA included.
commit 45a2c510c0a2a7bf6d2d6f5d82595e6389bcaf5a
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Fri Dec 18 15:51:40 2015 -0500
pmclient: restore old PMAPI version, plus add pmfg C and python clones
Restored the pre-fetchgroup pmclient machinery, so that the PCP_PG can
continue demonstrating it. The fetchgroup C code is renamed
pmclient_fg.c, and now a (partial) python clone pmclient_fg.py is
added for reference. All three tested on both -h and -a modes.
% wc -l *.c *.py
357 pmclient.c
279 pmclient_fg.c
165 pmclient_fg.py
commit ba4a1eecd9e5162a6a15a047f50c0bb9851ff8c6
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Fri Dec 18 15:30:41 2015 -0500
pmfg: special handling for intermittently unavailable PM_SEM_DISCRETE
metrics
When processing archives, a pmfg client program may encounter metrics
in an inconvenient order. A pmFetch in FORW mode will miss
once-logged values, and pmfg would report these as PM_ERR_VALUE
(absent) to the calling application. We can do better though for
DISCRETE semantics, documented in PCP_PG as "once observed it is
presumed the value will persist for an extended period". Even if
pmFetch gives us numval=0, pmfg can preserve the previous value in the
output variables/status.
This permits a pmfg application processing archives to easily tolerate
intermingling of occasionally-saved discrete and regularly-saved
instant/counter metrics.
commit 63e4742c2509466d31f8b22498dcf7ef021a1cdf
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Wed Dec 16 21:59:12 2015 -0500
pmfg qa: escape the doldrums of the low 800s
qa/XXX names are not reserved effectively for patchsets held back from
merging. Other merged work may take those XXX slots, causing a merge
conflict. Move the pmfg tests way out of the current 800ish kill
zone, so they may be safe from renumbering a while.
commit 8c1cb01837b66231110274d032aa7c5363ed582d
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Wed Dec 16 21:51:14 2015 -0500
man3/pmfetchgroup.3: spell out requirement for unique output pmAtomValues.
... else we might have results written on top of results.
commit 2723c095349eca9ed194b1dd0a0126627abfa300
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Wed Dec 16 21:48:21 2015 -0500
pmfg book additions: add new section on fetchgroup PMAPI, C+Python
The XML file was first pretty-printed for humane editing. The PDF was
rendered on Fedora22 BOOK_TOOLCHAIN=publican, with custom wkhtmltox
binaries from its upstream, replacing the crippled Fedora wkhtmltopdf.
commit d5314d02bbcfc09a2aa67ce107d233f05010c424
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Tue Dec 15 15:16:26 2015 -0500
pcp fetchgroup v3: with a more private context
As per review comments:
- a fetchgroup now owns a private pmapi context it creates via
pmNewContext, so pmDupContext is no longer relevant
- ... which it is willing to expose, with appropriate documentation
cautions
- which moots pmFetchGroupSetMode, so it's gone
- tests extended to cover interleaving,
- rate conversion failure due to missing history is PM_ERR_AGAIN'd
- a little bit more initialization is promised by a few few functions
pmstat, pmclient, pmmgr, python-bindings, test cases updated. pmmgr
more fully converted to pmfg (the container enumeration was previously
overlooked).
commit f14023bb89aca86489d6a8591bdd7c493291158f
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Tue Dec 8 09:27:21 2015 -0500
pmfg python bindings: type inference & arg reordering
As per Marko's suggestions, permit incoming mtype=None,
and reorder pmfg.extend_item arguments.
commit eac954bf975ac9809c16f84d32521422720b5c4e
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Mon Dec 7 13:30:23 2015 -0500
pmfg python testsuite: add some parameter-error cases
commit 26912713325307fd65e8ef3ab89294e719a1df94
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Mon Dec 7 13:12:27 2015 -0500
pmfg pmapi: use 'utf-8' rather than "utf-8" in encoding literals
commit 7d16c7b7b73ff60e98c5367c3c5346168b756f15
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Mon Dec 7 13:07:13 2015 -0500
pmfg python testcase: shrink the lambda
myllynen questioned whether the pmns traversal lambda structure was
necessary. It turns out in this case it was not, so it's gone.
commit 3529cea72f33f06e680b82767dcec964fd81c903
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Sun Dec 6 15:11:48 2015 -0500
pcp fetchgroups: python binding
Add an OO-flavoured interface to fetchgroups. It makes basic PMAPI
programming almost criminally easy. qa/801 = src/test_pmfg.python
test case included, which demonstrates basic plus PMNS-enumeration
techniques. Tested on python2 + python3.
commit 09af04d3dadedc594e452f5daf4512989d1dc6d3
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Wed Dec 2 19:04:15 2015 -0500
pmfg: pmstat conversion
Show off application of the fetchgroups api to pmstat. In old QA,
update diagnostic messages, and correct (!) a column of data.
commit d36bf1a1a5c08206f444922fcfecf44b17e61458
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Tue Dec 1 09:52:05 2015 -0500
pmfg docs: mention PR1129
Since pmDupContext is not really functional (PR1129), adjust the
fetchgroup documentation to advise that the context supplied to
pmCreateFetchGroup may be shared rather than copied, with the safety
constraints thus implied.
commit d61b4179545bcd8634abde039c668c78235240ce
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Tue Dec 1 09:24:20 2015 -0500
pmfg archive mode improvement: seek to end for lookup retry
The "pmclient -a" test qa/079 highlighted a problematic case both for
normal pmapi and fetchgroups: namely the lookup for metrics/indoms may
not be valid at the time origin when the operations are attempted.
The fetchgroup mechanism is now extended to retry such lookups by
temporarily seeking to the end of archive. This can still fail, if
the indom instance existed only briefly in the middle of the archive
timeline. The user is made aware of this possibility by new blurbage
in the man page.
commit 9172e20a88a6714ae062129c3f4bc2579a4d6990
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Sun Nov 29 19:28:49 2015 -0500
pmfg: qa/802
A C testsuite for the pmFetchGroup* API.
commit b361faac93e1952fcaeed4bd8ad5b643fe81c59c
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date: Wed Sep 30 20:56:57 2015 -0400
pmFetchGroup API - a simplification of pmLookup*/pmFetch/pmExtract*
This commit introduces a new C API in libpcp for simplified metric
fetch operations.
A man page is included and src/pmmgr and src/pmclient are converted.
Thanks to mgoodwin & myllynen for reviews.
|