Some small stuff hidden in amongst this, but 2 big changes ...
1. Going to PCP 3.10.0 broke the internal version encoding used in QA (was 4
digits, now needs to be 5 digits with 2 digits for the minor version number)
... this changes lots of QA files, but the changes should all be cosmetic.
2. (more significantly) Following Mark's investigations of arithmetic errors
triggered my pmiostat.py failures in QA, I audited the use of struct timeval in
the code base ... the result was not pretty. I've refactored all of this code,
added a couple of additional libpcp routines, and removed almost all references
to 1000000 in expressions involving conversions between double and struct
timeval and vice versa, except in the library routines __pmtimevalAdd(),
__pmtimevalSub(), __pmtimevalToReal(), __pmtimevalFromReal(). In the process
also removed all the implementations of a := a+b and a := a-b for struct
timevals a and b and replaced these by use of the new __pmtimevalInc() and
__pmtimevalDec() routines in libpcp. I believe we now have (a) consistent
arithmetic and (b) arithmetic that preserves as much precision as possible.
Changes committed to git://git.performancecopilot.org/kenj/pcp.git dev
qa/012 | 2
qa/019 | 2
qa/024 | 4
qa/028 | 4
qa/033 | 2
qa/049 | 2
qa/050 | 2
qa/052 | 4
qa/057 | 4
qa/070 | 2
qa/077 | 2
qa/078 | 2
qa/081 | 2
qa/082 | 6
qa/092 | 2
qa/094 | 2
qa/112 | 8
qa/119 | 6
qa/138 | 2
qa/146 | 2
qa/154 | 2
qa/155 | 2
qa/158 | 2
qa/159 | 8
qa/163 | 2
qa/171 | 2
qa/173 | 2
qa/176 | 2
qa/180 | 6
qa/181 | 2
qa/186 | 2
qa/200 | 2
qa/209 | 2
qa/217 | 2
qa/226 | 4
qa/237 | 2
qa/238 | 2
qa/239 | 2
qa/240 | 2
qa/241 | 2
qa/245 | 2
qa/248 | 2
qa/250 | 4
qa/254 | 6
qa/261 | 4
qa/266 | 2
qa/274 | 2
qa/278 | 2
qa/280 | 2
qa/283 | 2
qa/295 | 2
qa/299 | 4
qa/300 | 2
qa/311 | 2
qa/313 | 4
qa/316.out | 50 +--
qa/320 | 2
qa/322 | 5
qa/323 | 2
qa/324 | 2
qa/326 | 2
qa/346 | 2
qa/347 | 2
qa/348 | 2
qa/349 | 2
qa/357 | 2
qa/367 | 2
qa/368 | 4
qa/369 | 6
qa/374 | 2
qa/375 | 4
qa/382 | 2
qa/392 | 8
qa/403 | 2
qa/406 | 2
qa/411 | 2
qa/429 | 4
qa/430 | 2
qa/443 | 2
qa/444 | 4
qa/448 | 2
qa/456 | 2
qa/458 | 2
qa/461 | 2
qa/465 | 10
qa/466 | 4
qa/469 | 2
qa/471 | 2
qa/479 | 2
qa/479.out.2 | 1
qa/497 | 2
qa/498 | 4
qa/504 | 4
qa/511 | 2
qa/512 | 2
qa/513 | 2
qa/514 | 4
qa/520 | 4
qa/523 | 2
qa/526 | 2
qa/527 | 2
qa/528 | 2
qa/531 | 2
qa/537 | 2
qa/540 | 2
qa/556 | 2
qa/558 | 2
qa/565 | 2
qa/570 | 4
qa/575 | 2
qa/578 | 2
qa/592 | 2
qa/593 | 2
qa/597 | 2
qa/600 | 4
qa/605 | 2
qa/617 | 6
qa/628 | 6
qa/633 | 4
qa/638 | 2
qa/639 | 2
qa/642 | 6
qa/647 | 2
qa/648 | 2
qa/649 | 2
qa/651 | 2
qa/652 | 2
qa/653 | 2
qa/703 | 2
qa/705 | 2
qa/711 | 2
qa/715 | 2
qa/716 | 2
qa/719 | 2
qa/720 | 2
qa/721 | 2
qa/723 | 2
qa/726 | 2
qa/728 | 2
qa/733 | 2
qa/735 | 2
qa/744 | 19 +
qa/745 | 19 +
qa/768 | 2
qa/875 | 2
qa/common.check | 40 ++
qa/common.gfs2 | 2
qa/common.install.cisco | 2
qa/mk.pcpversion | 5
qa/src/churnctx.c | 7
qa/src/interp0.c | 16
qa/src/interp1.c | 3
qa/src/slow_af.c | 2
qa/src/xarch.c | 13
src/include/pcp/impl.h | 2
src/libpcp/src/AF.c | 37 --
src/libpcp/src/auxconnect.c | 4
src/libpcp/src/derive_fetch.c | 10
src/libpcp/src/discovery.c | 3
src/libpcp/src/exports | 6
src/libpcp/src/loop.c | 9
src/libpcp/src/pdu.c | 6
src/libpcp/src/rtime.c | 5
src/libpcp/src/tv.c | 44 ++
src/libpcp/src/util.c | 20 -
src/libpcp_pmda/src/open.c | 4
src/libpcp_qed/src/qed_app.cpp | 12
src/libpcp_qmc/src/qmc_metric.cpp | 12
src/libpcp_qmc/src/qmc_time.cpp | 21 -
src/libpcp_trace/src/pdu.c | 7
src/libpcp_trace/src/trace.c | 12
src/perl/PMDA/local.c | 3
src/pmchart/groupcontrol.cpp | 14
src/pmchart/main.cpp | 30 -
src/pmchart/main.h | 2
src/pmchart/recorddialog.cpp | 2
src/pmchart/tab.cpp | 4
src/pmchart/timecontrol.cpp | 2
src/pmchart/tracing.cpp | 12
src/pmdas/darwin/kernel.c | 8
src/pmdas/weblog/pmda.c | 3
src/pmdumptext/pmdumptext.cpp | 16
src/pmevent/pmevent.c | 3
src/pmie/src/dstruct.c | 14
src/pmie/src/dstruct.h | 4
src/pmie/src/pmie.c | 28 -
src/pmie/src/pragmatics.c | 9
src/pmieconf/rate-syscalls.c | 18 -
src/pmlogger/src/callback.c | 6
src/pmlogger/src/pmlogger.c | 7
src/pmlogger/src/ports.c | 7
src/pmlogreduce/pmlogreduce.c | 8
src/pmlogrewrite/pmlogrewrite.c | 12
src/pmlogsummary/pmlogcheck.c | 18 -
src/pmlogsummary/pmlogsummary.c | 34 --
src/pmtop/pmtop.c | 5
src/pmval/pmval.c | 3
src/python/pcp/pmcc.py | 620 --------------------------------------
src/python/pcp/pmsubsys.py | 355 ---------------------
src/python/pmda.c | 2
200 files changed, 490 insertions(+), 1537 deletions(-)
commit 196603d15303e8d53a462a2360dd70b066595b20
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Sep 15 07:31:13 2014 +1000
qa/common.check - tighter checks for pmlogger termination
In _wait_pmlogger_end() we were waiting for the control file to be
removed. But even after cleaning up the stdio buffer flushing in
pmlogger there is still a race, because the trailer "finished"
message is written from libpcp routines in an atexit() handler which
is after the control file is removed.
Extend the test to wait for the pmlogger process to have really
exited, using "kill -s 0 $pid" as the test.
commit 1e84106d5467d0bca703385851b075e7c2c40fed
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Mon Sep 15 07:29:25 2014 +1000
qa/744 & qa/745 - nvidia pmda testing fix
Need to make sure the compiled help text files exist before running
the tests.
commit c0bf02b31c788bf68b0afa495786db46e43069dd
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Sep 14 21:06:16 2014 +1000
Delete some Python files that were renamed, but not tracked in git
commit 6e56ba7a58f27b038a4048916a61cb5e3cb6b908
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Sep 14 15:10:25 2014 +1000
pmlogger - fix small race on exit condition
Flush stdio buffers before removing control file(s).
commit 3f0ed3008b1e2ed52447c82c2e3b88e6417a6f86
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Sep 14 09:33:19 2014 +1000
QA meets Y2K, sort of ...
Internally we've used a 4 digit number to encode the PCP version number
1 - major
2 - minor
3 & 4 - point
Going to PCP 3.10.0 broke this.
Convert to 5 digit numbers ...
1 - major
2 & 3 - minor
4 & 5 - point
There is nothing else to see in this commit, even though it touches
143 files.
commit 02c366ee1474c3fb987b75c36d12ce93c415ddae
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Sep 14 06:56:21 2014 +1000
timeval refactoring - part 5, QA output
This group of commits refactors the way we handle struct timeval
and in particular conversion to and from a double.
Use the libpcp routines in preference to inline code.
After all the changes, we need to amend our view of what is the
"correct" output for a couple of tests ... the previous .out files
were just wrong, and this had not been noticed.
commit 3592fa59fea2203fc372b992f4fc697766e318a1
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Sep 14 06:55:21 2014 +1000
timeval refactoring - part 4, QA applications
This group of commits refactors the way we handle struct timeval
and in particular conversion to and from a double.
Use the libpcp routines in preference to inline code.
The end result is we avoid arithmetic involving 1000000 appearing
all over the code base, and encapsulate the correct arithmetic in
just one place (tv.c of libpcp).
commit 9cc0a0c41924c451acffd1d4773124db8582a5dd
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sun Sep 14 06:54:13 2014 +1000
timeval refactoring - part 2, PMDAs and PMAPI clients
This group of commits refactors the way we handle struct timeval
and in particular conversion to and from a double.
Use the libpcp routines in preference to inline code.
The end result is we avoid arithmetic involving 1000000 appearing
all over the code base, and encapsulate the correct arithmetic in
just one place (tv.c of libpcp).
commit f56888346c7be10f7d534d0c57a50e8ec0fcfb0d
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Sep 13 17:53:28 2014 +1000
timeval refactoring - part 2, other libraries
This group of commits refactors the way we handle struct timeval
and in particular conversion to and from a double.
Use the libpcp routines in preference to inline code.
The end result is we avoid arithmetic involving 1000000 appearing
all over the code base, and encapsulate the correct arithmetic in
just one place (tv.c of libpcp).
commit 6c13d3afd85b5e89ae05d476365490b77173e222
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Sep 13 14:30:33 2014 +1000
libpcp_pmda - better diagnostics in pmdaInit
There are some integrity checks like ...
if ((nmetrics == 0 && metrics != NULL) || (nmetrics != 0 && metrics ==
NULL))
if ((nindoms == 0 && indoms != NULL) || (nindoms != 0 && indoms == NULL))
these are unaltered, but on failure the diagnostic now provides more useful
information as to why the check failed.
commit 30ee041a6e0cbe36e22d7db9aa61e129f9fd140e
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Sep 13 14:19:47 2014 +1000
timeval refactorting - part 1, <impl.h> and libpcp
This group of commits refactors the way we handle struct timeval
and in particular conversion to and from a double.
+ two new libc functions are introduced
void __pmtimevalInc(struct timeval *, const struct timeval *);
void __pmtimevalDec(struct timeval *, const struct timeval *);
to perform in place a := a + b and a := a - b arithmetic
+ lots of local utility functions, e.g. tvsub(), tvadd() are replaced
by uses of the libpcp routines above, or there "add/subtract and
return a double result" cousins:
double __pmtimevalAdd(const struct timeval *, const struct timeval *);
double __pmtimevalSub(const struct timeval *, const struct timeval *);
+ all of the "divide by 1000000" code to convert a struct timeval into a
double has been expunged and the correct high precision arithmetic is
encapsulated in one place, namely
double __pmtimevalToReal(const struct timeval *);
+ all of the double -> struct timeval conversions have been replaced by
calls to:
extern void __pmtimevalFromReal(double, struct timeval *);
The end result is we avoid arithmetic involving 1000000 appearing
all over the code base, and encapsulate the correct arithmetic in
just one place (tv.c of libpcp).
commit 58d3dfcf3df674dbbc138e0b55f402ad69f78fbe
Author: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Sat Sep 13 13:56:29 2014 +1000
qa/456 - filter out non-deterministic NOTICES rolling messages
|