Hi -
> > First of all, the existing test cases (qa/914 and qa/799) simply must
>
> (I think you mean qa/903 and qa/914)
No, in the lberk/papi branch, a qa/799 exists. 903 appears only
to run the pmda Install/Remove scripts.
> > stop relying on the -L local context this way. As we discussed on IRC
>
> [...]
> They exposed bugs in the authentication mechanism, exercise the install
> and remove mechanisms
Only barely so (see below): and not the "remove" mechanisms (since there
is nothing to remove from an -L blank slate).
> [...]
> > In fact, the only way anyone could have ever seen the pmda work and produce
> > counter *numbers*, was by -not- using the -L mode, which means its core
> > was only ever "hand tested" and qa/914 was to that extent a talisman.
>
> Heh, I see, seeking opportunity to grind the "hand testing" axe.
You drafted the term into service -as- an axe.
> Contrary to what you say here, the tests exposed a number of bugs & will
> continue to do so going forward.
It is no surprise at all that the very early snapshot of any code has
a number of bugs. The fact that trivial tests can tickle only one or
two entry points and fail is only evidence that the code was not
ready. It is not evidence of the thoroughness of QA, nor that the
code was working. It did not "assure quality", just "show
incompleteness". (At several points, not even hand-tests worked in
the merged code.)
> They are also small, fast smoke tests for initial sanity and will be
> handy for exposing PAPI problems related to enabling every possible
> hardware counter at once (i.e. the notorious short-lived client
> case).
No, they're not actually effective at even that. With the
pmStore-many-names model, as foreseen, there's no way for the user to
know which of multiple counters requested was erroneous, and which
counters were activated. And with the -L testing mode, you can't see
the aftereffects either.
> As discussed awhile back, I added these tests as just a start on
> testing, so that we had some basic initial coverage. So, please
> leave these tests as-is and build on this base with new tests.
> [...]
Simply switching them from -L to the default -h would actually get a
bit of new coverage of the core code. What's the disadvantage?
- FChE
|