Thanks for kicking this off Nathan.
On Mon, 2012-08-13 at 01:01 -0400, Nathan Scott wrote:
> Hi all,
>
> ...
> The current situation is:
> - A pcpqa.git tree exists containing all of the core PCP QA
> tests. On the surface it has loose association with any
> PCP release, and there's a mechanism allowing it to work
> for different PCP versions. However, I've found several
> of the tests to have *deep* knowledge of internals of PCP,
> and they can break in non-intuitive ways when the planets
> are not aligned.
I think this is an example of a brain failure on my part, not a reason
for merging the trees. I should have done a better job of isolating the
PCP version dependencies in the test artefacts that you tripped up on.
These are a very small minority of the total number of test artefacts.
Because PCP follows a fundamentally a client-server architecture it is
absolutely critical that the QA components are _not_ exposed to version
dependencies but rather accommodate version differences so that
cross-version inter-operation can be tested, e.g. new clients with old
pmcd and vice versa.
If merging the trees encourages (even implicitly) the creation of a
version of the QA packages that _only_ work with the same version of the
PCP packages then that would be a bad thing(tm) ... if QA continues to
be multi-version aware, then that would be a good thing(tm).
> - Ken did some initial work on the pcpqa tree a couple of
> months back & it can now be made produces a .deb package.
I stopped work on this when I could not decide what the QA package
should contain:
a) compiled binaries and scripts, or
b) source and scripts.
a) means you need one package for each QA platform of interest.
Since some of the tests require a toolchain (not may I agree), b) seems
simpler, means only one package for all platforms, and using tar to
package this would be simpler as we'd avoid all of the uglier parts of
the packaging "gooiness".
> ...
> I vastly favour the second option for simplicity - I'd like
> to end up with one pcp tree again at the end, and have the
> same model for both pcp and pcp-gui.
Other than my issue above, I don't really mind. I think it is highly
desirable to preserve the git history.
The one other "big" issue to be discussed is _what_ does unattended QA
really mean? I think I'd vote for a scheme where we had tests in the QA
suite that were _not_ run as part of the unattended run, so that we
could avoid the more difficult of the QA setup issues (like a working X
server, a close Cisco router, ...) ... and this would make much of
Tomas' issues with the setup checks just go away.
|