Re: Access to the PCP QA Suite

[Alan Hoyt <ahoyt@xxxxxxxxxxxxx>]

Yes, this is rather problematic.

Not having seen the code, I am flying blind here - but wouldnt large 
portions of the QA code base be relatively constant/stable?

Couldn't most of the hard coded dependencies be abstracted into 
configuration/initialization files to minimize architectural 
/topological dependencies?

In other words, could we substantially reduce the gatekeeper load 
through well conceived code changes/clean ups (sounds like a catch 22)?

If not, the simplest solution would be to release PCP along with the QA 
scripts that were used to validate it (i.e. without accepting QA script 
patches). At least the community would have tools to exhaustively verify 
PCP changes before they were submitted/deployed.

Possibly, you could monitor QA script changes made by the community and 
determine (at that time) whether there is any benefit for SGI to 
incorporate those changes.

- Alan -


Ken McDonell wrote:

> I've been lurking (and enjoying the sudden flurry of activity on the
> list!).
>
> The issues for the PCP QA are as follows:
>
> 0. The value to PCP developers and PCP in general is beyond dispute,
>   so this is _not_ an issue.
>
> 1. It is _big_ - 600+ scripts, 180+ source files with 42,000+ lines of
>   code.
>
> 2. The whole suite runs daily on a number of machines within SGI, and
>   has a fair degree of local customization ... hostnames where various
>   versions of PCP are running, ip addr of a friednly Cisco router,
>   hostname where a promiscuous X server is hiding, etc
>
> 3. It has a lot of multi-platform infrastructre with individual tests
>   that are skipped on IRIX or some IRIX versions, ia32 Linux, ia64
>   Linux, some versions of PCP, etc.  As more platforms are added to
>   the mix this will grow.
>
> 4. There is a large potential for gatekeeper load to answer questions
>   based on 1. to 3. above and to reconcile changes back from the
>   community.  There is little point in making the suite available
>   and then not following up with the gatekeeper function.
>
> 5. We have a potential takeback assignment problem ... the PCP QA suite
>   is used within SGI as QA for the proprietary version of PCP ... not
>   everyone would be happy to have changes taken back by SGI to be used
>   in this way.
>
> Note there is no issue associated with IP or releasing this to the open
> source community as a matter of principle (I'm a member of the SGI
> committee that judges these matters, so I'm pretty sure of this).
>
> The _big_ item is 4. ... I just do not have the resources to sign up to
> this at the level required to ensure the test suite remains useful to
> SGI and the community.
>
> I'm open to any creative suggestions that would help us more forward on
> this matter.
>
>
>