Hi -
rdoyle wrote:
> [...]
> I've created a branch "vagrant-qa" at the Aconex guthub repository (https://
> github.com/Aconex/pcp.git) if you'd like to have a look.
> [...]
Neat. One problem with an overly hardcoded approach like
<https://github.com/Aconex/pcp/blob/vagrant-qa/Vagrantfile> is that it
can encourage a monoculture of pcp test environments (a particular
ubuntu version etc. in this case). That can lead to false positive
test results, when new|old code avoids being run on actual new|old OS
kernels, architectures, ..., and so has a beautiful pass rate on the
VM but fails in real life. "All Of This Has Happened Before And Will
Happen Again."
If we can use Vagrant or such to be able to instantiate a wide range
of diverse environments (like kenj's farm on steroids), that'd address
this nicely.
> - Anyone can run the QA and they don't need to know how to setup the
> prerequisites. [...]
As far as building prerequisites go, this could be another instance of
the above problem: a monoculture of build configurations. PCP has
autoconf* machinery to detect presence or absence of numerous optional
libraries, and that stuff needs to be exercised within the cartesian
product / configuration space.
> - Set up of the environment required to run the
> QA test is captured as code
That is a great property. I believe lberk is working on improving
qa/check and pals to automate prerequisite setup in general.
- FChE
|