Hi Lukas,
----- Original Message -----
> [...]
> Several questions moving forwards:
>
> 1. What should constitute a 'failed' testsuite run?
We should aim for zero failures. Its not easy but you get there over
time. For persistently belligerent tests there is the "flakey" group
(-g option to qa/check) used by QE - we could skip that for CI too (I
know Ken & I at least don't skip those though, and ideally we wouldn't
here either - but ... maybe a helpful option for now).
> 2. How should we report (new) failures?
>
> Do we want to at all? Should it just be an the website for devs to
> check? Buildbot is capable of running an irc bot to inform those one
> the #pcp channel. However I know not everybody uses irc.
As discussed, its likely to be noisy & fail-filled - #pcp-ci seems to
be converging as a good option for this so people can opt-in if they
are interested.
> 3. Do we want warnings if the ./Makepkgs process has any 'Warnings' in
> the output (such as unused vars, etc)?
There are some platforms where there's unfixable warnings, unfortunately;
on RHEL6 the systemtap dtrace probe wrapper generates code which gives a
compiler warning, some versions of flex/yacc do the same IIRC, and some
of the non-Linux ports are also compiler-warning heavy. So, I'd like to
do that, but not sure its going to be practical.
cheers.
--
Nathan
|