webapp testing (was Re: pcp updates: python fix, build, qa, and docs tweaks (+lberk+mgoodwin+minnus merges))

[Nathan Scott <nathans@xxxxxxxxxx>]

----- Original Message -----
> [...]    (And for a real pcp+webjs release, one
> would normally prefer a good stable release for both rather than a
> snapshot, which brings us back to manual judgement & testing & c.)
> 

If the level of testing on upstream Vector releases is concerning you
(bear in mind there are many browsers that will be rendering Vector,
so its a hard problem) consider making a pre-release checklist.  This
could be just a manual set of checks to run through, if automating the
testing is too much effort to write/setup (selenium-webdriver?).

We've had some success in the past with the GUI tools (pmchart, pmtime)
with a checklist approach, back in the day, as an extra sanity-check to
do just prior to release.

I imagine this could be useful for both upstream Vector release testing
and for downstream distro integration testing; and possibly for people
working on web api fixes/features too.

At this stage our best strategy is getting new code out in a more timely
fashion, so that we increase the base of potential testers - but a more
formalised testing model (even just the simple checklist) would still be
valuable.  Likewise for grafana/graphite.

cheers.

--
Nathan