----- Original Message -----
> [warning, view in a wide window ... 8^)>]
>
> Here's a snapshot of my current QA landscape ... I'm about 3/4 of the way
> through a full pass across all hosts.
>
> Of the 276 failures, a small number look like systemic QA setup or QA test
> failures, but all of the others need individual inspection.
>
> It takes me between 1 and 10 minutes to triage and resolve a single QA
> failure ... so I can't clear the backlog before the next release is due and
> the situation is probably not sustainable, especially as this is a pro bono
> love job.
>
> Just FYI.
Thanks Ken. FWIW, I have little else pending for this release now. Max
has resurfaced and he's reported the new use of clock_gettime is causing
the Mac OS X build to fail, so I'll fix that up shortly. I've also been
working through the initial pmdapapi with Lukas and Frank and will merge
the work there so far (disabled by default). That's highlighting issues
with the configure.ac enabling of PMDAs, which I'll tackle also.
> Summary: 10249 run, 279 failed (2.72%)
Despite all the, wow, fail on the fail map - the above numbers are (IMO)
a truly incredible effort. FWIW we achieve only slightly better results
across the (now many) released/supported RHEL hardware platforms. There
is some jitter in tests/platforms/kernels that sometimes is unavoidable
I think (butterfly effect - some hardware platforms are flaky, and with
so many tests - complex-system factors creep in).
>From my own (far more limited) pcp-3.9.10 testing, I'm seeing roughly a
0.4% failure rate - but, yeah, thats only RHEL & Fedora boxen currently.
After the current release, I plan to dive into your farm setup and begin
helping you more proactively on more platforms and I will also start to
send out more of these failure-landscape-emails (and fixes!).
I'll need that pending mini-holiday before diving into that though. ;)
Thanks again for all your efforts, Ken, they are very much appreciated.
Having someone around with this level of dedication to quality is making
a *huge* difference here & you continue to set the standard for everyone
else to aspire to... so, thanks. :)
cheers.
--
Nathan
|