pcp
[Top] [All Lists]

Re: [pcp] State of PCP QA

To: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Subject: Re: [pcp] State of PCP QA
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Mon, 3 Aug 2015 01:44:09 -0400 (EDT)
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <20150802224512.2F615A2DD2@bozo>
References: <20150802224512.2F615A2DD2@bozo>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: qYl1csg5vxJUeWiMr7vUAKymWvAm6A==
Thread-topic: State of PCP QA
Hi Ken,

----- Original Message -----
> From my QA Farm ...
> 
> Executive Summary: 21108 run, 95 failed (0.45%)

Looking good - thanks to everyone who's been working on getting this
number down!  (special thanks to Lukas for a big effort the last few
releases incl. behind the scenes with the RHEL testing folks).

> 850: -a option problem, failing everywhere, fix under review
>      [21 failures]

We're expecting this to pass everywhere now - have checked py2+3 here.

> 975: qa comparison logic is not fuzzy enough, code is probably OK
>      [9 failures]

*nod* - I did make some initial tweaks here, but too conservative
it seems.  Its sensitive to non-QA activity on the test machine.

> 196: has become unstable, probably due to changes in default pmlogger
>      config and the broken pmcd this test uses ... needs test
>      reengineering
>      [6 failures]
> 
> 662: Python differences between platforms make this a challenge ...
>      the only option seems to be to expand the perimeter of the
>      older Python versions where tests like this are notrun
>      [6 failures]
> 
> 957: network.softnet induced failure (metric not supported), fixed in
>      qa test, but has not yet worked its way across the farm
>      [3 failures]

Ah, sorry - I missed this one during earlier commits in this vein.

> 964: multiple failure modes, not diagnosis as yet
>      [3 failures]
> 
> And then it is the normal noise of onsies and twosies.
> 

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>
  • State of PCP QA, Ken McDonell
    • Re: [pcp] State of PCP QA, Nathan Scott <=