Hi,
I'm bringing up a bunch of VMs to do QA testing, and am seeing
about 50 failures per machine. So far I've found 3 that are pretty common:
1. "bc" is required by a bunch of tests but the testsuite package
doesn't depend on it. The minimal VM's I'm starting with don't have
this installed by default. Is it reasonable to to make the testsuite
packages depend on "bc"? If so , I can add it to the build system for
the platforms I have. I have seen this on various Ubuntu and Centos VMs.
2. Some QA tests, for instance 276, try to run X based tests if the gui
package is installed regardless of whether an Xserver is running.
Should these not run? The relevant code seems to be in common.check:
**********
_check_display()
{
# note: non-X systems (MacOSX, Windows) must pass here unchallenged
which xdpyinfo >/dev/null 2>&1
if test $? -eq 0
then
DISPLAY=$PCPQA_CLOSE_X_SERVER xdpyinfo >/dev/null 2>&1 || \
_notrun "Failed sanity check on DISPLAY $PCPQA_CLOSE_X_SERVER"
export DISPLAY=$PCPQA_CLOSE_X_SERVER
fi
}
*************
In my case, I have the gui package installed, but no xdpyinfo. So
should the testsuite package depend on xdpyinfo, or should the lack of
xdpyinfo trigger a "notrun"? The comment seems to indicated that the
lack of xdpyinfo can't make the test fail. This is on Ubuntu 14.04.
3. Again on Ubuntu 14.04. Probably 20 tests fail with the following:
[4%] 039 - output mismatch (see 039.out.bad)
91a92,93
> __pmBind(/var/run/pcp/pmlogger.12496.socket): Permission denied
> __pmBind(/var/lib/pcp/.pcp/run/pmlogger.12496.socket): No such file
or directory
QA running as the pcpqa user. .bad file attached. I have not had any
luck in figuring this one out, since the actually QA test seems to run
fine, but the parsing of the log file at the end contains this error.
Thanks
Martins
039.out.bad
Description: Text document
|