SGI - Developer Central Open Source

Linux Test Project Glossary

Test
IEEE/ANSI [1]:
(i) An activity in which a system or component is executed under specified conditions, the results are observed or recorded, and an evaluation is made of some aspect of the system or component.
(ii) A set of one or more test cases.
Test Case
A test assertion with a single result that is being verified. This allows designations such as PASS or FAIL to be applied to a single bit of functionality. A single test case may be one of many test cases for testing the complete functionality of a system.
IEEE/ANSI [1]:
(i) A set of test inputs, execution conditions, and expected results developed for a particular objective.
(ii) The smallest entity that is always executed as a unit, from beginning to end.
Test Framework
A Test Framework is a mechanism for organising a group of tests. Frameworks may have complex or very simple API's, dirvers and result logging mechanisms. Examples of frameworks are TETware and DejaGnu.
Test Harness
A Test harness is the mechanism that connects a test program to a test framework. It may be a specification of exit codes, or a set of libraries for formatting messages and determining exit codes. In TETware, the tet_result() API is the test harness.
Test Suite
A collection of tests programs, assertions, cases grouped together under a framework.
Test Program
A single invokable program. A test program can contain one or more test cases. The test harness's API allows for reporting/analysis of the individual test cases.

[1] Kit, Edward, Software Testing in the Real World: Improving the Process. P. 82. ACM Press, 1995.