Estimation of quality of OSI test suites by using necessity rate: theory and applications
✍ Scribed by Tadao Wakasugi
- Publisher
- John Wiley and Sons
- Year
- 1999
- Tongue
- English
- Weight
- 115 KB
- Volume
- 82
- Category
- Article
- ISSN
- 8756-6621
No coin nor oath required. For personal study only.
✦ Synopsis
In the international collaboration on OSI conformance testing, it is necessary to establish an evaluation method for the test suite quality. This paper tries to evaluate the conformance test suite quality by an analysis using the concept of necessity rate. In other words, my purpose is not to develop a new test suite, but to evaluate the existing test suite and to improve the effectiveness of the test. By necessity rate is meant the extent to which faults can be detected. As a first step, based on the description of the purposes of the test, the concept of similar class is defined. For each of the specified test purposes, the evaluation score is defined and is used to represent the necessity rate of the test suite. The difficulty level is defined separately from the necessity rate, and the two are used to define the priority of test execution for each test purpose. Based on those ideas, a simple evaluation measure for the necessity rate, called pseudo-coverage, is proposed. The method is applied to the transport test suite constructed by ISO, and the necessity rate is represented by a numerical score. This example demonstrates that the proposed idea is practically useful.