[Back]

S. Kandl:
"Cost Effectiveness of Coverage-Guided Test-Suite Reduction for Safety-Relevant Systems";
Talk: Twenty-Third International Conference on Systems Engineering (ICSEng 2014), Las Vegas, Nevada, USA; 2014-08-19 - 2014-08-21; in: "Proceedings of the Twenty-Third International Conference on Systems Engineering (ICSEng 2014)", (2014).

Test-suite reduction targets to find a minimal subset of the test suite
that is sufficient regarding specific test requirements. In general, test-suite reduction
causes some kind of loss in the fault-detection ability of the reduced
test suite. Cost-Effectiveness for test-suite reduction is only given if the tradeoff
between the savings in the testing effort and the resulting degradation in the
fault-detection ability is reasonable. In this work we define a cost function, summarizing
the beneficial output of a test suite depending on the test effort and the
fault-detection ability. We apply this cost function to test suites that are defined
by different commonly used coverage criteria, like MCC (multiple condition coverage),
MC/DC (modified condition/decision coverage), and DC (decision coverage)
and evaluate the cost function for a safety-relevant system from automotive.
The results mainly demonstrate two fundamental aspects of test-suite reduction:
First, the cost effectiveness is very sensitive to the assumed costs for the test-case
execution and the penalty for undetected faults. Secondly, for non-safety-relevant
systems test-suite reduction for DC is still cost effective, but this does not hold for
safety-relevant systems. The cost effectiveness of test-suite reduction for safetyrelevant
systems for MCC and MC/DC is almost the same.