[Back]

S. Kandl, J. Forey:
"Fault-Detection Sensitivity Based Assessment of Test Sets for Safety-Relevant Software (Best Paper Award)";
Talk: Seventh International Conference on Dependability (DEPEND 2014), Lisbon, Portugal; 2014-11-16 - 2014-11-20; in: "Proceedings of the Seventh International Conference on Dependability (DEPEND 2014)", (2014), ISBN: 978-1-61208-378-0.

In testing it is, in general, not possible (or at least extremely time-consuming) to cover the complete input data space, therefore usually a test set is selected for a given coverage criterion (like decision coverage or similar). This restricted test set covers only a part of the complete input data space with a degraded fault-detection capability. The fault-detection capability of a test set is given by the number of detected faults in relation to the number of actual faults. In this work, we present a novel approach for the assessment of test sets based on their fault-detection sensitivity. The main goal of an efficient testing process is to reduce the test effort while targeting a maximum number of detected faults, i.e., an ideal test set requires a minimal execution time for the test run (defined by the number of the test cases and their individual run-times) with a maximum fault-detection sensitivity. Therefore, our proposed test-set selection process is not guided by a coverage criterion, but by the fault-detection capability of the different test cases using the output of the tool Certitude Functional Qualification System. We will apply our strategy on a safety-relevant (regarding ISO 26262) case study from the automotive domain focusing on the scalability of the test-set selection strategy. Our work presents a novel method to optimize the testing effort for software used in dependable systems with respect to a decreased testing effort while sustaining a high fault-detection capability.

Dependable Systems, Testing, Fault-Detection Sensitivity, Safety-Relevant Software