Prevalence of coincidental correctness and mitigation of its impact on fault localization

Abstract

The article examines the prevalence of coincidental correctness and mitigation of its impact on fault localization. Coverage-Based Fault Localization (CBFL) techniques seek to identify failure correlated program elements using test suites in which tests are tagged as failing or passing, that is, elements that are induced by all (or most) failing runs and not induced by any (or most) passing runs; and locate the faulty code using some examination strategy. To evaluate the effectiveness of research techniques, researchers empirically quantified their accuracy in identifying CC tests in test suites involving programs with seeded faults as well as real faults. The results were promising, for example, the better performing technique, using 105 test suites and statement coverage, exhibited 9% false negatives, 30% false positives, and no false negatives nor false positives in 14.3% of the test suites.