Doric: Foundations for Statistical Fault Localisation (1810.00798v1)

Abstract: To fix a software bug, you must first find it. As software grows in size and complexity, finding bugs is becoming harder. To solve this problem, measures have been developed to rank lines of code according to their "suspiciousness" wrt being faulty. Engineers can then inspect the code in descending order of suspiciousness until a fault is found. Despite advances, ideal measures --- ones which are at once lightweight, effective, and intuitive --- have not yet been found. We present Doric, a new formal foundation for statistical fault localisation based on classical probability theory. To demonstrate Doric's versatility, we derive cl, a lightweight measure of the likelihood some code caused an error. cl returns probabilities, when spectrum-based heuristics (sbhs) usually return difficult to interpret scores. cl handles fundamental fault scenarios that spectrum-based measures cannot and can also meaningfully identify causes with certainty. We demonstrate its effectiveness in, what is to our knowledge, the largest scale experiment in the fault localisation literature. For Defects4J benchmarks, cl permits a developer to find a fault after inspecting 6 lines of code 41.18% of the time. Furthermore, cl is more accurate at locating faults than all known 127 sbh. In particular, on Steimann's benchmarks one would expect to find a fault by investigating 5.02 methods, as opposed to 9.02 with the best performing sbh.