Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Leveraging Stack Traces for Spectrum-based Fault Localization in the Absence of Failing Tests (2405.00565v1)

Published 1 May 2024 in cs.SE

Abstract: Bug fixing is a crucial task in software maintenance to hold user trust. Although various automated fault localization techniques exist, they often require specific conditions to be effective. For example, Spectrum-Based Fault Localization (SBFL) techniques need at least one failing test to identify bugs, which may not always be available. Bug reports, particularly those with stack traces, provide detailed information on system execution failures and are invaluable for developers. This study focuses on utilizing stack traces from crash reports as fault-triggering tests for SBFL. Our findings indicate that only 3.33% of bugs have fault-triggering tests, limiting traditional SBFL efficiency. However, 98.3% of bugfix intentions align directly with exceptions in stack traces, and 78.3% of buggy methods are reachable within an average of 0.34 method calls, proving stack traces as a reliable source for locating bugs. We introduce a new approach, SBEST, that integrates stack trace data with test coverage to enhance fault localization. Our approach shows a significant improvement, increasing Mean Average Precision (MAP) by 32.22% and Mean Reciprocal Rank (MRR) by 17.43% over traditional stack trace ranking methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. (2024). Leveraging stack traces for spectrum-based fault localization in the absence of failing tests. https://zenodo.org/records/11062413.
  2. An evaluation of similarity coefficients for software fault localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06), pages 39–46.
  3. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007), pages 89–98. IEEE.
  4. Spectrum-based multiple fault localization. In 2009 IEEE/ACM International Conference on Automated Software Engineering, pages 88–99. IEEE.
  5. Improving test suites for efficient fault localization. In Proceedings of the 28th international conference on Software engineering, pages 82–91.
  6. What makes a good bug report? In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pages 308–318.
  7. Campos, J. et al. (2012). Gzoltar: an eclipse plug-in for testing and debugging. In Proceedings of the 27th IEEE/ACM international conference on automated software engineering, pages 378–381.
  8. Demystifying the challenges and benefits of analyzing user-reported logs in bug reports. Empirical Software Engineering, 26, 1–30.
  9. Pathidea: Improving information retrieval-based bug localization by re-constructing execution paths using logs. IEEE Transactions on Software Engineering, 48(8), 2905–2919.
  10. How useful is code change information for fault localization in continuous integration? In 37th IEEE/ACM International Conference on Automated Software Engineering, pages 1–12.
  11. Back to the future! studying data cleanness in defects4j and its impact on fault localization. arXiv preprint arXiv:2310.19139.
  12. Improving software fault localization by combining spectrum and mutation. IEEE Access, 8, 172296–172307.
  13. Ftmes: A failed-test-oriented mutant execution strategy for mutation-based fault localization. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE), pages 155–165. IEEE.
  14. Spectrum-based software fault localization: A survey of techniques, advances, and challenges. arXiv preprint arXiv:1607.04347.
  15. Interactive fault localization leveraging simple user feedback. In 2012 28th IEEE International Conference on Software Maintenance (ICSM), pages 67–76. IEEE.
  16. The importance of discerning flaky from fault-triggering test failures: A case study on the chromium ci. arXiv preprint arXiv:2302.10594.
  17. Museum: Debugging real-world multilingual programs using mutation analysis. Information and Software Technology, 82, 80–95.
  18. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, pages 273–282.
  19. Are mutants a valid substitute for real faults in software testing? In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 654–665.
  20. Defects4j: A database of existing faults to enable controlled testing studies for java programs. In Proceedings of the 2014 international symposium on software testing and analysis, pages 437–440.
  21. The future can’t help fix the past: Assessing program repair in the wild. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 50–61. IEEE.
  22. Practitioners’ expectations on automated fault localization. In Proceedings of the 25th international symposium on software testing and analysis, pages 165–176.
  23. The impact of rare failures on statistical fault localization: The case of the defects4j suite. In 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 24–28.
  24. Bug localization with combination of deep learning and information retrieval. In 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC), pages 218–229. IEEE.
  25. Theory and practice, do they match? a case with spectrum-based fault localization. In 2013 IEEE International Conference on Software Maintenance, pages 380–383.
  26. A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each. In 2012 34th International Conference on Software Engineering (ICSE), pages 3–13.
  27. Fault localization with code coverage representation learning. in 2021 ieee/acm 43rd international conference on software engineering (icse). IEEE, 661ś673.
  28. An optimal mutation execution strategy for cost reduction of mutation-based fault localization. Information Sciences, 422, 572–596.
  29. Boosting coverage-based fault localization via graph-based representation learning. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 664–676.
  30. Lutellier, T. et al. (2020). Coconut: combining context-aware neural translation models using ensemble for program repair. In Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis, pages 101–114.
  31. Ask the mutants: Mutating faulty programs for fault localization. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation, pages 153–162. IEEE.
  32. A model for spectra-based software diagnosis. ACM Transactions on software engineering and methodology (TOSEM), 20(3), 1–32.
  33. Using mutants to locate” unknown” faults. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pages 691–700. IEEE.
  34. Metallaxis-fl: mutation-based fault localization. Software Testing, Verification and Reliability, 25(5-7), 605–628.
  35. Are automated debugging techniques actually helping programmers? In Proceedings of the 2011 international symposium on software testing and analysis, pages 199–209.
  36. Evaluating & improving fault localization techniques. University of Washington Department of Computer Science and Engineering, Seattle, WA, USA, Tech. Rep. UW-CSE-16-08-03, page 27.
  37. Evaluating and improving fault localization. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), pages 609–620.
  38. Improving bug localization using structured information retrieval. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 345–355. IEEE.
  39. Lightweight fault-localization using multiple coverage types. In 2009 IEEE 31st International Conference on Software Engineering, pages 56–66.
  40. Do stack traces help developers fix bugs? In 2010 7th IEEE working conference on mining software repositories (MSR 2010), pages 118–121. IEEE.
  41. Why do developers neglect exception handling? In Proceedings of the 4th international workshop on Exception handling, pages 62–68.
  42. Silva, A. et al. (2021). Flacoco: Fault localization for java based on industry-grade coverage. arXiv preprint arXiv:2111.12513.
  43. Can higher-order mutants improve the performance of mutation-based fault localization? IEEE Transactions on Reliability, 71(2), 1157–1173.
  44. Historical spectrum based fault localization. IEEE Transactions on Software Engineering, 47(11), 2348–2368.
  45. Boosting bug-report-oriented fault localization with segmentation and stack-trace analysis. In 2014 IEEE international conference on software maintenance and evolution, pages 181–190. IEEE.
  46. Boosting bug-report-oriented fault localization with segmentation and stack-trace analysis. In 2014 IEEE International Conference on Software Maintenance and Evolution, pages 181–190.
  47. The dstar method for effective software fault localization. IEEE Transactions on Reliability, 63(1), 290–308.
  48. “automated debugging considered harmful” considered harmful: A user study revisiting the usefulness of spectra-based fault localization techniques with professionals using real bugs from large systems. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 267–278. IEEE.
  49. Neural program repair with execution-based backpropagation. In Proceedings of the 44th International Conference on Software Engineering, pages 1506–1518.
  50. Revisiting test cases to boost generate-and-validate program repair. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 35–46.
  51. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports. In 2012 34th International conference on software engineering (ICSE), pages 14–24. IEEE.
  52. What makes a good bug report? IEEE Transactions on Software Engineering, 36(5), 618–643.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets