Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding and Detecting Annotation-Induced Faults of Static Analyzers (2402.14366v1)

Published 22 Feb 2024 in cs.SE

Abstract: Static analyzers can reason about the properties and behaviors of programs and detect various issues without executing them. Hence, they should extract the necessary information to understand the analyzed program well. Annotation has been a widely used feature for different purposes in Java since the introduction of Java 5. Annotations can change program structures and convey semantics information without awareness of static analyzers, consequently leading to imprecise analysis results. This paper presents the first comprehensive study of annotation-induced faults (AIF) by analyzing 246 issues in six open-source and popular static analyzers (i.e., PMD, SpotBugs, CheckStyle, Infer, SonarQube, and Soot). We analyzed the issues' root causes, symptoms, and fix strategies and derived ten findings and some practical guidelines for detecting and repairing annotation-induced faults. Moreover, we developed an automated testing framework called AnnaTester based on three metamorphic relations originating from the findings. AnnaTester generated new tests based on the official test suites of static analyzers and unveiled 43 new faults, 20 of which have been fixed. The results confirm the value of our study and its findings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. adangel. 2019. NoPackage: False Negative for enums. https://github.com/pmd/pmd/issues/1782
  2. The dataflow pointcut: a formal and practical framework. In Proceedings of the 8th ACM international conference on Aspect-oriented software development. 15–26.
  3. Correlating automatic static analysis and mutation testing: towards incremental strategies. Journal of Software Engineering Research and Development 4, 1 (2016), 1–32.
  4. Nathaniel Ayewah and William Pugh. 2010. The Google FindBugs fixit. 241–252. https://doi.org/10.1145/1831708.1831738
  5. Evaluating static analysis defect warnings on production software. In Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering. 1–8.
  6. Belle. 2022. A false positive about rule RSPEC-2095. https://community.sonarsource.com/t/a-false-positive-about-rule-rspec-2095/67536
  7. Belle-PL. 2022. Parse error on array type annotations. https://github.com/pmd/pmd/issues/4152#issuecomment-1277447394
  8. Cristian Cadar and Alastair F Donaldson. 2016. Analysing the program analyser. In Proceedings of the 38th International Conference on Software Engineering Companion. 765–768.
  9. G Ann Campbell and Patroklos P Papapetrou. 2013. SonarQube in action. Manning Publications Co.
  10. Walter Cazzola and Edoardo Vacchi. 2014. @ Java: Bringing a richer annotation model to Java. Computer Languages, Systems & Structures 40, 1 (2014), 2–18.
  11. CheckStyle. 2022. Checkstyle. https://checkstyle.sourceforge.io/
  12. Understanding exception-related bugs in large-scale cloud systems. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 339–351.
  13. Metamorphic testing: A review of challenges and opportunities. ACM Computing Surveys (CSUR) 51, 1 (2018), 1–27.
  14. Maria Christakis and Christian Bird. 2016. What developers want and need from program analysis: an empirical study. In Proceedings of the 31st IEEE/ACM international conference on automated software engineering. 332–343.
  15. Testing static analyzers with randomly generated programs. In NASA Formal Methods Symposium. Springer, 120–125.
  16. Krysztof Czarnecki and Ulrich Eisenecker. 2000. Generative Programming: Methods, Tools, and Applications. Addison-Wesley, Boston, MA.
  17. Why Do Software Developers Use Static Analysis Tools? A User-Centered Study of Developer Needs and Motivations. IEEE Transactions on Software Engineering 48, 3 (2022), 835–847. https://doi.org/10.1109/TSE.2020.3004525
  18. fluxroot. 2019. False-positive with Lombok and inner classes. https://github.com/pmd/pmd/issues/1641
  19. Michael Gumowski. 2015. Annotations should be handled in all cases allowed by java 8. https://sonarsource.atlassian.net/browse/SONARJAVA-1167
  20. Michael Gumowski. 2016. Classes annotated with lombok’s @EqualsAndHashCode should be ignored. https://sonarsource.atlassian.net/browse/SONARJAVA-1513
  21. Michael Gumowski. 2017a. Annotations on type in a fully qualified name are not resolved. https://sonarsource.atlassian.net/browse/SONARJAVA-2205
  22. Michael Gumowski. 2017b. FP on S1128 (UselessImportCheck) when annotation is used in fully qualified name. https://sonarsource.atlassian.net/browse/SONARJAVA-2083
  23. Michael Gumowski. 2019a. Annotations are not always resolved when used in parameterized types. https://sonarsource.atlassian.net/browse/SONARJAVA-3045
  24. Michael Gumowski. 2019b. Generating starting states make analysis crash (OutOfMemory) when too many annotated parameters. https://sonarsource.atlassian.net/browse/SONARJAVA-3108
  25. A. Habib and M. Pradel. 2018. How Many of All Bugs Do We Find? A Study of Static Bug Detectors. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). 317–328. https://doi.org/10.1145/3238147.3238213
  26. Infer. 2022. Infer Static Analyzer. https://fbinfer.com/
  27. Quentin Jaquier. 2019. Support Java 11 Generated annotation. https://sonarsource.atlassian.net/browse/SONARJAVA-3174
  28. Quentin Jaquier. 2020. Consistently support Nullable/CheckForNull/Nonnull annotations in rules. https://sonarsource.atlassian.net/browse/SONARJAVA-3536
  29. Quentin Jaquier. 2023. S5122: ClassCastException when annotation is defined with an identifier. https://sonarsource.atlassian.net/browse/SONARJAVA-3438
  30. Why don’t software developers use static analysis tools to find bugs?. In 2013 35th International Conference on Software Engineering (ICSE). IEEE, 672–681.
  31. Josh Juneau. 2022. JSR 308 Explained: Java Type Annotations. https://www.oracle.com/technical-resources/articles/java/ma14-architect-annotations.html
  32. Learning to reduce false positives in analytic bug detectors. In Proceedings of the 44th International Conference on Software Engineering. 1307–1316.
  33. Studying Test Annotation Maintenance in the Wild. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 62–73. https://doi.org/10.1109/ICSE43902.2021.00019
  34. Differentially testing soundness and precision of program analyzers. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 239–250.
  35. konrad jamrozik. 2013. Regression: ERROR/dalvikvm(1854): Invalid type descriptor: ’dalvik.annotation.EnclosingClass’. https://github.com/soot-oss/soot/issues/123
  36. krzyk. 2015. SuppressWarnings should support CamelCase. https://github.com/checkstyle/checkstyle/issues/2202
  37. Compiler validation via equivalence modulo inputs. ACM Sigplan Notices 49, 6 (2014), 216–226.
  38. lgoldstein. 2017. EmptyBlock: NPE on annotation declaration. https://github.com/checkstyle/checkstyle/issues/4472
  39. Junjie Li and Jinqiu Yang. 2024. Tracking the Evolution of Static Code Warnings: the State-of-the-Art and a Better Approach. IEEE Transactions on Software Engineering (2024).
  40. An empirical study on the effectiveness of static C code analyzers for vulnerability detection. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 544–555.
  41. Dongmiao Liu. 2022. SONARJAVA-73 add more lombok’s used annotations for UnusedPrivateFieldCheck. https://github.com/SonarSource/sonar-java/pull/102#issuecomment-87545890
  42. A Comprehensive Study on Quality Assurance Tools for Java. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (Seattle, WA, USA) (ISSTA 2023). Association for Computing Machinery, New York, NY, USA, 285–297. https://doi.org/10.1145/3597926.3598056
  43. DeepAnna: Deep Learning based Java Annotation Recommendation and Misuse Detection. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 685–696.
  44. LynnBroe. 2021. UnnecessaryConstructor: false-positive with @Inject. https://github.com/pmd/pmd/issues/4487
  45. Metamorphic testing of Datalog engines. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 639–650.
  46. marcelmore. 2021. UnusedPrivateMethod violation for disabled class in 6.23. https://github.com/pmd/pmd/issues/2454
  47. martin. 2020. UnusedPrivateField cannot override ignored annotations property. https://github.com/pmd/pmd/issues/2876
  48. mjustin. 2020. Exception when using SuppressWarningsHolder with @SuppressWarnings as an annotation property. https://github.com/checkstyle/checkstyle/issues/7522
  49. ECSTATIC: An Extensible Framework for Testing and Debugging Configurable Static Analysis. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 550–562. https://doi.org/10.1109/ICSE48619.2023.00056
  50. msridhar. 2017. Eradicate not reading annotations from class file. https://github.com/facebook/infer/issues/559
  51. Marharyta Nedzelska. 2021. FP in S3077 when volatile is used with @Immutable and @ThreadSafe annotations. https://sonarsource.atlassian.net/browse/SONARJAVA-3804
  52. nrmancuso. 2020. Compact Constructor AST is missing annotations. https://github.com/checkstyle/checkstyle/issues/8734
  53. Mining Annotation Usage Rules: A Case Study with MicroProfile. In 2022 38th International Conference on Software Maintenance and Evolution, IEEE.
  54. Mining Annotation Usage Rules: A Case Study with MicroProfile. In 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 553–562.
  55. Fernando Rodriguez Olivera. 2017. FindBugs JSR305. https://mvnrepository.com/artifact/com.google.code.findbugs/jsr305
  56. Fernando Rodriguez Olivera. 2023. MvnRepository. https://mvnrepository.com/
  57. oowekyala. 2022. How PMD Works. https://docs.pmd-code.org/pmd-doc-6.53.0/pmd_devdocs_how_pmd_works.html
  58. Adoption and use of Java generics. Empirical Software Engineering 18, 6 (2013), 1047–1089.
  59. Nicolas Peru. 2015. Annotation on array type should be properly handled. https://sonarsource.atlassian.net/browse/SONARJAVA-1420
  60. Mutating code annotations: An empirical evaluation on Java and C# programs. Science of Computer Programming 191 (2020), 102418.
  61. PMD. 2022. PMD An extensible cross-language static code analyzer. https://pmd.github.io/
  62. William Pugh. 2022. JSR 305: Annotations for Software Defect Detection. https://jcp.org/en/jsr/detail?id=305
  63. rnveach. 2016. Java 8 Grammar: annotations on varargs parameters. https://github.com/checkstyle/checkstyle/issues/3238
  64. rnveach. 2021. AtclauseOrder: Falsely ignores method with annotation. https://github.com/checkstyle/checkstyle/issues/9941
  65. robtimus. 2021. OperatorWrap with token ASSIGN too strict for annotations. https://github.com/checkstyle/checkstyle/issues/10945
  66. Henrique Rocha and Marco Tulio Valente. 2011. How Annotations are Used in Java: An Empirical Study.. In SEKE. 426–431.
  67. A comprehensive study of deep learning compiler bugs. In Proceedings of the 29th ACM Joint meeting on european software engineering conference and symposium on the foundations of software engineering. 968–980.
  68. SonarQube. 2022. SonarQube Code Quality and Code Security. https://www.sonarqube.org/
  69. Soot. 2023. Soot - A framework for analyzing and transforming Java and Android applications. http://soot-oss.github.io/soot/
  70. SpotBugs. 2022. SpotBugs: Find bugs in Java Programs. https://spotbugs.github.io/
  71. Toward understanding compiler bugs in GCC and LLVM. In Proceedings of the 25th international symposium on software testing and analysis. 294–305.
  72. An empirical study on real bugs for machine learning programs. In 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE, 348–357.
  73. Testing static analyses for precision and soundness. In Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Optimization. 81–93.
  74. Static checking of safety critical Java annotations. In Proceedings of the 8th International Workshop on Java Technologies for Real-Time and Embedded Systems. 148–154.
  75. An empirical study of bugs in machine learning systems. In 2012 IEEE 23rd International Symposium on Software Reliability Engineering. IEEE, 271–280.
  76. David A Tomassi and Cindy Rubio-González. 2021. On the real-world effectiveness of static bug detectors at finding null pointer exceptions. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 292–303.
  77. Cohen’s kappa coefficient as a performance measure for feature selection. In International conference on fuzzy systems. IEEE, 1–8.
  78. Jules Villard. 2020. Infer workflow. https://fbinfer.com/docs/infer-workflow
  79. vovkss. 2018. [java] Processing error (ClassCastException) if a TYPE_USE annotation is used on a base class in the ”extends” clause. https://github.com/pmd/pmd/issues/1369
  80. Bug characteristics in blockchain systems: a large-scale empirical study. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 413–424.
  81. Find Bugs in Static Bug Finders. In 2022 IEEE/ACM 30th International Conference on Program Comprehension (ICPC). 516–527. https://doi.org/10.1145/3377811.3380380
  82. Characterizing the Usage, Evolution and Impact of Java Annotations in Practice. IEEE Transactions on Software Engineering 47, 5 (2021), 969–986. https://doi.org/10.1109/TSE.2019.2910516
  83. Statfier: Automated Testing of Static Analyzers via Semantic-preserving Program Transformations. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery (ACM).
  84. An empirical study on program failures of deep learning jobs. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 1159–1170.
  85. An empirical study on TensorFlow program bugs. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 129–140.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets