Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Java JIT Testing with Template Extraction (2403.11281v3)

Published 17 Mar 2024 in cs.SE

Abstract: We present LeJit, a template-based framework for testing Java just-in-time (JIT) compilers. Like recent template-based frameworks, LeJit executes a template -- a program with holes to be filled -- to generate concrete programs given as inputs to Java JIT compilers. LeJit automatically generates template programs from existing Java code by converting expressions to holes, as well as generating necessary glue code (i.e., code that generates instances of non-primitive types) to make generated templates executable. We have successfully used LeJit to test a range of popular Java JIT compilers, revealing five bugs in HotSpot, nine bugs in OpenJ9, and one bug in GraalVM. All of these bugs have been confirmed by Oracle and IBM developers, and 11 of these bugs were previously unknown, including two CVEs (Common Vulnerabilities and Exposures). Our comparison with several existing approaches shows that LeJit is complementary to them and is a powerful technique for ensuring Java JIT compiler correctness.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Azul Systems, Inc. 2018. AzulSystems/JavaFuzzer: Java* Fuzzer for Android*. https://github.com/AzulSystems/JavaFuzzer.
  2. Jonathan Bell and Gail Kaiser. 2014. Unit Test Virtualization with VMVM. In International Conference on Software Engineering. ACM, 550–561. https://doi.org/10.1145/2568225.2568248
  3. Korat: Automated Testing Based on Java Predicates. In International Symposium on Software Testing and Analysis. ACM, 123–133. https://doi.org/10.1145/566171.566191
  4. Bounded Exhaustive Test-Input Generation on GPUs. In International Conference on Object-Oriented Programming, Systems, Languages, and Applications. ACM, 94:1–94:25. https://doi.org/10.1145/3133918
  5. Finding Typing Compiler Bugs. In Programming Language Design and Implementation. ACM, 183–198. https://doi.org/10.1145/3519939.3523427
  6. A Survey of Compiler Testing. Comput. Surveys 53, 1 (2020), 4:1–4:36. https://doi.org/10.1145/3363562
  7. Deep Differential Testing of JVM Implementations. In International Conference on Software Engineering. IEEE, 1257–1268. https://doi.org/10.1109/ICSE.2019.00127
  8. Coverage-Directed Differential Testing of JVM Implementations. In Programming Language Design and Implementation. ACM, 85–99. https://doi.org/10.1145/2908080.2908095
  9. Wai-Mee Ching and Alex Katz. 1993. The Testing of an APL Compiler. In International Conference on APL. ACM, 55–62. https://doi.org/10.1145/166197.166205
  10. Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In International Conference on Functional Programming. ACM, 268–279. https://doi.org/10.1145/351240.351266
  11. Automated Testing of Refactoring Engines. In Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering. ACM, 185–194. https://doi.org/10.1145/1287624.1287651
  12. Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11, 4 (1978), 34–41. https://doi.org/10.1109/C-M.1978.218136
  13. Eclipse Foundation, Inc. 2024. The order of super interface initialization in J9 is strange - Issue #13242 - eclipse-openj9/openj9. https://github.com/eclipse-openj9/openj9/issues/13242.
  14. Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic Test Suite Generation for Object-Oriented Software. In Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering. ACM, 416–419. https://doi.org/10.1145/2025113.2025179
  15. Vectorizing Program Ingredients for Better JVM Testing. In International Symposium on Software Testing and Analysis. ACM, 526–537. https://doi.org/10.1145/3597926.3598075
  16. GitHub, Inc. 2023. GitHub. https://github.com.
  17. Test Generation through Programming in UDITA. In International Conference on Software Engineering. ACM, 225–234. https://doi.org/10.1145/1806799.1806835
  18. James Gosling and Greg Bollella. 2000. The Real-Time Specification for Java. Addison-Wesley.
  19. JavaParser.org. 2024. JavaParser - Home. https://javaparser.org.
  20. Detecting JVM JIT Compiler Bugs via Exploring Two-Dimensional Input Spaces. In International Conference on Software Engineering. IEEE, 43–55. https://doi.org/10.1109/ICSE48619.2023.00016
  21. Yue Jia and Mark Harman. 2008. Constructing Subtle Faults Using Higher Order Mutation Testing. In IEEE International Working Conference on Source Code Analysis and Manipulation. IEEE, 249–258. https://doi.org/10.1109/SCAM.2008.36
  22. JUnit. 2022. JUnit - About. https://junit.org/junit4/.
  23. Evaluating Fuzz Testing. In Conference on Computer and Communications Security. ACM, 2123–2138. https://doi.org/10.1145/3243734.3243804
  24. Validating JIT Compilers via Compilation Space Exploration. In Symposium on Operating Systems Principles. ACM, 66–79. https://doi.org/10.1145/3600006.3613140
  25. Random Testing for C and C++ Compilers with YARPGen. In International Conference on Object-Oriented Programming, Systems, Languages, and Applications. ACM, 196:1–196:25. https://doi.org/10.1145/3428264
  26. William M McKeeman. 1998. Differential Testing for Software. Digital Technical Journal 10, 1 (1998), 100–107. https://www.hpl.hp.com/hpjournal/dtj/vol10num1/vol10num1art9.pdf.
  27. Kazuhiro Nakamura and Nagisa Ishiura. 2016. Random Testing of C Compilers Based on Test Program Generation by Equivalence Transformation. In Asia Pacific Conference on Circuits and Systems. IEEE, 676–679. https://doi.org/10.1109/APCCAS.2016.7804063
  28. Oracle Corporation and/or its affiliates. 2021. The Java HotSpot Performance Engine Architecture. https://www.oracle.com/java/technologies/whitepaper.html.
  29. Oracle Corporation and/or its affiliates. 2023a. [JDK-8280126] C2: detect and remove dead irreducible loops - Java Bug System. https://bugs.openjdk.java.net/browse/JDK-8280126.
  30. Oracle Corporation and/or its affiliates. 2023b. [JDK-8280126] C2: detect and remove dead irreducible loops - Java Bug System. https://bugs.openjdk.org/browse/JDK-8280126?focusedCommentId=14476253&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14476253.
  31. Oracle Corporation and/or its affiliates. 2023c. openjdk/jdk: JDK main-line development. https://github.com/openjdk/jdk.
  32. OW2. 2024. ASM. https://asm.ow2.io.
  33. Feedback-Directed Random Test Generation. In International Conference on Software Engineering. IEEE, 75–84. https://doi.org/10.1109/ICSE.2007.37
  34. Priya Khaira-Hanks. 2023. What is a Java Uber-JAR and Why Is It Useful? https://blog.payara.fish/what-is-a-java-uber-jar.
  35. Emin Gün Sirer and Brian N. Bershad. 2000. Using Production Grammars in Software Testing. In Conference on Domain-Specific Languages. ACM, 1–13. https://doi.org/10.1145/331960.331965
  36. The Apache Software Foundation. 2023a. commons-codec/BinaryCodec.java. https://github.com/apache/commons-codec/blob/4de60e/src/main/java/org/apache/commons/codec/binary/BinaryCodec.java.
  37. The Apache Software Foundation. 2023b. commons-codec/StringUtils.java. https://github.com/apache/commons-codec/blob/4de60e/src/main/java/org/apache/commons/codec/binary/StringUtils.java.
  38. The Apache Software Foundation. 2023c. commons-math/AdamsNordsieckTransformer.java. https://github.com/apache/commons-math/blob/dff1a0/src/main/java/org/apache/commons/math4/ode/nonstiff/AdamsNordsieckTransformer.java.
  39. The Apache Software Foundation. 2023d. commons-math/Quaternion.java. https://github.com/apache/commons-math/blob/dff1a0/src/main/java/org/apache/commons/math4/complex/Quaternion.java.
  40. The Apache Software Foundation. 2023e. commons-text/StrBuilder.java. https://github.com/apache/commons-text/blob/e62203/src/main/java/org/apache/commons/text/StrBuilder.java.
  41. The Apache Software Foundation. 2023f. Maven - Welcome to Apache Maven. https://maven.apache.org/.
  42. The JUnit Team. 2023. JUnit 5. https://junit.org/junit5/.
  43. Growing A Test Corpus with Bonsai Fuzzing. In International Conference on Software Engineering. ACM, 723–735. https://doi.org/10.1109/ICSE43902.2021.00072
  44. JITfuzz: Coverage-guided Fuzzing for JVM Just-in-Time Compilers. In International Conference on Software Engineering. IEEE, 56–68. https://doi.org/10.1109/ICSE48619.2023.00017
  45. SJFuzz: Seed & Mutator Scheduling for JVM Fuzzing. In Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering. ACM, 1062–1074. https://doi.org/10.1145/3611643.3616277
  46. Finding and Understanding Bugs in C Compilers. In Programming Language Design and Implementation. ACM, 283–294. https://doi.org/10.1145/1993316.1993532
  47. GCC Bug List Found by Random Testing (Total 79). https://embed.cs.utah.edu/csmith/gcc-bugs.html.
  48. LLVM Bug List Found by Random Testing (Total 203). https://embed.cs.utah.edu/csmith/llvm-bugs.html.
  49. A Generative and Mutational Approach for Synthesizing Bug-exposing Test Cases to Guide Compiler Fuzzing. In Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering. ACM, 1127–1139. https://doi.org/10.1145/3611643.3616332
  50. Random Program Generator for Java JIT Compiler Test System. In International Conference on Quality Software. IEEE, 20–23. https://doi.org/10.1109/QSIC.2003.1319081
  51. Compiler Testing using Template Java Programs. In International Conference on Automated Software Engineering. ACM, 23:1–23:13. https://doi.org/10.1145/3551349.3556958
  52. Skeletal Program Enumeration for Rigorous Compiler Testing. In Programming Language Design and Implementation. ACM, 347–361. https://doi.org/10.1145/3140587.3062379
  53. History-Driven Test Program Synthesis for JVM Testing. In International Conference on Software Engineering. ACM, 1133–1144. https://doi.org/10.1145/3510003.3510059
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Zhiqiang Zang (3 papers)
  2. Fu-Yao Yu (1 paper)
  3. Aditya Thimmaiah (3 papers)
  4. August Shi (3 papers)
  5. Milos Gligoric (23 papers)
Citations (2)
View on Semantic Scholar