CrossLangFuzzer: Cross-Language JVM Testing
- CrossLangFuzzer is a differential-testing framework that exposes cross-language semantic bugs in JVM compilers through a unified intermediate representation.
- It applies seven targeted mutation operators to stress compiler interoperability in key areas such as generics, nullability, and override semantics.
- Empirical evaluation across major JVM compilers revealed 32 confirmed bugs, demonstrating its practical impact on detecting real-world compiler defects.
CrossLangFuzzer is a differential-testing framework for cross-language JVM compilation, designed to expose bugs that arise when code written in one JVM language is compiled and linked with code written in another. The framework targets the multilingual interoperability that is common in JVM software, especially among Java, Kotlin, Groovy, and Scala, and focuses on compiler failures caused by semantic mismatches at language boundaries rather than within isolated single-language compilation pipelines. In its 2026 form, CrossLangFuzzer uses a unified intermediate representation modeled on Kotlin compiler IR concepts, applies seven mutation operators, and evaluates generated programs across five major JVM compilers, uncovering 32 confirmed bugs in the latest actively maintained compiler versions (Ma et al., 26 Jun 2026).
1. Cross-language JVM compilation as a compiler-testing problem
Modern JVM systems frequently combine classes and libraries written in Java, Kotlin, Groovy, and Scala within a single application. In this setting, compiler correctness depends not only on parsing and translating a language’s own source code, but also on consuming artifacts emitted by other JVM languages and reconciling their semantics across language boundaries. CrossLangFuzzer is built around the claim that this interoperability layer is practically important and still under-tested, because most existing compiler fuzzers focus on one language at a time (Ma et al., 26 Jun 2026).
The framework’s problem setting is shaped by several classes of semantic mismatch. The paper identifies differences in type systems and generics, nullability, inheritance and override resolution, and cross-language metadata interpretation as particularly difficult areas. These are not superficial syntax-level incompatibilities. They concern whether one compiler correctly interprets declarations, type information, and inheritance structure produced by another compiler. A representative example in the paper shows a Java raw-type override that Kotlin mishandles: Kotlin rejects a program that Java accepts because the Kotlin compiler fails to connect a raw-type override with the generic interface contract. That example is explicitly presented as illustrative rather than as one of the newly reported 32 bugs, but it captures the broader class of cross-language semantic defects that CrossLangFuzzer is intended to detect (Ma et al., 26 Jun 2026).
This focus distinguishes cross-language compiler testing from single-language compiler fuzzing. A compiler may behave correctly on programs written purely in its own source language and still fail when required to interpret foreign-language class files, generics, raw or erased types, nullable or platform types, or override relations encoded through another language’s conventions. The earlier CrossLangFuzzer paper made the same general point in a more preliminary form, arguing that many bugs only appear when one language consumes classes, interfaces, or methods produced by another language, because the compilers must agree on inheritance, overriding, generics, raw types, and language-specific type features (Feng et al., 9 Jul 2025).
2. Unified IR and language-neutral program synthesis
The central design decision in CrossLangFuzzer is to generate cross-language test programs in a language-neutral internal form and then render them into multiple JVM source languages. The design is inspired by the Kotlin compiler’s unified intermediate representation because Kotlin IR can encode semantic information in a way that is not tied to a particular surface syntax. CrossLangFuzzer does not simply ask the Kotlin compiler to generate code; it implements its own IR abstraction modeled on these ideas so that it can synthesize semantically rich programs, retain one unified representation across languages, and print the same structure into Kotlin, Java, Groovy, or Scala source code (Ma et al., 26 Jun 2026).
The IR represents a program as a tree with Program at the root and multiple ClassDeclaration nodes beneath it. Each class records a target language tag, inheritance information through extends and implements, optional type parameters, and member functions. The type system of the IR supports five type kinds:
| Type kind | Example |
|---|---|
| Simple types | Int, String, Any |
| Parameterized types | List<Int> |
| Nullable types | String? |
| Platform types | String! |
| Type parameters | T <: Comparable<T> |
These type forms matter because many cross-language bugs arise where generics, nullability, and metadata conventions interact. The paper argues that Kotlin IR is a strong foundation precisely because it can naturally represent class hierarchies, generic parameters and bounds, nullability, platform types, and language-specific metadata needed for interoperability (Ma et al., 26 Jun 2026).
CrossLangFuzzer also defines two traversal mechanisms over this IR: a top-down visitor for collecting and validating structure and an in-place transformer for editing subtrees during mutation. These traversal primitives are used by the generator, mutator, and reducer. The result is an architecture in which generation, mutation, printing, reduction, and replay all operate over the same structural representation rather than over ad hoc language-specific source fragments (Ma et al., 26 Jun 2026).
A useful historical comparison is provided by the 2025 CrossLangFuzzer prototype. That earlier version also relied on a universal IR for JVM-based languages, but its formal description emphasized a source-level grammar for class/interface declarations, supertypes, methods, override relationships, and type parameters, while also arguing against bytecode as the primary abstraction because bytecode does not preserve all override information needed for source-level reasoning and introduces artifacts such as synthetic bridge methods and erasure-induced ambiguity (Feng et al., 9 Jul 2025). The 2026 system preserves the same source-level emphasis while broadening the IR around Kotlin IR concepts, especially for nullability and platform-type modeling.
3. Mutation operators and semantic stress dimensions
CrossLangFuzzer begins from a valid IR-generated program and then applies mutation to broaden coverage and improve bug-finding capability. The 2026 paper lists seven mutation operators:
mutateGenericArgumentInParentremoveOverrideMemberFunctionmutateGenericArgumentInMemberFunctionParametermutateParameterNullabilitymutateClassTypeParameterUpperBoundNullabilitymutateClassTypeParameterUpperBoundshuffleLanguage
These operators are grouped into four semantic stress areas. mutateGenericArgumentInParent, mutateGenericArgumentInMemberFunctionParameter, and mutateClassTypeParameterUpperBound target generic subtyping. mutateParameterNullability and mutateClassTypeParameterUpperBoundNullability target nullability. removeOverrideMemberFunction targets override resolution. shuffleLanguage targets language placement by reassigning which source language a class is emitted in while keeping structure fixed. Mutations are selected probabilistically, and multiple mutations can be chained on one program (Ma et al., 26 Jun 2026).
The operators are deliberately aligned with the fault lines of cross-language compilation. Replacing a type argument in a parent class or interface can alter inheritance compatibility across language-specific generic rules. Toggling nullability or the nullability of a type-parameter upper bound can stress the interaction between Kotlin’s explicit null model and the looser conventions of other JVM languages. Removing an overridden method body while preserving the signature disturbs override semantics without destroying the surrounding inheritance structure. Reassigning a class to a different source language changes which compiler becomes the producer or consumer of its declarations, thereby probing the interoperability layer itself rather than merely varying syntax (Ma et al., 26 Jun 2026).
The earlier 2025 version used three lighter-weight mutators—LangShuffler, FunctionRemoval, and TypeChanger—and reported that TypeChanger was the most effective, detecting 11 of the 24 compiler bugs found in that evaluation (Feng et al., 9 Jul 2025). The expansion from three mutators to seven in the 2026 system is explicitly connected to the improved bug yield: the paper notes that the improved IR, additional mutations, and automated reduction helped uncover 8 more bugs, increasing the total from 24 to 32 (Ma et al., 26 Jun 2026). This suggests that finer-grained mutation around generics and nullability increased the framework’s ability to expose cross-language miscompilations that are semantically subtle rather than structurally coarse.
4. Differential-testing workflow, execution modes, and reduction
The workflow has four main stages: generation, mutation, printing and running, and reduction with confirmation. A configuration file specifies the target languages, and the generator builds an initial program in the custom IR. A key property of this generator is that it is valid by construction: semantic constraints are checked during generation, class hierarchies and type assignments obey JVM inheritance rules, and the generated program should therefore be well-formed (Ma et al., 26 Jun 2026).
Mutation operates under a different contract. The valid IR is passed to the mutator, which applies the seven operators, but mutation does not enforce semantic well-typedness after every change. The paper therefore distinguishes two diagnostic regimes. For generator-produced programs, accept/reject divergence across compilers is a strong bug signal because the program is known to be valid. For mutated programs, divergence is only a candidate bug because invalid programs may legitimately be rejected differently by different compilers (Ma et al., 26 Jun 2026).
Printing is handled by three language-specific printers: KtIrClassPrinter for Kotlin, JavaIrClassPrinter for Java and Groovy, and ScalaIrClassPrinter for Scala. After printing, the runner invokes compilers in two modes. In NormalTest, it compiles with one compiler version and looks for crashes or internal errors. In DifferentialTest, it compares behaviors across compilers or versions; mismatches in acceptance or bytecode/output are treated as potential bugs (Ma et al., 26 Jun 2026).
When a candidate bug is found, CrossLangFuzzer serializes the program back into the IR and minimizes it using an optimized Delta Debugging Minimization-style reducer. An active validator checks that reductions preserve structural validity. After each reduction step, the reduced IR is reprinted and re-executed; if the bug still reproduces, the reduction is kept, otherwise it is rolled back. The minimized reproducer is then reported to the relevant compiler team for manual confirmation (Ma et al., 26 Jun 2026).
This automated reduction pipeline is a major methodological difference from the earlier CrossLangFuzzer work, where trigger programs were manually minimized after a discrepancy was found (Feng et al., 9 Jul 2025). The 2026 paper explicitly ties automated reduction to improved effectiveness, indicating that the revised system’s IR, mutations, and reduction together contributed to the discovery of additional bugs (Ma et al., 26 Jun 2026).
5. Evaluation and empirical findings
CrossLangFuzzer was evaluated on five major JVM compilers: the Kotlin compiler (Kotlinc), Groovy compiler (Groovyc), Scala 3 compiler (Scala3c), Scala 2 compiler (Scala2c), and Java compiler (Javac). The evaluation was conducted on the latest versions of these actively maintained compilers. The excerpt explicitly notes that the Kotlin runner requires JDK 17 and that the Groovy runner example can target versions such as 4.0.26 and 5.0.0-alpha-12 in command-line options shown in repository notes (Ma et al., 26 Jun 2026).
The paper reports 32 confirmed bugs across these five compilers:
| Compiler | Confirmed bugs | Status where stated |
|---|---|---|
| Kotlin | 15 | 1 fixed, 14 confirmed |
| Groovy | 4 | 4 fixed |
| Scala 3 | 7 | 7 confirmed |
| Scala 2 | 2 | 2 confirmed |
| Java | 4 | 4 confirmed |
All reported issues were minimized, sent to the corresponding compiler maintainers, manually validated, and confirmed as real bugs by the compiler teams (Ma et al., 26 Jun 2026).
The main metrics in the evaluation are the number of confirmed bugs, distribution by compiler, fix status where known, and an implied yield measure from differential behavior and crash detection. The paper emphasizes that these bugs were found in latest actively maintained compiler versions, which it uses to argue that cross-language inconsistencies remain common even in mature compilers (Ma et al., 26 Jun 2026).
The empirical progression from the earlier CrossLangFuzzer study is also informative. The 2025 paper reported 24 confirmed bugs: 10 in Kotlin, 4 in Groovy, 7 in Scala 3, 2 in Scala 2, and 1 in Java. It further reported that 22 bugs were found through differential testing, 2 through the original generator during normal testing, and 3 more during manual minimization, with TypeChanger as the most effective mutator (Feng et al., 9 Jul 2025). The 2026 result of 32 confirmed bugs therefore reflects both an expanded detection capability and a broader bug distribution, particularly for Kotlin and Java (Ma et al., 26 Jun 2026).
6. Representative defects, attribution, and limitations
The illustrative raw-type example in the 2026 paper demonstrates the class of defect that CrossLangFuzzer targets: a Java program using raw types and generics is valid, but Kotlin rejects it when a Kotlin class inherits from a Java class that interacts with a generic interface override through raw types. The issue stems from Kotlin’s interpretation of a Java raw type as a flexible or wildcard-like type, which prevents override recognition in a later subclass. The significance of this example is not that it belongs to the new 32-bug set—it does not—but that it shows how interoperability failures arise from semantic mismatch rather than parser failure or isolated frontend unsoundness (Ma et al., 26 Jun 2026).
The earlier paper provides more detailed representative cases. It describes a Kotlin bug with raw types from Java, a Kotlin bug with interface/class method conflict, a Groovy bug involving bridge methods after type erasure, and a Java compiler bug involving Kotlin’s Nothing type (Feng et al., 9 Jul 2025). These examples show that cross-language bugs may manifest as incorrect rejection, incorrect override analysis, erasure-induced bridge-method mishandling, or type-lowering failures at language boundaries.
That earlier work also introduces a responsibility distinction between the provider—the language/source that defines the class or interface being consumed—and the user—the language that references it. After manually inspecting 17 cross-language trigger programs, the authors found 9 bugs on the user side, 5 on the provider side, and 3 too complex to classify cleanly (Feng et al., 9 Jul 2025). A plausible implication is that interoperability defects are often concentrated in the consuming compiler, because it must faithfully adapt to foreign-language type systems, metadata, and inheritance semantics.
The 2026 paper states one explicit future-work direction: IR serialization could serve as a natural interface for LLM-assisted fuzzing, either by allowing an LLM to mutate serialized IR directly or by helping select mutation operators based on compiler error logs (Ma et al., 26 Jun 2026). The same paper also makes several limitations evident. Because mutations may produce ill-typed programs, candidate-versus-confirmed bug ambiguity remains an issue for mutated inputs and requires manual validation. The framework targets JVM languages in this work, specifically Java, Kotlin, Groovy, and Scala. Although the IR is unified, correctness still depends on the printers for each target language. Manual triage and compiler-team confirmation remain necessary. The evaluation scope—five compilers and latest versions—is strong but still bounded (Ma et al., 26 Jun 2026).
In that sense, CrossLangFuzzer establishes cross-language compilation as a distinct compiler-testing domain. Its technical contribution is not merely multilingual input generation, but a source-level, IR-based differential-testing workflow tuned to the semantic discontinuities of the JVM ecosystem: generics, nullability, override semantics, and language placement. The results indicate that these discontinuities remain a practical source of real compiler defects in contemporary toolchains (Ma et al., 26 Jun 2026).
