NullRepair: LLM-Assisted Nullability Repair

• NullRepair is an LLM-assisted system designed to repair residual Java nullability errors that remain after annotation inference.
• It integrates structured static analysis, project-wide usage indexing, and a manually-derived decision protocol to apply context-sensitive code transformations.
• Empirical evaluations on 12 projects show a reduction of remaining errors by up to 62% with improved semantic preservation compared to naïve LLM approaches.

Searching arXiv for NullRepair and closely related null-repair work to ground the article with current paper metadata and context. NullRepair is an LLM-assisted repair system for the residual nullability errors that remain after a Java project has already been processed by annotation inference. It targets the “last mile” of nullability-checker adoption: cases in which static nullability checkers expose genuine code-level nullability violations or false positives that require semantic changes rather than additional annotations. The system integrates LLMs into a structured workflow derived from a manual analysis of 200 real-world errors, and combines static analysis, project-wide usage indexing, decision logic, and iterative prompting to produce repairs that are more semantically aligned with project conventions than naively prompted code generation (Karimipour et al., 28 Jul 2025).

1. Problem setting and scope

NullRepair is defined in the context of modular, annotation-driven nullability checking in Java. Static nullability checkers such as NullAway, Nullsafe, and the Checker Framework can prevent null-pointer exceptions only when code is annotated accurately, and references are treated as non-null by default. In practice, annotation inference tools like Annotator can infer many missing annotations, but many errors still remain after inference because the checker has exposed genuine code-level nullability violations or false positives that require semantic changes rather than just more annotations.

The system is therefore not a general-purpose Java repair framework. Its scope is the subset of residual nullability errors that survive annotation inference. The paper emphasizes that adopting a nullability type system usually needs both annotations and code transformation, and positions NullRepair specifically on the code-transformation side of that process. Some residual errors require code restructuring, control-flow changes, or project-specific null-handling conventions. Others are false positives that are better handled by suppression. NullRepair is designed to automate this stage rather than asking an LLM to repair arbitrary Java code in an unconstrained way.

A central misconception addressed by the work is that residual nullability errors can be handled solely by inserting @Nullable or @Nonnull. The paper states that these residual errors are precisely the hard cases that developers end up fixing manually during checker adoption. NullRepair treats them as a distinct repair problem whose solution requires both program analysis and context-sensitive synthesis.

2. Human-derived decision protocol

A defining feature of NullRepair is that its control logic is derived from a preliminary manual analysis of 200 real residual nullability errors drawn from three benchmarks that had the least reduction after annotation inference. For each analyzed error, the study identified: the question that needed answering, the expected answer form, and the action that should follow from that answer. By iteratively expanding the analysis to cover every new case encountered, the paper distilled the results into a decision flowchart (Karimipour et al., 28 Jul 2025).

This flowchart is the backbone of the system. Each error is processed by walking through the chart node by node, where each node either asks the LLM a question, runs static analysis, or applies a transformation. The resulting repair process is explicitly presented as a structured decision protocol rather than a free-form “ask the LLM to fix it” strategy.

The flowchart proceeds in a fixed order. It first checks for trivial safe rewrites. It then determines whether the warning is actually a false positive. Next, it determines whether annotation insertion can solve the issue and whether that triggers more errors. Only if those steps fail does the system invoke context-rich LLM synthesis to generate a code patch. This ordering is significant because it constrains the LLM to cases where simpler and more local repairs have already been ruled out.

The paper’s characterization of the workflow implies a decomposition of the task into classification, validation, and synthesis stages. That decomposition is central to NullRepair’s identity: the LLM is used inside a repair framework, not as the framework itself.

3. Static analysis, usage regions, and project-wide evidence

The paper’s distinctive technical idea is to use static analysis not only to locate the error site, but also to index how the relevant symbol is used across the whole codebase. NullRepair partitions the codebase into “usage regions,” defined at method bodies and field declarations, and maps each symbol to the regions where it appears via the abstraction

$$\mathit{usageRegions} : Symbol \rightarrow \{Method \text{ or } Field Declaration\}.$$

For each region, the system records whether the usage is safe or unsafe with respect to nullability checking. Unsafe regions are those that trigger a checker error, while safe regions are those where the same symbol is used without error. This produces a project-wide, statically derived index of both positive and negative examples (Karimipour et al., 28 Jul 2025).

That safe/unsafe distinction is operationally important. If a symbol is used safely elsewhere in the project, NullRepair retrieves those examples and uses them as evidence of the intended null-handling pattern. If the triggering expression comes from third-party code, the system instead finds the closest project-defined receiver inside the expression and indexes usage of that receiver, because that reveals the relevant nullability discipline in the codebase.

The motivating example given in the paper illustrates the role of safe usage evidence. A dereference of user.getMapView() is unsafe in one class, but the system finds a safe pattern elsewhere in the codebase where getMapView() is checked against null and a domain-specific default value is returned. That safe usage is then inserted into the prompt, so the LLM can produce a patch aligned with existing project conventions rather than a generic null guard.

4. Prompting architecture and iterative repair loop

NullRepair uses two kinds of prompts: decision prompts and patch prompts. Decision prompts are used to traverse the flowchart. They ask narrow questions such as whether a method can return null or whether a dereference is guaranteed safe. The expected answers are structured as Yes/No/Unsure. When the model answers Unsure, it may request additional context, such as a method declaration, a field declaration, caller bodies, or usage regions. The system retrieves exactly that information and re-prompts. These prompts are stateless, but they are built on the current working copy of the code after any prior edits.

Patch prompts are used when the system reaches a terminal synthesis point. These prompts include the triggering expression, the enclosing method or field, and safe usage examples from the project. The model is expected to return a syntactically valid Java method enclosed in a structured XML response. If parsing fails, NullRepair enters a retry loop and prompts the model with the malformed output and parse error.

The repair loop itself has several explicit stages. First, the system checks for heuristic null-safe rewrites. The paper lists four built-in patterns guarded by preconditions.

Pattern	Preconditions
x.equals(y) → Objects.equals(x, y)	when x may be nullable
x.hashCode() → Objects.hashCode(x)	when x may be nullable
x.toString() → String.valueOf(x)	when x may be nullable
x.deref() → if (x == null) return null; x.deref();	when x is nullable, the enclosing method returns nullable, and safe call sites exist

If one of these patterns applies, NullRepair rewrites immediately and stops for that error.

If no heuristic rewrite applies, the system tries to determine whether the warning is a false positive. It begins with the method body, expands to caller bodies if necessary, and can then request additional declarations or usage regions. If the model concludes that the expression is guaranteed non-null, NullRepair inserts an explicit non-null cast at the error site to suppress the warning.

If the error is a true positive, the system asks whether annotation insertion can solve it. If the error is of the form “nullable value assigned to non-null location,” it adds a @Nullable annotation to the destination and reruns the checker. Any newly triggered errors are fed back into the same flowchart. The working copy is updated after each edit so later prompts reflect the current code rather than stale versions. Because this chain can in principle continue indefinitely, NullRepair imposes a maximum number of resolution attempts, with a per-error budget of 50 LLM queries and up to five retries for malformed outputs (Karimipour et al., 28 Jul 2025).

Only when annotation fixes and suppression decisions do not suffice does the system synthesize a patch using project-wide safe and unsafe usage evidence. In that stage, the LLM is asked to produce an actual code edit under explicit contextual constraints.

5. Empirical evaluation and patch quality

The evaluation covers 12 real-world Java projects and 1,137 residual nullability errors after annotation inference. The main result reported is that NullRepair resolves an average of 72% of these residual errors when patches are considered individually. In the combined, end-to-end setting—where only compileable patches that resolve their target issue are retained—the system reduces the remaining errors by 62% on average (Karimipour et al., 28 Jul 2025).

The paper separates raw warning removal from patch quality. At the patch level, NullRepair produces fewer compilation failures than the baseline LLM approach: about 5.78% of NullRepair patches fail to compile, compared with 15.22% for the naïvely prompted baseline. The raw repair rate is comparable, with NullRepair resolving about 74.67% of errors on average per patch and the baseline resolving about 77.67%. The paper explicitly notes that this does not capture patch quality.

The strongest contrast appears in semantic preservation. After applying all selected patches, all unit tests pass in 10 of the 12 projects, and in the remaining two projects at least 98% of tests pass. In the aggregate table, NullRepair’s combined test failures are 0.23% overall, while the baseline’s are 16.11% overall. NullRepair also triggers additional checker warnings more often during patching, because it tends to make deeper semantic changes; the baseline triggers fewer new errors but often by using simplistic suppressions or guards.

Manual review reinforces this interpretation. In a blinded human assessment of 73 patch pairs, NullRepair produced 38 likely acceptable patches, 23 needing work, and only 12 likely unacceptable. The baseline produced 11 likely acceptable patches and 38 likely unacceptable ones. Reviewer agreement was high, with Cohen’s kappa 0.86 initially and 0.95 after discussion. The cost profile is also reported: NullRepair averages 47 minutes per project and about 11.4k tokens per error, corresponding to roughly $0.05 per error.

A recurrent misconception addressed by these results is that higher raw warning removal necessarily indicates a better repair system. The paper’s comparison shows that a naively prompted LLM can sometimes remove more warnings, but often by generating over-aggressive or semantically inappropriate edits, such as fail-fast exceptions or broad guards that silence the checker while changing runtime behavior. NullRepair’s contribution is not primarily a higher raw repair rate, but a repair rate coupled with substantially stronger test preservation.

6. Position within null-repair research

NullRepair belongs to a line of work concerned with null-related program repair, but its target and mechanism differ sharply from prior runtime systems. NPEFix, for example, is a runtime repair framework for Java null dereferences that proposes 9 alternative execution semantics when a null pointer exception is about to happen, and evaluates runtime repair on 11 field null dereference bugs and 519 seeded failures (Cornu et al., 2015). BanditRepair extends the runtime perspective by systematically exploring runtime patches for null dereferences using bandit algorithms, treating repair as online exploration and exploitation over sequences of execution modifications (Durieux et al., 2016).

NullRepair addresses a different layer of the software lifecycle. It operates during nullability-checker adoption, after annotation inference and before deployment, rather than at failure time in production. Its inputs are checker-reported residual nullability errors rather than thrown null pointer exceptions. Its primary mechanisms are static analysis, project-wide usage indexing, and structured LLM interaction rather than runtime state modification or speculative execution.

This difference also clarifies its relation to annotation inference. NullRepair does not replace annotation inference; it assumes annotation inference has already been applied and focuses on the residual errors that remain. In that sense, it occupies a specific methodological niche: automating the code changes that are still required after state-of-the-art annotation inference has reduced the easy cases.

The paper’s broader implication is that nullability repair can be framed neither as pure type inference nor as unconstrained code generation. NullRepair instead presents it as a hybrid workflow in which static analysis classifies the situation, project-specific evidence constrains the synthesis space, and the LLM is invoked only within an explicit decision protocol.