Understanding Agent-Based Patching of Compiler Missed Optimizations

Abstract: Compiler missed optimizations refer to cases in which compilers failed to optimize certain code. It takes many compiler developers' efforts to implement or patch such missed optimizations. In this paper, we present a systematic study of how well agents patch compiler missed optimizations. We identify a significant challenge that patching a missed optimization requires more than just fixing the reported case, and instead requires generalizing to similar cases. We construct a benchmark of real-world LLVM missed optimization issues and compare agent-generated patches with patches from developers in terms of optimization scope. Our results show that coding agents often optimize the given examples, but many generated patches either cover only part of the developer-intended scope or partially overlap with it; in some cases, they further generalize beyond the reference patch. We further introduce historical-knowledge augmentation techniques that leverage prior LLVM optimization pull requests through retrieval and distillation, showing that they improve developer-aligned generalization and yield practical benefits when applied to real-world IR.

• The paper demonstrates that LLM-driven agents can synthesize patches for missed optimizations but often struggle to generalize to the developer’s intended scope.
• It employs a benchmark of 43 real LLVM issues and evaluates patch success by comparing agent-generated patches with golden developer patches using both golden tests and IR fuzzing.
• Historical knowledge augmentation through Retrieval Augmented Generation (RAG) and distillation significantly boosts patch alignment and real-world optimization effectiveness.

Agent-Based Patching of Compiler Missed Optimizations: An Expert Analysis

Introduction

The paper "Understanding Agent-Based Patching of Compiler Missed Optimizations" (2607.02370) delivers a comprehensive empirical study on the capabilities and generalization properties of LLM-driven agents tasked with generating patches for missed optimization cases in the LLVM compiler infrastructure. Differentiating from traditional bug-fixing, missed-optimization repair centers not only on correctness for the motivating test instance but on achieving alignment with developer-intended generalization scope: the range of similar IR patterns the patch should cover.

Benchmark and Problem Formulation

The authors formulate agent-based patching as a generalization-sensitive problem wherein the ultimate measure of patch success is its semantic scope alignment with developer fixes (golden patches), rather than mere instance satisfaction. They meticulously construct a benchmark of 43 rigorously validated real-world LLVM missed optimization issues, each with a verified developer patch and corresponding regression tests. The benchmark's structure reflects real LLVM repository organization and optimization pass distribution, with emphasis on InstCombine—the hub for IR canonicalization in LLVM.

Figure 1: The mapping between LLVM repository elements and the benchmark's key components highlights the focus on InstCombine and associated passes.

To approximate the latent optimization scope, the authors compare agent-generated patches and developer patches on both golden-reported and fuzz-generated test cases, enabling an operational classification of the scope relationship: agent under-generalization ($A \subset G$), partial overlap ($A \bowtie G$), agent over-generalization ($G \subset A$), or behavioral equivalence ($A \sim G$).

Agent Framework and Generalization Assessment

A controlled agent harness wraps each benchmark instance with a standardized toolset, context provision, validation workflow (including Alive2, llvm-mca, and lit checks), and ReAct-style agent interaction. Agents employ no privileged access to golden patches or regression tests, ensuring realistic deployment conditions. Generalization is evaluated post-hoc: golden tests reflect coverage of intended patterns, while IR fuzzing (driven by mutation and LLM-based synthesis) probes for over- or under-generalization divergences. This bidirectional test regime is central to robust scope assessment.

Figure 2: Fuzz-based generalization assessment showing how IR fuzzing supplements golden tests to probe patch scope divergence.

Baseline Study: Generalization Behavior of Agents

Assessment across GPT-5.5, DeepSeek-V4-Pro, Qwen3.5-Plus, and Kimi K2.5 under default instructions reveals that agents can reliably synthesize patches that optimize initial motivating cases. For GPT-5.5, 74% of issues result in applied patches under the budgeted setting. However, only approximately half of these align in scope with the respective golden patches ($A \sim G$), with the remainder split between under- and over-generalization, and partial overlap. Notably, attempting to remedy this via generic generalization instructions ("ensure the patch handles general patterns") yields little benefit and often degradation—showing increased failures or further scope misalignments without systematic improvement in intended generalization.

Historical-Knowledge Augmentation: Retrieval and Distillation

The authors hypothesize that LLM agents lack access to the rich, context-sensitive, and tacit developer experience that underpins robust optimization generalization. They introduce two methods to inject historical LLVM optimization knowledge into the agent workflow:

1. Retrieval Augmented Generation (RAG): Indexed corpus of 869 historical LLVM optimization PRs yields relevant source-to-target IR transformations and accompanying summaries for contextualization at patch time.
2. Distillation: Aggregating generalization behaviors from historical PRs, distilled high-level principles for each optimization pass are synthesized and provided as compact background knowledge.

   Figure 3: Workflow of RAG and distillation: retrieving similar prior optimizations and providing distilled generalization guidance during agent patching.

Effectiveness of Augmentation: Alignment and Practical Impact

Empirical results show clear scope alignment gains: for all models, historical-knowledge-augmented agents produce more patches with optimization behavior matching the developer patch ($A \sim G$). For GPT-5.5, RAG increases exact matches from 18 to 22; for Kimi K2.5, RAG and distillation both move the count from 10 to 13. The union of $A \sim G$ and $G \subset A$ (covering the golden scope) also improves, most notably for Kimi K2.5 (from 12 to 16). Augmentation benefits persist across models, reinforcement that prior developer experience is essential to generalization beyond instruction tuning alone.

Figure 4: Accumulated wins and losses: augmented agent patches versus baseline patches on real-world projects, demonstrating more frequent successes (wins) for RAG- and distillation-based augmentations.

The study verifies that these improvements are not merely theoretical: augmented patches trigger more optimization hits on real-world IR from open-source projects, as measured by frequency of fired optimization code paths. Distillation-based augmentation produces a significant increase in total hits—boosting from 20.2M to 24.3M over aggregate projects. RAG, while yielding smaller gains, still outperforms the baseline, suggesting augmentation strategies can have varying project-specific impact.

Figure 5: Project-level cumulative optimization hits across all issues and models, with both RAG and distillation outperforming the baseline in real-world IR coverage.

Case Study: Mechanism of Historical Context

A case study on LLVM Issue #158326 illustrates why historical context matters: retrieved RAG examples cue the agent to reconceptualize masked equality checks as range predicates, enabling it to implement an abstraction-level generalization that eludes both direct repair and generic instruction efforts. Visualization of the agent's reasoning shows how historical examples channel developer intuition and align patch structure to broader semantic patterns.

Implications and Future Directions

This work establishes that current LLM agents, even when effective at symbolic reasoning, cannot readily infer the generalization schema encoded in established compiler development practices. Robust generalization for missed-optimization patching requires integrating rich, project-specific, and pass-level historical experience, either via direct retrieval of analogous cases or via distilled abstraction of common generalization patterns.

Practically, this implies that effective deployment of agentic compiler patchers in production environments must maintain a continuously refreshed historical patch corpus and distill knowledge across evolving optimization landscapes. Theoretically, future systems would benefit from closed-loop learning architectures where successful patch generalizations are iteratively harvested, distilled, and injected.

Prospective research directions include:

• Automated construction and updating of historical-knowledge corpora through mining and abstraction tooling integrated with version control.
• Adaptive selection strategies for retrieval and distillation to maximize generalization alignment on a per-issue/per-pass basis.
• Extending the paradigm to cover backend code generation, architecture-specific optimizations, and non-IR code transformations.

Conclusion

"Understanding Agent-Based Patching of Compiler Missed Optimizations" rigorously demonstrates that LLM-driven agents can efficiently synthesize LLVM missed-optimization repairs for instance cases but do not, by default, generalize in lockstep with developer intent. Only through rich, targeted historical knowledge augmentation—by retrieval or distilled abstractions—can agents close the generalization gap, achieving stronger alignment with developer-scoped optimization space and yielding greater impact in practice.