Compiler-LLM Cooperation: Hybrid Integration
- Compiler-LLM cooperation is a hybrid approach where large language models serve as selectors, translators, or generators to complement conventional compiler infrastructures.
- The integration enables improved performance through techniques like function orchestration, feedback loops, and MCTS-driven optimization while managing correctness and scalability.
- Hybrid systems employ compiler feedback for error repair, bug isolation, and iterative code improvement, demonstrating that tool access can empower even smaller models.
Compiler-LLM cooperation denotes a family of hybrid systems in which LLMs interact with conventional compiler infrastructure, compiler-inspired orchestration layers, or compiler-derived feedback rather than replacing them outright. In the survey literature, this space is organized by Design Philosophy, LLM Methodology, Level of Code Abstraction, and Task Type, and its central tensions are correctness and scalability, with hybrid systems identified as the most promising path forward (Zhang et al., 5 Jan 2026).
1. Taxonomic foundations and formal models
A systematic account of the field distinguishes three design philosophies. In the Selector role, the LLM chooses among human-defined compiler actions; in the Translator role, it maps one program representation to another; and in the Generator role, it writes compiler-side artifacts such as passes, scripts, or transformation utilities (Zhang et al., 5 Jan 2026). The same survey further classifies work by whether it is training-required or training-free, by abstraction level spanning NL, Source, AST, IR, and ASM/Bin, and by task type including code generation, transpilation, optimization, repair, fuzzing, compilation, and decompilation (Zhang et al., 5 Jan 2026).
The formalization used in that survey makes the separation explicit. A selector is modeled as , where is the compilation context and is a finite set of compiler actions. A translator is modeled as at abstraction level . A generator is modeled as (Zhang et al., 5 Jan 2026). This framing is important because it locates the LLM either inside a bounded decision space, inside a semantics-preserving transformation loop, or inside compiler implementation itself.
| Design philosophy | Formal model | Operational role |
|---|---|---|
| Selector | Chooses predefined actions | |
| Translator | Transforms code representations | |
| Generator | Produces compiler-side code |
The same survey identifies three primary benefits: the democratization of compiler development, the discovery of novel optimization strategies, and the broadening of the compiler's traditional scope (Zhang et al., 5 Jan 2026). At the same time, it emphasizes that LLM-enabled compilation inherits the generative model’s hallucination risk, making verifiability a first-order systems concern rather than an auxiliary evaluation criterion (Zhang et al., 5 Jan 2026).
2. Compiler-inspired orchestration of tools and function calls
One prominent line of work treats tool orchestration itself as a compilation problem. "LLMCompiler" formulates natural-language tasks as a Directed Acyclic Graph of atomic function calls and executes them with three components: a Function Calling Planner, a Task Fetching Unit, and an Executor (Kim et al., 2023). The Planner emits tasks with placeholders such as $\$j03.7\times16.7\times2\sim 9\%$ compared to <a href="https://www.emergentmind.com/topics/react" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">ReAct</a> (<a href="/papers/2312.04511" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Kim et al., 2023</a>).</p> <p>A related but more explicitly compiler-front-end/compiler-back-end design appears in Singh et al.'s "An LLM-Tool Compiler for Fused Parallel Function Calling" (<a href="/papers/2405.17438" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Singh et al., 2024</a>). Its architecture is summarized as <strong>User query $\mathcal{C}$3 Fuser module $\mathcal{C}$4 Revised tool-set $\mathcal{C}$5 LLM agent call $\mathcal{C}$6 Executor module $\mathcal{C}$7 Actual tool executions $\mathcal{C}$8 Final answer, where the Fuser is a lightweight GPT-driven routine invoked once per user prompt and the Executor de-compiles a fused function call into constituent API calls run either concurrently or in the correct sequential order (Singh et al., 2024). The agent’s system prompt is unchanged; only the functions JSON array is rewritten, so Chain-of-Thought and ReAct logic remain unchanged (Singh et al., 2024).
The fusion heuristic is compiler-like in a literal sense. If $\mathcal{C}$9 is a group of $\mathcal{O}$0 tools of the same class, fusion is applied when
$\mathcal{O}\mathcal{O}$2 K geospatial queries, GPT-3.5/4 Turbo, and CoT and ReAct zero/few-shot settings, the system achieves up to four times more parallel calls, with token costs and latency reduced by up to $\mathcal{O}$3 and $\mathcal{O}$4, respectively; in a GPT-4 Turbo, ReAct zero-shot sample, latency improves from approximately $\mathcal{O}$5 s to $\mathcal{O}$6 s and tokens from approximately $\mathcal{O}$7 to $\mathcal{O}$8 (Singh et al., 2024).
Across these systems, the recurring structural pattern is a clean separation of concerns: the LLM determines which operations are needed and in what order, while the compiler layer handles grouping, dependency analysis, scheduling, and execution (Singh et al., 2024). A plausible implication is that some of the highest-value cooperation arises not when the LLM replaces compilation machinery, but when it is wrapped by it.
3. Feedback loops: from passive generator to active agent
Another major mode of cooperation gives the LLM access to a compiler as an external verifier. "From LLMs to Agents in Programming: The Impact of Providing an LLM with a Compiler" encapsulates the model as a single-model agent whose only external tool is a gcc compiler, with a short-lived memory of the task description, past code generations, and compiler error messages (Kjellberg et al., 17 Jan 2026). The loop runs for up to $\mathcal{O}$9 iterations: generate code, compile it with gcc -std=c17 -Werror, capture stderr on failure, extract the first error line, and prompt the model to fix the code (Kjellberg et al., 17 Jan 2026). On $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$0 RosettaCode C tasks evaluated across $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$1 LLMs from $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$2 million to $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$3 billion parameters, compiler access improves compilation success by $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$4 to $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$5 percentage units, syntax errors drop by $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$6, and undefined-reference errors drop by $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$7 for tasks where the agents outperform baselines (Kjellberg et al., 17 Jan 2026).
This study also records a counterintuitive size-performance relation. Qwen 3 4 B jumps from $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$8 to $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$9 compilation success, a gain of $\ell$0 pp, overtaking many larger models; the authors explicitly note that a $\ell$1 B-parameter model with compiler feedback can outshine a $\ell$2 B-parameter model with no feedback (Kjellberg et al., 17 Jan 2026). This directly challenges the common assumption that raw parameter count is the primary determinant of practical compilation performance.
A training-time analogue appears in "CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution" (Jana et al., 2023). CoTran combines compiler feedback with symbolic-execution feedback inside a reinforcement-learning loop. The forward model generates a candidate translation $\ell$3, compiler diagnostics are converted into a compilation reward, the translation is back-translated to $\ell$4, and a symbolic-execution engine generates a JUnit test suite on the original source to assess functional equivalence (Jana et al., 2023). The reward combines compilation and equivalence signals, and the policy is optimized with PPO while remaining close to a supervised reference model באמצעות a KL penalty (Jana et al., 2023). On AVATAR-TC with $\ell$5 K Java–Python pairs, CoTran achieves $\ell$6 FEqAcc and $\ell$7 CompAcc for Python-to-Java, while the nearest competing tool, PLBART-base, reaches $\ell$8 and $\ell$9; relative to the CodeT5 baseline, the system improves FEqAcc by $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$0 and CompAcc by $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$1 for Python-to-Java (Jana et al., 2023).
These systems instantiate the same underlying principle at different timescales. Inference-time loops use compiler messages to repair concrete outputs; training-time loops convert compiler and symbolic-execution feedback into rewards that reshape the model itself (Jana et al., 2023). This suggests that compiler diagnostics can function simultaneously as runtime critic, training signal, and search constraint.
4. Neural compilation, translation composability, and IR interoperability
The strongest form of cooperation asks whether the LLM can act as a compiler or a translation stage within a compiler stack. "Exploring the Feasibility of End-to-End LLM as a Compiler" defines LaaC as an end-to-end LLM that takes high-level source code as input and generates target assembly code directly, obviating the traditional compiler pipeline of frontend, middle-end, and backend (Zhang et al., 6 Nov 2025). To evaluate this claim, the paper introduces CompilerEval: $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$2 representative C kernels, each sampled $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$3 times, on x86, ARM, and RISC-V, with success measured both as executable sample and correct sample (Zhang et al., 6 Nov 2025). The paper concludes that mainstream LLMs exhibit basic capabilities as compilers but currently achieve low compilation success rates; prompt engineering yields an average gain of $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$4 pp across four LLMs, Claude gains $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$5 pp on average, GPT-o1 achieves $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$6 pp average success@1 over GPT-4o and up to $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$7 pp on kernels like rotate and resize, and cross-platform success@1 is $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$8 on x86, $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$9 on ARM, and $f_{\theta} : \mathcal{C} \to \mathcal{O}$0 on RISC-V (Zhang et al., 6 Nov 2025).
A more structured response to the same problem is "LEGO-Compiler: Enhancing Neural Compilation Through Translation Composability" (Zhang et al., 26 May 2025). Its central claim is that large programs should be split into composable control blocks, translated independently, then reassembled under formal guarantees that translation preserves semantics under concatenation (Zhang et al., 26 May 2025). The workflow includes variable renaming, type and layout analysis, variable mapping and allocation, part split, LEGO part translation, and part rebuild with final behavioral check; repair is iterative and limited to affected blocks (Zhang et al., 26 May 2025). Empirically, the system reports over $f_{\theta} : \mathcal{C} \to \mathcal{O}$1 on ExeBench, $f_{\theta} : \mathcal{C} \to \mathcal{O}$2 on industrial-grade AnsiBench, and a near one order-of-magnitude improvement on compilable code size scalability (Zhang et al., 26 May 2025). On the full $f_{\theta} : \mathcal{C} \to \mathcal{O}$3 ExeBench cases with DS-V3-0324, the paper reports Direct $f_{\theta} : \mathcal{C} \to \mathcal{O}$4 CoT $f_{\theta} : \mathcal{C} \to \mathcal{O}$5 LEGO $f_{\theta} : \mathcal{C} \to \mathcal{O}$6 (Zhang et al., 26 May 2025).
A domain-specific variant appears in "LLM-Aided Compilation for Tensor Accelerators" (Hong et al., 2024). That work proposes a two-phase workflow: Phase I: Functional Translation into a target accelerator DSL, followed by Phase II: Cost-Model-Driven Optimization (Hong et al., 2024). GPT-4 is guided by instructions, ISA specification, and source program, and outputs are verified on a simulator or hardware; translation and optimization are further decomposed into smaller subtasks and multi-step repair loops (Hong et al., 2024). On the TinyMPC mat-vec kernel, one-shot ICL plus NL annotations reaches $f_{\theta} : \mathcal{C} \to \mathcal{O}$7, $f_{\theta} : \mathcal{C} \to \mathcal{O}$8, and $f_{\theta} : \mathcal{C} \to \mathcal{O}$9; overall, $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$0 of $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$1 kernels are translated successfully after repair (Hong et al., 2024).
Interoperability at the IR level is addressed by "LLM Translation of Compiler Intermediate Representation" (Ramirez et al., 7 May 2026). IRIS-14B is fine-tuned to translate GIMPLE to LLVM IR and is explicitly positioned as an interoperability layer in hybrid neuro-symbolic compiler architectures (Ramirez et al., 7 May 2026). On CodeForces-IRIS it reaches $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$2 compile and $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$3 I/O test pass, and on ExeBench-IRIS $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$4 compile and $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$5 I/O test pass, outperforming widely used models by up to $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$6 percentage points (Ramirez et al., 7 May 2026). Unlike end-to-end replacement, this design preserves deterministic compilation and optimization in GCC and LLVM while inserting the LLM as a textual IR bridge (Ramirez et al., 7 May 2026).
Taken together, these results delimit a spectrum. At one end, LaaC probes end-to-end feasibility but reports low success rates (Zhang et al., 6 Nov 2025). In the middle, LEGO-Compiler and Gemmini-oriented workflows decompose compilation into verifiable subtasks (Zhang et al., 26 May 2025, Hong et al., 2024). At the other end, IRIS-14B inserts a learned translation layer between mature toolchains without modifying existing passes (Ramirez et al., 7 May 2026).
5. Optimization search, multi-agent compilation, and inference deployment
Compiler-LLM cooperation is also used to navigate optimization spaces that are too large for fixed heuristics. "Compiler Optimization via LLM Reasoning for Efficient Model Serving" formulates optimization as a finite-horizon Markov decision process whose state includes the current IRModule, hardware profile, and transformation history, while actions are hardware-aware transformations such as tiling, fusion, and vectorization (Tang et al., 2 Jun 2025). Its REASONING COMPILER architecture combines an LLM proposal mechanism with structured Monte Carlo tree search, using UCT for selection, LLM-guided expansion, cost-model simulation, and backpropagation (Tang et al., 2 Jun 2025). Across four kernels, LLM-MCTS reaches a 7 speedup after just 8 compiler evaluations on the Llama3 benchmark versus MetaSchedule or Ansor requiring 9 samples, attains a 0 speedup on Flux-Conv at 1 samples where the baseline needs more than 2, and achieves up to 3 end-to-end inference improvement over unoptimized code with only 4 queries (Tang et al., 2 Jun 2025).
"COLT: Lightweight Multi-LLM Collaboration through Shared MCTS Reasoning for Model Compilation" generalizes this search setting to multiple models (Tang et al., 2 Feb 2026). Its node state is a joint state 5, actions jointly choose a semantic-preserving compiler transformation and the next LLM to query, and rewards are back-propagated through a single shared MCTS tree (Tang et al., 2 Feb 2026). The model-aware UCT policy adds a normalized bias term 6 toward smaller LLMs, while a course-alteration mechanism escalates to the largest model after persistent regressions from small-model expansions (Tang et al., 2 Feb 2026). Empirically, COLT uses only approximately 7 of the large-model calls in the 8-model setting while achieving better final speedups, including up to 9 on GPU and $\$0 on CPU across five kernels; in end-to-end Llama-3 tuning, COLT$\$1 reaches $\$2 with only $\$3 samples, versus GPT-5-mini at $\$4 with $\$5 samples and GPT-5.2 at $\$6 with $\$7 samples (Tang et al., 2 Feb 2026).
A different form of cooperation appears in "Agentic Code Optimization via Compiler-LLM Cooperation" (Mikek et al., 5 Apr 2026). That work introduces a multi-agent system with LLM-based optimization agents at source, IR, and assembly levels, compiler constituents as tools, an LLM-based test generation agent, and a guiding LLM that allocates budget across levels of abstraction (Mikek et al., 5 Apr 2026). The reported outcome is that compiler-LLM cooperation outperforms both existing compiler optimizations and level-specific LLM baselines, producing speedups up to $\$8 (Mikek et al., 5 Apr 2026).
At inference deployment time, the cooperation may shift from code transformation to compiler-assisted scheduling of the LLM itself. "Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices" integrates speculative decoding into an IREE/MLIR workflow through offline quantization, profiling, analytical cost modeling, design-space exploration, and AOT compilation (Mesa et al., 8 Feb 2026). The cost model uses acceptance rate $\$9, draft length 00, and relative cost coefficient 01, with a necessary condition for speedup given by 02 (Mesa et al., 8 Feb 2026). On an NXP i.MX95 device with 03 A55 and a Mali-G310, the system reports a predicted peak speedup of approximately 04 at 05 and a measured speedup of approximately 06-07, about 08 below prediction (Mesa et al., 8 Feb 2026).
The unifying theme in these optimization papers is that the LLM is not treated as an optimizer in isolation. It is embedded inside search, budget allocation, profiling, or compiler runtime infrastructure, and its proposals are evaluated against cost models, tests, or measured performance (Tang et al., 2 Jun 2025, Mikek et al., 5 Apr 2026, Mesa et al., 8 Feb 2026). This suggests a shift from prompt-only optimization to tool-grounded optimization.
6. Testing, bug discovery, and reliability engineering
Compiler-LLM cooperation also extends to compiler testing and bug isolation. "LLM-Based Program Generation for Triggering Numerical Inconsistencies Across Compilers" presents LLM4FP, which combines Grammar-Based Generation and Feedback-Based Mutation to synthesize floating-point programs that produce differential behavior across compilers and optimization levels (Wang et al., 29 Aug 2025). The pipeline compiles each generated program with gcc, clang, and nvcc across six optimization levels, executes them on identical inputs, and marks a program as successful if any pair disagrees bitwise (Wang et al., 29 Aug 2025). Compared with Varity, LLM4FP reaches an inconsistency rate of 09 versus 10, producing 11 total mismatches versus 12; moreover, 13 of LLM4FP mismatches are 14 rather than extreme values such as infinities or NaN (Wang et al., 29 Aug 2025).
"LegoFuzz: Interleaving LLMs for Compiler Testing" separates LLM use into an offline phase and an online phase (Ni et al., 26 Aug 2025). Offline, the LLM transforms AnghaBench functions into small, feature-rich, valid C functions, while GCC/Clang front-ends, sanitizers, and CompCert validate them; online, these pre-validated components are composed into large, inter-dependent test programs without further LLM calls (Ni et al., 26 Aug 2025). On GCC and LLVM, this framework reports 15 bugs, of which 16 were previously unknown and 17 already fixed upstream; the set includes 18 crashes and 19 miscompilations (Ni et al., 26 Aug 2025). Throughput is reported as 20 s per test, about 21 s for 22 K tests, compared with approximately 23 s for Fuzz4All and approximately 24 s for WhiteFox (Ni et al., 26 Aug 2025).
Bug localization is addressed by "Improving Compiler Bug Isolation by Leveraging LLMs" (Qi et al., 21 Jun 2025). AutoCBI combines failing test programs, source-file function summaries generated by an LLM, suspicious file lists derived from SBFL formulas such as Ochiai and Wong2, and compilation configurations with related output messages, then uses a specialized prompt to rerank suspicious files (Qi et al., 21 Jun 2025). On 25 real-world bugs from GCC and LLVM, AutoCBI isolates 26, 27, and 28 more bugs than RecBi, DiWi, and FuseFL, respectively, in Top-1 ranked results for GCC/LLVM (Qi et al., 21 Jun 2025). The ablation study further reports that removing document summaries or failing test code cuts Top-1 performance by roughly 29-30, indicating that semantic context is not merely decorative but operationally central (Qi et al., 21 Jun 2025).
These reliability-oriented systems broaden the scope of compiler-LLM cooperation beyond code generation and optimization. Here the LLM supplies diversity, semantic summarization, or ranking judgment, while the compiler infrastructure supplies differential execution, sanitizer checks, symbolic criteria, or fault-localization evidence (Wang et al., 29 Aug 2025, Ni et al., 26 Aug 2025, Qi et al., 21 Jun 2025).
7. Limitations, misconceptions, and research trajectory
A recurrent misconception is that compiler-LLM cooperation is equivalent to replacing the compiler with an LLM. The current evidence is more qualified. LaaC reports that mainstream LLMs can generate simple, correct assembly but currently achieve low overall success rates, with failures including syntactic errors, register misallocations, and context length caps (Zhang et al., 6 Nov 2025). The survey literature therefore emphasizes correctness and scalability as the field’s critical hurdles and identifies hybrid systems as the most promising path forward (Zhang et al., 5 Jan 2026).
A second misconception is that cooperation matters only for large frontier models. The evidence is mixed. Compiler feedback allows smaller models to outperform larger baselines without feedback on program synthesis tasks (Kjellberg et al., 17 Jan 2026), and COLT shows that lightweight multi-LLM collaboration can exceed a single large model while using only a fraction of large-model calls (Tang et al., 2 Feb 2026). This suggests that tool access, orchestration, and search policy can be as important as model scale.
A third misconception is that cooperation is only about source-code generation. The cited systems span function-call orchestration (Kim et al., 2023, Singh et al., 2024), end-to-end or modular compilation (Zhang et al., 6 Nov 2025, Zhang et al., 26 May 2025), IR-to-IR translation (Ramirez et al., 7 May 2026), hardware-accelerator compilation (Hong et al., 2024), optimization search (Tang et al., 2 Jun 2025, Mikek et al., 5 Apr 2026), deployment-time inference compilation (Mesa et al., 8 Feb 2026), compiler testing (Ni et al., 26 Aug 2025, Wang et al., 29 Aug 2025), and compiler bug isolation (Qi et al., 21 Jun 2025). The field is therefore better understood as an expansion of compiler methodology rather than a single application niche.
The most consistent future directions are likewise hybrid. The survey highlights self-improving compilers and interactive developer-LLM tools (Zhang et al., 5 Jan 2026). LLM-Tool Compiler proposes extending compile-time graph analyses such as dependency DAGs and exploring local or open-source LLMs such as Llama 3 for both fuser and agent (Singh et al., 2024). IRIS-14B explicitly positions learned translation as an interoperability layer inside hybrid neuro-symbolic compiler architectures, while preserving deterministic compilation and optimization in existing toolchains (Ramirez et al., 7 May 2026). A plausible implication is that the mature form of compiler-LLM cooperation will not be a monolithic neural compiler, but a layered stack in which LLMs serve as selectors, translators, critics, search guides, and interoperability modules around deterministic compiler cores.