---
title: 'Compiler-LLM Cooperation: Hybrid Integration'
url: https://www.emergentmind.com/topics/compiler-llm-cooperation
type: topic
---

# Compiler-LLM Cooperation: Hybrid Integration

Compiler-LLM cooperation denotes a family of hybrid systems in which large language models interact with conventional compiler infrastructure, compiler-inspired orchestration layers, or compiler-derived feedback rather than replacing them outright. In the survey literature, this space is organized by Design Philosophy, LLM Methodology, Level of Code Abstraction, and Task Type, and its central tensions are correctness and scalability, with hybrid systems identified as the most promising path forward [2601.02045].

## 1. Taxonomic foundations and formal models

A systematic account of the field distinguishes three design philosophies. In the **Selector** role, the LLM chooses among human-defined compiler actions; in the **Translator** role, it maps one program representation to another; and in the **Generator** role, it writes compiler-side artifacts such as passes, scripts, or transformation utilities [2601.02045]. The same survey further classifies work by whether it is training-required or training-free, by abstraction level spanning NL, Source, AST, IR, and ASM/Bin, and by task type including code generation, transpilation, optimization, repair, fuzzing, compilation, and decompilation [2601.02045].

The formalization used in that survey makes the separation explicit. A selector is modeled as $f_{\theta} : \mathcal{C} \to \mathcal{O}$, where $\mathcal{C}$ is the compilation context and $\mathcal{O}$ is a finite set of compiler actions. A translator is modeled as $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$ at abstraction level $\ell$. A generator is modeled as $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$ [2601.02045]. This framing is important because it locates the LLM either inside a bounded decision space, inside a semantics-preserving transformation loop, or inside compiler implementation itself.

| Design philosophy | Formal model | Operational role |
|---|---|---|
| Selector | $f_{\theta} : \mathcal{C} \to \mathcal{O}$ | Chooses predefined actions |
| Translator | $T_\phi : \mathrm{Rep}_\ell \to \mathrm{Rep}_\ell$ | Transforms code representations |
| Generator | $G_\psi : \mathcal{D} \to \mathrm{Code}_{\text{tool}}$ | Produces compiler-side code |

The same survey identifies three primary benefits: the democratization of compiler development, the discovery of novel optimization strategies, and the broadening of the compiler's traditional scope [2601.02045]. At the same time, it emphasizes that LLM-enabled compilation inherits the generative model’s hallucination risk, making verifiability a first-order systems concern rather than an auxiliary evaluation criterion [2601.02045].

## 2. Compiler-inspired orchestration of tools and function calls

One prominent line of work treats tool orchestration itself as a compilation problem. "LLMCompiler" formulates natural-language tasks as a Directed Acyclic Graph of atomic function calls and executes them with three components: a Function Calling Planner, a Task Fetching Unit, and an Executor [2312.04511]. The Planner emits tasks with placeholders such as $\$j$ that induce dependencies, the Task Fetching Unit maintains a ready-queue and performs placeholder replacement, and the Executor runs independent tasks asynchronously in parallel [2312.04511]. On benchmarks including HotpotQA, Movie Recommendation, ParallelQA, Game of 24, and WebShop, the system reports latency speedup of up to $3.7\times$, cost savings of up to $6.7\times$, and accuracy improvement of up to $\sim 9\%$ compared to ReAct [2312.04511].

A related but more explicitly compiler-front-end/compiler-back-end design appears in Singh et al.'s "An LLM-Tool Compiler for Fused Parallel Function Calling" [2405.17438]. Its architecture is summarized as **User query $\to$ Fuser module $\to$ Revised tool-set $\to$ LLM agent call $\to$ Executor module $\to$ Actual tool executions $\to$ Final answer**, where the Fuser is a lightweight GPT-driven routine invoked once per user prompt and the Executor de-compiles a fused function call into constituent API calls run either concurrently or in the correct sequential order [2405.17438]. The agent’s system prompt is unchanged; only the functions JSON array is rewritten, so Chain-of-Thought and ReAct logic remain unchanged [2405.17438].

The fusion heuristic is compiler-like in a literal sense. If $G$ is a group of $k$ tools of the same class, fusion is applied when
$$
T_{\text{fuse\_overhead}} < (k-1)\cdot T_{RT}.
$$
At runtime, the Fuser issues a single GPT API call with `tool_choice: none`, returns a JSON schema of fused lists, passes a sanity check that only existing tool names are fused, and dynamically rewrites the tool-set visible to the LLM [2405.17438]. On GeoLLM-Engine with $5$ K geospatial queries, GPT-3.5/4 Turbo, and CoT and ReAct zero/few-shot settings, the system achieves up to four times more parallel calls, with token costs and latency reduced by up to $40\%$ and $12\%$, respectively; in a GPT-4 Turbo, ReAct zero-shot sample, latency improves from approximately $8.97$ s to $7.38$ s and tokens from approximately $32.55$ to $24.35$ [2405.17438].

Across these systems, the recurring structural pattern is a clean separation of concerns: the LLM determines which operations are needed and in what order, while the compiler layer handles grouping, dependency analysis, scheduling, and execution [2405.17438]. A plausible implication is that some of the highest-value cooperation arises not when the LLM replaces compilation machinery, but when it is wrapped by it.

## 3. Feedback loops: from passive generator to active agent

Another major mode of cooperation gives the LLM access to a compiler as an external verifier. "From LLMs to Agents in Programming: The Impact of Providing an LLM with a Compiler" encapsulates the model as a single-model agent whose only external tool is a `gcc` compiler, with a short-lived memory of the task description, past code generations, and compiler error messages [2601.12146]. The loop runs for up to $5$ iterations: generate code, compile it with `gcc -std=c17 -Werror`, capture `stderr` on failure, extract the first error line, and prompt the model to fix the code [2601.12146]. On $699$ RosettaCode C tasks evaluated across $16$ language models from $135$ million to $70$ billion parameters, compiler access improves compilation success by $5.3$ to $79.4$ percentage units, syntax errors drop by $75\%$, and undefined-reference errors drop by $87\%$ for tasks where the agents outperform baselines [2601.12146].

This study also records a counterintuitive size-performance relation. Qwen 3 4 B jumps from $18.0 \%$ to $97.4 \%$ compilation success, a gain of $+79.4$ pp, overtaking many larger models; the authors explicitly note that a $4$ B-parameter model with compiler feedback can outshine a $70$ B-parameter model with no feedback [2601.12146]. This directly challenges the common assumption that raw parameter count is the primary determinant of practical compilation performance.

A training-time analogue appears in "CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution" [2306.06755]. CoTran combines compiler feedback with symbolic-execution feedback inside a reinforcement-learning loop. The forward model generates a candidate translation $\hat y$, compiler diagnostics are converted into a compilation reward, the translation is back-translated to $\hat x$, and a symbolic-execution engine generates a JUnit test suite on the original source to assess functional equivalence [2306.06755]. The reward combines compilation and equivalence signals, and the policy is optimized with PPO while remaining close to a supervised reference model באמצעות a KL penalty [2306.06755]. On AVATAR-TC with $57$ K Java–Python pairs, CoTran achieves $48.68\%$ FEqAcc and $76.98\%$ CompAcc for Python-to-Java, while the nearest competing tool, PLBART-base, reaches $38.26\%$ and $75.77\%$; relative to the CodeT5 baseline, the system improves FEqAcc by $+14.89\%$ and CompAcc by $+8.14\%$ for Python-to-Java [2306.06755].

These systems instantiate the same underlying principle at different timescales. Inference-time loops use compiler messages to repair concrete outputs; training-time loops convert compiler and symbolic-execution feedback into rewards that reshape the model itself [2306.06755]. This suggests that compiler diagnostics can function simultaneously as runtime critic, training signal, and search constraint.

## 4. Neural compilation, translation composability, and IR interoperability

The strongest form of cooperation asks whether the LLM can act as a compiler or a translation stage within a compiler stack. "Exploring the Feasibility of End-to-End Large Language Model as a Compiler" defines **LaaC** as an end-to-end large language model that takes high-level source code as input and generates target assembly code directly, obviating the traditional compiler pipeline of frontend, middle-end, and backend [2511.04132]. To evaluate this claim, the paper introduces CompilerEval: $20$ representative C kernels, each sampled $10$ times, on x86, ARM, and RISC-V, with success measured both as executable sample and correct sample [2511.04132]. The paper concludes that mainstream LLMs exhibit basic capabilities as compilers but currently achieve low compilation success rates; prompt engineering yields an average gain of $+2.32$ pp across four LLMs, Claude gains $+7.5$ pp on average, GPT-o1 achieves $+5$ pp average success@1 over GPT-4o and up to $+30$ pp on kernels like rotate and resize, and cross-platform success@1 is $27.85 \%$ on x86, $35.02 \%$ on ARM, and $32.30 \%$ on RISC-V [2511.04132].

A more structured response to the same problem is "LEGO-Compiler: Enhancing Neural Compilation Through Translation Composability" [2505.20356]. Its central claim is that large programs should be split into composable control blocks, translated independently, then reassembled under formal guarantees that translation preserves semantics under concatenation [2505.20356]. The workflow includes variable renaming, type and layout analysis, variable mapping and allocation, part split, LEGO part translation, and part rebuild with final behavioral check; repair is iterative and limited to affected blocks [2505.20356]. Empirically, the system reports over $99\%$ on ExeBench, $97.9\%$ on industrial-grade AnsiBench, and a near one order-of-magnitude improvement on compilable code size scalability [2505.20356]. On the full $17\,121$ ExeBench cases with DS-V3-0324, the paper reports Direct $91.59\% \to$ CoT $94.71\% \to$ LEGO $99.37\%$ [2505.20356].

A domain-specific variant appears in "LLM-Aided Compilation for Tensor Accelerators" [2408.03408]. That work proposes a two-phase workflow: **Phase I: Functional Translation** into a target accelerator DSL, followed by **Phase II: Cost-Model-Driven Optimization** [2408.03408]. GPT-4 is guided by instructions, ISA specification, and source program, and outputs are verified on a simulator or hardware; translation and optimization are further decomposed into smaller subtasks and multi-step repair loops [2408.03408]. On the TinyMPC mat-vec kernel, one-shot ICL plus NL annotations reaches $\text{pass@1} = 46.0\%$, $\text{pass@10} = 88.8\%$, and $\text{pass@50} = 99.98\%$; overall, $8$ of $9$ kernels are translated successfully after repair [2408.03408].

Interoperability at the IR level is addressed by "LLM Translation of Compiler Intermediate Representation" [2605.08247]. IRIS-14B is fine-tuned to translate GIMPLE to LLVM IR and is explicitly positioned as an interoperability layer in hybrid neuro-symbolic compiler architectures [2605.08247]. On CodeForces-IRIS it reaches $73.2 \%$ compile and $63.3 \%$ I/O test pass, and on ExeBench-IRIS $86.9 \%$ compile and $79.1 \%$ I/O test pass, outperforming widely used models by up to $44$ percentage points [2605.08247]. Unlike end-to-end replacement, this design preserves deterministic compilation and optimization in GCC and LLVM while inserting the LLM as a textual IR bridge [2605.08247].

Taken together, these results delimit a spectrum. At one end, LaaC probes end-to-end feasibility but reports low success rates [2511.04132]. In the middle, LEGO-Compiler and Gemmini-oriented workflows decompose compilation into verifiable subtasks [2505.20356; 2408.03408]. At the other end, IRIS-14B inserts a learned translation layer between mature toolchains without modifying existing passes [2605.08247].

## 5. Optimization search, multi-agent compilation, and inference deployment

Compiler-LLM cooperation is also used to navigate optimization spaces that are too large for fixed heuristics. "Compiler Optimization via LLM Reasoning for Efficient Model Serving" formulates optimization as a finite-horizon Markov decision process whose state includes the current IRModule, hardware profile, and transformation history, while actions are hardware-aware transformations such as tiling, fusion, and vectorization [2506.01374]. Its REASONING COMPILER architecture combines an LLM proposal mechanism with structured Monte Carlo tree search, using UCT for selection, LLM-guided expansion, cost-model simulation, and backpropagation [2506.01374]. Across four kernels, LLM-MCTS reaches a $7.08\times$ speedup after just $36$ compiler evaluations on the Llama3 benchmark versus MetaSchedule or Ansor requiring $72$ samples, attains a $2\times$ speedup on Flux-Conv at $36$ samples where the baseline needs more than $600$, and achieves up to $2.5\times$ end-to-end inference improvement over unoptimized code with only $T=36$ queries [2506.01374].

"COLT: Lightweight Multi-LLM Collaboration through Shared MCTS Reasoning for Model Compilation" generalizes this search setting to multiple models [2602.01935]. Its node state is a joint state $\langle \text{program } P, \text{model } m \rangle$, actions jointly choose a semantic-preserving compiler transformation and the next LLM to query, and rewards are back-propagated through a single shared MCTS tree [2602.01935]. The model-aware UCT policy adds a normalized bias term $\phi_{\mathrm{small}}(m)$ toward smaller LLMs, while a course-alteration mechanism escalates to the largest model after persistent regressions from small-model expansions [2602.01935]. Empirically, COLT uses only approximately $23.9\%$ of the large-model calls in the $8$-model setting while achieving better final speedups, including up to $30.05\times$ on GPU and $10.86\times$ on CPU across five kernels; in end-to-end Llama-3 tuning, COLT$_4$ reaches $5.02\times$ with only $390$ samples, versus GPT-5-mini at $5.46\times$ with $660$ samples and GPT-5.2 at $5.01\times$ with $530$ samples [2602.01935].

A different form of cooperation appears in "Agentic Code Optimization via Compiler-LLM Cooperation" [2604.04238]. That work introduces a multi-agent system with LLM-based optimization agents at source, IR, and assembly levels, compiler constituents as tools, an LLM-based test generation agent, and a guiding LLM that allocates budget across levels of abstraction [2604.04238]. The reported outcome is that compiler-LLM cooperation outperforms both existing compiler optimizations and level-specific LLM baselines, producing speedups up to $1.25\times$ [2604.04238].

At inference deployment time, the cooperation may shift from code transformation to compiler-assisted scheduling of the LLM itself. "Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices" integrates speculative decoding into an IREE/MLIR workflow through offline quantization, profiling, analytical cost modeling, design-space exploration, and AOT compilation [2602.08060]. The cost model uses acceptance rate $\alpha$, draft length $\gamma$, and relative cost coefficient $c=t_{\text{draft}}/t_{\text{target}}$, with a necessary condition for speedup given by $c < \alpha$ [2602.08060]. On an NXP i.MX95 device with $6 \times$ A55 and a Mali-G310, the system reports a predicted peak speedup of approximately $1.68\times$ at $\gamma=5$ and a measured speedup of approximately $1.61$-$1.65\times$, about $4\%$ below prediction [2602.08060].

The unifying theme in these optimization papers is that the LLM is not treated as an optimizer in isolation. It is embedded inside search, budget allocation, profiling, or compiler runtime infrastructure, and its proposals are evaluated against cost models, tests, or measured performance [2506.01374; 2604.04238; 2602.08060]. This suggests a shift from prompt-only optimization to tool-grounded optimization.

## 6. Testing, bug discovery, and reliability engineering

Compiler-LLM cooperation also extends to compiler testing and bug isolation. "LLM-Based Program Generation for Triggering Numerical Inconsistencies Across Compilers" presents LLM4FP, which combines Grammar-Based Generation and Feedback-Based Mutation to synthesize floating-point programs that produce differential behavior across compilers and optimization levels [2509.00256]. The pipeline compiles each generated program with gcc, clang, and nvcc across six optimization levels, executes them on identical inputs, and marks a program as successful if any pair disagrees bitwise [2509.00256]. Compared with Varity, LLM4FP reaches an inconsistency rate of $26.56\%$ versus $11.93\%$, producing $4,781$ total mismatches versus $2,147$; moreover, $92\%$ of LLM4FP mismatches are $\{\text{Real},\text{Real}\}$ rather than extreme values such as infinities or NaN [2509.00256].

"LegoFuzz: Interleaving Large Language Models for Compiler Testing" separates LLM use into an offline phase and an online phase [2508.18955]. Offline, the LLM transforms AnghaBench functions into small, feature-rich, valid C functions, while GCC/Clang front-ends, sanitizers, and CompCert validate them; online, these pre-validated components are composed into large, inter-dependent test programs without further LLM calls [2508.18955]. On GCC and LLVM, this framework reports $66$ bugs, of which $58$ were previously unknown and $56$ already fixed upstream; the set includes $36$ crashes and $30$ miscompilations [2508.18955]. Throughput is reported as $0.02$ s per test, about $193$ s for $10$ K tests, compared with approximately $12{,}121$ s for Fuzz4All and approximately $28{,}284$ s for WhiteFox [2508.18955].

Bug localization is addressed by "Improving Compiler Bug Isolation by Leveraging Large Language Models" [2506.17647]. AutoCBI combines failing test programs, source-file function summaries generated by an LLM, suspicious file lists derived from SBFL formulas such as Ochiai and Wong2, and compilation configurations with related output messages, then uses a specialized prompt to rerank suspicious files [2506.17647]. On $120$ real-world bugs from GCC and LLVM, AutoCBI isolates $66.67\%/69.23\%$, $300\%/340\%$, and $100\%/57.14\%$ more bugs than RecBi, DiWi, and FuseFL, respectively, in Top-1 ranked results for GCC/LLVM [2506.17647]. The ablation study further reports that removing document summaries or failing test code cuts Top-1 performance by roughly $50$-$55\%$, indicating that semantic context is not merely decorative but operationally central [2506.17647].

These reliability-oriented systems broaden the scope of compiler-LLM cooperation beyond code generation and optimization. Here the LLM supplies diversity, semantic summarization, or ranking judgment, while the compiler infrastructure supplies differential execution, sanitizer checks, symbolic criteria, or fault-localization evidence [2509.00256; 2508.18955; 2506.17647].

## 7. Limitations, misconceptions, and research trajectory

A recurrent misconception is that compiler-LLM cooperation is equivalent to replacing the compiler with an LLM. The current evidence is more qualified. LaaC reports that mainstream LLMs can generate simple, correct assembly but currently achieve low overall success rates, with failures including syntactic errors, register misallocations, and context length caps [2511.04132]. The survey literature therefore emphasizes correctness and scalability as the field’s critical hurdles and identifies hybrid systems as the most promising path forward [2601.02045].

A second misconception is that cooperation matters only for large frontier models. The evidence is mixed. Compiler feedback allows smaller models to outperform larger baselines without feedback on program synthesis tasks [2601.12146], and COLT shows that lightweight multi-LLM collaboration can exceed a single large model while using only a fraction of large-model calls [2602.01935]. This suggests that tool access, orchestration, and search policy can be as important as model scale.

A third misconception is that cooperation is only about source-code generation. The cited systems span function-call orchestration [2312.04511; 2405.17438], end-to-end or modular compilation [2511.04132; 2505.20356], IR-to-IR translation [2605.08247], hardware-accelerator compilation [2408.03408], optimization search [2506.01374; 2604.04238], deployment-time inference compilation [2602.08060], compiler testing [2508.18955; 2509.00256], and compiler bug isolation [2506.17647]. The field is therefore better understood as an expansion of compiler methodology rather than a single application niche.

The most consistent future directions are likewise hybrid. The survey highlights self-improving compilers and interactive developer-LLM tools [2601.02045]. LLM-Tool Compiler proposes extending compile-time graph analyses such as dependency DAGs and exploring local or open-source LLMs such as Llama 3 for both fuser and agent [2405.17438]. IRIS-14B explicitly positions learned translation as an interoperability layer inside hybrid neuro-symbolic compiler architectures, while preserving deterministic compilation and optimization in existing toolchains [2605.08247]. A plausible implication is that the mature form of compiler-LLM cooperation will not be a monolithic neural compiler, but a layered stack in which LLMs serve as selectors, translators, critics, search guides, and interoperability modules around deterministic compiler cores.

Source: https://www.emergentmind.com/topics/compiler-llm-cooperation