- The paper introduces a multi-agent, evidence-guided framework that integrates LLM agents with compiler artifacts and runtime data for enhanced performance tuning.
- It employs specialized agents—Score, Analysis, Reasoning, and Evaluation—to iteratively refine optimization pipelines, achieving speedups up to 2.0× on ARM64 with reduced regressions.
- The approach underscores the robustness and adaptability of inference-only LLM systems, offering interpretable and architecture-aware compiler optimization across diverse platforms.
Motivation and Context
Modern compilers expose a large, combinatorial optimization space over hundreds of passes and parameters, with interaction effects that are highly nontrivial and dependent on platform microarchitecture, program structure, and target execution context. Traditional auto-tuning and profile-guided optimization (PGO) schemes either treat the compiler as a black box or restrict optimization to heuristically tuned pipelines, both approaches lacking robustness under distribution shift or in settings with strict evaluation budgets. Recent work attempts to use LLMs for code optimization, but prior approaches typically target deterministic objectives or lack grounding in runtime evidence and compiler-internal signals, resulting in weak alignment between static code-level outputs and measured hardware performance.
AutoPass addresses these fundamental limitations by integrating LLMs into a multi-agent, feedback-driven optimization loop, leveraging both compiler artifacts (e.g., LLVM IR, optimization remarks) and runtime measurements to iteratively refine optimization pipelines. The system operates inference-only, with no offline training or task-specific fine-tuning, enabling rapid adaptation to novel benchmarks and architectures.
Framework Design
AutoPass consists of four specialized LLM-driven agents:
- Score Agent: Prioritizes high-impact regions using static code features (block counts, loop counts, call frequency) and compiler logs, with agent-driven re-ranking supplanting standard profile-derived hotness.
- Analysis Agent: Extracts optimization-relevant semantic hints and compiler evidence, translating raw IR and diagnostics into structured summaries for downstream decision-making.
- Reasoning Agent: Proposes optimization pipelines based on compiler evidence, measured performance feedback, and target platform constraints, employing pass reordering, parameter tuning, and explicit rationale.
- Evaluation Agent: Validates candidate pipelines via compilation and runtime profiling, enforcing deterministic acceptance criteria and corrective feedback. If performance degrades, the system automatically discards the candidate and reverts to the strongest previously discovered pipeline.
This closed-loop, multi-agent architecture enables context-sensitive, evidence-guided pass ordering and parameter selection, targeted at minimizing execution time under practical optimization budgets.
Empirical Results
AutoPass is implemented atop LLVM and evaluated on both x86-64 and ARM64 architectures across diverse benchmarks, including embedded applications, scientific kernels, and synthetic stress suites (64 workloads). Key findings:
- Performance Effects: AutoPass consistently outperformed instrumented PGO, CSSPGO, AutoFDO, and OpenTuner under a fixed iteration budget. Geometric mean speedups were 1.043× (x86-64) and 1.117× (ARM64) over LLVM -O3, with peak improvements exceeding 2.0× for select workloads on ARM64. Variance and regression rates were substantially lower than classical autotuning and profile-guided baselines.
- Iteration Effects: Iterative feedback-driven refinement was critical; one-shot optimization yielded notably weaker results and higher regression counts. The evaluation agent provided constructive correction, stabilizing convergence and pruning ineffective strategies.
- Architectural Adaptation: Optimization strategies demonstrated platform-specific sensitivity. AutoPass increased instruction-level parallelism (loop unrolling, LICM coverage) consistently, but vectorization strategies (SLP, loop vectorizer) showed differentiated behavior between x86-64 and ARM64, reflecting microarchitectural profitabilities.
- Interpretability and Robustness: The use of compiler-internal signals and explicit rationale produced optimization decisions that were interpretable and diagnosable. Performance regressions were traceable to specific pipeline edits, which were iteratively corrected with targeted feedback.
- Ablation Study: The Reasoning Agent emerged as most critical for effective optimization; the Evaluation Agent supported robustness, and the Analysis Agent improved convergence efficiency.
- Backend Agnosticism: AutoPass exhibited robustness across multiple LLM backends, with all evaluated models achieving iterative performance gains following feedback-driven correction, demonstrating generalizability and model-agnostic applicability.
Implications for Practice and Theory
From a practical standpoint, AutoPass demonstrates that inference-only LLM agents, grounded in compiler evidence and runtime measurements, can reliably outperform established industrial optimization pipelines and classical autotuning within strict evaluation budgets. By not requiring expensive offline training, AutoPass is deployable in real-world scenarios with dynamic workloads and heterogeneous platforms.
Theoretically, the approach advances compiler optimization beyond static heuristics and black-box search by operationalizing semantic reasoning over structured compiler artifacts and runtime feedback. The multi-agent design, role specialization, and iterative correction embody a shift toward explainable, adaptive optimization frameworks that integrate both domain knowledge and empirical evidence.
The system's capacity for architecture-aware adaptation and selective optimization highlights deep potential for LLM-based frameworks to efficiently navigate sparsely effective regions of compiler optimization space, sidestepping exhaustive enumeration and profile overfitting.
Speculation on Future Developments
Future work may extend AutoPass with fine-grained cost modeling for cross-objective optimization (e.g., energy, code size), integration with verification-guided LLM approaches for stronger correctness guarantees, and scaling to distributed compilation environments. Additional advances may arise from improved semantic IR analysis, hybrid training regimes, or agentic pipelines combining symbolic reasoning with generative model outputs. Further exploration into explainability, trustworthiness, and robust deployment in safety-critical or real-time systems is warranted.
Conclusion
AutoPass introduces a multi-agent, evidence-guided LLM workflow for compiler performance tuning that surpasses classical auto-tuning and industrial PGO approaches under realistic constraints. Its interpretable, inference-only methodology, coupled with feedback-driven refinement, sets a technical precedent for future integration of LLMs into practical software optimization systems. The demonstrated adaptability, robustness, and explainability mark an important step in bridging semantic reasoning and empirical performance tuning in compiler design (2606.20373).