AI Coding Agents Need Better Compiler Remarks

Abstract: Modern AI agents optimize programs by refactoring source code to trigger trusted compiler transformations. This preserves program semantics and reduces source code pollution, making the program easier to maintain and portable across architectures. However, this collaborative workflow is limited by legacy compiler interfaces, which obscure analysis behind unstructured, lossy optimization remarks that have been designed for human intuition rather than machine logic. Using the TSVC benchmark, we evaluate the efficacy of existing optimization feedback. We find that while precise remarks provide actionable feedback (3.3x success rate), ambiguous remarks are actively detrimental, triggering semantic-breaking hallucinations. By replacing ambiguous remarks with precise ones, we show that structured, precise analysis information unlocks the capabilities of small models, proving that the bottleneck is the interface, not the agent. We conclude that future compilers must expose structured, actionable feedback designed specifically for the future of autonomous performance engineering.

• The paper demonstrates that structured compiler feedback significantly improves AI coding agents' vectorization success over vague, human-oriented remarks.
• Empirical results using TSVC benchmarks reveal that precise feedback raises success rates by up to 3.3×, highlighting the effectiveness of tailored agent workflows.
• The study emphasizes that redesigning compiler interfaces is essential for enabling reliable, autonomous code optimization in AI-driven engineering.

AI Coding Agents and Compiler Remarks: Interface Bottleneck for Automated Optimization

Motivation and Problem Statement

AI-driven code optimization has progressed considerably with LLMs evolving into autonomous performance engineers. Agents can refactor code to trigger reliable compiler optimizations, specifically auto-vectorization, thereby delegating semantic preservation to the compiler while exploring higher-level program transformations. However, current collaborative workflows are hampered by the legacy design of compiler interfaces. Optimization feedback is delivered as unstructured, lossy remarks intended for human intuition, largely obscuring deep analysis required by machine agents.

Empirical evaluation reveals that agents operate at a systemic disadvantage, possessing only partial visibility into the compiler's decision process. Vague remarks, or outright opacity, result in poor guidance for resolving failed optimizations. The lack of actionable signals—not agent capability—constitutes the dominant interface bottleneck.

Figure 1: Agentic workflow used in our evaluation, orchestrating source transformations based on compiler feedback.

Related Work

Prior research has focused on improving compiler heuristics and predicting optimal vectorization factors via ML and RL [xiao_graph-based_2025]. Systems such as "LLM-Vectorizer" [LLM-VECTORIZER:Taneja:2025] and Astra [ASTRA-MULTI-AGENT:Wei:2025] bypass compiler analyses using intrinsics but lack introspection into compiler failure reasoning. "VecTrans" [VECTRANS:Zheng:2025] established iterative agentic workflows guided by compiler feedback but treated the interface signal as immutable. Recent literature also investigates iterative optimization pipelines powered by LLMs [LLM-COMPILERS:Cummins:2025, LLM-COMPILER-FEEDBACK:Grubisic:2024], yet none address the explicit co-design of actionable, structured compiler feedback for agentic consumption.

Experimental Design

The paper proposes an agent-centric workflow where the "Engineer" agent receives both a C/C++ program and all compiler feedback (warnings, errors, optimization remarks). The agent is tasked with refactoring source code to enable vectorization, granted up to three syntax error corrections per iteration. Evaluation is conducted using the TSVC benchmark suite [TSVC:Maleki_Gao_Gazaran_Wong_Padua:2011], with validation via optimization record inspection and semantic differential testing.

The methodology isolates remark quality by employing a single-pass configuration, thereby controlling for agent iteration. Compiler feedback is deduplicated to minimize noise and maximize context fidelity during agent reasoning.

Quantitative Results: Signal Fidelity and Performance Multiplication

Vectorization success is strictly dependent on the presence and quality of remarks. Without feedback, agents exhibit negligible vectorization rates (<$1.5\%$ for Clang, <$3.7\%$ for Intel), regardless of sampling temperature. Remarks act as a performance multiplier; for Clang at $T=0.8$, success increases from $0.80\%$ to $2.68\%$ ($3.3\times$), and Intel rises from $2.38\%$ to $6.95\%$ ($2.9\times$).

Structured remarks, particularly those specifying explicit dependencies (output or anti), lead to substantial improvement (+26\%, +15.5\% delta, respectively). Vague remarks, especially Clang's NonReductionValue, are actively detrimental, triggering semantic-breaking hallucinations.

Temperature sensitivity analysis reveals that stochasticity ($T=1.2$) is required for satisfying complex constraints, while deterministic mode ($3.7\%$0) suffices for structural fixes when guided by precise feedback. For example, Intel's MultipleExits remark peaks in efficacy at low temperature, indicating deterministic transformation steps.

Hand-written precise remarks, exposing granularity such as read-after-write or write-after-read dependencies, unlock further gains; success rates climb by +45\% and +50\% at low and medium temperatures.

Figure 2: Vague remark (center) leads AI coding agent to hallucinate, breaking program semantics (right).

Qualitative Analysis: Ambiguity Hazards and Semantic Hallucination

Low-fidelity remarks do not merely fail to guide, but actively induce incorrect fixes and broken semantics. When provided with generic symptom-level feedback (e.g., "value could not be identified as reduction"), agents insert hallucinated temporaries and control branches, inadvertently destroying program logic even when syntactically valid. This effect is particularly pronounced at higher sampling temperatures where agent creativity overfits ambiguous signals.

Conversely, precise diagnostic remarks—especially those specifying the type and location of data dependencies—enable correct application of refactoring patterns such as scalar replacement, array privatization, and loop fission.

Implications: Interface Bottleneck and Compiler Co-Design

The primary constraint for autonomous optimization is the lack of structured, actionable compiler feedback. High-fidelity remarks make small models viable for tasks that previously required larger, highly capable reasoning models. The interface bottleneck is artificial and can be alleviated with co-designed diagnostic APIs exposing internal compiler analysis, dependency graphs, and source-level context.

This work proposes a paradigm shift for future compiler systems, advocating a transition from static, human-centric textual feedback to queryable, precise analysis interfaces geared toward agentic workflows. Such an evolution would empower agents to reliably navigate the semantics-preserving optimization search space while delegating correctness to trusted transformations.

Future Directions

Key avenues for research include:

• Exposing granular static and dynamic analysis (memory dependencies, runtime profiles) to facilitate complex architectural optimizations beyond local loop transformations.
• Integrating trusted transformation APIs for agents to orchestrate high-level refactorings with formal guarantees, merging LLM reasoning with compiler pass correctness [MEMOIR:MCMICHEN:2024].
• Adapting compiler signals to dynamically configure agent inference modes, matching stochastic exploration to semantic requirements and deterministic pruning to structural fixes.
• Investigating the role of remark fidelity in test-time scaling approaches and structured agent thinking, potentially reducing computational cost.

Conclusion

This paper defines the interface bottleneck between compilers and AI coding agents as the critical limiting factor for autonomous code optimization. Empirical evidence shows that actionable, structured feedback—not model size or capability—is decisive for success. Ambiguous remarks can actively degrade agent reasoning, while precise, context-rich signals unlock reliable transformation. The evolution of compiler interfaces toward agentic consumption is requisite for the next generation of autonomous performance engineering, facilitating both practical and theoretical advancements in AI-assisted code optimization.