AI-PROPELLER: Warehouse-Scale Interprocedural Code Layout Optimization with AlphaEvolve

Abstract: Post-link optimizers (PLOs) such as Propeller and BOLT have demonstrated that precise, profile-guided code layout can extract significant performance gains from heavily optimized binaries. However, these systems are currently restricted to intraprocedural techniques, leaving the global potential of interprocedural layout largely untapped. Interprocedural code layout is historically difficult due to a combinatorially intractable search space and complex call-return semantics that are challenging to model. Consequently, the performance potential of fine-grained interprocedural layout remains unproven in practice. AI-PROPELLER uses Magellan, an agentic workflow that evolves the compiler heuristic in Propeller into a fine-grained interprocedural optimizer and fine-tunes the resulting policy hyperparameters. To ensure high-fidelity, we move away from approximate static cost models and the agentic workflow generates multiple layout variants that are executed on actual hardware to measure real performance counters, providing a precise reward signal for the evolutionary loop. AI-PROPELLER has been evaluated on several benchmarks including large warehouse-scale applications and experiments show performance improvements of 0.23% to 1.6% optimized with state-of-the-art FDO and PLO which is significant for real-world binaries. This is the first time ever that large warehouse-scale applications in industrial settings have been optimized with fine-grained interprocedural code layout.

• The paper introduces an AI-driven framework leveraging AlphaEvolve and the Magellan workflow to optimize interprocedural code layout at warehouse scale.
• Experimental results demonstrate significant performance gains, including a 1.6% reduction in LLVM Clang execution time and reduced pipeline stalls across benchmarks.
• The method achieves these gains without additional build overhead and generalizes policy evolution across diverse industrial workloads.

AI-PROPELLER: Fine-Grained Interprocedural Code Layout Optimization at Scale

Introduction and Motivation

This paper presents Propeller, an agentic, AI-driven framework for interprocedural code layout optimization of warehouse-scale binaries. Existing post-link optimizers (PLOs) like Propeller and BOLT leverage profile-informed heuristics, but are constrained to intraprocedural reordering, missing the performance potential from interleaving hot code across function boundaries. The challenge of interprocedural code layout arises from its combinatorial complexity and requirement for precise modeling of call-return semantics. By integrating AlphaEvolve within the Magellan agentic workflow, Propeller evolves and tunes code layout heuristics, providing empirical hardware-validated performance signals as rewards.

Figure 1: Motivating example: Propeller rearranges hot code paths in LLVM Clang, dramatically improving spatial locality across function boundaries.

Methodology and Magellan Workflow

The core innovation is adaptation of the Magellan workflow, consisting of AlphaEvolve—an LLM-driven function-in-the-loop coder—and Vizier, a black-box optimizer for numerical heuristics. The evolutionary loop proposes, implements, recompiles, and empirically evaluates new layout policies using high-fidelity hardware counters. This bypasses the inaccuracies inherent to static cost models and enables stable, low-noise reward acquisition.

Figure 2: Overview of integration: Magellan applies AlphaEvolve and Vizier to Propeller’s code layout optimization loop.

Three principal hyperparameters, evolved by Propeller, govern interprocedural basic block placement: the Chain Split Threshold, controlling maximal sequence lengths for splitting; Forward/Backward Memory Offsets, regulating the maximum distance between basic block pairs connected by hot edges. Traditional heuristics use restrictive, intraprocedural settings, but Propeller discovers empirically that expanded ranges—e.g., increasing chain split threshold from zero (disallowing splits) to over 200, and memory offsets to several kilobytes—unlock non-trivial layout improvements.

Figure 3: Visualization of the discovered Propeller interprocedural layout policy as applied to LLVM Clang.

Experimental Results

Propeller is evaluated on several benchmarks: LLVM Clang, LevelDB, Redis, and a proprietary warehouse-scale search workload. Experiments utilize dedicated Skylake servers with extreme control over system noise (run-to-run variance below 0.05%), enabling accurate attribution of performance variation to code layout policy.

Performance improvements over state-of-the-art intraprocedural (Ext-TSP) and function-level (CDSort) heuristics are robust and statistically significant (p~0). For instance:

• LLVM Clang: 1.6% execution time reduction on top of highly optimized PGO+ThinLTO+Ext-TSP.
• LevelDB and Redis: 0.5–0.9% improvements.
• Search (warehouse-scale, ~1.4M functions, 40M basic blocks): 0.23% over production-optimized Ext-TSP.

Figure 4: Execution time reductions from Propeller compared to CDSort and Ext-TSP across real-world benchmarks.

Pipeline analysis attributes the gains to significant reduction in front-end pipeline stalls (as much as 15% relative in Redis), with a corresponding increase in the percent of CPU slots issuing useful instructions. Ablation reveals that ~80% of the total speedup of Propeller arises directly from interprocedural, rather than intraprocedural, block reordering—a strong confirmation of the practical value of the approach.

Figure 5: Fraction of CPU pipeline slots spent retiring instructions: Propeller reliably increases this metric, indicating more effective hardware utilization.

Policy Generalization and Scalability

Policies discovered and tuned on reduced-scale representative binaries generalize well to novel applications of similar structure (e.g., from 100 Clang modules to a full 3000-module build). They do not reliably generalize between structurally dissimilar workloads (e.g., Clang vs. warehouse-scale Search), motivating per-application (or per-family) policy evolution, which remains feasible due to the high automation of the Magellan loop and the stability of the training environments.

Unlike previous attempts at interprocedural layout, Propeller introduces no perceptible build time or space overheads, maintaining industrial scalability even for warehouse-scale binaries.

Related Work and Contrast

Propeller advances beyond static, function- or basic-block-level code layout heuristics by achieving practical interprocedural reordering at scale. Prior function-level approaches (e.g., HFSort, CDSort) cannot separate or co-locate hot code across function boundaries. Modern ML-guided compilers (e.g., MLGO, ACE) typically remain within the boundaries of intraprocedural or function-level transformations, or rely on cost models that underfit hardware nuances.

In contrast, Propeller’s empirical, agentic optimization mechanism extracts reward signals directly from hardware counters, enabling the discovery of globally non-intuitive but hardware-efficient layouts.

Implications and Future Directions

Propeller demonstrates that empirically validated, AI-discovered policies for complex, high-impact compiler problems—previously viewed as computationally intractable or not justifying engineering effort—yield measurable performance dividends even for mature industrial pipelines. Improved code locality and reduction in frontend stalls translate directly into reduced execution time and operational costs at hyperscale.

The approach opens further avenues:

• Extension to other binary or IR-level transformations (e.g., inlining, register allocation).
• Integration of richer, runtime-dependent profile information for context-adaptive tuning.
• Generalization of Magellan-like agentic optimization workflows to end-to-end compiler pipelines, not just late-stage PLOs.

Conclusion

Propeller provides the first solution for practical, fine-grained interprocedural code layout optimization at warehouse scale. By leveraging agentic, empirical learning via the Magellan framework, it achieves statistically significant and operationally meaningful performance improvements on top of existing highly-optimized compiler toolchains. These results demonstrate the viability and benefits of AI-guided agentic frameworks for automated discovery and tuning of complex compiler optimization heuristics in production settings.