AVO: Agentic Variation Operators for Autonomous Evolutionary Search

Abstract: Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a LLM to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware.

• The paper introduces AVO, a framework that employs self-directed coding agents to iteratively optimize kernel implementations and deliver significant performance gains.
• It replaces static variation mechanisms with interactive agents that leverage historical context, domain-specific knowledge, and real-time feedback.
• Empirical results on NVIDIA B200 GPUs reveal up to 10.5% improvement over state-of-the-art baselines for both causal and non-causal attention kernels.

Agentic Variation Operators: Autonomous Evolutionary Search for Performance-Critical Kernel Optimization

Overview and Motivation

The paper "AVO: Agentic Variation Operators for Autonomous Evolutionary Search" (2603.24517) introduces Agentic Variation Operators (AVO), a paradigm shift in evolutionary search that replaces static variation mechanisms (mutation, crossover, fixed heuristics) with self-directed coding agents. Classical evolutionary search leverages LLMs as candidate generators embedded in a fixed pipeline, with limited agency and no interactive diagnosis or iterative refinement. In contrast, AVO elevates the agent to the core of the variation step, empowering it to autonomously consult historical artifacts, domain-specific knowledge, and real-time evaluation feedback, thereby iteratively proposing, repairing, critiquing, and verifying kernel implementations.

Figure 1: Comparison of classical LLM-augmented evolutionary search frameworks and the proposed AVO, highlighting the transition from a fixed pipeline to autonomous, persistent agentic optimization.

This agentic approach enables discovery of optimizations that are unattainable in one-shot LLM frameworks, especially in highly optimized domains where further progress requires sustained, hardware-aware engineering. The central empirical target is the attention kernel, a performance-critical primitive in transformer architectures, on NVIDIA Blackwell (B200) GPUs. Both cuDNN and FlashAttention-4 (FA4) represent months of expert-level manual optimization; surpassing these kernels demands prolonged autonomous, iterative reasoning and hardware adaptation.

Formalization of Agentic Variation Operators (AVO)

AVO redefines the variation operator $Vary(\mathcal{P}_t)$ as an autonomous agentic process:

$$Vary(\mathcal{P}_t) = Agent(\mathcal{P}_t, \mathcal{K}, \mathbf{f})$$

where $\mathcal{P}_t$ is the lineage of prior solutions, $\mathcal{K}$ is the domain-specific knowledge base, and $\mathbf{f}$ encapsulates correctness and throughput evaluation. The agent operates as a persistent, interactive coding loop: consulting previous solutions, analyzing profiler outputs, leveraging technical documentation, implementing kernel-level optimizations, and verifying improvements across benchmarks with rigorous semantic and performance testing.

Figure 2: Schematic illustration of the agentic loop in AVO, encompassing planning, tool use, persistent memory, and direct interaction with feedback and resources.

A single AVO variation step subsumes sampling, generation, and evaluation. The agent autonomously decides which prior artifacts and knowledge to leverage, adapts optimization strategies as the search trajectory evolves, and commits new versions only upon passing correctness and exhibiting performance gains. This continuous, multi-day agentic process is safeguarded against stagnation through built-in self-supervision and conditional search redirection.

Experimental Results: Multi-Head and Grouped-Query Attention

The empirical evaluations focus on multi-head attention (MHA) and grouped-query attention (GQA) forward kernels, benchmarked on NVIDIA B200 hardware using BF16 precision, head dimension 128, and sequence lengths up to 32k tokens. Baselines include cuDNN (NVIDIA's production kernel) and FlashAttention-4 (open-source, state-of-the-art Blackwell-optimized kernel).

Figure 3: Throughput (TFLOPS) for multi-head attention forward-pass prefilling on B200, showing AVO outperforms baseline kernels across all key configurations.

AVO surpasses cuDNN by up to 3.5% and FA4 by up to 10.5% in causal MHA, consistently delivering higher throughput across all relevant configurations. In non-causal MHA, gains are modest ($+1.8\%$ to $+2.4\%$ over cuDNN at large sequence lengths) and within comparable variance at shorter lengths. The agent’s optimizations discovered during MHA evolution transfer robustly to GQA kernels: only 30 minutes of autonomous agentic effort adapts AVO’s MHA kernel for GQA, achieving up to $+7.0\%$ over cuDNN and $+9.3\%$ over FA4.

Figure 4: GQA throughput on B200 for two representative configurations, demonstrating rapid autonomous adaptation and performance gains over both baselines.

Evolution Trajectory and Optimization Patterns

The agentic search trajectory demonstrates intensive exploration over seven days, internally pursuing over 500 optimization directions and committing 40 kernel versions. Throughput improvements are realized as discrete jumps corresponding to architectural innovations:

• QK-PV interleaving with bitmask causal masking,
• Single-pass, online softmax computation,
• Branchless accumulator rescaling with non-blocking memory fences,
• Correction/MMA pipeline overlap,
• Register rebalancing across warp groups.

  Figure 5: Causal MHA evolution trajectory, tracking geometric mean throughput advances over 40 committed versions.

  

  Figure 6: Non-causal MHA trajectory shows compounding gains through agent-discovered schedule and resource refinements.

Diminishing returns are observed as kernel versions progress; initial architectural changes close the majority of the performance gap, while late-stage micro-architectural tuning extracts residual headroom.

Analysis of Agent-Discovered Micro-Architectural Optimizations

The ablation analysis evidences advanced hardware-level optimization strategies performed by the agent:

• Branchless Accumulator Rescaling: Eliminates warp-synchronization overhead by converting conditional logic to predicated, branchless computation. Measured $+8.1\%$ geomean throughput gain on non-causal MHA, $+1.6\%$ on causal.
• Correction/MMA Pipeline Overlap: Enables concurrent execution of correction and matrix multiplication warps, reducing idle time. Provides $+1.1\%$ geomean gain.
• Register Rebalancing: Dynamically reallocates registers from softmax to correction warps based on critical-path profiling, minimizing spill stalls. Gains reach $+2.1\%$ (non-causal).

These optimizations require intricate reasoning about GPU architectural resources (registers, memory barriers, warp scheduling, hardware ISA constraints) and reflect the agent’s ability to synthesize solutions beyond superficial code transformations.

Throughput Comparison with Official FA4 Baselines

Figure 7: AVO’s MHA throughput as measured versus official FA4 and cuDNN baselines, confirming consistent gains especially at shorter sequence lengths for causal attention.

The comparison validates the empirical results across multiple benchmarking environments, confirming AVO’s advantage in both internally measured and externally reported baselines.

Implications and Future Research Directions

AVO demonstrates that agentic variation operators enable autonomous discovery of micro-architectural optimizations surpassing hand-engineered kernels, with transferable gains across distinct attention variants. The framework is agnostic to domain, suggesting broad applicability to performance-critical software systems, scientific domains requiring persistent agentic exploration, and diverse hardware platforms.

The results motivate further investigation in several areas:

• Population-Level Branching: Extending AVO to archive-based or multi-lineage evolutionary regimes for broader exploration and diversity.
• Automated Knowledge Base Expansion: Incorporating dynamic domain knowledge acquisition to further enhance agent reasoning.
• Generalization Beyond Attention Kernels: Targeting other primitives (e.g., convolution, GEMM) and custom hardware architectures for comprehensive autonomous optimization.

Conclusion

AVO constitutes a substantive advance in autonomous optimization, replacing static evolutionary variation with persistent, self-directed agentic reasoning. Empirical results on state-of-the-art attention kernels establish significant throughput gains, rapid transferability, and deep architectural innovation. The theoretical and practical implications point toward scalable, agentic search as a paradigm for complex, hardware-aware software engineering and scientific discovery in AI.