MARCO: Multi-Agent Reactive Code Optimizer

• MARCO is a multi-agent framework that automates high-performance code optimization by integrating domain-specific transformation heuristics with real-time scholarly feedback.
• The system employs distinct agents—code optimizer, performance evaluator, and web-search—to iteratively refine code based on precise performance metrics and up-to-date optimization strategies.
• Empirical validations show MARCO achieving significant runtime and memory reductions, showcasing its scalable, cost-effective approach compared to monolithic model fine-tuning.

MARCO (“Multi-Agent Reactive Code Optimizer”) designates a framework for automating high-performance code optimization via a multi-agent, closed-loop architecture that integrates domain-specialized code transformation, empirical performance evaluation, and continuous real-time ingestion of up-to-date optimization knowledge from the scholarly literature. Developed to address the gap between general-purpose LLMs for code generation and the architecture-specific, resource-sensitive demands of high-performance computing (HPC), MARCO exemplifies a scalable, cost-effective methodology for achieving rigorous code improvements without monolithic model retraining or static fine-tuning (Rahman et al., 6 May 2025).

1. System Architecture and Agent Specialization

The MARCO framework is built around three tightly-integrated components: the Code Optimizer Agent, the Performance Evaluator Agent, and a Web-Search Engine. Each plays a distinct but interlinked role in a closed optimization loop.

• Code Optimizer Agent: Accepts an initial code snippet and applies HPC-specific transformation heuristics including cache blocking, loop tiling, OpenMP/MPI/CUDA directives, and hardware vector intrinsics. It also formulates structured NLP queries for external knowledge retrieval and iteratively refines the code based on feedback.
• Performance Evaluator Agent: Compiles and executes the candidate code on representative hardware platforms (multi-core CPUs, GPUs), collecting precise metrics—runtime, peak memory usage, FLOPS, estimated algorithmic complexity, and functional correctness (using test harnesses such as those from LeetCode).
• Web-Search Engine: Employs the Tavily tool to query scholarly repositories (IEEE, ACM, arXiv, ResearchGate), returning ranked lists of optimization strategies (e.g., platform-specific loop unrolling, memory access patterns) extracted from recently published literature.

This agent segregation enables an iterative feedback cycle that overcomes the “one-shot” limitations of monolithic LLM output by guiding code refinement through empirical performance reports and current optimization knowledge.

2. Iterative Optimization Process

MARCO formalizes the optimization process as a reactive loop. At each iteration $t$:

1. The generator infers a structured query $Q_t$ from the current code $C_t$ and feedback.
2. The web-search engine returns a knowledge set $K_t = \{ k_t^1, k_t^2, ... \}$.
3. The generator applies new techniques and its own heuristics, producing candidate code $C_t'$.
4. The evaluator benchmarks $C_t'$, obtaining metric vector $$M_t = \{ \text{Runtime}(C_t'), \text{Memory}(C_t'), ... \}$$.
5. The generator decides, based on feedback, either to halt or update $C_{t+1} \leftarrow C_t'$ and continue.

The objective is to minimize a cost function

$$\text{Cost}(C) = \alpha \cdot \text{Runtime}(C) + \beta \cdot \text{Memory}(C)$$

where $\alpha$ and $Q_t$0 are user-specified weights. An effective update is expressed as

$Q_t$1

where $Q_t$2 and $Q_t$3 regulate the step size and the reward for new-knowledge integration, respectively.

3. Real-Time Knowledge Integration

A defining feature of MARCO is the continuous use of a web-search agent to bridge the LLM’s pretraining cutoff and current HPC best practices. For each optimization cycle:

• The code optimizer auto-generates task-specific queries (e.g., “SSE4 vectorization for stencils on AMD Zen4”).
• The web-search tool extracts and summarizes relevant optimization methods from recent literature, injecting these summaries into the LLM’s prompt as additional “context windows.”
• Over successive cycles, MARCO builds a “knowledge cache” of high-impact strategies, enhancing adaptability and responsiveness to newly-published techniques (e.g., updated vectorization idioms or memory coalescing tactics).

This dynamic knowledge feed differentiates MARCO from workflows based on static fine-tuning.

4. Algorithmic Detail and Implementation

MARCO can be abstracted as follows:

$Q_t$4

Key sub-routines—FormulateQuery, WebSearch, GeneratorLLM, Evaluate, and UpdateCode—are implemented via the interaction of the LLM, Tavily web search tool, and automated benchmarking harnesses.

5. Empirical Validation and Benchmarks

Extensive evaluation on the LeetCode 75 suite—75 representative problems plus 10 hard problems—demonstrates:

Model	Mean Runtime (ms)	Peak Memory (MB)	Hard Problem Runtime (ms)
Claude 3.5	57.5	32.3	288.1
MARCO Base	49.1 (−14.6%)	28.8 (−10.8%)	138.5 (−51.9%)
MARCO + Web	39.8 (−30.9%)	—	—

These results establish that (1) iterative, closed-loop agentic optimization outperforms single-pass LLM generation, and (2) real-time web knowledge integration confers an additional ∼30% improvement over MARCO without web search.

6. Domain-Specific Techniques and Optimization Patterns

The Code Optimizer Agent systematically explores:

• Parallelism: Automated insertion of OpenMP pragmas, synthetic generation of MPI patterns, CUDA kernel synthesis with parameter tuning for grid/block dimensions and shared-memory usage.
• Memory Efficiency: Implementations of cache blocking, loop tiling, tailored prefetch intrinsics, and CUDA scratch-pad memory allocation for optimal data locality.
• Vectorization/Architecture-Awareness: Generation of AVX2/AVX-512 intrinsics and alignment directives, explicit control of SIMD kernel loop unrolling, and platform-specific calibration for hardware throughput.

Performance feedback is grounded in detailed hardware counters via FLOPS and memory-profiling tools, with a focus on empirically validated improvements versus superficial code transformation.

7. Implications, Limitations, and Future Directions

MARCO substantiates several key points:

• Efficacy: Multi-agent decoupling of code generation and evaluation—augmented by dynamic external knowledge feeds—effectively addresses the specialized demands of HPC code optimization.
• Cost-Effectiveness: MARCO achieves nontrivial runtime and memory reductions relative to both baseline LLMs and naive code optimization pipelines, without the expense of re-finetuning large LLMs.
• Extensibility: The framework’s modularity admits straightforward integration with additional languages, external libraries (e.g., Numba, Torch, MPI-GPU), and future domains as they arise.

Potential enhancements include richer, bidirectional agent communication, extended diagnostic and meta-evaluation beyond scalar metrics, and reinforcement learning-based agent control to further accelerate convergence. By automating both knowledge ingestion and rigorous empirical feedback, MARCO lowers the barrier to high-quality scientific computing and offers a generalizable blueprint for agentic code optimization (Rahman et al., 6 May 2025).