Agent-as-a-Router: Agentic Model Routing for Coding Tasks

Abstract: Real-world users typically have access to multiple LLMs from different providers, and these LLMs often excel at distinct domains, yet none dominate all. Consequently, routing each task to the most suitable model becomes critical for both performance and cost. Existing routers treat this as a static, one-off classification problem. However, we identify the performance bottleneck for these routers as information deficit: simply augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain, surpassing a heuristic router built on the same dimension-level priors. Motivated by this finding, we propose Agent-as-a-Router, a framework that formalizes routing as a C-A-F loop (Context->Action->Feedback->Context). It closes the information gap by accumulating execution-grounded experience during deployment. We instantiate this framework as ACRouter, composed of an Orchestrator, a Verifier, a Memory module, and introduce CodeRouterBench, an evaluation environment comprising ~10K task instances with verified scores from 8 frontier LLMs, enabling regret-based router comparison on streaming tasks. Experiments show that ACRouter achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic-programming tasks, demonstrating that our routing framework actively closes the information gap. Codes and benchmarks are released at https://github.com/LanceZPF/agent-as-a-router.

• The paper introduces an agentic routing framework that leverages a continuous Context–Action–Feedback loop to dynamically select optimal LLMs for coding tasks.
• The methodology integrates an Orchestrator, Verifier, and Memory module to adjust routing policies based on execution-verified performance statistics.
• Empirical results on CodeRouterBench demonstrate up to a 15.3% performance gain and superior cost-efficiency over traditional static routing methods.

Agent-as-a-Router: Agentic Model Routing for Coding Tasks

Motivation and Problem Statement

The proliferation of high-performing LLMs for code generation, debugging, and repository-level automation has created a multi-provider, multi-model environment. Each model—such as Claude Opus, GPT-5.4, Qwen3-Max, and GLM-5—exhibits strengths and specialization across different coding domains, but no single model achieves Pareto dominance across all key dimensions (e.g., code synthesis, bug fixing, data science pipelines). Given this model heterogeneity and the cost-performance trade-offs involved, optimal deployment requires dispatching each coding task to the most suitable backend model rather than statically committing to a single LLM.

Classical routing methods primarily treat model selection as a static, one-shot classification problem. The empirical evidence in this work demonstrates a fundamental limitation with such approaches: performance plateaus far below the per-task oracle, even when sophisticated reasoning engines (e.g., “zero-shot” LLM routers) are deployed. The core bottleneck is not in architectural model reasoning, but rather in information deficit—static routers lack mechanisms for accumulating task-verified, execution-grounded performance statistics and adapting their routing policy dynamically.

The Agent-as-a-Router Framework

This work introduces an explicit agentic formulation for model routing, conceptualized as a Context–Action–Feedback (C-A-F) loop. Each time a coding task arrives, the router:

1. Observes a rich context $c_i$ consisting of the task description, auxiliary metadata, and a growing memory of all previous routing outcomes;
2. Selects an action $a_i$ by choosing one of $M$ models;
3. Receives feedback $f_i$—the true, verifier-assessed score for the model’s output and the associated cost—after executing and validating the response;
4. Updates the context with the new triplet $(c_i, a_i, f_i)$ before the arrival of the next task.

Through this continual updating mechanism, the router incrementally closes the information gap, connecting C-A-F to online contextual bandit frameworks where cumulative regret with respect to the per-task oracle is the primary optimization objective.

Figure 1: Comparison of three routing strategies: static heuristics, static learned policy, and the proposed self-evolving Agent-as-a-Router with in-stream adaptation.

ACRouter Architecture

The practical instantiation of the agentic paradigm is ACRouter. It contains three compositional core modules:

• Orchestrator: Integrates routable information, blends prior statistics (e.g., dimension-level best models), retrieves similar task histories from the memory using vector embeddings, and fuses these via a fine-tuned LLM plus heuristics to make dispatch choices.
• Verifier: Performs sandboxed execution and structured validation of model outputs, aggregating multiple verification signals (e.g., AST parsing, execution results, LLM-judgement) into task- and type-conditioned quality scores.
• Memory: Online, embedding-based storage for recent routing episodes; retrieval via kNN links the current routing context to relevant previous experience.

  Figure 2: Schematic of the ACRouter system, implementing a continuous C-A-F loop with integrated Orchestrator, Verifier, and Memory modules.

The design explicitly supports fine-grained, content-aware routing. By encoding and retrieving task-representative embeddings—rather than just operating at a coarse dimension-grained level—the router adapts to content and distributional shifts in the task stream.

CodeRouterBench and Model Complementarity

Rigorous empirical validation is enabled by CodeRouterBench, a unified evaluation and benchmarking platform compiling ~10K tasks across 10 coding dimensions (e.g., generation, completion, bug fixing, test generation, agentic programming) from 15+ high-quality sources. Each task is graded by execution verification across eight strong LLMs, with both cost and score recorded per response. This enables regret-based benchmarking for routers under controlled in-distribution and OOD settings.

Figure 3: Pipeline of CodeRouterBench, from task unification to collection of per-model execution outcomes and multi-metric router comparison.

Empirical analysis of the per-dimension performance matrix shows substantial model complementarity (e.g., GLM-5 dominates Algorithm Design; Qwen3-Max leads in Test Generation; Kimi K2.5 excels in Data Science), with cost differentials often exceeding an order of magnitude between premium and budget models for similar task coverage.

Figure 4: (a) Per-dimension performance heatmap, (b) total cost comparison, (c) efficiency (performance-to-cost) ratios highlight the necessity of model-aware routing.

Empirical Results

Information Deficit as Primary Bottleneck

Ablation studies indicate a 15.3% relative performance gain over a zero-shot LLM router simply by augmenting with per-dimension prior statistics, and this statistic-aware router even surpasses a hand-coded dimension-best router. This direct evidence establishes information deficit—not a lack of model reasoning—as the main cause of the performance gap between static routers and the per-task oracle. Only architectures capable of accumulating execution-grounded feedback in deployment can close this gap.

In-Distribution and OOD Generalization

On in-distribution tasks, ACRouter achieves an AvgPerf of 49.98% and lowest cumulative regret (205.5), outperforming all static and adaptive routing baselines, including both heuristics (DimensionBest) and trained policies (Logistic Regression, BERT-based RouteLLM, kNN retrieval):

Figure 5: Cumulative regret analysis—static routers plateau rapidly, whereas ACRouter exhibits lower and steadily decreasing online regret as memory accrues more execution-grounded knowledge.

The generalization advantage of agentic routing becomes unequivocal in OOD “agentic programming” tasks (multi-step, repository-level coding workflows). Here, ACRouter achieves 62.5% resolved rate—exceeding both Always-Opus (57.14%) and all static-trained policies. Static learners suffer catastrophic degradation (falling to 8.9–21.4% resolved), indicating strong overfitting to the distributional profile of the probing set.

Figure 6: Cost-performance Pareto analysis—ACRouter extends the frontier in both AvgPerf and cost-efficiency, outperforming both dedicated premium and static routers under deployable cost constraints.

Theoretical and Practical Implications

This work delivers a decisive empirical refutation of static-(trained or heuristic)-policy routing as a sufficient mechanism for model selection in heterogeneous code LLM serving contexts, especially under non-stationary task streams or realistic OOD shifts. The central theoretical implication is that agentic, feedback-driven routing—architectures that actively accumulate and leverage execution-validated outcomes—achieve significantly lower regret and exhibit robust generalization.

Practically, this points toward several fundamental design principles for model-serving architectures in code assistants and agentic systems:

• Online self-improvement via memory enrichment: Statically trained policies rapidly saturate, whereas embedding-indexed memory with continual updates enables adaptation to streaming, non-iid task distributions.
• Verifier integration as a feedback gate: Execution sandboxing and aggregation of high-quality validation signals are essential for trustable evaluation and feedback, particularly as models’ hallucination and failure modes diverge across domains.
• Deployment optimization via agentic trade-off navigation: The Pareto frontier extension shown by ACRouter demonstrates practical cost savings, achieving higher resolved rates at lower cost than many default premium-only strategies.

Future Directions

The modular C-A-F agentic paradigm is extensible. Potential advancements include:

• Incorporating parameter-level memory or meta-learned context adaptation,
• Integrating more sophisticated Verifier architectures or crowdsourced feedback,
• Routing not only models but APIs/tools, prompt strategies, or agent subcomponents,
• Evolving CodeRouterBench with additional dimensions, models, and routing tactics for open community benchmarking.

This agentic routing principle provides a general foundation for next-generation LLM-based software engineering systems operating in heterogeneous, evolving environments.

Conclusion

The Agent-as-a-Router framework and its ACRouter instantiation provide decisive evidence for the primacy of information accumulation in model routing for coding tasks. By operationalizing routing as a C-A-F online learning loop with embedded memory and real execution feedback, ACRouter achieves superior performance and regret metrics both in-distribution and under substantial OOD drift, compared to all static heuristics and trained routing policies. This paradigm establishes a robust protocol for scalable, adaptive, cost-efficient LLM deployment in complex, user-centric coding agents.