ACRouter: Adaptive Model Routing

• ACRouter is an instantiation of the Agent-as-a-Router framework that uses an iterative Context→Action→Feedback→Context loop to address sequential model routing.
• It augments static routing by integrating task-dimension performance statistics, achieving a 15.3% relative gain over heuristic methods.
• Evaluated on CodeRouterBench, ACRouter minimizes cumulative regret in both in-distribution and out-of-distribution settings, balancing cost and performance.

Searching arXiv for the specified paper and closely related routing work. ACRouter is an instantiation of the Agent-as-a-Router framework for coding-task model selection, introduced in "Agent-as-a-Router: Agentic Model Routing for Coding Tasks" (Zhou et al., 22 Jun 2026). It addresses the setting in which real-world users have access to multiple LLMs from different providers, while no single model dominates all coding domains. In that setting, routing each task to the most suitable model becomes a joint performance-and-cost problem. The framework departs from routers that treat dispatch as a static, one-off classification problem, and instead formalizes routing as an iterative Context→Action→Feedback→Context loop. Its central premise is that the main bottleneck in existing routers is an information deficit: augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain and surpasses a heuristic router built on the same dimension-level priors. ACRouter is designed to close that deficit by accumulating execution-grounded experience during deployment, and it is evaluated on CodeRouterBench, a regret-ready streaming benchmark with verified outcomes from 8 frontier LLMs (Zhou et al., 22 Jun 2026).

1. Routing problem and the notion of information deficit

The routing problem is posed over a streaming task sequence $t_1,t_2,\dots,t_N$, where each task must be dispatched to one of $M$ models $\{m_1,\dots,m_M\}$. In this formulation, routing is not merely a model-ranking problem over a fixed prompt representation; it is a sequential decision process in which later decisions can benefit from earlier execution outcomes.

The motivating claim is that existing routers are constrained less by classifier capacity than by missing information. The reported 15.3% relative gain from augmenting a vanilla LLM router with task-dimension-level performance statistics indicates that performance priors substantially alter routing quality, even before online adaptation is introduced (Zhou et al., 22 Jun 2026). The same result also shows that dimension-level priors alone do not exhaust the available signal, because ACRouter is then constructed to gather execution-grounded feedback during deployment and feed that back into future routing decisions.

This framing places ACRouter between two unsatisfactory extremes. At one extreme are static learners, which route from prompt-side features and precomputed priors. At the other are single-model policies such as always choosing the strongest frontier model. The reported experiments indicate that neither extreme is sufficient once multiple specialists with distinct cost–quality tradeoffs coexist.

2. C–A–F loop formalism and regret objective

ACRouter formalizes routing as a four-stage loop executed at each step $i$:

• Context $c_i$
• Action $a_i \in \{1,\dots,M\}$
• Feedback $f_i$
• Context update $c_{i+1}$

The context is defined as

$c_i = (p_i,\; d_i,\; \mathcal H_{<i}),$

where $p_i$ is the task prompt, $M$0 is optional metadata such as coding dimension, difficulty, and language, and $M$1 is the Memory state summarizing prior $M$2 pairs for $M$3. The selected action $M$4 is the index of the chosen model. Feedback is defined as

$M$5

where $M$6 is the Verifier’s estimated correctness score and $M$7 is the monetary cost in USD computed from token usage.

The loop is explicitly cast as a contextual multi-armed bandit. The true per-task reward of choosing arm $M$8 on task $M$9 is

$\{m_1,\dots,m_M\}$0

Over a stream of length $\{m_1,\dots,m_M\}$1, a policy $\{m_1,\dots,m_M\}$2 attains average reward

$\{m_1,\dots,m_M\}$3

The per-task oracle, defined in hindsight by $\{m_1,\dots,m_M\}$4, achieves

$\{m_1,\dots,m_M\}$5

Performance is measured by cumulative regret: $\{m_1,\dots,m_M\}$6 Equivalently, with an outcome matrix $\{m_1,\dots,m_M\}$7 whose entries are $\{m_1,\dots,m_M\}$8, the reward matrix is

$\{m_1,\dots,m_M\}$9

and cumulative regret becomes

$i$0

In the reported experiments, $i$1 (Zhou et al., 22 Jun 2026).

3. System instantiation: Orchestrator, Verifier, and Memory

ACRouter is composed of an Orchestrator, a Verifier, and a Memory module. The overall control flow is: retrieve similar prior cases from Memory, combine them with metadata priors, choose a model through the Orchestrator, execute the selected model, score the result with the Verifier, and write the resulting experience back to Memory.

Module	Core function	Key specification
Orchestrator	Selects the model index $i$2	Cost-aware Qwen3.5-0.8B fine-tuned on the probing set, ensembled with heuristic rules
Verifier	Produces execution-grounded feedback	Sandbox execution, AST checks, in-prompt tests, and LLM-as-Judge for proxy dimensions
Memory	Stores and retrieves prior experiences	Vector store keyed by task embeddings, cosine-kNN top 10 with threshold 0.5, FIFO capacity 20 000

The Orchestrator takes $i$3, a DimensionBest prior, and top-$i$4 retrieved neighbors from Memory. Its core policy is a cost-aware Qwen3.5-0.8B LLM fine-tuned on the probing set and ensembled with simple heuristic rules, including preferring the cheaper model when estimated quality is approximately equal. Its output is the chosen model index $i$5.

The Verifier aggregates execution signals into a scalar score. When code is executable it runs the candidate’s code in a sandbox; it also uses AST checks, in-prompt tests, and an LLM-as-Judge for proxy dimensions. Its aggregation rule is

$i$6

where $i$7 is the set of tools for dimension $i$8. The Verifier emits a scalar $i$9 and the actual cost $c_i$0.

The Memory module is a vector store keyed by task embeddings produced by voyage-code-3 or BGE-large. Retrieval is cosine-kNN with top 10 neighbors and similarity threshold 0.5. Storage is an in-place FIFO buffer with capacity 20 000 entries. Each entry logs the embedding of $c_i$1 together with $c_i$2. The stated purpose of this module is to enable the Orchestrator to condition on actual execution results, thereby closing the feedback loop (Zhou et al., 22 Jun 2026).

4. CodeRouterBench as evaluation environment

CodeRouterBench is described as the first regret-ready streaming benchmark for code routing. It contains approximately 10 111 tasks across 10 coding dimensions:

1. Code Generation
2. Algorithm Design
3. Bug Fixing
4. Code Completion
5. Code Refactoring
6. Data Science
7. Multi-Language
8. Code Understanding
9. Test Generation
10. Agentic Programming (out-of-distribution)

The tasks are collected from more than 15 widely used benchmarks, including HumanEval+, MBPP, BigCodeBench, SWE-Bench, LiveCodeBench, DS-1000, and CodeXGLUE. Scoring differs by dimension: 7 dimensions use execution-based pass@1 in a sandbox, while 3 dimensions use proxy metrics plus LLM-as-Judge (Zhou et al., 22 Jun 2026).

The split structure is also central to the benchmark design. The probing set contains 7 080 tasks, divided into 60% train and 10% validation, and is used to build priors and train classifiers. The in-distribution test set contains 2 919 tasks and is used for the main evaluation over 9 single-turn dimensions. The out-of-distribution test set contains 176 Agentic Programming tasks that are multi-step, multi-file, and repository-level; no router sees these at train time.

The model pool comprises 8 frontier LLMs: Claude Opus 4.6, Sonnet 4.6, GPT-5.4, Qwen3-Max, Qwen3.5-Plus, GLM-5, Kimi-K2.5, and MiniMax-M2.7. For each task–model pair, an answer is generated, passed through the Verifier harness, and recorded as $c_i$3, yielding the full outcome matrix $c_i$4. Router evaluation then simulates a fixed task order, records each chosen model, and computes cumulative regret. Two additional metrics are reported: $c_i$5 and $c_i$6.

5. Empirical performance on in-distribution and OOD workloads

On the in-distribution evaluation over 2 919 tasks, the per-task oracle attains AvgPerf% $c_i$7, CumReg $c_i$8, and Perf/\$c_i$9 $a_i \in \{1,\dots,M\}$0, which is the lowest cumulative regret among the reported non-oracle routers (<a href="/papers/2606.22902" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Zhou et al., 22 Jun 2026</a>).</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>In-distribution router</th> <th style="text-align: right">AvgPerf%</th> <th style="text-align: right">CumReg</th> </tr> </thead><tbody><tr> <td>Oracle (per-task)</td> <td style="text-align: right">57.00</td> <td style="text-align: right">0</td> </tr> <tr> <td>ACRouter</td> <td style="text-align: right">49.98</td> <td style="text-align: right">205.5</td> </tr> <tr> <td>DimensionBest</td> <td style="text-align: right">47.50</td> <td style="text-align: right">277.4</td> </tr> <tr> <td>LogReg</td> <td style="text-align: right">47.26</td> <td style="text-align: right">284.4</td> </tr> <tr> <td>RouteLLM-BERT</td> <td style="text-align: right">47.22</td> <td style="text-align: right">285.5</td> </tr> <tr> <td>LinUCB</td> <td style="text-align: right">46.84</td> <td style="text-align: right">296.9</td> </tr> <tr> <td>LinTS</td> <td style="text-align: right">46.48</td> <td style="text-align: right">307.4</td> </tr> <tr> <td>Qwen3.5-0.8B–FT</td> <td style="text-align: right">46.41</td> <td style="text-align: right">309.1</td> </tr> <tr> <td>Always-Opus 4.6</td> <td style="text-align: right">43.83</td> <td style="text-align: right">387.1</td> </tr> <tr> <td>Always-Kimi-K2.5</td> <td style="text-align: right">36.66</td> <td style="text-align: right">593.3</td> </tr> </tbody></table></div> <p>Perf/\$ values on the same in-distribution setting show a different ordering: Always-Kimi-K2.5 reports $a_i \in \{1,\dots,M\}$1, Qwen3.5-0.8B–FT $a_i \in \{1,\dots,M\}$2, LogReg $a_i \in \{1,\dots,M\}$3, RouteLLM-BERT $a_i \in \{1,\dots,M\}$4, LinTS $a_i \in \{1,\dots,M\}$5, LinUCB $a_i \in \{1,\dots,M\}$6, ACRouter $a_i \in \{1,\dots,M\}$7, DimensionBest $a_i \in \{1,\dots,M\}$8, and Always-Opus 4.6 $a_i \in \{1,\dots,M\}$9. This makes explicit that regret minimization, average performance, and cost efficiency are related but non-identical objectives.

On the 176 out-of-distribution Agentic Programming tasks, the oracle attains AvgPerf% $f_i$0, CumReg $f_i$1, and Perf/\$f_i$2 $f_i$3, again yielding the lowest cumulative regret among the reported non-oracle routers (<a href="/papers/2606.22902" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Zhou et al., 22 Jun 2026</a>).</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>OOD router</th> <th style="text-align: right">AvgPerf%</th> <th style="text-align: right">CumReg</th> </tr> </thead><tbody><tr> <td>Oracle (per-task)</td> <td style="text-align: right">75.89</td> <td style="text-align: right">0</td> </tr> <tr> <td>ACRouter</td> <td style="text-align: right">62.50</td> <td style="text-align: right">17.0</td> </tr> <tr> <td>Always-Opus 4.6</td> <td style="text-align: right">57.14</td> <td style="text-align: right">26.7</td> </tr> <tr> <td>Qwen3.5-0.8B–FT</td> <td style="text-align: right">55.36</td> <td style="text-align: right">27.2</td> </tr> <tr> <td>LinUCB</td> <td style="text-align: right">49.82</td> <td style="text-align: right">31.1</td> </tr> <tr> <td>LinTS</td> <td style="text-align: right">46.43</td> <td style="text-align: right">35.9</td> </tr> <tr> <td>Random</td> <td style="text-align: right">31.25</td> <td style="text-align: right">50.4</td> </tr> <tr> <td>RouteLLM-BERT</td> <td style="text-align: right">21.43</td> <td style="text-align: right">59.4</td> </tr> <tr> <td>LogReg</td> <td style="text-align: right">19.64</td> <td style="text-align: right">61.8</td> </tr> <tr> <td><a href="https://www.emergentmind.com/topics/quantum-k-nearest-neighbour-knn" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">kNN</a> Retrieval</td> <td style="text-align: right">14.29</td> <td style="text-align: right">66.7</td> </tr> </tbody></table></div> <p>The associated Perf/\$ values on OOD tasks are: kNN Retrieval $f_i$4, RouteLLM-BERT $f_i$5, ACRouter $f_i$6, LogReg $f_i$7, LinUCB $f_i$8, Random $f_i$9, LinTS $c_{i+1}$0, Qwen3.5-0.8B–FT $c_{i+1}$1, and Always-Opus 4.6 $c_{i+1}$2. The principal empirical claim is therefore not that ACRouter dominates every scalar metric, but that it minimizes cumulative regret in both the in-distribution and OOD settings.

6. Interpretation, misconceptions, and scope

A recurring misconception in model routing is that the problem can be adequately solved as prompt-only classification. The ACRouter results are presented specifically against that view. The reported gain from adding task-dimension performance statistics, followed by further gains from execution-grounded feedback, supports the argument that the central bottleneck is missing information rather than only classifier choice.

A second misconception is that online adaptation alone is sufficient. Classical bandits such as LinUCB and LinTS do adapt online, but the reported interpretation is that they lack fine-grained context reasoning and therefore still lag ACRouter. Conversely, static learners and heuristic priors can match or slightly beat DimensionBest in-distribution, yet collapse under OOD shift: on the agentic tasks, RouteLLM-BERT drops to AvgPerf% $c_{i+1}$3, LogReg to $c_{i+1}$4, and kNN Retrieval to $c_{i+1}$5, whereas ACRouter remains at $c_{i+1}$6 (Zhou et al., 22 Jun 2026).

A third misconception is that always using the strongest individual model is either performance-optimal or cost-efficient. The reported results reject that simplification. Always-Opus 4.6 trails ACRouter in cumulative regret in both in-distribution and OOD settings, and it also reports lower Perf/\$ than several alternatives. The stated conclusion is that always choosing the single strongest model is neither performance- nor cost-efficient once multiple specialists exist.

Within the scope of the reported study, the broader significance of ACRouter lies in treating model routing as an iterative decision process with verified feedback rather than a one-off dispatch rule. A plausible implication is that robustness under distribution shift depends not only on prior statistics or online reward updates in isolation, but on maintaining a memory of execution-grounded experience and conditioning future choices on that memory.