Agent-as-a-Router: Routing Specialists

• Agent-as-a-Router is a design pattern that delegates tasks to specialized experts instead of solving tasks end-to-end.
• It leverages diverse signals—from runtime telemetry to graph-structured data—to decide which model, tool, or agent should act.
• The pattern supports various routing mechanisms, including hard, soft, and hierarchical approaches, for optimized execution and adaptive performance.

Searching arXiv for papers on “Agent-as-a-Router” and closely related routing frameworks across agents, tools, models, and context. Agent-as-a-Router denotes an architectural pattern in which an agentic component is optimized not primarily to solve a task end-to-end, but to decide which specialist should act, which model or tool path should execute, which context should be exposed, and, in some systems, whether execution should occur at all. Across recent work, the routed object may be an operating-system scheduler, a QA agent, an MCP tool or agent, a model tier for coding or web tasks, a structured workflow, or a subset of shared memory; the common abstraction is delegation under task, state, and systems constraints rather than monolithic inference (Zhou et al., 22 Jun 2026, Wang et al., 7 Nov 2025, Zhang et al., 6 Oct 2025, Cheng et al., 8 Apr 2026, Liu et al., 6 Aug 2025).

1. Conceptual basis and scope

The central intuition is that many modern systems are no longer bottlenecked by the existence of a single strong solver, but by the inability to match heterogeneous situations to heterogeneous specialists. In operating systems, this appears as a policy-selection problem over schedulers specialized for fairness, throughput, and latency (Wang et al., 7 Nov 2025). In multi-agent question answering, it appears as the need to allocate responsibility among agents with different backbones and prompting strategies, because larger models are not always superior and no single agent uniformly dominates (Zhang et al., 6 Oct 2025). In coding-task model routing, it appears as an information-deficit problem: routing quality improves when the router can condition on execution-grounded experience rather than frozen one-shot classification features (Zhou et al., 22 Jun 2026).

This literature treats routing as a first-class learned or structured capability. The router may observe runtime telemetry, graph structure, dialogue history, model outcomes, tool schemas, or shared memory state; it then maps that state to a downstream action such as expert activation, weighted collaboration, escalation, or direct response. Some systems make routing hierarchical, with separate decisions for intent, workflow, model backend, and edge-versus-cloud execution (Yao et al., 11 Apr 2025). Others cast routing as conditional selection over a generalized candidate space $$\mathcal{C} \in \{\mathcal{T}, \mathcal{A}, \ldots\}$$, so that tools and agents become instances of the same dispatch problem (Yao et al., 13 Jan 2026).

A recurring misconception is that Agent-as-a-Router is simply classical mixture-of-experts under another name. The papers do not support that simplification. Some systems use hard selection among external experts rather than differentiable blending; the Adaptive Scheduling Agent selects one active scheduler at a time via sched_ext, and its “experts” are whole scheduling policies rather than neural submodules (Wang et al., 7 Nov 2025). Other systems use soft routing distributions over external agents and then aggregate outputs by weighted voting, which is closer to instance-wise collaboration than to internal MoE (Zhang et al., 6 Oct 2025). Still others route memory rather than experts, as in role-aware context routing for multi-agent LLM systems (Liu et al., 6 Aug 2025). The term therefore names a design pattern, not a single algorithmic family.

2. Routed objects and architectural variants

Recent papers instantiate the pattern over different routed entities and execution layers.

Setting	Routed object	Output form
OS scheduling	Scheduler portfolio	Hard scheduler switch
Multi-agent QA	QA agents	Soft routing distribution
Agent Web / MCP	Tools or agents	Next-candidate selection
Web or coding inference	Model/backend	Model or tier choice
Multi-agent memory routing	Shared-memory subset	Agent-specific context

In operating-system scheduling, the router observes runtime signals and routes execution to one expert scheduler from a portfolio containing scx_p2dq, scx_bpfland, scx_nest, scx_lavd, scx_simple, scx_flash, scx_rusty, and EEVDF; the representation is a workload-pattern estimate, and the final decision is translated through a machine-specific mapping table into a scheduler switch (Wang et al., 7 Nov 2025). In collaborative QA, the router may instead learn a graph-conditioned score $s(q,a)$, normalize it into $p_\theta(a \mid q,\mathcal{G})$, and use the resulting distribution to weight multiple agent outputs rather than selecting a single winner (Zhang et al., 6 Oct 2025). In NG-Router, the same pattern is specialized to nutritional QA by inserting agent nodes directly into a domain knowledge graph and learning query-specific routing distributions over them (Shi et al., 10 Oct 2025).

The routed object can also be a search or workflow policy rather than an expert model. SRSA routes contextual user queries into Direct Search, Parallel Search, or Planning Search, thereby treating strategy selection itself as the primary action (Wang et al., 2024). The Super Agent System similarly places an intent router and planner at the top of a hierarchy that can either route directly to a specialized task agent or automatically generate an agentic workflow, with an additional model router choosing between on-device SLMs and cloud LLMs (Yao et al., 11 Apr 2025). AgentGate further sharpens this structured view by decomposing routing into action decision—single-agent invocation, multi-agent planning, direct response, or safe escalation—and structural grounding into executable outputs such as target agents, arguments, or plans (Cheng et al., 8 Apr 2026).

A further extension is routing over infrastructure itself. The Workload–Router–Pool architecture defines routing as the mapping from workload state to a $(model, pool, configuration)$ tuple, so the router mediates among models, serving pools, and execution policies under latency, cost, energy, and safety constraints (Chen et al., 22 Mar 2026). In this sense, Agent-as-a-Router can operate simultaneously at semantic, orchestration, and systems levels.

3. Signals, representations, and decision variables

The representations used by agent-routers are as heterogeneous as the domains they govern. In systems routing, the observation space can be high-dimensional telemetry. ASA’s routing inputs span CPU, memory, disk, process, scheduling, and network features collected from kernel eBPF programs, procfs, and GNOME Shell, including user/system utilization, hot-spot core ratio, run queue length, lock contention, wakeup latency, bytes sent and received, retransmission rate, and focused-window process statistics (Wang et al., 7 Nov 2025). Its learned representation is a workload class prediction, initially a probability distribution over classes such as CPU-intensive computing, I/O-bound operations, interactive applications, and mixed workloads, which is then temporally aggregated before a discrete scheduler choice is made.

In QA and domain reasoning, the dominant representation is graph-structured. AgentRouter constructs a heterogeneous graph with query nodes, agent nodes, and entity nodes, plus query–entity, entity–entity, agent–entity, and trainable query–agent edges; after type-aware message passing, the router computes query–agent scores and produces a routing distribution over agents (Zhang et al., 6 Oct 2025). EvolveRouter retains this graph-based routing formulation but uses the router’s learned weights as diagnostics for targeted prompt refinement and for adaptive collaboration-size control (Huang et al., 6 Apr 2026). NG-Router extends the same idea to nutritional QA and adds gradient-based subgraph retrieval, where entity salience is defined by the gradient norm of the routing loss with respect to entity embeddings (Shi et al., 10 Oct 2025).

In open-ended ecosystems, the representation shifts from domain graphs to history-conditioned candidate selection. ToolACE-MCP defines routing as modeling $\pi_\theta(c \mid Q, H, \mathcal{C})$, where $Q$ is the current query, $H$ is dialogue history, and $\mathcal{C}$ is the candidate set of tools or agents; the routed entities are represented by structured specifications $\phi(c)$, and the decisive signal is multi-turn trajectory state rather than single-turn lexical matching (Yao et al., 13 Jan 2026). The paper’s history ablation—accuracy dropping from 53% to 48% on MCP-Universe and from 60% to 52% on MCP-Mark when history is removed—makes explicit that routing is “inherently a dynamic, history-dependent reasoning process” in this setting (Yao et al., 13 Jan 2026).

Coding and agentic-programming routers similarly depend on stateful representations. Agent-as-a-Router for coding tasks defines the context at task $t_i$ as $s(q,a)$0, where $s(q,a)$1 is the task prompt, $s(q,a)$2 is metadata, and $s(q,a)$3 is accumulated memory from previous loops; its C-A-F formulation,

$s(q,a)$4

makes the decision variable a model-selection action conditioned on deployment history rather than a one-off label (Zhou et al., 22 Jun 2026). TwinRouterBench formalizes the same shift at the benchmark level by defining the routable unit as each step’s “router-visible prefix” $s(q,a)$5, not the user request as a whole (Yang et al., 14 May 2026). This moves model routing from prompt classification to sequential control over an evolving prefix of system messages, tool outputs, logs, retrieval snippets, and partial code edits.

4. Learning, supervision, and adaptation

A major fault line in the literature is whether routing is decoupled from experts or trained jointly with them. Many practical systems are explicitly decoupled. ASA trains a hardware-agnostic workload recognizer offline, then uses a machine-specific mapping table from recognized workload class to optimal scheduler; the schedulers themselves are pre-existing expert policies and are not learned jointly with the router (Wang et al., 7 Nov 2025). WebRouter likewise learns a query-specific distribution over candidate LLM backbones using a cost-aware variational information bottleneck objective rather than end-to-end optimization of the downstream web agent (Li et al., 13 Oct 2025). In the Super Agent System, the intent router is described as an LLM-based controller using function calling, while the model router and edge-cloud router remain qualitative policy layers rather than jointly trained components (Yao et al., 11 Apr 2025).

Other systems learn routing from empirical downstream performance. AgentRouter and NG-Router run all candidate agents on each training query, score outputs against gold answers using F1, convert those scores into temperature-scaled soft target distributions $s(q,a)$6, and train the router with KL divergence to align predicted routing distributions with these empirical targets (Zhang et al., 6 Oct 2025, Shi et al., 10 Oct 2025). This makes routing supervision instance-specific and non-binary: several agents may receive nontrivial target mass if they perform similarly well. EvolveRouter retains this soft supervision but changes the target distribution over time by improving the prompts of selected agents, so that router training and agent quality co-evolve in alternating rounds (Huang et al., 6 Apr 2026).

Open-world tool and agent routing often relies on synthetic or self-generated supervision. ToolACE-MCP constructs a dependency-rich candidate graph over an initial bank of 627 MCP tools, expands it to 2,005 tools via mutation, synthesizes multi-turn trajectories with role-based LLM simulation, and extracts routing labels from each candidate invocation step (Yao et al., 13 Jan 2026). STRMAC generates successful multi-agent execution paths through solution-aware pruning and router-guided search, then trains a lightweight router encoder with a contrastive objective so that the state embedding aligns with the best next-agent embedding (Wang et al., 4 Nov 2025). TCAndon-Router instead treats routing as structured generation: the router emits a natural-language reasoning chain $s(q,a)$7 and a candidate-agent subset $s(q,a)$8, is first trained by supervised fine-tuning, and is then optimized with DAPO using a reward that balances precision-like correctness, coverage, and penalties for over-predicting too many agents (Zhao et al., 8 Jan 2026).

Adaptation during deployment is another defining feature of the pattern. Agent-as-a-Router for coding tasks identifies information deficit as the bottleneck and addresses it by storing execution-grounded outcomes—observed score, cost, and verification traces—in a memory module that conditions future routing decisions (Zhou et al., 22 Jun 2026). FlyRoute adapts not the router parameters but the descriptions of the agents themselves: successful query-agent pairs are stored in per-agent success stores, periodically distilled into learned capability descriptions, and then used together with BM25-retrieved exemplars to strengthen future routing (Li et al., 21 May 2026). The Workload–Router–Pool vision paper generalizes this further, proposing bandit adaptation, RL-based model selection, user-feedback-driven routing adaptation, and closed-loop self-adaptation as mature or near-term routing regimes (Chen et al., 22 Mar 2026).

5. Inference patterns, collaboration regimes, and evaluation

Agent-as-a-Router systems differ sharply in how routing decisions are executed. Some perform hard routing to one expert at a time. ASA’s runtime pipeline is observations $s(q,a)$9 class-probability estimates $p_\theta(a \mid q,\mathcal{G})$0 temporally smoothed confidence $p_\theta(a \mid q,\mathcal{G})$1 discrete class decision $p_\theta(a \mid q,\mathcal{G})$2 scheduler activation; its Time-Weighted Probability Voting mechanism suppresses low-confidence predictions, applies exponential recency weighting, and uses a sliding window with $p_\theta(a \mid q,\mathcal{G})$3 to prevent route flapping (Wang et al., 7 Nov 2025). ToolACE-MCP’s Light Routing Agent similarly chooses one next candidate at each step and then dispatches it, repeating this loop as history evolves (Yao et al., 13 Jan 2026). STRMAC is also single-next-agent routing: at each step it selects exactly one next agent from a pool based on state–expert similarity (Wang et al., 4 Nov 2025).

Others perform soft or subset routing. AgentRouter produces a full distribution $p_\theta(a \mid q,\mathcal{G})$4 over 24 candidate agents and aggregates answer strings by weighted voting; optional top-$p_\theta(a \mid q,\mathcal{G})$5 clipping improves performance by suppressing the long tail of weak agents (Zhang et al., 6 Oct 2025). EvolveRouter adds adaptive collaboration size through router-weighted answer agreement, querying agents in router-ranked order and stopping once weighted agreement exceeds threshold $p_\theta(a \mid q,\mathcal{G})$6 subject to $p_\theta(a \mid q,\mathcal{G})$7 and $p_\theta(a \mid q,\mathcal{G})$8 (Huang et al., 6 Apr 2026). TCAndon-Router performs subset prediction rather than top-1 dispatch, invokes all selected expert agents in parallel, and passes their outputs to a dedicated Refining Agent for synthesis (Zhao et al., 8 Jan 2026). RCR-Router does not route among agents at all; instead, it routes per-agent context slices $p_\theta(a \mid q,\mathcal{G})$9 under token budgets, recomputing those slices each round as the shared memory store evolves (Liu et al., 6 Aug 2025).

The evaluation literature reflects these differences. For OS scheduling, ASA reports that it outperforms the default Linux scheduler EEVDF in 86.4% of scenario–machine runs, with a global average improvement of $(model, pool, configuration)$0 and 95% CI $(model, pool, configuration)$1, and that its selected scheduler is among the top three in 78.9% of cases (Wang et al., 7 Nov 2025). For QA routing, AgentRouter reaches 70.54 F1 / 57.33 EM on HotpotQA and 63.36 F1 / 51.00 EM on TriviaQA, while EvolveRouter pushes these to 72.28 / 60.33 and 69.83 / 60.00 respectively through co-evolution and adaptive collaboration (Zhang et al., 6 Oct 2025, Huang et al., 6 Apr 2026). For open agent ecosystems, ToolACE-MCP reports 53.44 on MCP-Universe, 60.00 on MCP-Mark, strong robustness under noise and scale perturbations, and 91.6% accuracy on the Agent Route Benchmark without additional fine-tuning (Yao et al., 13 Jan 2026).

Model-routing evaluations have increasingly shifted from one-shot classification to sequential or cost-aware settings. WebRouter reports 82.3% average accuracy at \$(model, pool, configuration)$$20.98, corresponding to an 87.8$$(model, pool, configuration)$354.73 for unrouted Opus 4.6 (Yang et al., 14 May 2026). These results collectively suggest that routing quality is increasingly measured not only by local match accuracy but by end-to-end task success under realistic execution constraints.

6. Misconceptions, limitations, and open questions

A common overstatement is that routing simply solves heterogeneity once a sufficiently strong router is trained. The surveyed papers are more cautious. Many routers are only as good as their candidate pools. ASA cannot produce a scheduler better than the available portfolio (Wang et al., 7 Nov 2025). AgentRouter assumes a fixed manually designed pool of 24 agents and does not handle open-ended agent generation (Zhang et al., 6 Oct 2025). FlyRoute shows that even when the router backbone is held fixed, stale or weak capability descriptions can dominate routing quality until enough deployment evidence accumulates to rewrite them (Li et al., 21 May 2026). This suggests that agent quality, profile quality, and router quality are tightly coupled, not separable.

Another misconception is that routing is universally best handled by a single powerful LLM. Several papers argue against that view. AgentGate shows that structured routing quality depends less on generic free-form generation and more on constrained action prediction, candidate awareness, and grounding into executable outputs; compact 3B–7B models can be competitive when the problem is formulated as structured dispatch rather than unrestricted text generation (Cheng et al., 8 Apr 2026). The coding paper makes the complementary claim that the limiting factor is information deficit, not reasoning deficit: simply adding per-dimension performance statistics yields a 15.3% relative gain for a vanilla LLM router, and the full C-A-F loop gains further by accumulating verified experience during deployment (Zhou et al., 22 Jun 2026). These findings suggest that architecture and state are often more important than raw model scale.

Open problems are equally consistent across the literature. Several systems rely on scenario-driven labels or curated graphs and therefore leave open-world generalization unresolved. ASA’s workload labels are effectively tied to predefined benchmark scenarios (Wang et al., 7 Nov 2025). AgentRouter and NG-Router depend on graph extraction quality and on supervision from running all agents on each training example (Zhang et al., 6 Oct 2025, Shi et al., 10 Oct 2025). ToolACE-MCP relies heavily on synthetic, environment-free trajectories; while this enables scale, it leaves open how faithfully the simulations capture real API failures, latency, and side effects (Yao et al., 13 Jan 2026). The Super Agent System paper is explicit that its contribution is a blueprint rather than an empirically validated algorithmic specification (Yao et al., 11 Apr 2025).

The hardest unresolved questions concern closed-loop deployment. TwinRouterBench argues that one-shot prompt routing benchmarks are insufficient because they ignore how a cheap decision at step $(model, pool, configuration)$4 can poison later states, yet its own static labels still target the cheapest sufficient tier per step rather than the globally optimal joint policy (Yang et al., 14 May 2026). The Workload–Router–Pool paper pushes this further by identifying sparse or open interaction cells—especially Router $(model, pool, configuration)$5 Pool and closed-loop three-way co-adaptation—as major research directions, including pool-state inference, output-length-aware pool routing, energy as a routing objective, and governance-as-code for safety and authorization (Chen et al., 22 Mar 2026). Taken together, these papers suggest that the mature form of Agent-as-a-Router is neither a pure classifier nor a pure planner, but a constrained, stateful control layer whose competence depends on candidate quality, memory, feedback, and execution-aware evaluation.