Budget-Aware Agentic Routing

• The paper introduces a framework that balances computational cost and quality by dynamically routing tasks based on both hard and soft budget constraints.
• It advocates for using cheaper models for low-stakes components and expensive models for high-complexity steps to optimize overall performance.
• The approach emphasizes transparent state representations and audit logs to guide model selection and facilitate continuous, feedback-driven improvements.

Searching arXiv for papers on budget-aware agentic routing and closely related routing frameworks. Budget-aware agentic routing denotes a class of control policies for agentic systems in which model selection, tool use, memory access, search depth, or orchestration topology is chosen under explicit resource limits rather than by a fixed “best model” heuristic. In the cited work, the problem arises because modern agents are not single-turn predictors: they execute sequences of subtasks, accumulate state, observe intermediate failures, and must remain within budgets on money, time, probes, or other resources. The resulting objective is typically not raw accuracy alone, but a cost–quality or cost–success frontier, and in several formulations the budget or deadline becomes part of the success criterion itself (Okamoto et al., 4 Apr 2026, Liu et al., 12 Feb 2026, Wang et al., 7 May 2026).

1. Problem setting and scope

A common premise across the literature is that “best model” is not a global answer once workflows decompose into heterogeneous subtasks. Topaz states the point directly: a cheap model may be sufficient for low-stakes subtasks, while a more capable model is justified for high-complexity, high-quality-sensitivity steps. Its contribution is to make those trade-offs visible and auditable rather than leaving them as silent model assignments (Okamoto et al., 4 Apr 2026).

Several papers sharpen this into a sequential control problem. BAAR characterizes agentic routing as path-dependent: early mistakes compound, rewards are often terminal, and deployments may require strict per-task spending limits (Zhang et al., 4 Feb 2026). Harness-Native agentic routing makes the same distinction against single-turn routing, arguing that in an execution harness the relevant state includes observation, compressed and raw context, tool and artifact history, recovery status, action space, and verifier signals, so the best model is a function of the current step and harness state rather than only of the original query (Liu et al., 13 Jul 2026). MTRouter similarly treats long-horizon routing as a turn-level choice under a fixed episode budget, with model invocations accumulating cost over the full episode (Zhang et al., 26 Apr 2026).

“Budget” is correspondingly heterogeneous. In INTENT, each instance is $I=(q,B,M)$, where $B$ is a hard monetary budget and $M$ is a market snapshot of tools and prices (Liu et al., 12 Feb 2026). In MCPP, the instance is $I=(G,M,\Phi,B,D)$, with a workflow DAG, total budget $B$, and deadline $D$; success is completion within both constraints (Wang et al., 7 May 2026). ShardMemo uses a probe budget $B_{\mathrm{probe}}$ for shard selection and a separate budget $R$ for skill retrieval (Zhao et al., 29 Jan 2026). BCAS enforces max_searches and max_total_tokens as hard search and token caps inside an agentic RAG loop (McCleary et al., 9 Mar 2026). The certified search framework of “Auditable Early Stopping for Agentic Routing” further layers price, latency, and $(\epsilon,\delta)$-aware control on top of run-wise search certificates (Akhauri, 9 Sep 2025).

A plausible implication is that budget-aware agentic routing is less a single algorithmic family than a shared systems problem: how to allocate scarce compute across branching, stateful execution so that expensive capability is used where it changes the trajectory rather than where it is merely available.

2. Optimization criteria and formal objectives

Two objective patterns recur. The first is a soft cost-penalized objective. BAAR writes this as

$$J_{\text{soft}}(\theta)=\mathbb{E}_{\tau\sim\pi_\theta}\!\left[\mathbb{I}(\mathrm{success}(\tau))-\lambda\sum_{t=0}^{|\tau|-1} c(a_t)\right],$$

which traces a cost–success frontier by varying $B$0 (Zhang et al., 4 Feb 2026). The theorem-proving router uses the same scalarization at the decision level,

$B$1

continuing a proof trajectory only when predicted marginal success exceeds cost penalty (Rögnvaldsson et al., 3 Jun 2026). Harness-Native routing likewise uses the scalarized trajectory objective $B$2, with realized cost accumulated over all selected models along the trajectory (Liu et al., 13 Jul 2026).

The second pattern is a hard feasibility constraint. BAAR formulates

$B$3

and enforces the constraint at inference with budget-constrained decoding that forces the cheap model when the expensive one would exceed remaining budget (Zhang et al., 4 Feb 2026). INTENT makes feasibility even more explicit by defining reward as

$B$4

so a trajectory that exceeds budget receives zero reward even if the answer is otherwise correct (Liu et al., 12 Feb 2026). MCPP replaces generic reward maximization with constrained completion probability,

$B$5

which changes the goal from average efficiency to maximizing the probability that the entire workflow completes successfully within specified budget and deadline constraints (Wang et al., 7 May 2026).

Topaz occupies both regimes. Its objective-based router balances quality and cost continuously, while its budget-based router maximizes total quality subject to a hard spend limit. The paper emphasizes the distinction: one router is for planning under strict budgets, the other for general heuristic optimization (Okamoto et al., 4 Apr 2026). A similar duality appears in ShardMemo, where Top-$B$6 gives a hard cap while Adaptive Top-$B$7 spends more probes only when the query is uncertain, still respecting the cap (Zhao et al., 29 Jan 2026).

These formulations agree on one point: local cheapness is not enough. INTENT models future retry cost through

$B$8

and accepts a tool action only when the immediate call is affordable and the risk-adjusted projected future plan remains within remaining budget (Liu et al., 12 Feb 2026). MCPP models repeated parallel attempts through

$B$9

so allocating more width now increases immediate success probability but consumes budget and time needed by downstream nodes (Wang et al., 7 May 2026). This suggests that budget-aware routing is fundamentally prospective: it prices not only the current action, but the downstream states that action makes feasible or infeasible.

3. Information signals and state representations

The routing signal is rarely just prompt text. Topaz builds a shared skill space $M$0, profiles public benchmarks into benchmark-to-skill weights, and synthesizes per-skill capability profiles from normalized leaderboard scores. Each workflow subtask is separately profiled into required skill weights $M$1, complexity $M$2, estimated input/output tokens, and quality sensitivity $M$3. The paper also reports a capability calibration factor $M$4 to avoid saturated satisfaction ratios, so routing remains sensitive to skill mismatch rather than collapsing into uniformly satisfied tasks (Okamoto et al., 4 Apr 2026).

Other work replaces skill profiling with trajectory evidence. The theorem-proving router takes as state the failed-attempt history for a theorem or lemma target and learns $M$5 with logistic regression from proof similarity, error diversity, and attempt count encoded as $M$6; cost $M$7 is estimated by average prior output-token count (Rögnvaldsson et al., 3 Jun 2026). SWE-Router uses the partial weak-model trajectory

$M$8

and learns a value head $M$9 to predict whether the cheap model is likely to solve the task after a few exploratory turns (Son et al., 30 Jun 2026). MTRouter serializes history as

$I=(G,M,\Phi,B,D)$0

encodes it with a frozen text encoder, and concatenates the history embedding with a model embedding built from metadata such as context length, knowledge cutoff, pricing, and a learned residual; the outcome estimator then predicts the expected final episode outcome for each history–model pair (Zhang et al., 26 Apr 2026).

Several systems enrich the state with structural or scoped metadata. CASTER uses a dual-signal router with semantic embeddings plus a 6-dimensional structural vector consisting of a 4-dimensional one-hot agent-role vector, context length, and a high-risk keyword indicator; implementation dimensions are $I=(G,M,\Phi,B,D)$1, $I=(G,M,\Phi,B,D)$2, $I=(G,M,\Phi,B,D)$3, $I=(G,M,\Phi,B,D)$4, and $I=(G,M,\Phi,B,D)$5 (Liu et al., 27 Jan 2026). RGAO extracts a 5-dimensional structural complexity vector

$I=(G,M,\Phi,B,D)$6

from a retrieved code subgraph and routes among FastPath, SubAgent, MultiAgent, and DeepResearch based on thresholded repository structure rather than query text alone (Talluri et al., 7 May 2026). ShardMemo computes an eligible shard set $I=(G,M,\Phi,B,D)$7 before any semantic routing, then scores only eligible shards using query embeddings, structured query features, and shard summaries (Zhao et al., 29 Jan 2026). BudgetMem routes at the module level using the current query, the current module input or intermediate state, and a module descriptor, which makes routing a sequence of local tier choices rather than a single global decision (Zhang et al., 5 Feb 2026).

A recurring claim is that prompt-only routing suffers an information deficit. ACRouter reports that simply augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain over Vanilla, which the paper interprets as evidence that better information matters at least as much as more elaborate reasoning (Zhou et al., 22 Jun 2026).

4. Routing mechanisms and architectural patterns

The simplest routing action is binary model choice. CASTER computes $I=(G,M,\Phi,B,D)$8 and compares it to a threshold $I=(G,M,\Phi,B,D)$9, dispatching the strong model when $B$0 and the weak model otherwise (Liu et al., 27 Jan 2026). BAAR emits one of two decision tokens, SMALL or LARGE, and under strict budgets replaces the learned choice with the cheap model when the expensive call would violate remaining budget (Zhang et al., 4 Feb 2026). Switchcraft takes a different route: it predicts, for each candidate model, whether that model will answer a function-calling request correctly, then selects the cheapest model predicted to be correct; if none exceed threshold, it falls back to the argmax model (Agarwal et al., 8 May 2026). MTRouter scores all candidate models with $B$1 and greedily selects

$B$2

subject to the episode budget and horizon (Zhang et al., 26 Apr 2026).

A second pattern is continue-versus-restart or cheap-explore-then-escalate. The theorem-proving agent always samples two proof attempts before its first routing decision, then either Attempts again on the current target or Terminates the trajectory and restarts from a new decomposition depending on the sign of $B$3 (Rögnvaldsson et al., 3 Jun 2026). SWE-Router deliberately spends $B$4 cheap exploratory turns on file discovery, localization, and error inspection, then continues with the weak model if a value head exceeds threshold and otherwise escalates to the strong model, which restarts from the original prompt (Son et al., 30 Jun 2026). BCAS implements a related mechanism on the tool side: once the search allowance is exhausted, search_database is removed from the available action list and the agent must answer from accumulated evidence (McCleary et al., 9 Mar 2026).

A third pattern generalizes routing beyond model calls. ShardMemo treats shard probing as masked mixture-of-experts routing over eligible shards, with optional cost bias $B$5, Top-$B$6 or Adaptive Top-$B$7 selection, and evidence-to-shard supervision (Zhao et al., 29 Jan 2026). BudgetMem routes among $B$8 tiers for each memory module using a PPO-trained policy and can realize tiers through implementation, reasoning, or capacity changes (Zhang et al., 5 Feb 2026). GraphPlanner makes the action a pair $B$9, jointly choosing an agent role—Planner, Executor, or Summarizer—and the backbone LLM, with a dynamic action mask to disallow semantically invalid steps (Feng et al., 26 Apr 2026). RGAO routes not among models but among orchestration topologies, and static budget verification determines whether the chosen delegation tree is admissible before any LLM call (Talluri et al., 7 May 2026). MCPP chooses, for every ready workflow node, both a model $D$0 and a parallel width $D$1, executing simultaneously executable subtasks while preserving remaining budget and time (Wang et al., 7 May 2026).

Two papers make the routing action explicitly adaptive over deployment. EvoRoute retrieves analogous step-level records from a self-evolving experience base, filters to Pareto-optimal candidate models over performance, cost, and delay, and then uses Thompson sampling to preserve exploration among non-dominated choices (Zhang et al., 6 Jan 2026). Agent-as-a-Router formalizes the same logic as a Context $D$2 Action $D$3 Feedback $D$4 Context loop, in which the selected model, verified score, and monetary cost become part of the next routing context (Zhou et al., 22 Jun 2026).

5. Auditability, feedback loops, and learned control

Topaz is the clearest statement of auditable routing. It records a structured explanation log containing user configuration, intermediate calculations such as skill-match scores and cost penalties for each model–task pair, and final assignments with objective scores. Developer-facing explanations are then generated from the log by an LLM prompt that explicitly avoids formulas, variables, and raw numbers, and the paper emphasizes that the explanations are faithful because they derive from the actual numerical routing trace rather than from post-hoc rationalization (Okamoto et al., 4 Apr 2026).

“Auditable Early Stopping for Agentic Routing” extends auditability from explanations to run-wise certificates. In Exact mode the halting rule is based on

$D$5

where $D$6 is the minimum exponential-race arrival time among leaves under node $D$7; in Surrogate mode the router uses upper bounds $D$8, conservative provisional keys, and validator-side tightening

$D$9

The router logs NDJSON ledgers with deterministic replay domains, and the validator recomputes keys, $B_{\mathrm{probe}}$0, stop conditions, and downgrades to NoCert when certification assumptions fail (Akhauri, 9 Sep 2025).

A different lineage treats routing logs as training data. Harness-Native routing defines a typed arena record containing the query, harness state, candidate slate, routing action, router-side scores, execution trace, verification events, outcomes, cost, duration, and provenance. The paper’s “harness-native data flywheel” claim is that environment-supplied labels—success, failure, recovery path, latency, and cost—can train progressively stronger routers and harness-native models (Liu et al., 13 Jul 2026). ACRouter operationalizes this at smaller scale with an Orchestrator, a Verifier that aggregates execution-based signals into a unified score, and an online vector-store Memory bounded at 20K entries, retrieved by cosine kNN over top-10 neighbors (Zhou et al., 22 Jun 2026). EvoRoute stores per-step records $B_{\mathrm{probe}}$1 and continuously expands its knowledge base $B_{\mathrm{probe}}$2 with actual executions (Zhang et al., 6 Jan 2026). GraphPlanner uses GARNet to combine workflow memory and historical memory in a heterogeneous graph, so current routing decisions can exploit previous query–role–response interactions (Feng et al., 26 Apr 2026).

Learning under sparse rewards has generated its own budget-aware techniques. BAAR’s Boundary-Guided Training anchors learning to two boundary policies, always-small and always-large, uses stratified sampling to synthesize lower-cost successful hard-task trajectories for SFT, and then applies Boundary-Guided Policy Optimization with task-relative normalized cost and a reference-guided advantage to avoid degenerate cheap-failure solutions (Zhang et al., 4 Feb 2026). This suggests that in agentic routing, data engineering and objective shaping are often as important as the router architecture itself.

6. Reported empirical effects

The empirical literature is heterogeneous in benchmark, budget type, and reported metric, but a common pattern is frontier movement rather than raw dominance by one fixed model or one fixed architecture. Reported gains range from lower monetary cost at matched success, to lower latency at matched accuracy, to higher constrained completion probability under explicit budget–deadline caps.

System	Setting	Reported outcome
“Optimizing the Cost-Quality Tradeoff of Agentic Theorem Provers in Lean” (Rögnvaldsson et al., 3 Jun 2026)	PutnamBench subset	25.8% average cost reduction at parity accuracy; 7.8% accuracy improvement at parity cost
“ShardMemo” (Zhao et al., 29 Jan 2026)	LoCoMo, fixed $B_{\mathrm{probe}}$3	+6.87 F1 over cosine-to-prototype routing; VecScan $B_{\mathrm{probe}}$4; p95 latency $B_{\mathrm{probe}}$5 ms
“CASTER” (Liu et al., 27 Jan 2026)	Graph-based MAS	up to 72.4% inference cost reduction while matching success rates
“MTRouter” (Zhang et al., 26 Apr 2026)	ScienceWorld / HLE	58.7% lower total cost than GPT-5 on ScienceWorld; 43.4% lower total cost than GPT-5 on HLE
“Switchcraft” (Agarwal et al., 8 May 2026)	Five function-calling benchmarks	82.94% accuracy; 84% cost reduction; over $3,600 per million queries saved
“RGAO” (Talluri et al., 7 May 2026)	Multi-agent code generation	proxy-measured misrouting reduced from 30.1% to 8.2%
“Agentic Routing: The Harness-Native Data Flywheel” (Liu et al., 13 Jul 2026)	PinchBench singleton routing	93.14 vs 93.35 for fixed Opus/OpenClaw, with cost reduced from $B_{\mathrm{probe}}$60.0204
“EvoRoute” (Zhang et al., 6 Jan 2026)	GAIA / BrowseComp+ and other benchmarks	up to 80% cost reduction and over 70% latency reduction
“GraphPlanner” (Feng et al., 26 Apr 2026)	14 diverse LLM tasks	accuracy improved by up to 9.3% while GPU cost reduced from 186.26 GiB to 1.04 GiB

Several qualitative findings are equally important. Topaz reports that as global cost sensitivity rises, high-stakes tasks such as Technical Diagnosis remain assigned to stronger models even under substantial cost pressure, while lower-sensitivity subtasks move to cheaper models (Okamoto et al., 4 Apr 2026). BCAS finds that accuracy improves reliably up to about three searches, that hybrid retrieval with re-ranking gives the largest average gains in its HotpotQA ablation grid, and that larger completion budgets matter most on HotpotQA-style synthesis (McCleary et al., 9 Mar 2026). BudgetMem reports that implementation and capacity tiering span a broader cost range, whereas reasoning tiering is a finer-grained knob with a narrower spread; capacity tiering is strongest in high-budget, performance-first settings (Zhang et al., 5 Feb 2026). INTENT reports perfect feasible rate, $B_{\mathrm{probe}}$7, on both evaluated backbones while improving pass-rate-related metrics under dynamic market shifts (Liu et al., 12 Feb 2026).

The literature also repeatedly rejects a simple “bigger is better” picture. Switchcraft reports that larger or more expensive models do not consistently do better on tool use, and that nominally cheaper models can incur higher total cost due to token-intensive reasoning (Agarwal et al., 8 May 2026). Harness-Native routing argues similarly that a locally cheap model can be expensive at the trajectory level if it triggers retries, repair, or verification failure (Liu et al., 13 Jul 2026). This suggests that the core empirical advantage of budget-aware routing is not merely cheapness, but selective expenditure at high-leverage states.

7. Limitations, controversies, and open directions

The first limitation is scope of transparency. Topaz explicitly notes that it explains which model was selected and why, but not the downstream behavior of that model’s output; routing transparency is only one layer of end-to-end agent auditing (Okamoto et al., 4 Apr 2026). The certificate-based search framework likewise guarantees stopping soundness only under its structural assumptions—local finiteness, child-partition structure, and admissible race-independent bounds—and otherwise downgrades to NoCert, where stopping is heuristic rather than certified (Akhauri, 9 Sep 2025).

The second limitation is dependence on supervision or calibration. ShardMemo reports that an untrained masked MoE router performs much worse, which the paper interprets as evidence that evidence-to-shard supervision is essential (Zhao et al., 29 Jan 2026). MCPP assumes estimated single-attempt success probabilities $B_{\mathrm{probe}}$8 and expected generation lengths $B_{\mathrm{probe}}$9, explicitly treating their estimation as separate from the routing problem; robustness experiments show larger degradation under success-rate noise than under token-length noise (Wang et al., 7 May 2026). Switchcraft is trained and evaluated in-distribution on a fixed eight-model pool and notes that retraining is required when the pool changes (Agarwal et al., 8 May 2026). SWE-Router states that collecting and training on trajectory data is more expensive than prompt-only routing data, and its current exploration budget $R$0 is fixed rather than adaptive (Son et al., 30 Jun 2026).

A third limitation concerns transfer to strict constraints and safety. BAAR reports that soft-budget training generalizes only partially to hard inference-time caps: small $R$1 policies can exhaust budget too early, while large $R$2 policies can become too conservative. The paper explicitly identifies conditioning on remaining budget during training as the next step for production systems (Zhang et al., 4 Feb 2026). SWE-Router raises a separate safety concern: trajectory-based routing could be used to bypass stronger, safer models for malicious tasks, so future routers may need safety-aware objectives in addition to cost and performance (Son et al., 30 Jun 2026). Agent-as-a-Router further shows that static routers can collapse on out-of-distribution agentic-programming tasks, whereas online adaptive methods preserve lower cumulative regret, indicating that deployment-time feedback remains central under shift (Zhou et al., 22 Jun 2026).

Across these works, a stable research direction is visible. Budget-aware agentic routing increasingly combines three requirements that were often separate in earlier routing systems: explicit constraints, trajectory-aware decision state, and inspectable or learnable feedback loops. This suggests that future systems will continue to move away from prompt-only one-shot dispatch toward routers that plan, verify, adapt, and justify their allocations over the full agentic execution trace.