Confidence-Aware Routing System

• Confidence-aware routing is a system that bases routing decisions on estimated reliability, uncertainty, or trust rather than static labels.
• It employs various techniques such as empirical self-performance, semantic uncertainty, and signal aggregation to choose between local and escalated processing.
• Applications include LLM orchestration, cognitive routing in networks, and multimodal systems, yielding improved efficiency and reduced latency.

Searching arXiv for papers on confidence-aware routing and related routing frameworks. arxiv_search(query="confidence-aware routing system LLMs routing cascading SATER confidence-aware rejection", max_results=5) arxiv_search(query="confidence-aware routing system LLMs routing cascading SATER", max_results=10) A confidence-aware routing system is a routing architecture in which the routing decision is conditioned on an explicit estimate of confidence, uncertainty, reliability, or trust, rather than only on static task labels, destination identifiers, or purely local heuristics. Across the literature, the routed object varies—from a user query, reasoning step, and tool invocation to a packet flow, route request, or traveler demand—and the action space varies accordingly, including local inference, escalation to a larger model, retrieval augmentation, refusal, alternate network-path selection, and route verification. What unifies these systems is the use of a confidence-bearing control signal to manage the trade-off among quality, cost, latency, safety, or congestion (Shen et al., 4 Oct 2025, Zhang et al., 16 Feb 2025, Nair et al., 2010).

1. Conceptual scope and formalization

In contemporary LLM routing, the canonical formulation is thresholded deferral. SATER studies the decision of whether query $i$ should be answered by a small LLM $M_s$ or routed to a larger model $M_l$, with routing function

$$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$

where $s_i$ is the SLM’s confidence score and $\tau \in [0,1]$ is the routing threshold. In that formulation, higher thresholds send more queries to the LLM, improving quality while increasing cost and latency. The paper explicitly argues that routing decisions based only on task or domain classification miss a central variable—difficulty—and that the relevant question is whether the SLM can solve the instance with high confidence and short outputs (Shen et al., 4 Oct 2025).

Related formulations appear in other domains with different routed entities. In the MANET setting, EMP computes a link duration estimate $LDT_{(i,j)}$ and discards a route request when $LDT_{(i,j)} < \varepsilon$, where $\varepsilon$ is a confidence level derived from Kalman-filter uncertainty; here routing is confidence-aware because link viability must be sufficiently certain, not merely positive (Vu et al., 2016). In pre-generation hallucination mitigation, the routing module maps a scalar confidence score to one of four pathways—local generation, retrieval-augmented generation, larger model, or human review—so the routing action itself becomes multi-valued rather than binary (M, 23 Sep 2025). In stochastic transport routing, the output is not only a shortest path but also a confidence interval for its cost, making uncertainty quantification part of the routing product rather than only an internal control variable (Chrétien et al., 2018).

These formulations suggest a broad definition: a confidence-aware routing system is one in which route selection is a function of estimated reliability of future performance. The estimate may target answer correctness, semantic stability, downstream channel health, mobility prediction error, or user compliance. The route may be a model choice, a path through a communication network, or a recommendation to a traveler. The common design move is to replace blind commitment with contingent execution.

2. Confidence signals and calibration mechanisms

The literature does not treat confidence as a single primitive. Instead, confidence is operationalized through several distinct estimator families. One family uses empirical self-performance. In SATER, Stage II resamples the model 10 times per question, assigns an empirical accuracy score on a 0 to 1 scale, and defines 10 discrete confidence thresholds from 0.1 to 1.0; the model is then trained to answer only when its estimated accuracy exceeds the prompted threshold, otherwise emitting a refusal template (Shen et al., 4 Oct 2025).

A second family uses semantic uncertainty. The Confidence-Driven LLM Router defines semantic entropy over meaning clusters,

$$SE(\mathbf{x}) = -\frac{1}{|C|} \sum_{i=1}^{|C|} \log p(C_i \mid \mathbf{x}),$$

after clustering sampled outputs via bidirectional entailment. Lower semantic entropy indicates fewer semantically distinct plausible answers and thus higher confidence. This framework uses semantic entropy not only as an analysis tool but as the source of preference labels for training lightweight routers (Zhang et al., 16 Feb 2025).

A third family uses comparative separation rather than calibrated probability. CARGO defines confidence as the gap between the top two predicted model scores,

$M_s$0

and invokes a second-stage binary classifier only when the gap is small. In that system, uncertainty is not estimated from softmax entropy or posterior calibration, but from the separability of the top-ranked and runner-up experts under an embedding-based regressor (Barrak et al., 18 Sep 2025).

A fourth family aggregates heterogeneous internal signals. The pre-generation hallucination-mitigation framework combines semantic alignment, layerwise convergence, and a learned confidence estimator into a unified score,

$M_s$1

with validation-tuned weights. Higher confidence routes to local generation; lower confidence triggers progressively more expensive or conservative pathways (M, 23 Sep 2025). Robust-TO similarly calibrates tool confidence by multiplying a tool’s intrinsic score by the reliability of the input frames, $M_s$2, thereby making confidence depend jointly on model self-assessment and input trustworthiness (He et al., 25 Jun 2026).

A fifth family derives confidence directly from model internals at generation time. OI-MAS defines raw confidence as average token log-probability over a generated sequence and then normalizes it per backbone model to make scores comparable across heterogeneous models (Wang et al., 8 Jan 2026). STEER uses the small model’s maximum token logit as token-level confidence, aggregates it to the reasoning-step level, and fits a two-component Gaussian mixture distinguishing “confident” and “unconfident” steps (Lee et al., 9 Nov 2025). In sexism detection, the routing score is simply the classifier’s maximum posterior probability, optionally augmented by the top-two margin in multi-class settings; temperature scaling is applied before threshold selection so that trust-vs-escalate decisions rely on calibrated probabilities rather than raw logits (Alajmi et al., 21 Dec 2025).

Beyond LLMs, confidence can denote route compliance or network health. TASR treats trust $M_s$3 as the probability that a traveler group accepts a routing recommendation, turning confidence into a probabilistic compliance parameter (Brown et al., 2024). In the HMM-based cognitive routing architecture, the Forward Channel Performance Index is the operative confidence signal: an 8-bit summary of downstream channel health derived from bandwidth, delay, jitter, and loss, then predicted forward in time by a first-order HMM (Nair et al., 2010).

3. Routing granularities and control architectures

Confidence-aware routing systems differ sharply in when they make the routing decision. Query-level routing decides before or immediately after a full candidate response. SATER distinguishes pre-generation routing from cascade routing. Pre-generation routing decides before any SLM answer is generated, which is cheaper and lower-latency but depends on predicting difficulty from the input alone. Cascade routing lets the SLM answer first and then decides whether to accept it or fall back to the LLM, which is typically more accurate and robust but can suffer from “latency polarization” when long, low-quality outputs are rejected after generation (Shen et al., 4 Oct 2025).

Other systems refine this timing further. PRISM separates offline strategy planning from online targeted execution. A lightweight Strategy Adapter outputs a confidence distribution over four reasoning strategies—NLR, CAR, TIR, and EBR—and the router chooses among single-strategy execution, dual-strategy verification, and full multi-strategy exploration according to thresholds $M_s$4 and $M_s$5. Here confidence controls not just which expert to use, but how much computation to allocate to verification and exploration (Qi et al., 29 Sep 2025).

Still finer control appears in stepwise and agentic systems. OI-MAS implements a hierarchical router in which a Role Router selects agent roles and a Model Router assigns a backbone model to each selected role, with EarlyStop as a routable role and role selection accumulated to threshold $M_s$6. Confidence acts as a state-dependent weight on computational cost, so high-confidence states penalize expensive role-model pairs more strongly than low-confidence states (Wang et al., 8 Jan 2026). STEER moves routing inside the chain of thought itself: after each reasoning step, it estimates the posterior probability that the step came from the confident component of a Gaussian mixture and then decides whether the next step should be generated by the small or large model. This is routing at the reasoning-step level rather than the query level (Lee et al., 9 Nov 2025).

Tool-routing and evidence-routing systems introduce another granularity. Robust-TO first decomposes a question into atomic sub-queries, then routes each sub-query to a perception tool chosen by semantic type and corruption profile, after first filtering video frames by a reliability-relevance score $M_s$7. It then synthesizes tool outputs through high/medium/low evidence tiers driven by calibrated reliability (He et al., 25 Jun 2026). EverydayGPT routes before expensive generation but after retrieval and extraction, using a joint policy over minimum retrieval distance and extraction adequacy; the routed options are rag, gpt, and refuse, so routing is also a safety gate (Nahal, 24 Apr 2026).

These architectures indicate that confidence-aware routing is not restricted to a single defer-or-answer binary. It can govern pre-generation model selection, post-generation rejection, strategy selection, role allocation, tool orchestration, evidence weighting, and multi-stage refusal. A plausible implication is that the central design variable is routing granularity: earlier routing tends to save more compute, while later routing can condition on richer evidence.

4. Language-model and multimodal systems

The most extensive recent development of confidence-aware routing occurs in LLM systems. SATER is a dual-mode compatible routing and cascading framework that trains an SLM to prefer short correct answers over long incorrect answers and to refuse complex questions at user-specified confidence levels. Across three SLMs—Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Qwen2.5-3B-Instruct—and six datasets, it reports comparable performance with over 50\% computational cost reduction in pre-generation routing, over 50\% lower average generation latency, and over 80\% lower average routing overhead latency in cascade routing. The reported token reductions are substantial: average tokens drop from 202.5 to 93.3 for Llama-3.1-8B-Instruct, from 254.5 to 170.5 for Qwen2.5-7B-Instruct, and from 319.8 to 195.0 for Qwen2.5-3B-Instruct (Shen et al., 4 Oct 2025).

Confidence-aware routing has also been used to mitigate hallucination before generation. The multi-signal pre-generation hallucination framework routes high-confidence queries to local generation, medium-confidence queries to RAG, low-confidence queries to larger models, and very low-confidence queries to human review, with thresholds $M_s$8, $M_s$9, and $M_l$0. On knowledge-intensive QA benchmarks, it reports hallucination detection of 0.74 versus 0.42 for the baseline, F1 of 0.82 versus 0.61, false positive rate of 0.09, and computational cost of 1.6x compared with 2.8x for always-on RAG; the paper also claims about 40\% computational cost reduction relative to post-hoc methods (M, 23 Sep 2025).

Hybrid GPT–RAG conversational QA provides a different operating point. EverydayGPT formalizes Confidence-Gated Routing as a joint policy over retrieval distance and extraction adequacy. With $M_l$1 and a stable operating point around $M_l$2, it avoids invoking the costly GPT pathway for 85 percent of queries by resolving them through fast RAG extraction at about 45 ms, versus about 5.9 s for GPT generation. On a 500-question in-domain benchmark, it reports $M_l$3, compared to 0.171 for GPT-only and 0.210 for unconditional RAG, together with a 6.3x mean latency reduction and a routed-query speedup of over 120x on the majority of queries (Nahal, 24 Apr 2026).

Strategy-routing and multi-agent routing extend the same principle to computational orchestration. PRISM reports improvements ranging from 0.9\% to 7.6\% across different base models, with a highlighted result of 53.2\% accuracy on MATH, beating the best single-strategy baseline by 3.1 points and the ensemble baseline by 2.5 points (Qi et al., 29 Sep 2025). OI-MAS reports accuracy improvements by up to 12.88\% while reducing cost by up to 79.78\%, and in the reported main table reaches 78.23\% average accuracy versus 70.55\% for MasRouter (Wang et al., 8 Jan 2026). Robust-TO, although centered on video understanding rather than text QA, is structurally homologous: it routes only trustworthy frames and confidence-calibrated evidence, reaching 56.4\% average accuracy on clean inputs and 54.3\% under five realistic corruption types while showing the smallest clean-to-corrupted accuracy drop among compared methods (He et al., 25 Jun 2026).

Taken together, these systems show that confidence-aware routing is now used not only for expert selection but also for controlling output length, fallback timing, retrieval invocation, strategy breadth, agent allocation, and multimodal evidence synthesis. The empirical pattern is consistent: confidence is used to preserve the cheap path on easy or well-supported cases and to justify escalation only when uncertainty is high.

5. Communication, mobile, and transport-network implementations

Confidence-aware routing predates LLM orchestration and has a separate lineage in communication and transport networks. In the cognitive router architecture based on HMM learning, each router is divided into a Cognitive Domain and a Communication Domain. The Cognitive Domain exchanges low-rate cognitive packets, constructs an 8-bit Forward Channel Performance Index from channel-level evaluations of bandwidth, delay, jitter, and loss, and predicts future neighbor health with a first-order HMM. Routing then uses predicted downstream channel quality rather than blindly forwarding according to immediate local state; FCPI is therefore the confidence signal, and the HMM is the prediction engine that turns past observations into forwarding advice (Nair et al., 2010).

In mobile ad hoc networks, the confidence-aware element is estimator uncertainty. EMP corrects noisy location and velocity estimates with a discrete Kalman filter, derives confidence from posterior covariance, and discards a Route Request whenever the estimated link duration is smaller than the confidence threshold. Simulations in NS-2.34 use 100 nodes over a $M_l$4 area, 900 s simulation time, Random Waypoint mobility, and Gaussian location errors with standard deviation from 3 m to 50 m. The reported qualitative conclusion is that EMP remains robust and consistently outperforms both MP and AODV as location error increases, whereas MP degrades quickly and can be worse than AODV when errors are large (Vu et al., 2016).

In intelligent transport systems, confidence enters as uncertainty over path cost rather than a hard accept/reject gate. The online shortest-path framework based on Online Frank-Wolfe repeatedly computes shortest paths in a time-varying random graph and then estimates a confidence interval for the final path cost by rerunning the stochastic optimization $M_l$5 times and forming an interval $M_l$6 around the empirical mean. The paper is explicit that this is an approximate uncertainty-quantification mechanism based on diffusion-style reasoning rather than an exact posterior interval, but it makes route uncertainty an output of the routing system itself (Chrétien et al., 2018).

TASR introduces a trust-aware variant of Stackelberg routing in which traveler groups accept recommendations with trust probability $M_l$7. The routing platform therefore optimizes recommendations under probabilistic compliance rather than a fixed compliant fraction. In the single-commodity Sioux Falls experiments, TASR improves average per-unit travel time by roughly $M_l$8 for $M_l$9, $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$0 for $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$1, and $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$2 for $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$3 relative to the next best algorithm, Scale. In repeated interaction, starting from $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$4, trust reaches full compliance after about 12 interactions for $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$5 and about 23 interactions for $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$6 (Brown et al., 2024).

A related but distinct thread is trustworthy routing verification. RVaaS does not primarily choose among alternate routes using an uncertainty score; instead, it lets clients query and verify routing properties despite an untrusted provider control plane by combining passive monitoring, active monitoring, logical data plane verification, and in-band Packet-In/Packet-Out tests. The paper is explicit that the aim is detection of misbehavior rather than prevention. This distinguishes verification-centric trustworthy routing from proactive confidence-aware routing, even though both address uncertainty about network behavior (Schiff et al., 2016).

6. Limitations, conceptual boundaries, and open directions

A recurrent limitation is that many confidence signals are operational proxies rather than fully calibrated probabilities. EverydayGPT states explicitly that its extraction confidence is a weighted heuristic, not a calibrated probability, chosen because the routing decision budget is less than 1 ms (Nahal, 24 Apr 2026). CARGO’s uncertainty measure is the predicted score gap between the top two models, not a probabilistic calibration score (Barrak et al., 18 Sep 2025). SATER’s refusal behavior is tied to empirical accuracy estimates and prompted thresholds rather than a formal posterior over correctness (Shen et al., 4 Oct 2025). Robust-TO calibrates tool confidence by input reliability, but its three-tier synthesis still depends on fixed thresholds such as HIGH for $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$7 and $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$8 (He et al., 25 Jun 2026). A common misconception is therefore that “confidence-aware” automatically implies probabilistic calibration; the literature shows that confidence is often a task-specific decision variable.

Threshold sensitivity is another shared issue. SATER notes that prompt-based refusal training struggles when thresholds fall below 0.1 and is studied mainly in a single SLM–single LLM setup, with cost analysis tied to a simplified pricing ratio of 1:13.75 and additional analyses at 1:25, 1:50, and 1:100 (Shen et al., 4 Oct 2025). The pre-generation hallucination-mitigation framework uses static thresholds fixed after validation and is evaluated on a 360M-parameter model with a balanced training set of 72 examples, which the paper identifies as a scale limitation and a source of domain-specific recalibration requirements (M, 23 Sep 2025). STEER depends on the choice of threshold $$r(i) = \begin{cases} 1, & \text{if } s_i < \tau \text{ (routed to } M_l\text{)} \ 0, & \text{if } s_i \geq \tau \text{ (routed to } M_s\text{)} \end{cases}$$9, on test-time Gaussian-mixture fitting, and on the assumption that the larger model is better in expectation than the smaller one (Lee et al., 9 Nov 2025). CARGO identifies dependence on judge LLMs, task imbalance, domain drift, and lower separability in summarization and creative writing as central limitations (Barrak et al., 18 Sep 2025).

The boundary between routing and verification remains important. Proactive systems such as SATER, STEER, and the pre-generation hallucination framework attempt to prevent wasted computation or unreliable answers by intervening before or during generation (Shen et al., 4 Oct 2025, Lee et al., 9 Nov 2025, M, 23 Sep 2025). RVaaS, by contrast, is a misbehavior-detection system and does not guarantee that bad routing cannot occur (Schiff et al., 2016). This distinction matters because a system can be trustworthy without being confidence-routed, and confidence-routed without providing auditable guarantees.

Across domains, the open research direction suggested by the surveyed work is not a single universal estimator but a general control principle: route cheaply when evidence is strong, escalate when uncertainty is consequential, and make the uncertainty signal commensurate with the action space. The remaining hard problems—domain shift, threshold transfer, model-family comparability, multi-model scaling, and the gap between heuristic and calibrated confidence—are recurring precisely because confidence-aware routing couples epistemic judgment to systems control.