EntroRouter: Learning Efficient Model Routing via Entropy Regulation

Abstract: Model routing balances solution accuracy and computational cost by selecting among models of varying capabilities. While recent multi-round frameworks interleave reasoning and planning, we identify a structural failure mode termed Trust Region Collapse. We demonstrate that the deep coupling of reasoning and routing, exacerbated by the dominance of strong pre-training priors under sparse supervision, leads to degenerate local optima where capable experts are systematically suppressed. To decouple these processes, we propose $\textbf{EntroRouter}$, a single-round routing framework that treats entropy regulation as a core objective. We first initialize the policy via Soft Supervision, fitting a distribution of suitable models to establish a high-entropy prior for exploration. Subsequently, we stabilize Reinforcement Learning using a Soft Anchor, which utilizes offline capability estimates to orchestrate controlled entropy contraction within a safe trust region. Extensive experiments demonstrate that EntroRouter retains 98.3% of the strongest expert's accuracy while reducing computational costs by 48.25%.

• The paper presents EntroRouter, which addresses premature entropy collapse by decoupling model routing from reasoning through entropy regulation.
• It employs a two-stage training process using soft supervision followed by RL with soft anchor regularization to balance cost and accuracy.
• Empirical results demonstrate near-expert accuracy (88.6%) and a 48.25% cost reduction, validating robust and scalable model routing in LLM systems.

EntroRouter: Learning Efficient Model Routing via Entropy Regulation

Introduction and Failure Analysis of Existing Model Routing Paradigms

Efficient deployment of LLMs necessitates resolving the tension between the superior reasoning performance of large models and the prohibitive computational cost associated with their use. Model routing frameworks are designed to dynamically assign queries to an optimal model from a diverse pool, balancing accuracy and computational expenditure. However, both multi-round agents and single-round classifiers exhibit a fundamental failure mode: premature entropy collapse.

In multi-round architectures, such as Router-R1, the RL process rapidly suppresses the probability of invoking external experts, resulting in what the authors formalize as Trust Region Collapse. This is empirically illustrated by the drastic and early probability collapse post-initial fine-tuning, where reliance on stronger external models falls to zero, effectively abandoning their utility (Figure 1).

Figure 1: Empirical observation of Trust Region Collapse—after cold-start fine-tuning, RL optimization collapses expert call probability to zero, eliminating expert solver utility.

This degeneration is rooted in the strong pre-training priors of the router, which override information from external expert traces, especially under sparse supervision. Consequently, external model calls are interpreted as high-cost/zero-gain actions, enforcing deterministic, non-exploratory routing policies.

Single-round classifiers also suffer from label-space entropy collapse due to overuse of hard supervision, prematurely eliminating the uncertainty required to accommodate nuanced query difficulty and forcing deterministic decisions that degrade out-of-distribution (OOD) generalization.

The EntroRouter Framework: Decoupling Routing and Reasoning via Entropy Regulation

The authors introduce EntroRouter, which re-conceptualizes model routing as a single-round policy optimization problem where entropy regulation is a principal objective. EntroRouter’s training is architected in two explicit stages:

1. Stage I: Supervised Fine-Tuning (SFT) via Soft Supervision.
2. Stage II: RL Optimization with a Soft Anchor regularizer.

This framework decouples planning (routing) from downstream solution execution, positioning the router as a meta-analyzer rather than an interleaved reasoner.

Stage I: SFT via Soft Supervision

To mitigate entropy collapse during policy initialization, EntroRouter employs soft supervision using adaptive convex combinations of hard and soft targets. For simple queries, the training signal is one-hot, targeting the most economical qualified model. For complex queries, the supervision is softened: probability mass is spread uniformly over the top-$k$ cheapest models that surpass a reliability threshold in Monte Carlo ground-truth passes. This approach maintains a high-entropy prior for the routing policy (Figure 2), preserving exploration capacity for ambiguous instances.

Figure 2: SFT via Soft Supervision—query difficulty determines whether the training target is a one-hot label or a soft distribution over top-performing models, maintaining exploration entropy.

Stage II: Reinforcement Learning with Soft Anchor Regularization

RL fine-tuning further sharpens cost-accuracy trade-offs. EntroRouter regularizes policy updates using a Soft Anchor: a reference distribution, parameterized by offline capability (pass rate) and model cost estimates, constructed via a Boltzmann transform. The optimization objective includes dual regularization penalties: KL-divergence to the reference (anchoring within a safe trust region) and an explicit entropy penalty to drive the policy toward confident decisions within this region, avoiding both over-exploration and collapse. Empirically, this mechanism enables controlled contraction of the policy entropy and stable convergence.

Empirical Results: Cost-Accuracy Pareto Dominance and Robustness

EntroRouter is extensively benchmarked on seven mathematical reasoning tasks, deploying Qwen3-4B/30B/235B as the candidate pool. Results indicate:

• Accuracy: 88.6%, which is 98.3% of the strongest fixed-expert baseline (Qwen3-235B)
• Cost reduction: 48.25% relative to always using the strongest expert

Contrasted with multi-round (Router-R1, xRouter) and single-round (RouteLLM, FORC) baselines, EntroRouter robustly avoids Trust Region Collapse: multi-round methods degenerate into exclusive reliance on weak internal routers, whereas EntroRouter consistently invokes stronger experts proportionally to the anticipated problem complexity.

Ablation studies confirm:

• Removal of soft supervision or gating mechanisms causes significant accuracy drops and increased computational waste.
• Disabling KL anchoring or expectation-based rewards undermines entropy regulation and policy convergence.
• The method’s cost-accuracy Pareto efficiency is tunable via the cost-sensitivity hyperparameter $\alpha$.

Routing behavior analysis (Figure 3) shows adaptive sharpening: low-complexity queries are routed to the smallest model, with the routing distribution shifting to larger models as complexity rises, and average API cost monotonically increasing with difficulty, empirically validating the intended entropy contraction dynamics.

Figure 3: Routing Distribution and Token Cost Efficiency—routing becomes more selective and expensive as query difficulty increases, indicating adaptive entropy contraction.

Theoretical Formulation: Trust Region Collapse and the Role of Entropy Anchoring

The authors formalize Trust Region Collapse under sparse supervision and strong prior dominance, proving that the effective trust region for escalating to stronger models is empty when the observed reward improvement does not exceed the escalation cost. This validates the necessity of decoupling routing from execution and controlling policy entropy contraction. The theoretical framework also demonstrates that Soft Anchor regularization is mathematically equivalent to solving a constrained RL objective with KL and entropy penalties, operationalized via reward shaping for efficient policy optimization.

Broader Implications and Future Directions

EntroRouter establishes that small models are most effectively utilized as meta-controllers—estimating query difficulty and dispatching requests without being encumbered by strong internal reasoning priors. This approach achieves Pareto-efficient routing across a range of problem complexities and maintains robustness to noisy offline capability estimates and heterogeneous candidate pools.

Implications include:

• Scalability: The single-round, high-entropy initialization and trust-region regularization are conducive to deployment over large, mutable model pools.
• Transferability: Empirical generalization to OOD benchmarks, including GPQA and MMLU-Pro, demonstrates practical relevance beyond mathematical reasoning.
• Orthogonality: The framework can be integrated with process-level supervision and verification modules to further stabilize multi-round routing.

Future research should explore the integration of stronger auxiliary supervision signals (e.g., process rewards, verification feedback), dynamic candidate pool profiling, and generalization to multi-step aggregation modules.

Conclusion

EntroRouter clearly demonstrates that explicit entropy regulation and cognitive decoupling are prerequisites for effective cost-aware model routing in LLM systems. The framework achieves near-expert accuracy at substantially reduced computational cost, robustly managing exploration-exploitation trade-offs and avoiding structural failures associated with prior routing paradigms. These findings directly inform the design of scalable, sample-efficient, and robust AI meta-control systems.