CR^2: Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference

Abstract: As LLMs move from centralized clouds to mobile edge environments, efficient serving must balance latency, energy consumption, and accuracy under constrained device-edge resources. Query-level routing between lightweight on-device models and stronger edge models provides a flexible mechanism to navigate this trade-off. However, existing routers are designed for centralized cloud settings and optimize token-level costs, failing to capture the dynamic latency and energy overheads in wireless edge deployments. In this paper, we formulate mobile edge LLM routing as a deployment-constrained, cost-aware decision problem, and propose CR^2, a two-stage device-edge routing framework. CR^2 decouples a lightweight on-device margin gate from an edge-side utility selector for deferred queries. The margin gate operates on frozen query embeddings and a user-specified cost weight to predict whether local execution is utility-optimal relative to the best edge alternative under the target operating point. We further introduce a conformal risk control (CRC) calibration procedure that maps each operating point to an acceptance threshold, enabling explicit control of the marginal false-acceptance risk under the full-information utility reference. Experiments on the routing task show that CR^2 closely matches a full-information reference router using only device-side signals before deferral. Compared with strong query-level baselines, CR^2 consistently improves the deployable accuracy-cost Pareto frontier and reduces normalized deployment cost by up to 16.9% at matched accuracy.

• The paper introduces CR², which implements a two-stage routing process that optimizes local versus edge inference decisions through precise cost and risk calibration.
• It employs a margin gate, teacher selector, and CRC calibration to regulate false acceptance risks while achieving significant cost savings, up to 16.9% at high accuracy.
• Experimental results confirm that CR² outperforms baseline methods by maintaining low error rates and minimizing energy and latency costs in mobile-edge settings.

Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference: An Expert Analysis

Device-Edge LLM Inference: Motivation and System Architecture

LLMs are increasingly deployed across heterogeneous environments, necessitating efficient collaborative inference across user equipment (UE) and edge servers for latency, privacy, and continuity requirements. The device-edge paradigm introduces new constraints: variable wireless conditions, real-time device limitations, and asymmetric decision risks. Query-level routing—deciding whether to execute a request locally or escalate to a stronger edge model—serves as the fundamental optimization mechanism. However, routing approaches designed for centralized, cloud-based model pools are insufficient in mobile-edge deployments. They typically optimize token-level costs and lack explicit modeling of state-dependent latency and energy incurred through wireless links.

Figure 1: Two-tier device-edge inference and routing flow.

This architecture enforces a two-stage routing process. The UE makes an immediate accept/defer decision using only local signals; deferred queries are then routed among edge models, potentially exploiting system state information unavailable to the UE at decision time.

Formulation of Deployment-Constrained Routing

Routing in the device-edge context is formalized as a budgeted optimization problem over accuracy and normalized deployment cost, which aggregates latency and energy consumption with application-specific weights. Selection policies must observe the decentralized information structure: the UE can access only its own embeddings and explicit operator preferences, but not edge-side utilities or runtime states prior to deferral.

The proposed framework, CR², factorizes routing: a local margin gate (UE) decides accept/defer via a utility-margin estimate conditioned on query embeddings and operator cost weights; if deferred, an edge-side utility selector chooses among available models using teacher-estimated utilities incorporating wireless state and normalized costs.

CR² Methodology: Margin Gate, Utility Selector, and CRC Calibration

CR² implements three coupled components:

1. Margin Gate: A lightweight, FiLM-conditioned MLP trained on a frozen encoder, regresses a continuous local-versus-edge utility margin. The gate’s output is temperature-scaled for downstream calibration (Figure 2).
2. Teacher Selector: Multi-head binary classifier producing per-model correctness estimates, used both for offline distillation and runtime edge-side selection.
3. Conformal Risk Control (CRC) Calibration: CRC maps each operator-defined cost weight to an acceptance threshold, regulating the marginal risk of false local acceptance independently of the gate's internal scoring scale.

   Figure 2: Overview of CR² including offline training, CRC-based calibration, and online two-stage routing.

The calibration step is critical: false local acceptance irreversibly forgoes edge alternatives, necessitating explicit risk control at the UE. CRC calibration ensures that the probability of accepting a query locally when a strictly better edge model exists is bounded by a target $\alpha$.

Experimental Evaluation and Numerical Results

CR² and baselines (KNN, MLP, EmbedLLM, LLMRank) are evaluated on a routing dataset spanning MMLU, BBH, GPQA, and MBPP, using Qwen3-$1.7$B (device) and Qwen3-$4$B/$8$B/$14$B (edge) models. Deployment cost models are profiled for latency/energy across Jetson AGX Orin, RTX 4070 Ti, RTX 4090, and A40 platforms.

Figure 3: Accuracy--cost Pareto curves: (a) full range, (b) zoomed operating region.

CR² establishes the strongest deployable accuracy-cost frontier in the relevant cost bands. At fixed accuracy, CR² yields normalized cost savings versus KNN: $4.8\%$ at $0.79$, $16.9\%$ at $0.81$ accuracy. Marginal false acceptance risk is tightly controlled across CRC calibration levels.

Figure 4: Fixed-accuracy cost comparison.

Figure 5: Accuracy--cost curves for threshold-selection rules.

Figure 6: Marginal false-acceptance rate under CRC-calibrated thresholds.

Figure 7: Local-model selection rate.

Per-benchmark, CR² outperforms competitors across pooled accuracy at fixed cost targets, and maintains low false local acceptance rates (<$1.5\%$) while improving overall gate error alignment as cost increases.

Ablations, Deferred-Branch Variants, and Gate Error Analysis

Component ablations indicate that margin regression loss and monotonicity regularization improve gate accuracy and stability. Deferred branch variants (edge-only, inclusive, fallback) show that device-edge separation closely tracks the full-information reference, with residual gap attributable to gate over-escalation rather than deferred selection.

Figure 8: Deferred-branch variants.

Gate-error decompositions reveal that false deferral decreases with increasing cost, while false local acceptance remains consistently low.

Figure 9: Gate-error decomposition.

Router complexity measurements confirm that the CR² device-edge gate incurs minimal overhead (one frozen embedding and small MLP), making it practical for mobile environments.

Implications and Future Research

CR² rigorously addresses deployment-constrained routing for collaborative device-edge LLM inference, introducing a calibrated mechanism for controlling asymmetric decision risks inherent to decentralized settings. The two-stage routing structure, explicit utility-margin estimation, and CRC calibration collectively advance accuracy-cost trade-offs compared to prior query-level routers. Practically, CR² enables service providers to manage energy and latency budgets while preserving reliability guarantees in mobile applications.

This framework establishes theoretical precedents for embedding coverage and bounded-risk conformal calibration into decentralized inference systems. Future directions include extending CRC calibration to simultaneous guarantees across multidimensional operating points, integrating wireless-informed semantic verification, and deploying cost-aware routing in distributed edge-served MoE and context-caching regimes.

Conclusion

The CR² methodology, through its factorized routing structure, margin gate, and CRC-calibrated thresholding, offers a principled solution to device-edge collaborative LLM inference under state-dependent deployment constraints. Empirical results demonstrate close tracking of full-information reference policies and significant cost savings at matched accuracy, validating the practical and theoretical contributions for collaborative AI inference (2605.12001).