LASER: Load-Aware Serving with Early-Exit for Reasoning LLMs at the Edge

Abstract: Large reasoning models (LRMs) such as DeepSeek-R1 have achieved strong performance through extended chain-of-thought (CoT) generation. However, deploying them on edge devices raises a conflict between long CoT sequences and constrained resources. Recent confidence-based early exit methods reduce CoT length for individual requests, yet they apply fixed thresholds from a single-request perspective, ignoring multi-request concurrency and load fluctuation in edge serving. To bridge this gap, we propose \underline{L}oad-\underline{A}ware \underline{S}erving with \underline{E}arly-exit for \underline{R}easoning (LASER). LASER couples two complementary designs: (1) a load-aware adaptive exit threshold that adjusts the confidence bar based on real-time system load within an empirically validated robust range, and (2) a difficulty- and load-aware reasoning budget pre-allocation that assigns compute resources by request difficulty and system capacity. We formulate the problem as a joint optimization of reasoning quality and service latency. Experiments on two reasoning models, four benchmarks, and diverse load conditions show that LASER reduces average latency by 17--38\% and improves service-level objective (SLO) satisfaction by 3--6\% over fixed-threshold baselines, at an average accuracy cost of only 1\%.

• The paper introduces LASER, which implements dynamic early exit and reasoning budget allocation to balance latency and accuracy in edge LLMs.
• It employs an EMA-smoothed load estimate and a tanh-based adaptive threshold, achieving 17–38% lower latency and improved SLO satisfaction.
• Empirical results across benchmarks demonstrate minimal accuracy loss (<1%) with significant compute savings, enabling efficient edge deployments.

LASER: Load-Aware Dynamic Early Exit for Edge Reasoning LLMs

Motivation and Problem Setting

The trend of deploying complex large reasoning models (LRMs) on edge devices exposes a prominent latency-vs-accuracy challenge, as extended chain-of-thought (CoT) sequences impose significant compute overhead and queueing delays. Modern LRMs like DeepSeek-R1 and Qwen3 produce solutions with deep slow-thinking reasoning, often exceeding 10k tokens per query, which is infeasible for latency-constrained, single-server edge deployments. The overthinking phenomenon compounds this bottleneck, as verbose chains yield negligible or even negative marginal accuracy per compute cost. Existing confidence-based early exit methods reduce reasoning depth at the single-request level but fail to coordinate serving under concurrent, fluctuating loads, leading to overall suboptimal latency and SLO (Service-Level Objective) satisfaction.

LASER System Overview

LASER (Load-Aware Serving with Early-Exit for Reasoning) is proposed to align inference quality and service efficiency under edge constraints. The system elevates early exit from isolated, request-level optimization to a global, queuing-aware controller by coupling two core mechanisms:

1. Load-aware adaptive exit threshold: The system computes a dynamic confidence threshold for each exit decision. The threshold is modulated in real-time, based on an EMA-smoothed load estimate, lowering the bar under high load and maximizing reasoning quality under low load.
2. Difficulty- and load-aware reasoning budget pre-allocation: Before inference, each request is assigned a hard cap on allowed reasoning steps. This budget reflects both request difficulty (approximated by prompt length) and concurrent system load, preventing runaway compute on hard samples during overload scenarios.

These two controls operate in tandem: the threshold acts as a soft boundary for early exit, while the hard budget ensures compute fairness and predictability.

Figure 1: LASER system architecture details managing request queues, early exit, and adaptive reasoning budgets at the edge.

Algorithmic Design and Implementation

LASER’s online algorithm proceeds as follows: For each incoming request, a budget allocator sets the maximum allowed reasoning steps as a linear function of prompt length (difficulty proxy), modulated by current system load. As the model generates each reasoning chunk, the system updates the load estimate via EMA and recalculates the adaptive exit threshold using a tanh-based function, which maintains a bounded, smooth response to load variations. Reasoning halts once the model’s local answer confidence exceeds the threshold or when the hard budget is depleted, whichever comes first.

This design yields:

• $O(1)$ per-decision overhead, reliant solely on active request count and local prompt length, requiring no extra neural inference.
• Parameter robustness, with results stable over a broad range of thresholds, scaling factors, and EMA smoothing weights.
• No model retraining or special prompt modifications.

Figure 2: High-level algorithmic workflow of LASER, illustrating decision points and interaction between adaptive thresholds and budgets.

Empirical Evaluation

Rigorous experiments cover two state-of-the-art LRMs (DeepSeek-R1-Distill-Qwen-7B, Qwen3-4B) on four benchmarks (GSM8K, MATH-500, AMC 2023, GPQA). Examined serving scenarios include steady loads and bursty arrival patterns, under FIFO single-GPU service, with real request traces simulated. Key baselines are:

• Vanilla: No early exit (maximal CoT length).
• Fixed-High/Low: Static threshold early exit.
• NoThinking: No reasoning (lower bound).

LASER consistently produces 17–38% lower average latency and 3–6 percentage points higher SLO satisfaction compared to fixed-threshold baselines, with mean accuracy loss <1%. Notably in some configurations, LASER preserves or even exceeds fixed-threshold accuracy at a fraction of the compute cost, particularly on competitive math and science benchmarks.

Figure 3: Accuracy and average token count for LASER and baselines across benchmarks, demonstrating substantial compute savings with minimal quality degradation.

LASER also robustly improves under burst traffic, dynamically contracting reasoning chain lengths during overloads and restoring depth as load alleviates, leading to best-in-class SLO adherence and throughput.

Component Analysis

Ablation studies demonstrate that budget pre-allocation is the main driver of latency reduction, providing coarse-grained predictability to control tail response times, while the adaptive threshold recovers accuracy by adjusting exit sensitivity under varying queue conditions. The fusion yields superior efficiency/quality trade-offs compared to either mechanism in isolation.

Figure 4: Ablation analysis highlights the specific impacts of budget allocation and adaptive thresholding on latency and accuracy.

Implications and Prospects

LASER establishes methodology for leveraging system-level serving state to dynamically control inference cost in edge LLM deployments, contrasting with prior work which treats chain-of-thought length as static or decoupled from load. The framework is training-free, compatible with mainstream LLMs, and generalizes across benchmarks and traffic regimes. Theoretical implications include the utility of confidence polarization for global service policies and the value of lightweight request difficulty estimation in practical compute assignment.

Practically, LASER enables edge inference deployments to meet latency SLOs with minimal hardware, supporting conversational and problem-solving workloads that would otherwise overwhelm edge GPUs. The system presents a foundation for multi-device extension (e.g., coordinated multi-GPU budget sharing), integration with model selection/routing policies, and further theoretical analysis of joint compute allocation and exit confidence calibration.

Conclusion

LASER advances the state-of-the-art in edge LLM serving with an integrated, load-aware early exit framework. By bringing adaptive, global scheduling control—anchored in empirical load signals and lightweight request difficulty features—LASER yields significant reductions in latency and increases in SLO satisfaction while nearly maintaining single-request optimal reasoning accuracy. This methodology offers a robust path toward deploying sophisticated reasoning agents in latency-critical, resource-constrained edge environments.