Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing

Abstract: Emerging computation-intensive applications impose stringent latency requirements on resource-constrained mobile devices. Mobile Edge Computing (MEC) addresses this challenge through task offloading. However, designing effective policies remains difficult due to dynamic task arrivals, time-varying channels, and the spatio-temporal coupling of server queues. Conventional heuristics lack adaptability, while Deep Reinforcement Learning (DRL) suffers from limited generalization and architectural rigidity, requiring retraining when network topology changes. Although LLMs offer semantic reasoning capabilities, standard Supervised Fine-Tuning (SFT) yields myopic policies that greedily minimize immediate latency without accounting for long-term system evolution. To address these limitations, we propose COMLLM, a generative framework that enables foresighted decision-making in MEC systems. COMLLM integrates Group Relative Policy Optimization (GRPO) with a Look-Ahead Collaborative Simulation (LACS) mechanism, which performs multi-step Monte Carlo rollouts while jointly modeling server queue dynamics. By incorporating these rollouts into the reward design, the framework captures the long-term impact of current decisions on future system states. Experimental results demonstrate that COMLLM achieves near-optimal latency and improved load-balancing fairness. Notably, it exhibits zero-shot topological scalability, allowing a model trained on small-scale networks to generalize to larger, unseen topologies without retraining, outperforming SFT, DRL, and heuristic baselines.

• The paper introduces COMLLM, a framework that integrates generative LLMs with GRPO and LACS for future-aware, topology-agnostic task offloading in MEC.
• It employs semantic state serialization and virtual look-ahead simulations to optimize load balancing and minimize latency under dynamic network conditions.
• Experimental results demonstrate near-optimal latency, zero task drop rates, and zero-shot generalization outperforming traditional DRL methods.

Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing

Introduction

Emerging computation-intensive applications—such as augmented reality, real-time video analytics, and digital twins—impose stringent latency constraints on resource-constrained mobile devices. Mobile Edge Computing (MEC) addresses these requirements by offloading computationally intensive tasks to edge servers, reducing service delay compared to centralized cloud architectures. However, optimal task offloading in MEC remains challenging due to dynamically arriving tasks, time-varying channels, and the complex spatio-temporal coupling of server queues. Existing approaches based on heuristics and Deep Reinforcement Learning (DRL) suffer from limited adaptability and generalization, especially as network topology changes. LLMs, though promising for flexible reasoning, have exhibited myopic policies that minimize immediate latency without accounting for future queue congestion or long-term system evolution.

This paper introduces the COMLLM framework, which leverages generative LLMs and advanced reinforcement learning to enable foresighted, topology-agnostic task offloading policies in MEC. COMLLM employs Group Relative Policy Optimization (GRPO) combined with a Look-Ahead Collaborative Simulation (LACS) mechanism, effectively incorporating Monte Carlo rollouts to capture the long-term impact of decisions. The proposed system achieves near-optimal latency, improved load balancing, and exhibits zero-shot generalization to new network topologies without retraining.

Figure 1: Overview of the COMLLM method framework, integrating semantic state serialization, reinforcement learning, and look-ahead simulation for future-aware MEC task offloading.

System Model and Problem Formulation

The MEC environment is modeled with heterogeneous edge servers and stochastic task arrivals. Each task has input size, computational density, and latency deadline. Offloading decisions are binary, assigning a task either to a local device or one of the edge servers. Each server’s computation rate is modulated by processor-sharing, with queue evolution tightly coupled to task assignments, thereby inducing strong temporal dependencies.

Communication delay is modeled as a function of the uplink channel parameters, while local and edge computation delays are determined by respective CPU frequencies and current queue backlogs. The objective is to design an offloading policy $\pi$ that minimizes cumulative generalized task cost, including deadline violation penalties, subject to binary assignment and queue constraints. The optimization problem is of mixed discrete, nonlinear flavor, with the state space rapidly expanding as server count increases.

Methodology: COMLLM Architecture

COMLLM reformulates MEC task offloading as an MDP, enabling sequential policy optimization. The state is semantically serialized into textual prompts, allowing LLMs to reason over variable server sets without architectural modification. The system integrates three stages:

1. Supervised Fine-Tuning (SFT): The LLM is initialized with domain-specific knowledge via oracle-labeled instruction-following datasets. Semantic serialization includes unit tagging, dynamic topology abstraction, and temporal workload encoding.
2. Group Relative Policy Optimization (GRPO): The SFT-initialized LLM is further refined using GRPO, which samples action groups for advantage estimation and uses KL-divergence regularization to ensure stable updates. This avoids dependency on separate critic networks.
3. Look-Ahead Collaborative Simulation (LACS): To address myopic optimization, LACS performs virtual transitions by simulating queue updates and sampling future tasks. Oracle evaluation under these perturbed states yields a future-impact cost, which is smoothly integrated into the reward function. This reshaping ensures the RL policy internalizes both immediate and downstream effects.

Numerical Performance and Comparative Evaluation

COMLLM demonstrates superior performance across several axis: average latency, task drop rate, performance ratio relative to oracle, and load balancing index. Notably, COMLLM consistently achieves the lowest latency (e.g., 3.07 for average latency), zero task drop rate, and highest load balancing fairness (index > 70%). GRPO-7B is competitive but consistently outperformed by COMLLM, particularly under heavy workloads and varying topology.

Increasing task sizes induce congestion; COMLLM maintains robust performance, limiting drop rates below 3% even under maximum workload, while imitation-based SFT or low-capacity models deteriorate rapidly (drop rates rising > 30%). Under varying server topologies, COMLLM achieves near-oracle performance and zero-shot generalization, maintaining latency and fairness without retraining. Load balancing analysis illustrates that COMLLM allocates tasks across servers more evenly than baselines, substantiating its long-term, congestion-aware optimization.

Prompt robustness tests reveal that LLM-based models can be sensitive to prompt semantics. COMLLM is notably resilient to text perturbations, parameter shuffling, and unit variation, retaining high task completion and balanced load metrics, indicating that semantic state serialization and future-aware reward shaping foster robust, physically meaningful representations.

Theoretical Implications and Future Outlook

COMLLM’s design advances both practical and theoretical understanding in AI-driven MEC optimization. The use of semantic prompts for topology-flexible policy design addresses a significant limitation in existing DRL frameworks, eliminating the need for model retraining as network topology scales. The integration of LACS into RL reward structure provides a tractable approach for incorporating long-horizon dependencies and congestion effects, previously difficult to optimize efficiently.

From a theoretical perspective, the GRPO process, coupled with KL-penalized policy updates and future-aware reward shaping, retains stability and formal improvement guarantees reminiscent of PPO-style RL methods while adapting to the generative LLM context. This opens avenues for extending RL fine-tuning of LLMs to other structured sequential decision tasks with variable state-action spaces.

Practically, COMLLM offers a scalable method for real-time adaptive task offloading in MEC, supporting dynamic environments and heterogeneous topologies. The zero-shot generalization and robust performance under prompt perturbations facilitate deployment in real-world, non-stationary settings where traditional DRL methods would falter.

Conclusion

COMLLM is a generative AI framework for MEC task offloading that combines semantic state serialization, reinforcement RL fine-tuning, and look-ahead collaborative simulation for future-aware resource allocation. Experimental evidence demonstrates strong numerical results for latency reduction, robustness, and load balancing, attaining zero-shot adaptability to new network topologies. This methodological advance extends the applicability of LLMs to dynamic, combinatorial optimization tasks in edge computing. Further research may explore integrating COMLLM with federated learning or secure aggregation, and extending LACS to incorporate multi-agent system dynamics for wider distributed AI applications.