LLM-Enhanced Deep Reinforcement Learning for Task Offloading in Collaborative Edge Computing

Abstract: Collaborative edge computing uses edge nodes in different locations to execute tasks, necessitating dynamic task offloading decisions to maintain low latency and high reliability, especially under unpredictable node failures. Although deep reinforcement learning (DRL) and LLMs have shown promise for task offloading, DRL often suffers from high sample inefficiency and local optima, whereas LLMs struggle with real-time decision-making. To address these limitations, we propose \textbf{LeDRL}, a hybrid decision framework that couples a \emph{lightweight LLM} with self-attention-enhanced DRL for real-time task offloading. LeDRL constructs structured, context-aware prompts capturing node status, task semantics, and link dynamics to derive high-level strategy priors. These are selectively processed by a self-attention-based alignment module for context-aware policy optimization. A reflective evaluator distills semantic feedback from past trajectories to guide future prompts, enabling more informative and temporally generalizable LLM queries. Extensive experiments show that LeDRL outperforms baselines in task success rate, convergence speed, and real-time responsiveness across diverse network scales, achieving over 17\% improvement in success rate. Furthermore, we deploy LeDRL on Jetson-based edge devices using our prototype system \textit{CoEdgeSys}, demonstrating its robustness and feasibility under resource constraints. Our code is available at:https://github.com/GalleyG5/LeDRL.git.

• The paper introduces LeDRL, a hybrid framework integrating lightweight LLM guidance with DRL to optimize scalable task offloading in dynamic edge networks.
• It leverages self-attention fusion and reflective memory to enhance sample efficiency and robustness against node failures and topology changes.
• Experimental results demonstrate up to 17% improvement in task success rates and near real-time performance on resource-constrained edge devices.

LLM-Enhanced Deep Reinforcement Learning for Collaborative Edge Task Offloading

Introduction and Motivation

Collaborative Edge Computing (CEC) leverages distributed edge nodes to execute latency-sensitive and reliability-critical tasks, substantially improving resource utilization and user experience. The core operational challenge in this domain is efficient and adaptive task offloading: dynamically selecting whether to execute tasks locally or forward them through multi-hop overlays, in response to nonstationary resource conditions, node failures, link disruptions, and topology changes. Traditionally, task offloading relies on static heuristics or standard Deep Reinforcement Learning (DRL) policies, which demonstrate limited adaptability in the complex, decentralized, and failure-prone environments characteristic of edge systems.

This paper introduces LeDRL, a hybrid learning and inference architecture that synergizes the context-aware abstraction and generalization capabilities of LLMs with the online adaptation and interaction-driven optimization strengths of DRL. The design hypothesis is that lightweight LLMs can inject robust semantic priors and failure reasoning into DRL agents, while self-attention-based policy fusion enables selective semantic adaptation, mitigating high sample complexity and local optima that plague conventional DRL. Notably, LeDRL is engineered for real-time deployment on resource-constrained devices and explicitly models NP-hard, dynamic offloading as a decentralized POMDP.

Problem Formulation and System Architecture

The system is formalized as a time-varying undirected graph $\mathcal{G}(t) = (\mathcal{V}(t), \mathcal{E}(t))$, with each node $v_i$ featuring device-specific compute and communication resources and subject to variable task arrivals, execution delays, and multiple failure modes (software, hardware, and transmission). The offloading control variable $x_{ij}(t)$ determines dispatch locations under global constraints of end-to-end delay, reliability, and resource feasibility. The objective is maximization of global task success rate, where success is defined jointly by deadline satisfaction and reliability thresholds across all decision epochs.

Figure 2: System overview depicting distributed edge task arrival, local and multi-hop pipelines, and decision interfaces for offloading and scheduling.

Task offloading is shown to be NP-hard due to its reduction from Bin Packing, underscoring the need for sample-efficient, scalable online solutions rather than exact combinatorial optimization in practical deployments.

LeDRL Framework: Architecture and Mechanisms

The LeDRL pipeline is architected as a multi-component hierarchically-guided system:

• LLM-Guided Policy Prior: A lightweight LLM receives a structured prompt encoding node states, task features, and dynamic link statuses. Time-local and time-generalized semantic memory are retrieved and distilled using custom scoring for contextual, domain-specific retrieval. Reflective introspection is employed: on violation or failure, a risk summary is constructed and incorporated into the long-term memory, closing a semantic feedback loop for robust episodic guidance.
• Self-Attention Policy Fusion: The hybrid RL agent fuses LLM priors with raw environment observations via self-attention. Feature representations are embedded from both the LLM and the observation spaces, projected into a shared latent space, and selectively weighted using attention mechanisms. This approach supports nonstationary action spaces, dynamic topological changes, and partial observability.
• Hybrid Optimization: The overall learning objective combines policy gradient updates (MAPPO) and attention-driven distillation gradients, balancing environmental reward maximization with LLM alignment. The fusion coefficient $\lambda_{\text{LLM}}$ is dynamically scheduled to control semantic integration versus learned adaptation. Exploration is further regularized via entropy bonuses.

  Figure 1: Architecture of LeDRL—semantic guidance from a lightweight LLM is fused with DRL policy gradients via a self-attention fusion module to construct hybrid offloading decisions.

Experimental Evaluation

Simulation Results

LeDRL is rigorously evaluated in simulated decentralized edge environments (10–20 nodes, stochastic arrivals, heterogeneous workloads, link/node failures) and compared against six baselines including VDN-TO, MAPPO-TO, MASAC-TO, heuristic search (RATC, AGSP), and LLM-guided Reflexion.

Figure 3: Comparative learning curves for task success rates across network scales; LeDRL outperforms all baselines with higher convergence speed, stability, and final success rate.

• Sample Efficiency and Robustness: LeDRL demonstrates superior convergence rates and lower variance, offering at least 7–17% higher final average success rates over DRL and heuristic baselines as network size or perturbation complexity grows. The introduction of LLM priors steers early exploration and recovers rapidly from topology changes, mitigating instability prevalent in pure DRL.
• Failure and Load Perturbations: Across sweeping variations in task size, computational intensity, execution, and transmission failure rates, LeDRL maintains the highest robustness. In high failure regimes, the method achieves up to 17% improvement in robustness over MAPPO-TO, validating the practical utility of LLM-driven semantic adaptation.

Figure 4: Task success rate for different (a) input sizes, (b) computational intensities, (c) execution failure rates, and (d) transmission failure rates; LeDRL consistently yields superior robustness.

• Topology Sensitivity: When evaluated on random versus ring topologies, LeDRL maintains its performance edge. Ablation shows the combined reflective memory, context-aware prompting, and attention fusion are necessary—self-attention alone or MLP-based fusion without context yields suboptimal learning or myopic policies.

Figure 5: Comparative analysis of network topology impact and key module ablations, confirming both the robustness and necessity of self-attentional fusion with LLM guidance for nonstationary environments.

• Latency–Accuracy Trade-off: LeDRL preserves real-time inference compatibility. For 10–20 node topologies, it achieves the best task success rate (up to 63.8%) with sub-second additional inference latency ($<$1s), outperforming DRL and LLM baselines while satisfying strict deadline constraints.

Real-World Prototype and Deployment

A full-stack distributed prototype, CoEdgeSys, is deployed on heterogeneous Jetson hardware, integrating the LeDRL scheduler and runtime agent with YOLOv8 detection workloads and COCO dataset benchmarks.

Figure 6: CoEdgeSys architecture on a Jetson edge device, depicting local scheduler, model process queue, and offloading interface.

Dynamic DoS conditions and strict latency requirements are enforced. LeDRL obtains up to 60% real-world success, outperforming DRL and heuristic methods by 13.5%, maintaining near-constant latency even under sharp increases in node failure probability or workload intensity. The transferability of the LLM-guided attention and reflective feedback loop from simulation to hardware environments is empirically demonstrated.

Figure 7: Real-world evaluation—LeDRL task success rate under varying detection confidence thresholds, demonstrating highest resilience compared to baselines.

Theoretical and Practical Implications

LeDRL’s framework establishes a flexible hybridization paradigm for embedding LLMs as dynamic, context-aware priors within online RL-style agents, rather than as static policy oracles. This advances state-of-the-art decentralized task offloading by offering scalable, robust learning under NP-hard, dynamic and partially observable CEC constraints. The attention fusion mechanism and reflective memory design are broadly extensible to other domains where combining episodic reasoning with sample-driven online adaptation is necessary.

Practically, the deployment of LeDRL on resource-constrained edge devices demonstrates feasibility of running LLM-guided hybrid control in real-world, delay-sensitive environments, provided that lightweight LLMs are used and semantic fusion is computationally efficient. This suggests next-generation edge AI systems will increasingly incorporate LLMs as embedded decision priors, with hybrid training strategies to balance latency and robustness.

Conclusion

The proposed LeDRL framework achieves significant advances in decentralized, reliable, and sample-efficient edge task offloading by integrating lightweight LLM guidance with attention-fused DRL. The method outperforms state-of-the-art DRL, heuristic, and prior LLM approaches in both simulation and real-world Jetson deployments, realizing robust adaptability to nonstationary topologies, failures, and workloads. Core innovations—the reflective episodic reasoning module and robust self-attention fusion—are general paradigms for edge intelligence, opening research directions in scaling hybrid LLM-RL agents, LLM distillation for embedded inference, and principled memory-driven meta-adaptation in resource-constrained systems.