Digital Twin-Assisted In-Network and Edge Collaboration for Joint User Association, Task Offloading, and Resource Allocation in the Metaverse

Abstract: Advancements in extended reality (XR) are driving the development of the metaverse, which demands efficient real-time transformation of 2D scenes into 3D objects, a computation-intensive process that necessitates task offloading because of complex perception, visual, and audio processing. This challenge is further compounded by asymmetric uplink (UL) and downlink (DL) data characteristics, where 2D data are transmitted in the UL and 3D content is rendered in the DL. To address this issue, we propose a digital twin (DT)-based in-network computing (INC)-assisted multi-access edge computing (MEC) framework that enables real-time synchronization and collaborative computing via URLLC. In this framework, a network operator manages wireless and computational resources for XR user devices (XUDs), while XUDs autonomously offload tasks to maximize their utilities. We model the interactions between XUDs and the operator as a Stackelberg Markov game, where the optimal offloading strategy constitutes an exact potential game with a Nash Equilibrium (NE), and the operator's problem is formulated as an asynchronous Markov decision process (MDP). We further propose a decentralized solution in which XUDs determine offloading decisions based on the operator's joint UL-DL optimization of offloading mode (INC-E or MEC only) and DL power allocation. A Nash-asynchronous hybrid multi-agent reinforcement learning (AMRL) algorithm is developed to predict the UL user-associated and DL transmission power, thereby achieving NE. Simulation results demonstrate that the proposed approach considerably improves system utility, uplink rate, and energy efficiency by reducing latency and optimizing resource utilization in metaverse environments.

• The paper introduces a DT-assisted INC-E framework that integrates digital twins with multi-access edge computing to facilitate joint user association, task offloading, and energy allocation.
• The paper employs a hierarchical Stackelberg Markov game with decentralized MARL strategies, enabling adaptive decision-making for interference-aware channel selection and partial offloading.
• The paper demonstrates significant performance gains—improved utility, energy efficiency, and reduced latency—compared to conventional baselines in dynamic XR environments.

Digital Twin-Assisted In-Network and Edge Collaboration in the Metaverse: Joint User Association, Task Offloading, and Resource Allocation

Introduction and System Motivation

The proliferation of extended reality (XR) applications in the metaverse context has imposed stringent requirements on wireless networks and computing infrastructures, specifically for real-time, low-latency transformation of 2D content into complex 3D objects. This process is characterized by intense computation and pronounced uplink (UL) and downlink (DL) asymmetry—lightweight 2D input data uploaded by users and bandwidth-intensive 3D output rendered to user devices. Existing multi-access edge computing (MEC) strategies, though impactful, face critical scalability and efficiency limits under these conditions due to resource contention and inadequate handling of wireless interference. The integration of in-network computing (INC) nodes and digital twin (DT) technologies into edge architectures presents new opportunities for dynamic and intelligent XR task orchestration.

The paper proposes a DT-assisted INC-enabled MEC (INC-E) system in which INC nodes collaborate with traditional edge servers. This design supports joint task offloading, user association, and resource allocation, emphasizing the need for adaptive, decentralized decision-making to handle dynamic XR workloads and interference-aware wireless access.

Figure 1: DT-assisted INC-E architecture illustrating the placement of XR user devices, in-network computing nodes, MEC servers, and their orchestration via a digital twin control layer.

Theoretical Modeling: Game-Theoretic and Reinforcement Learning Foundations

At the core of the framework lies a hierarchical Stackelberg Markov game. The network operator adopts the leader’s role, optimizing key network parameters—offloading mode (OFMO) and downlink power allocation (POAL)—by leveraging multi-agent reinforcement learning (MARL). XR user devices (XUDs), as followers, autonomously select transmission channels (INC association) and partial offloading ratios in response to operator decisions. The interaction structure yields an exact potential game among XUDs, admitting a pure-strategy Nash equilibrium (NE) which is efficiently computable.

Partial offloading is formalized, allowing joint processing of a task between multiple INC nodes and the edge server, subject to stringent latency and capacity constraints. The DT layer maintains virtual replicas of physical resources and user states, enabling real-time, low-latency optimization and control through predictive analytics and processing rate estimation.

Figure 2: Nash-AMRL scheme for DT-assisted INC-E system highlighting the Stackelberg interaction, asynchronous agent updates, and hybrid (UL/DL/global) critic design.

The operator’s problem is cast as an asynchronous Markov decision process (AMDP), where the state space is decomposed into UL agent and DL agent components, together capturing the system’s stochastic wireless load, task arrivals, and resource availability. The reward structure is hybrid: UL efficiency, DL energy, and overall system utility are separately defined and used in actor-critic learning.

Solution Architecture and Algorithms

The Nash-AMRL framework brings together decentralized best-response dynamics among XUDs and centralized operator coordination based on MARL. The user-side interaction is an interference-aware channel selection and task splitting game, underpinned by (i) decentralized channel association, (ii) real-time self-organization to mitigate interference and balance loads, and (iii) convex optimization of partial offloading under latency and energy constraints.

Operator-side, AMRL agents optimize UL OFMO and DL POAL via hybrid critic architectures. Three policy structures are evaluated:

• Asynchronous Hybrid Multiagent RL (AHMRL): Incorporates both local (UL, DL) and global critic heads, facilitating granular credit assignment and enhanced knowledge transfer.
• Multi-Actor Shared Critic (MASC): Employs a single critic shared between agents, leveraging collective global feedback.
• Actor-Critic (AC): Assigns each agent an independent local critic, resulting in fully decentralized policy learning.

The OFMO optimization is formulated as a binary knapsack problem solved via dynamic programming, applying DT-predicted user preferences and ensuring feasibility under INC node capacity limits. The DL agent samples continuous POAL values, exploring the power allocation space while adhering to operational constraints.

Numerical Results and Comparative Analysis

Empirical evaluations focus on multi-metric performance in dynamic XR environments, examining both user and operator perspectives and benchmarking against non-learning heuristics (randomized, equal, and proportional policies).

Figure 3: Training for different models and metrics demonstrating convergence across global/UL/DL rewards, utility, UL rate, energy consumption, end-to-end latency, and game utility.

The proposed architecture demonstrates rapid convergence to high-utility equilibria and substantially outperforms baselines in system utility, uplink rate, energy efficiency, and fairness. AHMRL achieves the most balanced improvement, with notable increases in utility (+6.24% over MASC and +4.07% over AC in data-intensive settings), maximum UL rates, and lower E2E latency (up to 3.73% reduction over MASC). While MASC performs best in compute-intensive scenarios, AHMRL’s superior robustness in communication-heavy workloads is evident.

Figure 4: Influence of task type and number of user equipments (UE) on utility, latency, and reward metrics, showcasing the model’s adaptability to different workload profiles.

Under varied operator-side resource configurations (INC node scaling), AHMRL consistently secures the highest global reward for moderate INC capacities, but its advantage diminishes with minimal or excessive node allocations.

Figure 5: Influence of INC node count and user density on operator-level performance, highlighting global reward, latency, and energy trends for each AMRL variant.

Rigorous robustness tests illustrate that AHMRL’s cost gain is both higher and more uniformly distributed at user and operator levels, surpassing heuristic approaches by a wide margin.

Figure 6: CDFs of user-level cost gains indicating strong fairness and consistency under learning-based policies.

Figure 7: CDFs of operator-level cost gains under different policies, affirming the efficiency gain realized by the Nash-AMRL scheme.

Practical and Theoretical Implications

The results substantiate the efficacy and scalability of joint DT-assisted INC-E collaboration combined with hybrid MARL for large-scale, low-latency XR task orchestration. The framework’s modularity allows for flexible adaptation to different operator deployment scenarios, varying INC capacities, and user heterogeneity. Notably, the Stackelberg game-theoretic modeling captures the inherent hierarchical interaction between operators and users in intelligent networks, facilitating decentralized self-organization and efficient resource utilization.

The comparative analysis of critic architectures also yields key theoretical insights: while hybrid decomposition (AHMRL) provides nuanced advantage assignment and improved coordination, global-only critics (MASC) sometimes offer superior stability due to the dominating global signal in highly coupled environments. These findings inform future MARL design for hierarchical resource allocation and multi-stage decision processes in networked AI systems.

Future Directions

Potential extensions include broadening the diversity and dynamism of user-task profiles, advancing critic design for further scalability with increasing agent populations, and integrating real-world wireless channel and hardware emulation. Exploration of federated or privacy-preserving MARL approaches could be particularly relevant for sensitive metaverse applications. The DT layer’s predictive analytics could be harnessed for proactive pre-fetching and adaptive task migration, further reducing latency and enhancing QoS. The hierarchical Stackelberg framework offers a promising generalization for broader AI-driven cyber-physical infrastructures.

Conclusion

This work establishes a formal framework for DT-assisted joint user association, task offloading, and energy allocation in INC-enabled metaverse edge computing. By modeling XR orchestration as a hierarchical Markov Stackelberg game and solving it with asynchronous MARL, the approach achieves significant gains in utility, efficiency, and fairness over structured and unstructured baselines. Analysis of critic architectures and task scaling reveals valuable guidance for future implementations of collaborative edge intelligence in virtualized immersive environments.

(2604.02938)