An Intelligent eUPF for Time-Sensitive Path Selection in B5G Edge Networks

Abstract: In Beyond 5G (B5G) networks, intelligent, flexible traffic management is essential to meet the stringent speed and reliability requirements of new applications. This paper presents an improved User Plane Function (eUPF) design that uses a Deep Q-Network (DQN) agent for real-time path selection between Multi-access Edge Computing (MEC) and cloud endpoints. The path selection problem is formulated as a Partially Observable Markov Decision Process (POMDP). We propose a novel passive delay measurement method that uses eBPF programs to link TEID-based timestamps in GTP-U traffic, allowing for low-cost delay estimation without active testing. Experiments show that the DQN agent substantially outperforms a random baseline, with lower average latency, more stable rewards, and more reliable low-delay path choices. These results demonstrate the effectiveness of AI-driven control in B5G core networks and the promise of reinforcement learning for modern network management.

• The paper demonstrates that an eUPF, integrated with a DQN agent and TEID-aware passive telemetry, significantly reduces RTT compared to a random baseline.
• The methodology employs eBPF/XDP for real-time data plane observation and validates improvements on a physical testbed using Free5GC and Kubernetes.
• Experimental results highlight a median RTT reduction from 454 ms to 6.95 ms, proving DRL's effectiveness in adaptive path steering for time-sensitive applications.

Intelligent eUPF for Time-Sensitive Path Selection in B5G Edge Networks

Introduction and Problem Statement

The proliferation of delay-sensitive and compute-intensive services in Beyond-5G (B5G) architectures necessitates fine-grained, low-latency, and programmable core network elements. Central to this requirement is the User Plane Function (UPF), which orchestrates data forwarding in 5G/6G environments and, in contemporary deployments, is increasingly leveraged for edge/cloud service steering. Traditional UPF designs rely on rudimentary, static path selection and lack mechanisms to react to dynamic, slice-dependent service requirements or stochastic network degradations. This work rearchitects the UPF as an intelligent, programmable entity (eUPF), integrating Deep Reinforcement Learning (DRL) for real-time, time-sensitive path selection between Multi-access Edge Computing (MEC) and cloud egress.

Figure 1: Schematic of the proposed eUPF integrating DQN for adaptive data plane path selection leveraging TEID-correlated delay observation.

The path selection challenge is formulated as a partially observable Markov decision process (POMDP) with stochastic and memory-rich interface degradation. This abstraction captures the operational characteristics of both MEC and cloud connections as subject to random failures and recoveries, as well as measurement noise.

Related Work

State-of-the-art methods in 5G/6G networks—reviewed with emphasis on resource allocation, UPF steering, and AI4Net—primarily employ path selection and VNF placement optimization via DRL [Shokrnezhad et al., Tran et al., Sasithong et al., Kibalya et al.]. However, most studies are simulation-based, do not utilize dataplane signals such as TEID, and rarely deploy on real testbeds. The specific integration of TEID-awareness with RL-driven path control at the UPF, validated in physical testbeds, responds directly to these gaps.

Technical Architecture and Innovations

The eUPF design introduces several notable mechanisms:

• DQN-Based Path Selection: A DQN agent maps per-session, TEID-scoped delay proxies to egress interface choices across N6 links. Observations are reduced to a single scalar: the passively measured round-trip time (RTT) proxy for each TEID, acquired with no intrusive probe injection.
• Passive Telemetry via eBPF/XDP: TEID-correlated timestamps for each GTP-U PDU session are extracted and paired within the programmable data plane using eBPF. This provides near-instantaneous RTT estimation at the granularity required for session-specific path selection.
• Agent ⇄ Dataplane Interface: XDP maps expose observations and receive forwarding actions. This API between the DQN agent and the UPF ensures that adaptation occurs at the required temporal resolution for low-latency applications.

  Figure 2: Experimental configuration for evaluating eUPF with DQN in a real testbed using Free5GC, MEC and cloud endpoints, and in-kernel data collection.

The problem is formalized as adaptive interface choice, with stochastic state transitions, observation noise, and a reward signal reflecting (negated and normalized) end-user RTT. Episodes are 60 seconds, and learning dynamics are stabilized via experience replay and target networks.

Experimental Setup and Evaluation

The methodology was implemented on the FABRIC testbed with high-performance CPUs, orchestrated by Kubernetes. The agent interacts with the actual 5G core (Free5GC), and paths are artificially degraded according to controlled probabilities and durations—a critical step beyond simulation.

Reward and Policy Evolution:

Comparison with a random, non-learning baseline shows stark improvement from DQN: the baseline's reward is non-convergent and highly variable.

Figure 3: Raw and smoothed cumulative rewards of the non-learning baseline policy over 400 training episodes.

Figure 4: Raw and smoothed cumulative rewards showing rapid convergence and stability for the DQN agent.

The DQN agent quickly exploits the lower-latency MEC path while intermittently probing less reliable routes, reflecting correct temporal-difference learning and policy stabilization.

Action Distribution:

The baseline policy distributes actions uniformly. In contrast, the DQN converges to a strong preference for the MEC path as expected from reward structure and path degradation statistics.

Figure 5: Action selection frequencies for each interface, with DQN learnings shifting decisively toward the low-latency path.

Latency and Consistency:

Average RTT for DQN versus baseline is closer to 95.24 ms versus 199.37 ms across all episodes, with even greater divergence in variance and stability over the final training epochs.

Figure 6: Average round trip time across all and last 50 episodes, summarizing the impact of learned versus random path selection.

Forwarding Realization:

Data plane forwarding counts reinforce policy learning: DQN commands the transmission of the majority of packets towards MEC, while the baseline produces a near-equal split.

Figure 7: Distribution of output packets per interface, reflecting concrete steering due to DQN policy convergence.

Numerical results reveal superior median and mean RTT, with the DQN policy attaining median RTT of 6.95 ms versus 454 ms for the baseline in stabilized episodes, demonstrating the practical, not merely theoretical, gain of learning-driven data plane control.

Implications and Future Directions

This study advances programmable mobile core networks by demonstrating that RL can be tightly coupled with dataplane programmability (eBPF/XDP) to realize closed-loop, session-specific path selection at the UPF. The approach is directly applicable to scenarios demanding sub-10 ms E2E latency, such as XR/Metaverse workloads and mission-critical edge applications. Eliminating the need for active probes addresses operational and scaling burdens, and the runtime autonomy aligns with evolving 6G architectural principles [Alberti et al., 2024].

Two outstanding research avenues arise:

1. Algorithmic Exploration: Experimentation with DRL variants (e.g., PPO, Actor-Critic, LSTM-based agents) may further stabilize policy learning with richer state features, better coping with partial observability.
2. E2E and Cross-Domain Control: Application of this architectural paradigm to both uplink and downlink, multi-slice contexts, and integration with predictive analytics (e.g., network load forecasting) represents an actionable extension.

Conclusion

The introduction of an RL-powered, TEID-aware eUPF substantively advances the field of time-sensitive B5G path selection. The integration of fine-grained passive telemetry, in-kernel observation, and DRL-based adaptive control delivers consistent, superior low-latency service delivery in edge-to-cloud environments. The demonstrated reduction in mean and median RTT, as well as the practical data plane programmability, establishes a clear path forward for real-time, programmable 5G/6G core networks capable of meeting stringent application SLAs.

Reference:

"An Intelligent eUPF for Time-Sensitive Path Selection in B5G Edge Networks" (2605.01475)