Digital Twin-assisted belief-state reinforcement learning for latency-robust ISAC in 6G networks

Abstract: Integrated Sensing and Communication (ISAC) enables joint data transmission and environmental perception for sixth-generation (6G) networks, but centralized and virtualized RAN control loops introduce telemetry latency that yields stale observations and unstable control. This paper proposes a Digital Twin-assisted belief-state reinforcement learning framework for latency-robust ISAC. A Digital Twin (DT) reconstructs a synchronized belief state from delayed telemetry using an Extended Kalman Filter, and a Proximal Policy Optimization agent performs joint beamforming and power allocation for communication and sensing. Closed-loop simulations with telemetry delays up to 100 ms demonstrate consistent performance gains over latency-unaware deep reinforcement learning (DRL) and heuristic baselines. At 50 ms latency, the proposed method improves median throughput by 12% and reduces sensing error by 7% relative to a DT-only controller, while achieving an order-of-magnitude reduction in reliability violations. Even at 100 ms latency, the proposed approach retains approximately 88% of its zero-latency throughput. These results show that Digital Twin-assisted belief-state control enables stable and efficient ISAC operation under realistic telemetry delays in 6G networks.