Towards enabling reliable immersive teleoperation through Digital Twin: A UAV command and control use case

Abstract: This paper addresses the challenging problem of enabling reliable immersive teleoperation in scenarios where an Unmanned Aerial Vehicle (UAV) is remotely controlled by an operator via a cellular network. Such scenarios can be quite critical particularly when the UAV lacks advanced equipment (e.g., Lidar-based auto stop) or when the network is subject to some performance constraints (e.g., delay). To tackle these challenges, we propose a novel architecture leveraging Digital Twin (DT) technology to create a virtual representation of the physical environment. This virtual environment accurately mirrors the physical world, accounting for 3D surroundings, weather constraints, and network limitations. To enhance teleoperation, the UAV in the virtual environment is equipped with advanced features that maybe absent in the real UAV. Furthermore, the proposed architecture introduces an intelligent logic that utilizes information from both virtual and physical environments to approve, deny, or correct actions initiated by the UAV operator. This anticipatory approach helps to mitigate potential risks. Through a series of field trials, we demonstrate the effectiveness of the proposed architecture in significantly improving the reliability of UAV teleoperation.