360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network (2401.10560v1)

Abstract: To enhance the performance and effect of AR/VR applications and visual assistance and inspection systems, visual simultaneous localization and mapping (vSLAM) is a fundamental task in computer vision and robotics. However, traditional vSLAM systems are limited by the camera's narrow field-of-view, resulting in challenges such as sparse feature distribution and lack of dense depth information. To overcome these limitations, this paper proposes a 360ORB-SLAM system for panoramic images that combines with a depth completion network. The system extracts feature points from the panoramic image, utilizes a panoramic triangulation module to generate sparse depth information, and employs a depth completion network to obtain a dense panoramic depth map. Experimental results on our novel panoramic dataset constructed based on Carla demonstrate that the proposed method achieves superior scale accuracy compared to existing monocular SLAM methods and effectively addresses the challenges of feature association and scale ambiguity. The integration of the depth completion network enhances system stability and mitigates the impact of dynamic elements on SLAM performance.

• The paper introduces 360ORB-SLAM, a visual SLAM system that integrates panoramic triangulation with a depth completion network to significantly improve scale estimation.
• It employs a depth completion network to convert sparse depth data into dense maps, reducing tracking failures in dynamic lighting and high-motion scenarios.
• Experimental results in the Carla simulation environment confirm the system's enhanced pose estimation and robustness, underscoring its potential for AR/VR and autonomous driving applications.

Introduction

Simultaneous Localization and Mapping, commonly referred to as SLAM, is a crucial technological element in the fields of robotics and computer vision, with significant implications for AR/VR applications, autonomous driving, and robot navigation. SLAM’s primary objective is to generate maps of an unknown environment while concurrently estimating the location of an agent within it. A key challenge in developing robust vSLAM systems is dealing with the limitations posed by monocular cameras, such as scale uncertainty and narrow fields of view. To address these challenges, a new SLAM system tailored for panoramic images, titled 360ORB-SLAM, has been introduced, equipped with a depth completion network aimed at improving scale accuracy and overall system performance.

Methodology

At the core of 360ORB-SLAM is the integration of a panoramic triangulation module with a state-of-the-art depth completion network. The triangulation module generates sparse depth information by managing the unique characteristics of panoramic cameras that otherwise tend to struggle with large FOVs and non-linear distortions. Further enhancement of the system’s capabilities is achieved by introducing a depth completion network that transforms sparse depth information into dense depth maps. Through this process, the system enhances stability and reduces the influence of dynamic elements, providing a more robust and accurate SLAM experience.

Experimental Results

The system's effectiveness was rigorously tested using a specially constructed panoramic dataset, rendered in the Carla simulation environment. 360ORB-SLAM demonstrated superior performance in scale accuracy and showed promise in resolving issues commonly experienced with monocular systems, such as tracking failures due to rapid motion or changing lighting conditions. The depth completion network played a pivotal role in this success by generating dense depth maps that significantly reduced scale drift and improved pose estimation accuracy. The quantitative results underscored the system's robustness, maintaining performance across various sequences and environmental conditions.

Implications and Future Work

The introduction of 360ORB-SLAM signals a step forward in overcoming the constraints of traditional monocular vSLAM systems. The system's performance in feature detection, scale accuracy, and environmental mapping highlights its potential for a range of applications, from enhancing AR/VR experiences to bolstering the reliability of autonomous vehicles. While the current focus is on the application of the system within intelligent driving scenarios, the methodologies and results discussed in this paper lay the groundwork for future advancements and optimizations that can be extended to other domains and real-world implementations.