Sparse Depth Enhanced Direct Thermal-infrared SLAM Beyond the Visible Spectrum (1902.10892v1)

Abstract: In this paper, we propose a thermal-infrared simultaneous localization and mapping (SLAM) system enhanced by sparse depth measurements from Light Detection and Ranging (LiDAR). Thermal-infrared cameras are relatively robust against fog, smoke, and dynamic lighting conditions compared to RGB cameras operating under the visible spectrum. Due to the advantages of thermal-infrared cameras, exploiting them for motion estimation and mapping is highly appealing. However, operating a thermal-infrared camera directly in existing vision-based methods is difficult because of the modality difference. This paper proposes a method to use sparse depth measurement for 6-DOF motion estimation by directly tracking under 14- bit raw measurement of the thermal camera. In addition, we perform a refinement to improve the local accuracy and include a loop closure to maintain global consistency. The experimental results demonstrate that the system is not only robust under various lighting conditions such as day and night, but also overcomes the scale problem of monocular cameras. The video is available at https://youtu.be/oO7lT3uAzLc.

Sparse Depth Enhanced Direct Thermal-infrared SLAM Beyond the Visible Spectrum

Sparse Depth Enhanced Direct Thermal-infrared SLAM (DEDT-SLAM) emerges as a promising solution for refined ego-motion estimation and mapping in challenging environments where traditional RGB-based systems falter. The method leverages the advantages of thermal-infrared cameras, specifically their durability under adverse conditions such as fog, smoke, and variable lighting, and combines them with sparse measurement of depth from LiDAR.

In practice, RGB cameras face significant difficulties in scenarios characterized by fog, dust, or darkness. Thermal-infrared cameras provide perceptual robustness against these limitations but integrating them into existing visual perception frameworks presents several obstacles. This paper introduces a DEDT-SLAM framework, which enhances direct SLAM methodologies by coupling thermal-infrared cameras with sparse LiDAR data, maintaining 14-bit raw thermal data without conversion to traditional 8-bit formats. This approach circumvents the heuristics involved in preprocessing conversions, thereby mitigating potential degradation due to photometric inaccuracies inherent in adjusted 8-bit thermal images.

The method focuses on core components; it ensures robustness in 6-DOF estimation by directly tracking sparse depth measurements on a raw thermal image. The introduction of loop closure for comprehensive global consistency solidifies positions temporally, accounting also for thermographic variations potentially induced by environmental shifts. A key addition is the automatic calibration method between the thermal camera and LiDAR, ensuring minimal user intervention and optimally utilizes time-based data streams for enhanced extrinsic parameter precision.

For tracking, DEDT-SLAM applies a thermographic residual error model on the 14-bit image, using patches to collect robust data even amid signal noise or motion blur. Despite computational demands, this improves model resilience significantly. Local accuracy is enhanced by employing recent keyframes for refinement; these are processed via map-based coordinates, overcoming immediate temporal drifts.

The implications of this research are substantial both practically and theoretically within the field of applied visual SLAM. By expanding SLAM applications beyond visible light parameters, it optimizes autonomous navigation systems struggling in complex visual environments. In advancing this framework, further research could explore more intricate sensor integrations, improving environmental adaptability and potentially extending usage to realms such as complete darkness scenarios or environments fraught with unpredictable visual dynamics.

Experimental results affirm the robustness of this methodology against ORB-SLAM and traditional SLAM approaches under both daylight and nocturnal sequences, with the novel system consistently outperforming in both accuracy and practical applicability. The provision for 3D thermographic mapping envisages new vistas in visual SLAM applications, embedding environmental temperature readings into the spatial mapping continuum.

Sparse Depth Enhanced Direct Thermal-infrared SLAM not only fills the traditional gaps in SLAM applications, aligning more robustly with real-world navigational demands, but also opens modalities for future explorations in autonomous system perceptions across varied environments.