ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals (1905.02082v3)

Abstract: Mapping and localization are essential capabilities of robotic systems. Although the majority of mapping systems focus on static environments, the deployment in real-world situations requires them to handle dynamic objects. In this paper, we propose an approach for an RGB-D sensor that is able to consistently map scenes containing multiple dynamic elements. For localization and mapping, we employ an efficient direct tracking on the truncated signed distance function (TSDF) and leverage color information encoded in the TSDF to estimate the pose of the sensor. The TSDF is efficiently represented using voxel hashing, with most computations parallelized on a GPU. For detecting dynamics, we exploit the residuals obtained after an initial registration, together with the explicit modeling of free space in the model. We evaluate our approach on existing datasets, and provide a new dataset showing highly dynamic scenes. These experiments show that our approach often surpass other state-of-the-art dense SLAM methods. We make available our dataset with the ground truth for both the trajectory of the RGB-D sensor obtained by a motion capture system and the model of the static environment using a high-precision terrestrial laser scanner. Finally, we release our approach as open source code.

• The paper introduces ReFusion, a novel TSDF‐based SLAM method that uses registration residuals to robustly detect and exclude dynamic elements.
• It employs a point-to-implicit alignment leveraging depth and color cues to optimize camera pose tracking while dynamically carving free space.
• Experimental evaluations show improved trajectory accuracy over existing methods, particularly in challenging scenarios with complex dynamic environments.

An Expert Review of "ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals"

The paper "ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals" presents a novel approach aimed at enhancing simultaneous localization and mapping (SLAM) in dynamic environments using RGB-D cameras. The authors propose ReFusion, a dense SLAM methodology that employs a truncated signed distance function (TSDF) representation to reconstruct static 3D models while effectively handling dynamic elements without reliance on deep neural networks or pre-defined semantic segmentation.

The primary challenge addressed is the inherent difficulty SLAM systems face in dynamic settings where conventional methods fail due to incorrect pose estimation caused by moving objects. ReFusion effectively tackles this challenge by introducing geometric filtering techniques that leverage registration residuals and dynamically manage free space within the scene. This approach is contrasted against existing methods which either disregard dynamic components altogether or employ computationally intensive semantic segmentation frameworks.

Methodology

ReFusion uses a voxel-based TSDF which captures the signed distance from a surface within each voxel, enabling implicit modeling of the environment. The proposed system efficiently computes registration residuals while avoiding the dependency on depth or key point features, thereby supporting robust camera pose tracking in environments with dynamic interactions.

Key steps in the ReFusion pipeline involve:

1. Pose Estimation: Utilizes a point-to-implicit approach where incoming depth information is directly aligned with the SDF, optimized through a joint error function incorporating both depth and color cues.
2. Residual-Based Dynamics Detection: A threshold-based mechanism determines dynamic scene components by analyzing registration residuals, forming a basis for identifying transient objects.
3. Integration and Carving: Adjusts the TSDF model by discarding measurements from dynamic objects and updates regions classified as free space, ensuring further robustness.

Experimental Evaluation

The effectiveness of ReFusion is substantiated through rigorous testing on both the established TUM RGB-D dataset and a newly compiled Bonn RGB-D dynamic dataset. ReFusion consistently demonstrates performance at least on par with or surpassing contemporary dense SLAM systems like StaticFusion and DynaSLAM in terms of absolute trajectory error (ATE) across a range of dynamic scenarios. Importantly, the evaluation illustrates that the proposed geometric filtering is exceptionally effective against false-positive inclusions of dynamic elements, thus maintaining the accuracy of the static environment model.

Specifically, the analysis reveals:

• The alignment of depth and color information in the absence of deep learning classifiers facilitates robust trajectory estimation;
• The ability to dynamically carve and manage free space contributes to the integrity of the reconstructed model;
• A notable accuracy in static background quantification, outperforming alternatives in geometrically complex settings.

Implications and Future Directions

The implications of ReFusion are significant for robotics operating in unpredictable and dynamic settings, such as autonomous vehicles or drones navigating dense, urban spaces. By maintaining high fidelity models that exclude transient objects, ReFusion enhances the dependability of robotic perception and planning systems.

Further theoretical developments could explore hybrid approaches that marry this geometric-based filtering with lightweight machine learning models, potentially improving resilience to a wider array of textures and shapes.

Ultimately, the ReFusion framework signifies a considerable contribution to the advancement of SLAM technology, promoting more robust and adaptable robotic systems in dynamic real-world environments. The publicly available dataset and source code offer a platform for advancing SLAM research endeavors, encouraging further refinement and innovation in the field.