Fusion++: Volumetric Object-Level SLAM (1808.08378v2)

Abstract: We propose an online object-level SLAM system which builds a persistent and accurate 3D graph map of arbitrary reconstructed objects. As an RGB-D camera browses a cluttered indoor scene, Mask-RCNN instance segmentations are used to initialise compact per-object Truncated Signed Distance Function (TSDF) reconstructions with object size-dependent resolutions and a novel 3D foreground mask. Reconstructed objects are stored in an optimisable 6DoF pose graph which is our only persistent map representation. Objects are incrementally refined via depth fusion, and are used for tracking, relocalisation and loop closure detection. Loop closures cause adjustments in the relative pose estimates of object instances, but no intra-object warping. Each object also carries semantic information which is refined over time and an existence probability to account for spurious instance predictions. We demonstrate our approach on a hand-held RGB-D sequence from a cluttered office scene with a large number and variety of object instances, highlighting how the system closes loops and makes good use of existing objects on repeated loops. We quantitatively evaluate the trajectory error of our system against a baseline approach on the RGB-D SLAM benchmark, and qualitatively compare reconstruction quality of discovered objects on the YCB video dataset. Performance evaluation shows our approach is highly memory efficient and runs online at 4-8Hz (excluding relocalisation) despite not being optimised at the software level.

• The paper introduces an object-level SLAM framework that reconstructs 3D objects using distinct volumetric TSDF models.
• It integrates Mask R-CNN instance masks to fuse semantic labeling with efficient volumetric mapping and pose graph optimization.
• Experimental results show robust loop closure and memory efficiency on indoor benchmarks, paving the way for dynamic scene applications.

Analysis of "Fusion++: Volumetric Object-Level SLAM"

The paper under review, "Fusion++: Volumetric Object-Level SLAM," authored by researchers affiliated with Imperial College London, introduces an innovative approach to Simultaneous Localization and Mapping (SLAM) with a focus on object-level volumetric representation. This work is noteworthy for several reasons, particularly its attempt to bridge the gap between map representation and semantic scene understanding using RGB-D data, which is critical for practical deployments in robotic applications.

Core Contributions

The authors present a SLAM system that reconstructs object instances as distinct volumetric models in 3D space, diverging from more conventional dense reconstruction methods. The system's core features and contributions are summarized as follows:

• Object-Oriented Mapping: Unlike traditional dense SLAM approaches, this system focuses on object-instance mapping using rigid Truncated Signed Distance Function (TSDF) volumes. This allows the system to exclude free space, thereby enhancing memory efficiency and potential reconstruction fidelity.
• Integration of Instance Masks: The system employs Mask R-CNN to obtain 2D instance masks, which are subsequently fused into a 3D object-centric representation. This technique not only offers semantic labeling but also integrates volumetric reconstruction.
• Pose Graph Optimization: By incorporating detected objects as landmarks within a pose graph framework, the system enhances loop closure capabilities and global consistency. An innovative strategy for information matrix computation ensures robust pose graph optimization.
• Scene Dynamics and Tracking: While the system assumes a static environment, its architecture naturally paves the way for future advancements toward dynamic scene understanding and real-time applicability.

Evaluation and Results

The authors conduct various experiments to evaluate the system, including:

• Memory and Efficiency: The proposed system benefits from a highly efficient memory utilization strategy due to its object-centric map. The researchers demonstrate the feasibility of maintaining up to 2,500 object volumes without significantly compromising run-time performance and memory constraints, with an operational capability of 4-8Hz pipelines on provided sequences.
• Benchmark Validation: Utilizing sequences from the established RGB-D SLAM Benchmark, the authors illustrate improved trajectory accuracy over a baseline coarse TSDF odometry approach. Notably, robust loop closure capabilities and enhanced error metrics reinforce the system's efficacy in complex indoor scenarios.
• Reconstruction Quality: On the YCB video dataset, the system exhibits competitive reconstruction quality compared to ground truth models, highlighting the effective capture of object surfaces and geometry despite challenges presented by occlusions and detection misses.

Implications and Future Work

"Fusion++: Volumetric Object-Level SLAM" offers substantial contributions to advancing object-oriented SLAM methodologies, particularly in indoor environments. The object-level representation paradigm provides a rich semantic layer that is highly relevant to enhancing autonomous robotic operations, human-robot interaction, and augmented reality applications.

The proposed SLAM method brings to the forefront the practical implications of integrating scene semantics into SLAM, emphasizing the need for scalable, memory-efficient solutions. The separation of scene elements into object-level volumes allows for improved handling of occlusions and relocalization tasks.

Potential future directions suggested by this research include the incorporation of dynamic object tracking, which would further align the system with real-world applications where scene elements are seldom static. The integration of more sophisticated object models, potentially enhanced by databases such as ShapeNet, could significantly refine the system's reconstruction and semantic mapping capabilities.

In summary, the paper presents a methodically sound and technically advanced approach to SLAM, offering meaningful advancements in semantic object reconstruction and memory efficiency. Future expansions on this foundation could greatly enhance deployment across varying domains requiring detailed and dynamic environmental awareness.