VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold (2505.12549v2)

Abstract: We present VGGT-SLAM, a dense RGB SLAM system constructed by incrementally and globally aligning submaps created from the feed-forward scene reconstruction approach VGGT using only uncalibrated monocular cameras. While related works align submaps using similarity transforms (i.e., translation, rotation, and scale), we show that such approaches are inadequate in the case of uncalibrated cameras. In particular, we revisit the idea of reconstruction ambiguity, where given a set of uncalibrated cameras with no assumption on the camera motion or scene structure, the scene can only be reconstructed up to a 15-degrees-of-freedom projective transformation of the true geometry. This inspires us to recover a consistent scene reconstruction across submaps by optimizing over the SL(4) manifold, thus estimating 15-degrees-of-freedom homography transforms between sequential submaps while accounting for potential loop closure constraints. As verified by extensive experiments, we demonstrate that VGGT-SLAM achieves improved map quality using long video sequences that are infeasible for VGGT due to its high GPU requirements.

VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold

The paper introduces VGGT-SLAM, a novel approach designed to enhance RGB Simultaneous Localization and Mapping (SLAM) specifically for uncalibrated, monocular setups. The innovation hinges upon leveraging both incremental and holistic alignment of submaps within SLAM operations while utilizing only monocular cameras without needing pre-calibrated data. This facilitates recovery of a consistent scene reconstruction across submaps by optimizing over the SL(4) manifold, thus highlighting the inadequacies of solely relying on similarity transforms.

Core Concepts and Contributions

Feed-forward Reconstruction Paradigm:

The architecture of VGGT-SLAM is heavily reliant upon feed-forward networks like VGGT, which generate point clouds from uncalibrated input images. Previous models, such as DUSt3R, would handle pairs of images, but VGGT branches out to process arbitrary numbers of frames, creating an unprecedented dense reconstruction.

Projective Ambiguity Resolution:

A significant stride in this paper is the addressing of reconstruction ambiguity. Generally, uncalibrated monocular setups can only reconstruct up to a 15-degrees-of-freedom transform of the true scene structure. VGGT-SLAM successfully mitigates this by optimizing over the SL(4) manifold, thus enabling improved transformations which include shear, stretch, and perspective adjustments.

Global Optimization with Loop Closures:

Integral elements include efficient submap generation and incorporation of loop closures for ensuring consistent global scene mapping. This is undertaken by estimating homographic transforms between sequential submaps, incorporating loop constraints which aid in rectifying potential drifts.

Technical Evaluation

Quantitative evaluations reveal that VGGT-SLAM competes admirably against state-of-the-art methods like DROID-SLAM and MASt3R-SLAM, particularly across benchmarks such as 7-Scenes and the TUM RGB-D datasets. This includes promising results in dense reconstruction as gauged through metrics like accuracy, completion, and Chamfer distance.

Despite its prowess, VGGT-SLAM encounters challenges like potential degeneracy situations where planar scenes induce ambiguous transformations. Such instances merit future investigative efforts, especially in enhancing robustness through alternative strategies like ray-based matching akin to MASt3R-SLAM’s approach.

Implications and Future Directions

VGGT-SLAM propels forward the application potential of SLAM systems in scenarios where camera calibration is unavailable, making it particularly valuable for real-time operations in dynamic environments. The introduction of optimization on SL(4) spearheads navigating the nuances of uncalibrated setups while offering a promising framework for handling projective transformations which similarity transforms cannot resolve adequately.

Future studies can explore ways to harmonize SLAM operations within scenarios showcasing projective ambiguity versus scenarios where only similarity transformations suffice. There are also prospects in refining computational efficiencies or enhancing resistance to adversarial depth measurement outliers, which currently challenge homography estimations.

Conclusion

The VGGT-SLAM framework offers substantial advancements in dense SLAM with uncalibrated monocular feeds, optimizing submap alignment through innovative use of the SL(4) manifold. This presents a new paradigm within the discourse of visual SLAM applications and underpins future explorations in expanding scalability and enhancing robustness of SLAM methodologies within real-world applications.