Evaluating geometric accuracy of NeRF reconstructions compared to SLAM method (2407.11238v2)

Abstract: As Neural Radiance Field (NeRF) implementations become faster, more efficient and accurate, their applicability to real world mapping tasks becomes more accessible. Traditionally, 3D mapping, or scene reconstruction, has relied on expensive LiDAR sensing. Photogrammetry can perform image-based 3D reconstruction but is computationally expensive and requires extremely dense image representation to recover complex geometry and photorealism. NeRFs perform 3D scene reconstruction by training a neural network on sparse image and pose data, achieving superior results to photogrammetry with less input data. This paper presents an evaluation of two NeRF scene reconstructions for the purpose of estimating the diameter of a vertical PVC cylinder. One of these are trained on commodity iPhone data and the other is trained on robot-sourced imagery and poses. This neural-geometry is compared to state-of-the-art lidar-inertial SLAM in terms of scene noise and metric-accuracy.

• The paper demonstrates that NeRF reconstructions, especially using iPhone imagery, can reduce noise comparably to LiDAR-based SLAM methods.
• It employs a controlled experiment with a vertical PVC cylinder, analyzing metrics like PSNR, SSIM, and LPIPS to assess reconstruction quality.
• Findings indicate that while SLAM offers more precise diameter estimates, NeRF provides a cost-effective alternative for accessible 3D mapping.

Evaluating Geometric Accuracy of NeRF Reconstructions Compared to SLAM Method

The paper "Evaluating geometric accuracy of NeRF reconstructions compared to SLAM method" presents a comparative analysis between Neural Radiance Fields (NeRF) and Simultaneous Localization and Mapping (SLAM) methods, focusing on their geometric accuracy in 3D scene reconstructions. The paper aims to highlight NeRF's potential as a cost-effective and efficient alternative to traditional LiDAR-based 3D mapping systems.

Introduction and Context

3D mapping has significant applications in fields such as construction, urban planning, and forestry. Traditionally, LiDAR sensors have dominated this area due to their high precision in measuring distances and creating detailed point clouds. However, LiDAR systems are expensive, often ranging from $80,000 to$120,000, limiting their accessibility. SLAM algorithms have ameliorated some issues by aligning numerous LiDAR scans but still require costly hardware.

NeRFs offer a substantial shift by leveraging neural networks trained on sparse image sets to reconstruct 3D scenes. Unlike photogrammetry, which necessitates dense image datasets and complex computations, NeRFs achieve superior results with fewer inputs. This paper focuses on the geometric accuracy of NeRF in real-world applications, specifically in comparing its performance to state-of-the-art SLAM-based LiDAR systems.

Methodology

The experiment revolves around reconstructing a vertical PVC cylinder of 40 cm diameter and three meters height situated in an outdoor courtyard. Three different methods were employed:

1. NeRF trained on iPhone 14 imagery: Using a free iOS application (NeRFCapture) for camera poses.
2. NeRF trained on robot-collected imagery: Poses derived from LiDAR-inertial SLAM (LIOSAM).
3. State-of-the-art LiDAR-inertial SLAM (LIOSAM): Using a Unitree B1 quadruped robot equipped with an Ouster OS0-128 LiDAR and Inertialsense IMX-5 IMU.

Each method's output was evaluated based on the quality and noise of the point clouds, as well as the accuracy of the reconstructed cylinder's diameter.

Results

The paper's results are summarized in Table 1, which presents the metrics for both NeRF reconstructions and SLAM. Key observations include:

• The iPhone NeRF method outperformed the robot NeRF in terms of Peak Signal-to-Noise Ratio (PSNR) and Learned Perceptual Image Patch Similarity (LPIPS), although the robot NeRF showed better Structural Similarity (SSIM).
• The SLAM point cloud exhibited more significant deviation due to scan misalignment, likely caused by suboptimal LiDAR-IMU calibration.
• Despite higher observed noise levels, the SLAM method produced the most accurate diameter estimate (within 5 mm). The iPhone and robot NeRF estimates were within a 2.2-2.7 cm margin (5.68-7.12% error).

Visual inspection of the point clouds further illustrated these findings, with the NeRF reconstructions showing less noise deviation compared to the SLAM reconstruction.

Discussion and Implications

The numerical results suggest that NeRF-based methods can indeed achieve metric relevance comparable to state-of-the-art SLAM methods, albeit with some limitations in diameter accuracy. The reduced noise levels in NeRF reconstructions highlight one of its salient advantages—mitigation of complex sensor alignment processes inherent in LiDAR-based approaches.

The practical implications are significant. The ability to use commodity mobile devices for detailed 3D reconstructions democratizes access to high-quality mapping solutions, potentially impacting fields like forestry, urban planning, and real-time environmental monitoring. Moreover, integrating NeRF with existing SLAM systems could enhance mapping accuracy by reducing accumulated errors through inference models.

Future Directions

Future research could explore several avenues:

• Data Collection Consistency: Evaluating the consistency of NeRF and SLAM methods across diverse environments and object types.
• Integration with SLAM: Augmenting SLAM pipelines with NeRF components to balance data-driven accuracy with sensor-fusion capabilities.
• NeRF Optimization: Improving NeRF's algorithm for better pose estimation and exposure control, especially under challenging lighting conditions.

Conclusion

This investigation underscores NeRF's capability as an accessible and accurate alternative to traditional LiDAR-based 3D mapping systems. While there are areas where SLAM still holds an edge, particularly in precise diameter measurements, NeRF's advantages in noise reduction and cost-efficiency cannot be overlooked. The findings pave the way for broader application of neural rendering in practical, large-scale mapping tasks.