SwiftNDC: Fast Neural Depth Correction for High-Fidelity 3D Reconstruction

Abstract: Depth-guided 3D reconstruction has gained popularity as a fast alternative to optimization-heavy approaches, yet existing methods still suffer from scale drift, multi-view inconsistencies, and the need for substantial refinement to achieve high-fidelity geometry. Here, we propose SwiftNDC, a fast and general framework built around a Neural Depth Correction field that produces cross-view consistent depth maps. From these refined depths, we generate a dense point cloud through back-projection and robust reprojection-error filtering, obtaining a clean and uniformly distributed geometric initialization for downstream reconstruction. This reliable dense geometry substantially accelerates 3D Gaussian Splatting (3DGS) for mesh reconstruction, enabling high-quality surfaces with significantly fewer optimization iterations. For novel-view synthesis, SwiftNDC can also improve 3DGS rendering quality, highlighting the benefits of strong geometric initialization. We conduct a comprehensive study across five datasets, including two for mesh reconstruction, as well as three for novel-view synthesis. SwiftNDC consistently reduces running time for accurate mesh reconstruction and boosts rendering fidelity for view synthesis, demonstrating the effectiveness of combining neural depth refinement with robust geometric initialization for high-fidelity and efficient 3D reconstruction.

• The paper introduces a neural depth correction field that refines monocular and multi-view depth estimates using a dual-stage (global then local) training approach.
• It leverages affine alignment with sparse SfM points and rigorous outlier filtering to achieve sub-millimeter accuracy in mesh reconstruction and improved novel-view synthesis metrics.
• The approach reduces optimization iterations in 3D Gaussian Splatting, offering scalable, real-time 3D asset generation for applications in robotics and computer vision.

SwiftNDC: Fast Neural Depth Correction for High-Fidelity 3D Reconstruction

Motivation and Context

Depth-guided 3D reconstruction is increasingly favored for its efficiency compared to optimization-heavy radiance-field methods, yet persistent issues—scale drift, multi-view inconsistencies, and the need for intensive refinement—block the achievement of high-fidelity geometry. SwiftNDC introduces a paradigm that leverages a neural depth correction field to produce geometrically reliable and cross-view-consistent depth maps, subsequently generating dense point clouds for robust mesh reconstruction and superior novel-view synthesis. This approach bridges the gap between feed-forward depth pipelines and radiance field optimization-centric methods, providing both speed and accuracy.

Methodological Contributions

SwiftNDC's pipeline integrates monocular and multi-view dense depth estimates for each image, aligns them via affine fitting to sparse structure-from-motion (SfM) points, and applies a lightweight, pixel-level neural correction field for further refinement. The corrected depths are back-projected into dense point clouds, filtered via rigorous reprojection-error analysis across views to guarantee geometric consistency. This dense point cloud initialization ensures fewer optimization steps are required for downstream mesh reconstruction in 3D Gaussian Splatting (3DGS), and also substantially enhances 3DGS-based view synthesis quality.

Figure 1: SwiftNDC pipeline: from raw depth maps to dense, corrected point clouds employed for both mesh reconstruction and 3DGS-based view synthesis.

The neural depth correction field operates as a compact, scene-specific MLP, trained using sparse SfM anchors to minimize pixel-wise depth bias. Training is amortized in a global-then-local schedule: first, a scene-level correction is learned to capture systematic bias; then, per-image fine-tuning quickly eliminates local deviations. This procedure supports rapid depth refinement, scalable to scenes with moderate view counts (<60 images).

Quantitative and Qualitative Results

Extensive experiments across five datasets—including DTU and Tanks and Temples for mesh reconstruction and MipNeRF360, Tanks and Temples, and Deep Blending for view synthesis—demonstrate that SwiftNDC consistently outperforms state-of-the-art methods in both runtime and fidelity. Mesh reconstruction using SwiftNDC achieves comparable or superior accuracy to leading neural implicit and 3DGS-based pipelines, with mean Chamfer Distance (CD) errors in the sub-millimeter range (0.75 mm with no 3DGS, 0.59 mm with 1k 3DGS steps on DTU) while running orders of magnitude faster (as little as one minute per scene).

SwiftNDC's dense geometry initialization notably reduces the required optimization iterations for 3DGS, facilitating mesh extraction and rendering in sparse and weakly observed regions.

Figure 2: Depth accuracy directly drives surface quality; SwiftNDC achieves sub-centimeter residuals versus persistent multi-view inconsistencies in previous methods, producing visually smooth and metrically accurate meshes.

For novel-view synthesis, SwiftNDC's initialization consistently boosts metrics such as PSNR, SSIM, and LPIPS across benchmarks, outperforming both NeRF and 3DGS variants initialized from sparse SfM (e.g., PSNR 29.33 on MipNeRF360, LPIPS 0.19).

Ablation Analysis

Ablation studies confirm that every pipeline component—global multi-view depth, monocular fine detail, pixel-level correction, and reliable dense geometry initialization—contributes significantly to the final accuracy and efficiency. Dual-depth synergy yields a 25% accuracy gain relative to single-modality correction. Omitting pixel-level correction or robust outlier filtering causes substantial degradation in mesh quality and runtime. The two-stage training (global, then local) amortizes computation, preserving fidelity while slashing optimization time by an order of magnitude.

Practical and Theoretical Implications

SwiftNDC's approach demonstrates substantial advantages for real-time applications, scalable 3D asset generation, and robotics. The drop-in nature of its dense geometric prior facilitates integration with existing splatting pipelines, accelerating convergence and improving output quality without extensive hyperparameter tuning. The theoretical implication is the validation of pixel-level neural correction fields as a powerful mechanism for resolving local geometric noise, pushing the boundaries of rapid, reliable geometry extraction from feed-forward estimates.

Further, the empirical findings suggest that reliable dense geometry initialization reduces the need for iterative regularization and surface consensus checking typical in radiance-field approaches, potentially transforming the architecture of future hybrid 3D representation learning systems.

Limitations and Future Directions

SwiftNDC's dependencies include accurate camera poses and solid initial monocular/MVS depth estimation. Although processing time remains sub-minute for moderate-scale scenes, preprocessing may become a bottleneck in very large datasets. Currently, depth correction and radiance-field optimization remain separate; joint end-to-end approaches may further enhance fidelity and efficiency.

There is clear future research potential in end-to-end systems coupling neural depth correction directly with radiance-field optimization, as well as broader adaptation to scene collections with thousands of views or challenging indoor/outdoor transitions.

Conclusion

SwiftNDC establishes a robust and efficient framework for high-fidelity 3D reconstruction and novel-view synthesis. By converting sparse SfM points into a dense, pixel-accurate geometric prior—through global multi-view reasoning, monocular detail, and rigorous filtering—the method offers significant runtime reductions and quality improvements for mesh and radiance-field pipelines. With broad practical compatibility and theoretical significance, SwiftNDC marks a substantial advancement in the automation and scalability of dense geometry and radiance learning in computer vision.