SwiftNDC: Fast Depth-Guided 3D Reconstruction

• SwiftNDC is a depth-guided 3D reconstruction framework that uses a Neural Depth Correction field to ensure cross-view depth consistency.
• It employs a three-stage pipeline: per-view depth estimation and alignment, pixel-level neural correction, and robust dense point-cloud generation.
• The approach accelerates downstream mesh extraction and novel-view synthesis by reducing optimization iterations while improving geometric fidelity.

Searching arXiv for the SwiftNDC paper and closely related referenced methods (3DGS, SuGaR, Neuralangelo, Splatfacto, PGSR) to ground citations. SwiftNDC is a framework for depth-guided 3D reconstruction that centers on a Neural Depth Correction field designed to produce cross-view consistent depth maps and, from them, a dense point cloud suitable for downstream reconstruction. The method is presented as a response to depth-guided pipelines that remain fast but still exhibit scale drift, multi-view inconsistencies, and substantial refinement requirements before high-fidelity geometry is obtained. Its stated contribution is a three-stage workflow—per-view depth estimation and coarse alignment, pixel-level neural depth correction, and dense point-cloud construction with robust filtering—that accelerates mesh reconstruction and improves novel-view synthesis by providing a reliable dense geometric initialization (Han et al., 26 Feb 2026).

1. Problem setting and methodological rationale

SwiftNDC is motivated by three failure modes identified in existing feed-forward depth-guided 3D reconstruction pipelines: scale drift, cross-view inconsistencies, and over-smoothing or holes. In the formulation given for the method, scale drift arises because per-image or per-patch depth predictions have unknown or drifting metric scale; cross-view inconsistencies arise because local biases in each depth map cause misalignment when fusing multiple views; and over-smoothing or holes arise because learning-based MVS often blurs fine edges, while classical TSDF fusion of noisy depths yields ripples, pinholes, and holes (Han et al., 26 Feb 2026).

The method is also situated against optimization-heavy radiance-field approaches. NeRF, SuGaR, and 3DGS are described as delivering high fidelity but requiring hours of scene-specific optimization to recover accurate geometry, which motivates a fast, learnable correction mechanism operating on feed-forward depth maps rather than replacing them outright (Han et al., 26 Feb 2026). This framing makes SwiftNDC a hybrid system: it retains learned depth estimation as the source of dense geometric cues, but inserts a scene-specific correction stage and a geometric filtering stage before radiance-field or mesh-based downstream reconstruction.

The central design requirement is explicitly threefold: align and refine feed-forward depth maps at pixel resolution, enforce multi-view geometric consistency, and provide a high-quality dense initialization that reduces downstream optimization time. This suggests that SwiftNDC should be understood less as a standalone reconstructor than as a geometry-conditioning layer for reconstruction pipelines that otherwise depend on lengthy optimization.

2. End-to-end pipeline

The framework is organized into three broad stages: per-view depth estimation and coarse alignment; pixel-level neural depth correction; and dense point-cloud construction, robust filtering, and downstream 3DGS or mesh extraction (Han et al., 26 Feb 2026). The pipeline begins from calibrated images $\{I_i\}$, COLMAP poses $(K_i, R_i, t_i)$, and a sparse SfM point cloud $P=\{x_k\}$.

Two complementary depth sources are used at input. VGGT provides globally consistent MVS depths $D_i^{vggt}$, while VDA provides detailed monocular depths $D_i^{mono}$ (Han et al., 26 Feb 2026). For each view $i$, SwiftNDC fits per-view affine transforms $(s_t, b_t)$ to align both depth sources to sparse COLMAP depths at projected sparse points:

$$\min_{s,b}\sum_k\bigl(s\,D_i^t(p_{ik})+b - d_{ik}^{\mathrm{col}}\bigr)^2,\quad t\in\{\mathrm{vggt},\mathrm{mono}\}.$$

The resulting affine-aligned depths are denoted $\widetilde D_i^{v}$ and $\widetilde D_i^{m}$ (Han et al., 26 Feb 2026).

A lightweight neural depth-correction field is then trained per scene and fine-tuned per view, yielding corrected maps $(K_i, R_i, t_i)$0 (Han et al., 26 Feb 2026). Every pixel in $(K_i, R_i, t_i)$1 is back-projected into 3D,

$(K_i, R_i, t_i)$2

after which multi-view reprojection-error filtering removes outliers. The retained points are uniformly downsampled to form a clean dense point set. That point set is used either for TSDF plus Marching Cubes mesh extraction or as initialization for 3D Gaussian Splatting, with the latter requiring only $(K_i, R_i, t_i)$3–$(K_i, R_i, t_i)$4 iterations rather than tens of thousands (Han et al., 26 Feb 2026).

The pipeline’s structure reflects a deliberate separation of concerns. VGGT contributes global consistency, VDA contributes local detail, sparse COLMAP points supply geometric supervision, and the correction field reconciles these signals into a depth representation intended for cross-view fusion.

3. Neural Depth Correction field

The Neural Depth Correction field is the core learned component. For each sparse anchor $(K_i, R_i, t_i)$5 in view $(K_i, R_i, t_i)$6, the model takes as input

$(K_i, R_i, t_i)$7

where $(K_i, R_i, t_i)$8 and $(K_i, R_i, t_i)$9 are affine-aligned depths, $P=\{x_k\}$0 are normalized pixel coordinates, and $P=\{x_k\}$1 is the normalized view index (Han et al., 26 Feb 2026). A sinusoidal positional encoding $P=\{x_k\}$2 is applied, and the encoded vector is passed to an MLP $P=\{x_k\}$3 with six hidden layers of width $P=\{x_k\}$4 and ReLU activations.

The network outputs four scalars,

$P=\{x_k\}$5

which parameterize corrected anchor depths as

$P=\{x_k\}$6

The method also states that the correction can be viewed as a residual field $P=\{x_k\}$7 such that, for any pixel $P=\{x_k\}$8,

$P=\{x_k\}$9

where $D_i^{vggt}$0 is the input affine-aligned depth (Han et al., 26 Feb 2026).

Only sparse COLMAP depths are used for supervision, through an $D_i^{vggt}$1 reprojection loss:

$D_i^{vggt}$2

Training follows a two-stage schedule: a global stage optimizing $D_i^{vggt}$3 over all views for $D_i^{vggt}$4 steps of AdamW, followed by a local stage that fine-tunes per view from the global optimum $D_i^{vggt}$5 for $D_i^{vggt}$6 steps (Han et al., 26 Feb 2026).

Within the paper’s own interpretation, this schedule enforces multi-view consistency through the global objective and local refinement through the per-view pass. The exponential parameterization and the small MLP are described as acting as an implicit regularizer. A plausible implication is that SwiftNDC trades model capacity for stability and scene-specific adaptation rather than attempting large-scale depth inference from scratch.

4. Dense geometry generation and robust filtering

After correction, SwiftNDC converts the corrected depth maps into dense geometry by back-projection:

$D_i^{vggt}$7

This is followed by a reprojection-error filtering stage intended to enforce geometric reliability across views (Han et al., 26 Feb 2026). For each candidate point $D_i^{vggt}$8, a neighbor view $D_i^{vggt}$9 is selected; the point is projected to that view as $D_i^{mono}$0; the corrected depth $D_i^{mono}$1 is sampled and back-projected to $D_i^{mono}$2; and this point is then reprojected back into view $D_i^{mono}$3 to obtain $D_i^{mono}$4.

The reprojection error is defined as

$D_i^{mono}$5

Points are discarded if their average $D_i^{mono}$6 over a small set of neighbors exceeds $D_i^{mono}$7 px (Han et al., 26 Feb 2026). Uniform downsampling is then applied to produce what the method describes as a well-distributed, reliable dense point set.

This point set is significant because it is treated as the final geometry initialization for all downstream stages. The paper emphasizes that the initialization is both clean and uniformly distributed, which is presented as the reason subsequent optimization can be reduced substantially. In this sense, the filtering stage is not merely a cleanup heuristic; it is the mechanism by which corrected depths are converted into a representation usable by both surface extraction and Gaussian-based rendering pipelines.

5. Integration with downstream reconstruction and rendering

SwiftNDC supports two downstream uses of its dense initialization. The first is direct mesh extraction via TSDF plus Marching Cubes. The second is initialization for 3D Gaussian Splatting, in which each Gaussian center is placed at a filtered 3D point, an appropriate covariance is set, and initial color estimates are assigned from the nearest image (Han et al., 26 Feb 2026). Because the geometry is already accurate and uniformly sampled, the optimization is restricted to Gaussian opacity and radiance parameters under a standard photometric loss.

The reported implication is a substantial reduction in 3DGS optimization length. The method states that high-fidelity meshes can be obtained in as few as $D_i^{mono}$8 Gauss-splatting iterations on DTU or $D_i^{mono}$9 iterations on Tanks & Temples, compared with $i$0 in baselines (Han et al., 26 Feb 2026). For novel-view synthesis, the dense initialization is inserted into existing 3DGS-based view-synthesis pipelines such as Splatfacto, with no changes to the rendering loss; only the Gaussian initialization differs (Han et al., 26 Feb 2026). The stated rationale is that occluded and weakly observed regions are seeded with better geometry, allowing optimization to converge to higher PSNR and SSIM and lower LPIPS, especially in grazing-angle and under-sampled regions.

The framework therefore occupies an intermediate position between depth-prediction systems and radiance-field optimizers. It does not modify the downstream rendering objective for novel-view synthesis, and it does not replace 3DGS; rather, it changes the initial conditions under which 3DGS operates. This suggests that the principal contribution lies in geometric conditioning rather than in radiance modeling.

6. Empirical results, scope, and limitations

SwiftNDC reports experiments across five datasets, including two for mesh reconstruction and three for novel-view synthesis (Han et al., 26 Feb 2026). The mesh reconstruction results on DTU and Tanks & Temples are given explicitly, as are representative novel-view synthesis results on MipNeRF 360, with additional improvements noted on Tanks & Temples and Deep Blending.

Setting	Reported result	Runtime
DTU, “Ours-Depth”	mean Chamfer Dist = 0.75 mm	1 min
DTU, “Ours-3DGS”	0.59 mm	3 min
DTU, PGSR baseline	0.53 mm	30 min
Tanks & Temples	mean F1 = 0.50	26 min

For DTU, “Ours-Depth” denotes TSDF plus Marching Cubes on $i$1, and “Ours-3DGS” denotes the same geometry followed by $i$2 Gaussian iterations (Han et al., 26 Feb 2026). On Tanks & Temples, the paper states that the method attains $i$3 in $i$4 minutes, matching the best baselines, identified as Neuralangelo and PGSR, at half the runtime (Han et al., 26 Feb 2026).

For novel-view synthesis on MipNeRF 360, the dense initialization plus Splatfacto achieves $i$5 versus $i$6, $i$7 versus $i$8, and $i$9 versus $(s_t, b_t)$0 (Han et al., 26 Feb 2026). On Tanks & Temples and Deep Blending, the method reports improvements in all three metrics by approximately $(s_t, b_t)$1–$(s_t, b_t)$2 dB in PSNR and $(s_t, b_t)$3–$(s_t, b_t)$4 in SSIM, while lowering LPIPS by approximately $(s_t, b_t)$5–$(s_t, b_t)$6 (Han et al., 26 Feb 2026).

The paper also identifies several limitations. SwiftNDC depends on accurate SfM poses and on the baseline quality of VGGT and VDA depths. Additional preprocessing, specifically depth correction and reprojection filtering, can become non-negligible for very large-scale collections involving thousands of views. The method further notes that depth correction and radiance-field optimization remain decoupled, and suggests that a joint, end-to-end scheme could further boost fidelity (Han et al., 26 Feb 2026). Future work is said to include integrating per-pixel uncertainty into 3DGS weighting and co-training the MLP and 3DGS in a single loop.

These limitations are important for interpreting the method’s scope. SwiftNDC does not remove the need for high-quality camera calibration or robust upstream depth estimates, and it does not yet formulate a unified optimization over corrected depth and radiance-field parameters. A plausible implication is that its strongest use case is as a scene-level accelerator and stabilizer for pipelines that already possess reliable poses and feed-forward depth estimates, rather than as a general replacement for structure-from-motion or multi-view stereo.