Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes

Abstract: We introduce S2C-3D, a novel sparse-view 3D reconstruction framework for high-fidelity and complete scene reconstruction from as few as six to eight images. Our framework features three components: a specialized diffusion model for scene-specific image restoration, a training-free view-consistency conditioned sampling process in the diffusion model for refined Gaussian optimization, and a camera trajectory planning scheme to ensure comprehensive scene coverage. The specialized diffusion model is developed by finetuning a pretrained architecture on the input views and their corresponding degraded counterparts. The adaptation to the scene distribution allows the model to repair Gaussian renderings while effectively eliminating domain gaps. Meanwhile, the trajectory planning scheme optimizes scene coverage by connecting each newly sampled camera to its two nearest neighbors. By iteratively constructing paths and retaining only those that significantly enhance visibility, the scheme establishes a trajectory that covers the entire scene. To address multi-view conflicts, the view-consistency conditioned sampling process quantifies the consistency between neighboring repaired images. This information is injected as a condition into the sampling process of the frozen diffusion model, facilitating the generation of view-consistent images without additional training. Consequently, our approach produces high-fidelity 3D Gaussians that are robust to artifacts. Experimental results demonstrate that S2C-3D outperforms state-of-the-art methods, constructing high-quality scenes that are free from missing regions, blurring, or other artifacts with very sparse inputs. The source code and data are available at https://gapszju.github.io/S2C-3D.

• The paper introduces S2C-3D, a pipeline that reconstructs complete 3D scenes from sparse unposed images.
• It finetunes a specialized diffusion model and enforces view-consistency via a training-free, energy-based sampling method.
• The approach employs submodular-greedy camera trajectory planning to maximize scene coverage, outperforming state-of-the-art metrics on PSNR, SSIM, and LPIPS.

Sparse-to-Complete: High-Fidelity 3D Scene Reconstruction from Sparse Image Captures

Introduction

"Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes" (2605.05664) introduces S2C-3D, a pipeline for reconstructing high-quality, complete 3D scenes from a minimal set (as few as 6–8) of unposed images. This approach directly addresses three limitations found in prior sparse-view 3D scene reconstruction methods: (1) view inconsistency in rendered images repaired with pretrained diffusion models, (2) domain gap between off-the-shelf diffusion models and the specific target scene, and (3) poor camera coverage due to naive trajectory generation. S2C-3D integrates a specialized diffusion model, a training-free view-consistency conditioned sampling framework, and a camera trajectory planning algorithm, contributing significant empirical improvements over state-of-the-art baselines.

Figure 1: Overview of S2C-3D. (a) Sparse unposed images are processed to estimate camera poses and produce an initial point cloud; (b) The specialized diffusion model is finetuned and a trajectory covering the scene is planned; (c) Gaussian-rendered images are repaired and refined under a view-consistency constraint.

Methodology

Initial Geometry Estimation

Given unposed, sparse images, S2C-3D leverages $\pi^{3}$, a feed-forward geometry network, to estimate camera poses and a point cloud that initializes the scene. This overcomes the inaccuracy of traditional SfM strategies under extreme sparsity. The resulting set of 3D Gaussians, $G_0$, formed from the predicted geometry, is typically noisy and incomplete but sufficient for initializing the subsequent diffusion-based refinement procedure.

Specialized Diffusion Model

Unlike prior works relying on pretrained diffusion priors, S2C-3D finetunes a diffusion model (using Difix3D+ as baseline) directly on the sparse input images and synthetic degraded variants that simulate both noise and occlusion. This finetuning bridges the domain gap, tailoring image restoration to the target scene's visual statistics. The loss function includes $\ell_2$, LPIPS, and Gram-matrix supervision, ensuring both photometric and perceptual fidelity in render repair. Ablation demonstrates the necessity of both noise and occlusion-based augmentation.

Camera Trajectory Planning

To maximize reconstructed region coverage, S2C-3D implements a submodular-greedy planning scheme. Surface points (including bounding box faces to address incomplete meshes) are sampled, and each candidate camera's contribution is evaluated by the incremental area of novel surface coverage it provides—termed information gain.

Figure 2: Information gain computation for trajectory planning and formulation of the view-consistency energy function for multi-view correspondence.

Cameras are iteratively introduced to the planned path based on maximizing this gain, with pairwise interpolation to ensure continuity. Unlike simple linear interpolation, this process guarantees high scene coverage, with coverage empirically demonstrated to outperform previous interpolation-based planning.

Training-Free View-Consistency Conditioned Diffusion Sampling

To enforce inter-view consistency in repaired images, S2C-3D introduces a training-free, energy-based condition into the diffusion sampling. The view-consistency signal is formulated as a pixel-wise correspondence energy, using geometry-informed pixel matches (depth and camera pose) between neighboring virtual views. The loss enforces consistency between the repaired image and its projections from adjacent views, effectively regularizing the diffusion process to remove multi-view conflicts.

Figure 3: Qualitative comparison of novel view rendering from S2C-3D and baselines on ScanNet++ highlights S2C-3D’s artifact-free reconstructions under high input sparsity.

Figure 4: Ablation study visualizing the impact of each algorithmic component. Superior scene coverage and appearance are observed with the full framework.

Quantitative and Qualitative Evaluation

Comprehensive experiments are conducted on ScanNet++, Replica, and S2C-Scene datasets with varying input sparsity (4–8 views). S2C-3D consistently achieves the highest PSNR/SSIM and lowest LPIPS across all datasets. Notably, S2C-3D achieves an SSIM up to 0.721 and PSNR up to 20.36 on the Replica 6-view setting, representing a statistically significant advance over all compared state-of-the-art diffusion and prior-based methods.

Ablation studies confirm the criticality of each pipeline component:

• The diffusion model’s finetuning for the target scene directly leads to improved repair of unseen and occluded regions.
• Camera trajectory planning yields higher geometric coverage and scene completeness compared with standard interpolation approaches.
• The view-consistency conditioned diffusion process removes visible multi-view artifacts and blurring.

Theoretical and Practical Implications

By integrating domain adaptation, coverage-optimized sampling, and view-consistency regularization, S2C-3D systematically closes the gap between sparse multi-view image capture and data-intensive, professional 3D scene scan requirements. The method is robust to non-posed sparse image sets without strong priors on initial coverage, suggesting applicability in VR/AR scene digitization, rapid environment scanning for robotics, and low-effort media/content creation.

Practically, S2C-3D reduces the manual burden of exhaustive image acquisition in complex or cluttered spaces, bringing high-fidelity geometry and texture recovery into the regime of consumer-level scene captures. Theoretically, the work motivates further integration of geometric priors, adaptive generative modeling, and active planning in underconstrained inverse graphics pipelines.

Future research directions include:

• Improving trajectory planning with global optimality and physical feasibility constraints,
• Extending the pipeline for outdoor, unbounded, or highly non-convex environments,
• Addressing semantic alignment and dynamic scenes,
• Joint optimization strategies for image, camera, and Gaussian parameter estimation.

Conclusion

S2C-3D presents an effective pipeline for 3D scene reconstruction with minimal input requirements, outperforming previous sparse-view approaches both qualitatively and quantitatively. Through its tailored diffusion, coverage-driven planning, and energy-based consistency enforcement, it realizes artifact-free, complete 3D Gaussian scene models from as few as 4–8 images. The results have immediate implications for democratizing scene digitization and inspire future research in generative, geometry-aware vision.