Sparse-View 3D Gaussian Splatting in the Wild

Abstract: We propose a 3D novel sparse-view synthesis framework for unconstrained real-world scenarios that contain distractors. Unlike existing methods that primarily perform novel-view synthesis from a sparse set of constrained images without transient elements or leverage unconstrained dense image collections to enhance 3D representation in real-world scenarios, our method not only effectively tackles sparse unconstrained image collections, but also shows high-quality 3D rendering results. To do this, we introduce reference-guided view refinement with a diffusion model using a transient mask and a reference image to enhance the 3D representation and mitigate artifacts in rendered views. Furthermore, we address sparse regions in the Gaussian field via pseudo-view generation along with a sparsity-aware Gaussian replication strategy to amplify Gaussians in the sparse regions. Extensive experiments on publicly available datasets demonstrate that our methodology consistently outperforms existing methods (e.g., PSNR - 17.2%, SSIM - 10.8%, LPIPS - 4.0%) and provides high-fidelity 3D rendering results. This advancement paves the way for realizing unconstrained real-world scenarios without labor-intensive data acquisition. Our project page is available at $\href{https://robotic-vision-lab.github.io/SaveWildGS/}{here}$

• The paper proposes Save WildGS, a novel framework that leverages reference-guided diffusion and opacity-driven Gaussian replication for robust sparse-view 3D reconstruction.
• The method achieves significant gains in PSNR, SSIM, and LPIPS while maintaining geometric consistency in unconstrained, real-world conditions.
• The approach offers practical benefits for robotics, AR/VR, and autonomous driving by enabling high-fidelity novel view synthesis from limited imagery.

Sparse-View 3D Gaussian Splatting in the Wild: Technical Analysis

Introduction

"Sparse-View 3D Gaussian Splatting in the Wild" (2604.27422) addresses the limitations of state-of-the-art 3D scene reconstruction methods in unconstrained, real-world conditions characterized by sparse input views and the presence of distractors/transient elements. Unlike dense-view synthesis, which assumes abundant imagery and mostly static environments, this paper proposes Save WildGS—a sparse-view synthesis framework that delivers robust, high-fidelity novel view rendering for real-world scenarios without increasing time complexity. The method integrates reference-guided view refinement, diffusion models, and a novel sparsity-aware Gaussian replication mechanism to mitigate artifacts and preserve geometric consistency.

Motivation and Related Work

Traditional approaches for 3D reconstruction (e.g., NeRF [32], 3DGS [16]) rely on dense image sets, which are impractical for many real-world applications such as robotics, autonomous driving, and VR/AR. Sparse-view synthesis methods exhibit degraded performance due to ambiguities in geometry and depth, issues exacerbated in unconstrained settings with distractors. Existing solutions include multi-stage training [9, 69, 77], depth regularization [23, 79, 19], and diffusion-based techniques [63, 36, 59], but their efficacy is limited in scenarios with sparse, unconstrained imagery. Masking distractors using foundation models (SAM [18], DINO [35]) remains problematic when image collections are limited. Save WildGS differentiates itself by explicitly solving the distractor challenge in sparse, unconstrained scenarios.

Methodology

Reference-Guided View Refinement

Save WildGS leverages a redesigned one-step diffusion model for view refinement, conditioning on both a reference view and a transient mask. Grounded SAM [42] is used to generate transient masks, providing generalization across diverse distractors by leveraging text descriptions that specify dynamic objects. The refinement process selectively applies cross-attention in the diffusion model to error regions, with the rest of the image handled via self-attention, thus efficiently mitigating artifacts without impacting computational complexity.

Reference-Guided Pseudo-Label Synthesis

The technique addresses geometric inconsistencies and overfitting, typical of sparse-view initialization, by generating refined pseudo-labels from other camera perspectives, aided by transient masks. This approach distills multi-view consistency and improves the 3D representation, optimizing loss both on reconstructed views and pseudo-labels.

Sparsity-Aware Gaussian Replication (SAGR)

Standard densification in Gaussian fields typically uses positional gradients with depth supervision, but this entangles color and opacity, leading to misalignment and artifacts. Save WildGS introduces opacity-driven Gaussian replication, where opacity maps guide the spatial distribution of new Gaussians, directly targeting sparse regions. This efficiently fills deficient areas in the Gaussian field, maintaining high-frequency detail and avoiding blur.

Regularization and Optimization

To stabilize optimization and enforce geometric consistency, LoRA [12] is applied in the VAE decoder with Score Distillation Sampling (SDS) loss. The total loss aggregates photometric, pseudo-label, and SDS losses, with periodic Gaussian replication and densification during training.

Experimental Evaluation

Save WildGS was evaluated on NeRF On-the-go [43], Photo Tourism [51], and LLFF [31] datasets, with initialization via COLMAP [47]. The method was compared against 3DGS [16], Mip-Splatting [71], RegNeRF [34], DiffusionNeRF [62], FreeNeRF [66], ReconFusion [61], FSGS [79], DropoutGS [37], Difix3D+ [59], NeRF-W [30], Ha-NeRF [4], CR-NeRF [68], WildGaussians [20], GS-W [73], RobustSplat [10], DroneSplat [53], and SparseGS-W [25].

Quantitative Performance

Save WildGS achieves notable gains across PSNR, SSIM, and LPIPS metrics in unconstrained settings, outperforming prior methods by up to 17.2% in PSNR, 10.8% in SSIM, and 4.0% in LPIPS. In both constrained and unconstrained scenarios, it maintains competitive rendering quality, avoiding the 3D inconsistency and sparsity issues observed in baselines. Ablation studies confirm the robustness and efficacy of reference-guided refinement and SAGR, with cross-attention mechanisms yielding improved artifact mitigation and geometric consistency.

Qualitative Results and Robustness

In scenarios featuring transient distractors (e.g., moving objects, dynamic occlusions), competing systems produce blurred, artifact-ridden outputs. Save WildGS delivers sharper, consistent renderings with correctly suppressed distractors. Grounded SAM enhances mask generation robustness, especially when the "dynamic" keyword is used in text descriptions. Memory usage is higher than less capable baselines but justified by superior rendering outcomes and uniform Gaussian distribution.

Limitations

Save WildGS is susceptible to challenges in detecting static distractors, such as parked vehicles or immobile pedestrians. Overlapping camera perspectives may limit the efficacy of reference-guided refinement, potentially degrading representation quality. The diffusion-based refinement can introduce mild noise, and some artifacts evade existing mask generators.

Implications and Future Directions

Save WildGS advances 3D scene rendering for real-world, sparse-view settings, enabling applications in robotics, AR/VR, and autonomous systems where dense imagery is unattainable and distractors are frequent. The plug-and-play nature of the reference-guided refinement allows integration with constrained and unconstrained pipelines. Future work may focus on mesh-based techniques, improved transient segmentation through prior knowledge or super-resolution models, and further adaptation for physical AI scenarios where artifact suppression and geometric integrity are paramount.

Conclusion

This paper presents Save WildGS, a sparse-view 3D Gaussian splatting framework robust to distractors and unconstrained environments. Through reference-guided diffusion refinement, opacity-driven Gaussian replication, and advanced regularization strategies, the framework establishes a new standard for high-fidelity, consistent novel view synthesis from minimal real-world imagery (2604.27422). Its contributions open new directions in scalable, artifact-resistant 3D reconstruction, with practical relevance for numerous vision and graphics domains.