AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings

Abstract: Generating high-quality novel views at real-time frame rates remains a central challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric rendering leads to high computational cost and slow inference. In contrast, Gaussian Splatting methods achieve real-time rendering through rasterization, but their optimization is highly sensitive to the quality of the initial geometry. This sensitivity becomes especially problematic in sparse-view settings, where limited observations often lead to incomplete or noisy point-cloud reconstructions. In this work, we present AugSplat, a simple framework for improving Gaussian Splatting in sparse-view regimes using radiance-field-based view augmentation. We first train a radiance field on the sparse input views and use it to synthesize additional images from nearby novel viewpoints, increasing the effective view-space coverage available for supervision. These synthetic views are then used as auxiliary supervision during Gaussian Splatting optimization. We study two variants: Staged AugSplat, which uses synthetic views for an initial optimization phase before switching to real images, and Dual AugSplat, which jointly trains on real and synthetic views with a decaying synthetic loss weight. Experiments on sparse-view mip-NeRF 360 scenes show that AugSplat improves reconstruction quality over standard Gaussian Splatting. Staged AugSplat achieves the strongest average performance, while Dual AugSplat provides a closely performing formulation that keeps real-image supervision active throughout training, and both variants preserve real-time rendering at inference.

• The paper introduces AugSplat, a hybrid approach that uses NeRF-generated synthetic views to augment GSplat optimization and improve sparse-view reconstructions.
• The methodology combines staged and dual variants to effectively manage real-image and synthetic supervision, achieving superior geometric fidelity and artifact suppression.
• Experimental results on the mip-NeRF 360 dataset show enhanced reconstruction quality, demonstrating lower average error and robust performance in challenging scenes.

Radiance Field-Informed Gaussian Splatting for Sparse-View 3D Reconstruction

Motivation and Context

Sparse-view novel-view synthesis presents numerous challenges in 3D vision. Neural radiance fields (NeRFs) are robust under minimal input images but suffer from slow inference due to volumetric rendering. Conversely, Gaussian Splatting (GSplat), as introduced in previous works [kerbl20233d], achieves real-time rendering but is critically sensitive to point cloud initialization, which is especially problematic in sparse-view scenarios. Traditional structure-from-motion often yields incomplete or noisy reconstructions in such cases, impeding downstream GSplat optimization.

This paper proposes AugSplat, a framework that leverages a NeRF-based prior to synthesize auxiliary training views, thus augmenting the supervision available for GSplat optimization. This hybridization enhances spatial coverage, mitigating initialization deficiencies inherent to sparse-view settings.

Figure 1: Overview of the AugSplat pipeline.

Methodology

Radiance Field-Driven View Augmentation

AugSplat utilizes an ensemble of NeRFs trained on the sparse input set to generate synthetic views from interpolated camera poses. Ensemble-based pixel confidence is computed as the per-pixel variance across NeRF renderings, normalized via robust percentiles and transformed into confidence weights using a power law. This weighting suppresses unreliable synthetic regions, ensuring that GSplat optimization benefits from stable predictions.

Figure 2: Example NeRF-rendered synthetic view and corresponding confidence map used for weighted supervision.

These weighted synthetic images are then incorporated as auxiliary supervision in GSplat optimization. The primary intuition is that NeRF-generated views stabilize early-stage geometry by providing geometric and photometric constraints beyond those present in sparse real images.

Gaussian Splatting Variants

Two distinct optimization strategies are introduced:

1. Staged AugSplat: Warm-up phase with synthetic only; subsequently switches to real-image supervision.
2. Dual AugSplat: Simultaneous training on real and synthetic supervision, with the synthetic weight decaying exponentially per step.

Both variants maintain real-time inference by utilizing explicit Gaussian primitives, and neither alters the underlying GSplat representation.

Experimental Results

Experiments were conducted on the mip-NeRF 360 dataset, subsampled to 30 images per scene. The augmentation protocols used an ensemble of 5 NeRF models and 200 synthetic views per scene. PSNR, SSIM, and LPIPS metrics were reported, with checkpoints selected at lowest average error (geometric mean of MSE, $\sqrt{1-\text{SSIM}}$, and LPIPS).

The results demonstrate consistent improvement in average error for both AugSplat variants compared to baseline GSplat. Staged AugSplat exhibited the strongest reconstruction quality, while Dual AugSplat provided slightly more stable performance and mitigated scene-level variance by retaining real-image supervision throughout training.

A qualitative comparison reveals that AugSplat better preserves fine scene details and corrects artifacts stemming from incomplete initialization. On complex scenes with substantial spatial gaps, AugSplat is able to reconstruct missing geometric features that baseline GSplat fails to resolve.

Figure 3: Visual comparison between Staged AugSplat and standard GSplat on kitchen and garden scenes; AugSplat delivers superior fidelity and artifact suppression.

Training Dynamics and Analysis

The average-error minimization reached optimality at a compromise between pixel-level fidelity (PSNR, SSIM) and perceptual metrics (LPIPS). Metrics illustrate that GSplat and AugSplat peak on fidelity early in training, tapering off as perceptual similarity improves at the expense of structural accuracy—a behavior characteristic of sparse-view overfitting.

Figure 4: Training dynamics on the stump scene, showing optimal compromise for image-fidelity and perceptual metrics.

Figure 5: PSNR and SSIM peak early while LPIPS improves later; this trade-off is captured in the average-error metric.

AugSplat achieves lower average error at equivalent training stages, confirming that gains are not simply due to extended optimization but arise from improved early-stage geometry and appearance modeling.

Implications and Future Directions

AugSplat demonstrates that radiance field-informed view augmentation distinctly improves GSplat optimization in sparse-view reconstruction. The method leverages the spatial completeness of implicit representations for training, while maintaining real-time explicit rendering at inference. The dual approach allows continuous refinement via real-image loss, while staged variants exploit synthetic views for rapid stabilizing of coarse primitives.

Broadly, this hybrid framework suggests that explicit–implicit representation fusion is a promising avenue for scene reconstruction, particularly when data collection is constrained. The ongoing development of uncertainty weighting and progressive supervision strategies will further strengthen robustness to initialization bias and scene complexity.

Future research may focus on adaptive synthetic pose generation, higher-order ensemble confidence modeling, and direct integration of implicit priors into explicit primitive initialization. The applicability of AugSplat to dynamic or articulable scenes, as well as its scalability to larger environments, remains an open question.

Conclusion

Radiance field-informed augmentation as instantiated by AugSplat addresses fundamental deficiencies in sparse-view explicit reconstruction. The framework combines NeRF synthetic view supervision with GSplat’s efficient rendering, yielding demonstrable improvements in both early-stage and final reconstruction quality. The results strongly support the integration of implicit- and explicit-model driven strategies for robust, scalable, and real-time 3D vision, with significant practical implications for VR, robotics, and large-scale spatial modeling.