---
title: Sparse-View Gaussian Splatting
url: https://www.emergentmind.com/topics/sparse-view-gaussian-splatting-cd3aab0d-03c9-47b1-8fb4-7f379f51d7d5
type: topic
---

# Sparse-View Gaussian Splatting

Searching arXiv for the supplied papers and closely related sparse-view Gaussian splatting work.
Sparse-view Gaussian Splatting denotes the family of methods that adapt Gaussian Splatting to image collections with severely limited viewpoint coverage, typically where standard dense-view assumptions break down and reconstruction quality degrades through poor initialization, incomplete geometry, floaters, background collapse, holes, or unstable optimization. Across the recent literature, the common setting is that a scene is represented by explicit Gaussian primitives and optimized from as few as 2, 3, 4, 5, 6, 8, 12, or 24 input images, depending on the benchmark and task; the central technical problem is that sparse supervision is insufficient to reliably place, shape, and regularize these primitives without additional priors or training strategies [2312.00206], [2408.00254], [2410.18822], [2505.20729], [2602.03327], [2606.31556].

## 1. Foundations and problem formulation

Sparse-view Gaussian Splatting inherits the explicit scene parameterization of 3D Gaussian Splatting. In the standard 3D formulation, a scene is represented by anisotropic Gaussian primitives parameterized by a mean, covariance, opacity, and color; rendering proceeds by projecting Gaussians into screen space, sorting them by depth, and applying front-to-back alpha compositing. Representative formulations write a Gaussian density as
$$
G_i(x)=\exp\!\left(-\tfrac12(x-\mu_i)^T\Sigma_i^{-1}(x-\mu_i)\right),
$$
with pixel color accumulated as
$$
C=\sum_i c_i\,\alpha_i\prod_{j<i}(1-\alpha_j),
$$
or equivalent variants using transmittance terms and projected opacities [2506.10335], [2506.07670], [2410.18822], [2312.00206].

The sparse-view regime is defined less by a new renderer than by a change in the optimization regime. When only a few calibrated images are available, classical Structure from Motion often yields too few reliable points, geometric supervision becomes weak, and the Gaussian field can overfit the training views rather than recover consistent 3D structure. Multiple papers identify recurrent failure modes: floaters, background collapse, large holes, incomplete geometry in unobserved regions, and degraded novel-view quality [2312.00206], [2505.20729], [2506.10335]. In surface-oriented variants, the difficulty is sharpened further because sparse supervision must support not only view synthesis but also depth consistency, normal fidelity, and watertight or TSDF-fusable geometry [2505.19854], [2412.15400], [2511.14633].

A second foundational distinction is between volumetric Gaussians and surface-oriented primitives. Some works retain 3D anisotropic Gaussians for novel-view synthesis and scene rendering [2505.20729], [2506.10335], [2606.31556], whereas others use 2D Gaussian splatting, Gaussian surfels, or flattened Gaussians aligned to local tangent planes for surface reconstruction [2505.19854], [2604.08370], [2412.15400], [2511.14633]. This suggests that sparse-view Gaussian Splatting is not a single algorithmic template but a representation family whose specific geometry prior is chosen according to task.

## 2. Initialization as the dominant bottleneck

A large fraction of the literature treats initialization as the primary sparse-view bottleneck. The baseline failure mode is explicit: standard Gaussian Splatting commonly initializes primitive centers from sparse SfM point clouds, but with only 3–5 images such point clouds are often too sparse for stable optimization [2505.19854]. SparseGS likewise identifies dependence on sparse COLMAP points as a limitation in extremely sparse settings [2312.00206].

Recent methods replace or augment SfM with dense point or depth prediction from foundation models. Sparse2DGS uses DUSt3R together with COLMAP MVS to generate a highly accurate and dense 3D point cloud, then converts each point into a 2D Gaussian by fitting a local tangent plane, estimating local variances from neighbors, and initializing opacity and color from nearby observations [2505.19854]. Intern-GS uses DUSt3R to produce a dense, globally aligned point cloud, followed by a redundancy-free sampling step that adds Gaussians only for pixels satisfying a mask derived from accumulated support and depth disagreement [2505.20729]. PointGS uses VGGT to jointly estimate accurate camera poses and dense per-view point clouds, which are bundle-adjusted and fused into a single global cloud before subsequent learned appearance modeling [2506.10335]. Pi-GS replaces COLMAP/SfM with the reference-free point-cloud-and-pose network $\pi^3$, which predicts per-pixel depth, confidence, and camera extrinsics directly from the sparse image set [2602.03327].

Other methods push initialization into feed-forward prediction. FSFSplatter uses a large Transformer to predict monocular depth maps, camera intrinsics and extrinsics, pixel-aligned features, and a dense Gaussian scene initialization via a self-splitting Gaussian head; it then prunes local floaters through contribution-based pruning before fast joint optimization [2510.02691]. SurfelSplat performs feed-forward prediction of pixel-aligned Gaussian surfels from sparse-view images and is explicitly designed to be generalizable rather than per-scene optimized [2604.08370]. ProSplat adopts a feed-forward two-stage design in which a 3DGS generator first predicts Gaussian primitives and a second stage improves rendered views through a one-step diffusion model [2506.07670].

An alternative line densifies initialization during optimization rather than before it. LoopSparseGS introduces loop-based Progressive Gaussian Initialization: after optimizing Gaussians for one stage, it renders pseudo-views near the training cameras, reruns COLMAP on the union of real and pseudo images, and reinitializes Gaussians from the densified point set in an outer loop [2408.00254]. SparseGS-W similarly relies on DUSt3R for dense initialization from very sparse outdoor images, including a second DUSt3R pass after masking user-specified occluders [2503.19452].

These works collectively indicate that sparse-view Gaussian Splatting shifted rapidly from SfM-dependent initialization toward dense point-cloud, depth, and pose estimation from DUSt3R, VGGT, and $\pi^3$ [2505.19854], [2505.20729], [2506.10335], [2602.03327]. A plausible implication is that initialization quality increasingly determines whether later regularization acts as refinement or as damage control.

## 3. Regularization, priors, and optimization strategies

Once initialized, sparse-view methods introduce additional supervision to constrain geometry and appearance in under-observed regions. A recurrent design is to combine the standard photometric objective with depth, normal, feature, or pseudo-view losses.

Several methods formulate explicit depth and normal constraints. Sparse2DGS refines 2D Gaussian parameters under a total loss
$$
L=L_c+\alpha L_d+\beta L_n,
$$
with $\alpha=1000$ and $\beta=0.05$, where $L_d$ penalizes inconsistent depths among overlapping Gaussians on each ray and $L_n$ aligns Gaussian normals with depth-map normals [2505.19854]. Pi-GS introduces uncertainty-guided depth supervision via a confidence-weighted Pearson correlation, a masked normal consistency loss, and a depth-warping pseudo-view loss; its total objective combines these with PGSR losses and a planar scale penalty [2602.03327]. PointGS uses photometric reconstruction, monocular depth regularization, and an edge-aware depth-smoothness term, while learning appearance-aware Gaussian attributes through feature aggregation and self-attention [2506.10335].

Pseudo-view supervision is another dominant strategy. Intern-GS creates pseudo-views by slight camera rotations and imposes both depth-correlation and color losses on these views, where pseudo-view appearance is refined using a diffusion model [2505.20729]. SparseGS-W alternates Gaussian updates with a Constrained Novel-View Enhancement module that generates diffusion-enhanced pseudo ground truth for novel views, and an Occlusion Handling module that inpaints training views containing transient occluders [2503.19452]. SaveWildGS also uses a one-step diffusion model for reference-guided view refinement and pseudo-view synthesis, combined with an SDS regularizer and a sparsity-aware Gaussian replication strategy [2604.27422]. AugSplat approaches the same problem from the opposite direction: it first trains a radiance-field ensemble on the sparse real images, renders synthetic views from nearby poses, computes per-pixel confidence from ensemble variance, and then uses those synthetic views as auxiliary supervision for Gaussian optimization in either staged or dual schedules [2606.31556].

Several papers tackle overfitting or instability through structural regularization of the Gaussian field itself. PairDropGS samples two dropped Gaussian subsets from the same shared field, applies a Gaussian-blur low-pass filter, and minimizes a low-frequency consistency loss
$$
L_{lfc}^{pair}=\|\tilde I_1-\mathrm{sg}(\tilde I_2)\|_1,
$$
with a progressive weight schedule $\lambda(t)=\lambda_{\max}\min(1,t/T_{warm})$ and $\lambda_{\max}=0.05$ [2605.12072]. LoopSparseGS proposes Sparse-friendly Sampling, which identifies top-error pixels, locates the dominant contributing Gaussian, and splits oversized primitives into two children; it couples this with Depth-alignment Regularization that aligns rendered depth with sparse SfM depth and sliding-window Pearson correlation against monocular depth [2408.00254]. Binocular-Guided 3D Gaussian Splatting dispenses with external priors and instead uses disparity-guided binocular stereo consistency between rendered and shifted views, together with multiplicative opacity decay $\alpha_i\leftarrow\lambda_{\mathrm{decay}}\alpha_i$ using $\lambda_{\mathrm{decay}}=0.995$ [2410.18822].

Surface-reconstruction methods add more explicitly geometric penalties. Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views fixes appearance during a geometry stage, renders MVS features via splatting, and imposes cosine feature consistency as well as disk-sampling-based cross-view regularization over primitive position, orientation, and scale [2504.20378]. SparseSurf augments flattened Gaussians with Stereo Geometry-Texture Alignment and Pseudo-Feature Enhanced Geometry Consistency, activating stereo priors from iteration 500 and pseudo-feature losses from iteration 3000 in a 7000-iteration schedule [2511.14633]. SolidGS replaces the Gaussian kernel itself with a generalized-exponential “solid” kernel
$$
G_i^{\mathrm{solid}}(x)=\exp\!\Bigl(-\tfrac12\bigl[(x-c_i)^T\Sigma_i^{-1}(x-c_i)\bigr]^{\beta_g/2}\Bigr),
$$
and combines this with depth distortion, normal consistency, and monocular normal supervision [2412.15400].

The broader pattern is clear: sparse-view Gaussian Splatting is no longer trained solely by image reconstruction. It is trained by image reconstruction plus priors, with those priors variously drawn from monocular depth, stereo, diffusion, radiance fields, feature correspondence, dropout consistency, or kernel design [2505.20729], [2606.31556], [2605.12072], [2412.15400].

## 4. Representation variants and task-specific branches

The topic spans several distinct subdomains rather than a single benchmark culture. A compact taxonomy is useful.

| Branch | Representative methods | Core emphasis |
|---|---|---|
| Sparse-view novel-view synthesis | SparseGS, LoopSparseGS, Binocular-Guided 3DGS, PointGS, PairDropGS, CuriGS, AugSplat | Real-time rendering, anti-overfitting regularization, pseudo-view or feature priors |
| Sparse-view surface reconstruction | Sparse2DGS, SolidGS, SparseSurf, SurfelSplat, FSFSplatter | Geometry accuracy, depth/normal consistency, TSDF or mesh extraction |
| In-the-wild or distractor-aware reconstruction | SparseGS-W, SaveWildGS, Sparse View Distractor-Free Gaussian Splatting | Occlusion handling, transient masking, reference-guided diffusion, semantic priors |
| Feed-forward or wide-baseline systems | ProSplat, SurfelSplat, FSFSplatter | Per-scene optimization reduction, wide-baseline robustness, fast inference |
| Domain-specific inverse problems | GR-Gaussian | Sparse-view CT reconstruction |

Novel-view-synthesis papers usually evaluate PSNR, SSIM, and LPIPS under sparse splits. SparseGS addresses unbounded 360° scenes with depth priors, unseen-viewpoint regularization, and floater pruning [2312.00206]. LoopSparseGS uses outer-loop densification and depth alignment [2408.00254]. PointGS introduces multiscale 2D appearance features and a point interaction network based on local self-attention [2506.10335]. CuriGS turns pseudo-view generation into a curriculum over perturbation magnitudes, promoting only the best-performing student views under a multi-signal metric [2511.16030]. PairDropGS reinterprets dropout as a consistency-regularization problem [2605.12072]. AugSplat uses NeRF-generated synthetic views as auxiliary supervision while preserving Gaussian rasterization at inference [2606.31556].

Surface-reconstruction papers are typically stricter about geometry. Sparse2DGS demonstrates that dense DUSt3R plus COLMAP MVS initialization can support reconstruction from only three images [2505.19854]. Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views stages optimization from geometry to appearance and uses feature splatting and disk regularization [2504.20378]. SolidGS introduces a shared solidness factor for all Gaussians to consolidate geometry across views [2412.15400]. SparseSurf argues that flattened Gaussians alone can exacerbate overfitting under sparse supervision and therefore adds stereo and pseudo-feature consistency [2511.14633]. SurfelSplat reframes the problem as generalizable feed-forward Gaussian surfel prediction and explicitly analyzes Nyquist sampling constraints [2604.08370]. FSFSplatter combines learned dense initialization, differentiable camera refinement, and geometry-enhanced optimization in a fast pipeline [2510.02691].

The in-the-wild and distractor-aware branch extends sparse-view Gaussian Splatting beyond controlled datasets. SparseGS-W handles large-scale outdoor scenes from as few as five training images through constrained diffusion priors and user-provided reference images [2503.19452]. SaveWildGS adds Grounded-SAM-based transient masking, reference-guided diffusion refinement, pseudo-view synthesis, and sparsity-aware Gaussian replication [2604.27422]. Sparse View Distractor-Free Gaussian Splatting integrates VGGT geometry, attention-based semantic entity matching, VLM confirmation of large static regions, and a warm-up phase with non-learnable mask priors inside RobustGS [2603.01603].

A final branch adapts Gaussian splatting to sparse-view CT. GR-Gaussian models volumetric density as a sum of radiative Gaussians, introduces denoised point-cloud initialization from FDK, and uses a graph-aware gradient augmentation for splitting decisions [2508.02408]. This broadens the topic from image-based scene synthesis to tomography and suggests that the same sparse-view difficulties recur whenever Gaussian primitives are optimized under limited projections.

## 5. Empirical benchmarks and observed performance trends

The benchmark landscape is heterogeneous but consistent in its stress tests. LLFF, DTU, Mip-NeRF360, Tanks and Temples, BlendedMVS, PhotoTourism, NeRF-on-the-go, RealEstate10K, DL3DV-10K, Replica, and X-3D appear repeatedly, usually under 2-, 3-, 4-, 5-, 6-, 8-, 12-, or 24-view settings [2408.00254], [2505.20729], [2506.07670], [2503.19452], [2511.14633], [2510.02691], [2508.02408].

On forward-facing or object-centric three-view benchmarks, several papers report state-of-the-art or near-state-of-the-art performance with different inductive biases. Intern-GS reports, under 3-view training, LLFF $20.49/0.693/0.212$, DTU $20.34/0.851/0.163$, and Tanks and Temples $22.67/0.736/0.191$ in PSNR/SSIM/LPIPS [2505.20729]. PointGS reports LLFF 3-view performance of $20.54$ dB, $0.72$, and $0.25$, with ablations showing degradation when removing the point interaction module, variance fusion, depth loss, or smoothness loss [2506.10335]. Binocular-Guided 3D Gaussian Splatting reports LLFF 3-view PSNR $21.44$, SSIM $0.751$, LPIPS $0.168$, and DTU 3-view PSNR $20.71$, SSIM $0.862$, LPIPS $0.111$ [2410.18822]. CuriGS reports averaged LLFF 3-view $21.10/0.732/0.193$ and DTU 3-view $20.35/0.870/0.131$, while its ablation without curriculum degrades substantially on representative scenes [2511.16030]. PairDropGS reports LLFF 3-view improvement from DropGaussian’s PSNR $20.33$ to $20.89$, and also reduces PSNR standard deviation across 10 seeds from approximately $0.1724$ dB to approximately $0.0726$ dB [2605.12072].

For 360° or unbounded sparse-view scenes, SparseGS reports on Mip-NeRF360 with 12 views PSNR $17.26$, SSIM $0.507$, LPIPS $0.474$, improving over base 3DGS while retaining 120+ FPS [2312.00206]. AugSplat reports average Mip-NeRF360 performance over nine scenes of SSIM $0.630$, PSNR $18.39$, LPIPS $0.632$, and Avg $0.183$ for Staged AugSplat, compared with standard GSplat at SSIM $0.622$, PSNR $17.88$, LPIPS $0.625$, and Avg $0.192$ [2606.31556]. On Mip-NeRF360 with 24 views, PointGS reports $23.48$ dB, $0.72$, and $0.24$ [2506.10335].

Surface-reconstruction metrics commonly use Chamfer distance. Sparse2DGS reports mean Chamfer distance approximately $1.51$ mm over 12 challenging DTU scenes using only 3 views, compared with approximately $2.60$ mm for SparseNeuS, approximately $2.69$ mm for ReTR, approximately $2.77$ mm for COLMAP MVS, and approximately $2.54$ mm for 2DGS with SfM initialization [2505.19854]. Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views reports DTU mean Chamfer distance $1.13$ mm under 3 views, versus $2.81$ mm for 2DGS, $2.08$ mm for PGSR, and $1.27$ mm for SparseNeus, while taking approximately 10 minutes per scene [2504.20378]. SolidGS reports DTU 3-view mean Chamfer distance $1.27$ with PSNR $21.32$ in $3.1$ min, compared with $1.83$ for PGSR, $1.71$ for 2DGS, and $1.55$ for MVPGS [2412.15400]. SparseSurf reports DTU 3-view Chamfer distance $1.05$ mm under little overlap and $0.89$ mm under large overlap, along with DTU sparse NVS metrics PSNR $21.31$, SSIM $0.886$, LPIPS $0.089$, and AVGE $0.067$ [2511.14633]. SurfelSplat reports mean DTU 2-view Chamfer-D2S $1.12$ mm with 1-second inference, compared with $1.18$ for FatesGS and much longer runtimes for several optimization-based baselines [2604.08370]. FSFSplatter reports DTU surface CD $1.58$ mm after 1000 iterations and Replica surface CD $3.37$ cm, with LPIPS $0.113$ on DTU and $0.095$ on Replica after full optimization [2510.02691].

In-the-wild benchmarks emphasize perceptual and no-reference metrics. SparseGS-W reports on PhotoTourism with five input views PSNR $19.01$, SSIM $0.55$, LPIPS $0.31$, FID $48$, ClipIQA $0.51$, and MUSIQ $66.98$, and on Tanks and Temples with three input views PSNR $21.58$, SSIM $0.68$, LPIPS $0.22$, FID $55$, ClipIQA $0.50$, and MUSIQ $69.80$ [2503.19452]. SaveWildGS reports relative gains averaged over splits of $+17.2\%$ PSNR, $+10.8\%$ SSIM, and $-4.0\%$ LPIPS on NeRF-on-the-go, and states that it outperforms GS-W, WildGaussians, and Difix3D+ by 3–4 dB in sparse-view settings [2604.27422].

These results do not isolate a single universally dominant method. Instead, they show benchmark-dependent specialization: diffusion- and reference-guided methods excel in unconstrained scenes [2503.19452], [2604.27422]; geometry-prioritized and surfel-based methods excel in surface reconstruction [2504.20378], [2604.08370], [2511.14633]; and dropout, curriculum, or augmentation methods stabilize general sparse-view NVS [2605.12072], [2511.16030], [2606.31556].

## 6. Limitations, misconceptions, and open directions

A common misconception is that sparse-view Gaussian Splatting is simply “3DGS with fewer images.” The literature indicates otherwise. Sparse-view performance depends on changing the initialization, the supervision, the regularization schedule, or even the primitive itself; several papers explicitly show that vanilla 3DGS degrades sharply under sparse inputs [2312.00206], [2505.19854], [2506.10335]. Another misconception is that sparse-view improvements come only from stronger appearance priors. In fact, many of the strongest gains in reconstruction accuracy are attributed to dense geometry initialization, depth correlation, normal consistency, feature correspondence, or kernel redesign rather than purely photometric enhancement [2504.20378], [2412.15400], [2602.03327].

The literature also converges on several limitations. Extrapolation beyond the convex hull of training views remains weak in diffusion-guided systems; Intern-GS states that diffusion cannot hallucinate far-out regions reliably [2505.20729]. Depth priors remain vulnerable to scale ambiguity and monocular noise [2505.20729], [2602.03327]. MVS- or foundation-model-based initialization can propagate errors under heavy occlusion, specularity, or textureless regions [2504.20378]. Dropout and pseudo-view methods improve stability, but still degrade under extreme sparsity such as two views in some settings [2506.10335]. Feed-forward methods trade per-scene optimization for learned inductive bias and may require strong pretraining or architecture-specific assumptions [2506.07670], [2604.08370], [2510.02691]. In distractor-aware pipelines, semantic matching and mask priors reduce dependence on color residuals, but introduce additional foundation-model and VLM dependencies [2603.01603], [2604.27422].

Several open directions recur explicitly across papers. These include scene extrapolation with stronger geometric constraints [2505.20729]; dynamic-scene or video extensions [2505.20729], [2508.02408]; learned Gaussian placement and redundancy reduction [2505.20729]; joint optimization of camera poses and Gaussians under sparse supervision [2312.00206], [2510.02691]; stronger integration of radiance-field or diffusion priors with explicit geometry [2606.31556], [2506.07670]; adaptive or learned graph structures for inverse problems [2508.02408]; and faster, generalizable surface reconstruction without per-scene optimization [2604.08370].

Taken together, the field suggests a broader shift in the role of Gaussian Splatting under sparse supervision. Rather than serving as a standalone explicit renderer, it is increasingly used as the optimization core inside hybrid systems that import priors from stereo foundation models, monocular depth estimators, diffusion models, radiance fields, semantic matchers, or curriculum-based data augmentation [2505.20729], [2503.19452], [2606.31556], [2511.16030]. A plausible implication is that future sparse-view Gaussian Splatting research will be defined less by the rasterizer itself than by how effectively explicit Gaussian primitives can absorb and reconcile heterogeneous priors while preserving the speed advantages that originally motivated Gaussian Splatting.

Source: https://www.emergentmind.com/topics/sparse-view-gaussian-splatting-cd3aab0d-03c9-47b1-8fb4-7f379f51d7d5