VisDom: Sparse Novel View Synthesis with Visible Domain Constraint

Abstract: Sparse novel view synthesis (NVS) remains challenging due to the ambiguity of recovering 3D geometry from few input views. While NeRF- and Gaussian Splatting (GS)-based methods perform well with dense supervision, they often overfit in sparse settings, producing floating artifacts and inconsistent geometry. Silhouette consistency is commonly used as a regularizer, but it remains insufficient, as silhouette-consistent regions can extend beyond the true object geometry. We introduce VisDom, a learning-free geometric constraint that augments classical carving-based visual hull reconstruction by enforcing a minimum multi-view visibility requirement. Specifically, we define a visible domain as the subset of 3D space observed by at least $K$ views and use it as an additional filtering criterion on top of standard silhouette-based reconstruction. This provides a stronger spatial prior in sparse-view settings. We integrate VisDom into both implicit (NeRF) and explicit (GS) pipelines by restricting volumetric sampling and guiding Gaussian placement during optimization. Experiments on three challenging datasets show consistent improvements in sparse-view NVS, enabling high-quality object-centric reconstruction from as few as four input images. Our method is domain-agnostic, requires only silhouettes, and introduces no learned parameters, making it a simple complement to existing approaches. Applying VisDom on top of GaussianObject further improves performance on Omni3D and MipNeRF360, while matching or surpassing it at 22 $\times$ lower training cost.

• The paper introduces a learning-free geometric prior that uses a multi-view visibility constraint to tighten visual hulls in sparse novel view synthesis.
• It integrates with NeRF and 3D Gaussian Splatting pipelines, significantly reducing artifacts and improving PSNR by up to 90% in sparse regimes.
• The method enhances reconstruction quality with minimal computation overhead, enabling efficient 3D capture in challenging sparse-view scenarios.

VisDom: Sparse Novel View Synthesis with Visible Domain Constraint

Motivation and Problem Statement

Sparse novel view synthesis (NVS) is fundamentally ill-posed when only a limited set of input views are available, as ambiguity in recovering 3D geometry increases dramatically with reduced supervision. State-of-the-art radiance field (NeRF) and 3D Gaussian Splatting (3DGS) methods exhibit strong results in dense regimes but fail in sparse-view settings, with artifacts such as floaters and inconsistent geometry. Previous regularization strategies employ learned priors, depth constraints, or diffusion guidance, often requiring domain-specific training data and adding considerable complexity. Silhouette constraints, though easy to extract via off-the-shelf segmentation models, do not provide sufficient geometric specificity in the extreme sparse regime; traditional visual hulls built from silhouette intersections are too permissive and admit ambiguous regions, resulting in suboptimal reconstruction.

Visible Domain Constraint: Methodology

VisDom introduces a learning-free geometric prior that augments classical carving-based visual hulls by enforcing a minimum multi-view visibility requirement. Rather than retaining all space silhouette-consistent in any view, the visible domain requires regions of interest to be observed in at least $K$ input views, thereby substantially tightening the reconstructed hull Figure 1.

The constraint is applied alongside standard silhouette carving. Voxels are projected into training image silhouettes, with occupancy and visibility votes accumulated. Only voxels seen by at least $K$ cameras and passing occupancy criteria are retained, while spurious regions—visible in few or only one view—are suppressed. This visible domain hull forms the spatial prior for downstream NeRF and 3DGS optimization.

Integration with NeRF and 3DGS Pipelines

NeRF Integration

For NeRF pipelines, VisDom restricts the volumetric ray sampling bounds to only regions inside the sparse visual hull. Let $\mathbf{r}(t) = \mathbf{o} + t\mathbf{d}$ represent the camera ray, $\mathbf{p}^*$ the intersection with the hull, and an $\epsilon$ margin for precise bounding. Sampling bounds are set to $$[t_n, t_f] = [\mathbf{p}^* - \epsilon\mathbf{d}, \mathbf{p}^* + \epsilon\mathbf{d}]$$, thus constraining density allocation strictly to geometric regions supported by multiple views. Loss functions combine base photometric objectives with binary cross entropy mask consistency, using the masked hulls for supervision.

3D Gaussian Splatting Integration

In 3DGS, VisDom regularizes the initialization stage and ongoing optimization. The constraint filters Gaussian placements, enforcing that explicit primitives lie within the visible domain hull. Additionally, visibility-induced mask losses penalize opacity in regions outside the hull when rendering from interpolated views. The cumulative loss penalizes Gaussians that appear opaque outside the hull, refining the spatial support for reconstructed surfaces.

Empirical Evaluation

Experiments span ActorsHQ (human subjects), MipNeRF360, and Omni3D datasets, each presenting challenging object-centric scenarios. Quantitative results demonstrate that VisDom enhances reconstruction quality across all architectures and datasets, even enabling general-purpose NeRF methods to recover from failure modes at extreme sparsity (4 views), producing up to $\sim$90% PSNR gains, thereby matching or surpassing sparse-view dedicated baselines.

Qualitative comparisons show marked reduction of floaters, sharper structural boundaries, and more consistent geometry when VisDom constraint is applied Figure 2.

Notably, 3DGS+VD achieves competitive or superior quality at up to $22\times$ lower training cost, as the learning-free hull computation takes only two seconds per scene and adds no runtime overhead.

Ablation Studies

Ablating the minimum observing camera count $K$ reveals trade-offs: $K=1$ is too permissive, retaining the ambiguous union of all viewing frusta; $K$0 or $K$1 increasingly tighten spatial support and eliminate floaters. Empirical results favor $K$2 for ZipNeRF, balancing overly aggressive carving and excessive retention, with $K$3 peaking for 3DGS-GO due to method-specific sensitivity.

Additional mask dilation experiments demonstrate that small dilation can further improve PSNR, especially when mask boundaries are imprecise, but performance plateaus early—suggesting robustness of the constraint to segmentation noise Figure 3.

Comparison and Theoretical Implications

VisDom is domain-agnostic, relying exclusively on silhouette masks and multi-view camera poses, and introduces no learned parameters or object priors. By enforcing geometric consistency across sparse views, it removes reconstruction ambiguities inherent in silhouette-only regularization. The method refines traditional visual hulls, outperforming unit-sphere heuristics and unsupervised segmentation-based constraints, neither requiring object centering nor resorting to generative priors. It is complementary with learned shape and diffusion priors, enabling hybrid pipelines (e.g., combining VisDom with GaussianObject's diffusion regularizer) to further boost quality.

The theoretical implication is that visible domain co-visibility is a sufficient spatial prior for geometric regularization in sparse-view NVS, independently of explicit photometric or generative priors. This geometric constraint is also robust across domains, with minimal assumptions about object scale or pose.

Practical Implications and Future Directions

Practically, VisDom enables high-fidelity NVS from as few as four training images, unlocking casual 3D capture, robotics, and AR/VR settings where dense capture is infeasible. Integration is trivial—a single preprocessing step and a modification of ray sampling bounds or Gaussian initialization—making it a drop-in solution for existing pipelines.

Future directions include combining the visible domain hull constraint with learned generative priors to push further into the extreme sparse regime, developing adaptive strategies for $K$5 selection, and exploring its impact in dynamic scene or non-rigid reconstruction and real-time applications.

Conclusion

VisDom introduces a principled, learning-free geometric constraint for sparse-view novel view synthesis, generalizing classical visual hulls with a multi-view co-visibility requirement. Empirical results show dramatic improvements in PSNR and visual quality, recovering failed methods and advancing the state of the art among explicit and implicit pipelines. The method is efficient, robust, and complementary with existing priors, setting a strong foundation for future research in sparse NVS and geometric scene regularization.

Figure 4: Effect of mask dilation on ZipNeRF+VisDom for MipNeRF360: mean PSNR increases with mask dilation, indicating robustness and opportunity for fine-tuning segmentation-induced hulls.