InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis

Abstract: Single-image feed-forward 3D Gaussian Splatting (3DGS) aims to directly generate a renderable 3D scene representation from one input image, avoiding the cost of multi-view capture and per-scene optimization. However, existing methods are often constrained by a pixel-aligned representation, where Gaussians are predicted from fixed image-grid locations. Such pixel-aligned primitives can produce promising nearby-view renderings, but they remain weakly coupled to underlying scene surfaces and struggle to preserve coherent structures under large viewpoint shifts. We present InfiniSplat, a feed-forward single-image 3DGS framework that moves from a pixel-aligned representation toward a surface-aligned representation. InfiniSplat constructs this representation by first using geometry-guided sampling to place 2D supports according to depth-induced local surface structure, and then applying a query-conditioned implicit decoder to predict Gaussian attributes from the image features queried at these supports. By grounding support locations in geometry while decoupling Gaussian prediction from fixed pixel centers, InfiniSplat produces Gaussian layouts that better follow scene surfaces and reduce scattered primitives caused by grid discretization. Across multiple cross-dataset NVS evaluations, InfiniSplat achieves state-of-the-art performance compared with single-image feed-forward baselines, and demonstrates zero-shot generalization from Hypersim indoor synthetic training to complex open-world scenes. Project page: https://zju3dv.github.io/InfiniSplat.

• The paper introduces a novel surface-aligned representation with a two-stage pipeline combining geometry-guided sampling and an MLP-based implicit Gaussian decoder.
• It demonstrates superior performance over baselines, achieving notable improvements in PSNR and SSIM on diverse indoor and outdoor datasets under large viewpoint shifts.
• Extensive ablation studies confirm that learned Gaussian updates, dual-branch feature fusion, and regularization are key to enhancing scene coherence and rendering quality.

InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis

Overview and Motivation

InfiniSplat introduces a feed-forward framework for single-image 3D Gaussian Splatting (3DGS), targeting robust large-baseline novel view synthesis (NVS) given only a single image. Unlike prior feed-forward single-image 3DGS methods, which generate Gaussian primitives at fixed image-grid locations (pixel-aligned representations), InfiniSplat employs a surface-aligned representation. The explicit goal is to generate 3D scene structures that remain coherent and visually plausible under large viewpoint shifts, addressing fundamental limitations of pixel-aligned approaches that suffer from surface tearing, structural breakage, and geometric artifacts, especially at object boundaries and planar regions.

Methodology

The core methodological innovation of InfiniSplat is its two-stage pipeline for geometry-aware Gaussian generation:

1. Geometry-Guided Sampling:

InfiniSplat leverages pretrained monocular depth estimators to provide dense scene depth maps. By back-projecting image pixels using predicted depth and camera intrinsics, it estimates 3D surface areas and then samples 2D supports via area-weighted strategies. Local 3D triangles with greater area (i.e., surface slant/size) receive proportionally more samples, enabling effective allocation of representation capacity to regions with higher geometric complexity or relevance.

2. Implicit Gaussian Decoding:

An MLP-based decoder predicts bounded updates to each base Gaussian, conditioned on a support-specific, fused feature vector. Features are queried bilinearly at each support location from two parallel encoder branches:

• A DINO (ViT backbone) branch for high-level semantic context.
• A CNN branch for low-level texture and boundary information.

Feature fusion is achieved via a gated mechanism, balancing semantic and appearance cues. The decoder outputs residuals for position, scale, rotation, color, and opacity, composed onto the base Gaussians generated during geometry-guided sampling. This design enables adaptation to spatially irregular, surface-aligned supports, decoupling placement from the pixel grid (see (Figure 1)).

Figure 1: InfiniSplat pipeline — geometry priors and image features are fused with sampled supports to implicitly decode surface-aligned 3D Gaussians for feed-forward NVS.

The joint effect of geometry-guided sampling and implicit decoding is critical. The system is not merely upsampling or lifting grid points but aligning supports with depth-induced local surface patches, resulting in more coherent surfaces and fewer disconnected splats.

Experimental Results

Quantitative Evaluation:

InfiniSplat establishes state-of-the-art results in cross-dataset zero-shot NVS benchmarks spanning ETH3D, ScanNet++, Tanks-and-Temples, and DL3DV. For the RGB-only (monocular) setting, InfiniSplat-RGB outperforms SHARP and other baselines by substantial margins:

• On average: $+1.9$ PSNR, $+0.048$ SSIM over SHARP; improvements relative to Flash3D/Flash3D-DepthPro often exceed $+3.0$ PSNR.
• For LiDAR-conditioned settings, InfiniSplat-LiDAR surpasses ADGaussian by over $+10$ PSNR and $+0.2$ SSIM.

These numerical results are consistent across both indoor and outdoor scenes and hold even for large-baseline camera motions.

Qualitative Analysis:

InfiniSplat yields fewer visible cracks, maintains structural coherence across planar regions and at object boundaries, and generates smoother, more continuous normal maps compared to SHARP and Flash3D (see (Figure 2) and (Figure 3)). Under the challenging LiDAR-prompted setting, InfiniSplat-LiDAR significantly surpasses ADGaussian regarding stability and visual fidelity (see (Figure 4)). It also generalizes capably to "in-the-wild" images, demonstrating resilience to complex geometry and lighting variations (see (Figure 5)).

Figure 2: InfiniSplat-RGB shows superior scene coherence and reduced cracks under large viewpoint shifts compared to prior feed-forward methods.

Figure 3: Surface and normal comparison: InfiniSplat produces cleaner, more coherent surface organization than SHARP.

Figure 4: InfiniSplat-LiDAR produces stronger novel-view rendering and surface integrity than ADGaussian with sparse LiDAR conditioning.

Figure 5: InfiniSplat demonstrates robustness and high detail on in-the-wild monocular images.

Model Ablations

A comprehensive ablation study disentangles contributions from key architectural choices:

• Learned Gaussian Updates: Disabling residual updates leads to degraded sharpness and loss of local detail, validating the necessity for learned parameter refinement (see (Figure 6)).
• Dual-branch Feature Encoders: Removing the DINO branch impairs semantic consistency and cross-dataset transfer; removing the CNN branch diminishes edge definition and appearance quality (see (Figure 7)).
• Gaussian Regularization: Regularization stabilizes scale and opacity, preventing degenerate Gaussian attributes and enhancing surface continuity (see (Figure 8)).
• Geometry-Guided Sampling & Implicit Decoding: Reverting to pixel-aligned supports or removing implicit decoding re-introduces cracks, holes, and scene breakages, particularly under large-baseline viewpoints (see (Figure 9)).

  Figure 6: Without learned Gaussian updates, scenes remain coarse, blurry, and lacking in local structure.

  

  Figure 7: Ablations of feature branches — removal of CNN or DINO weakens rendering detail and generalization, respectively.

  

  Figure 8: Absence of Gaussian regularization leads to degenerate, unstable rendered regions.

  

  Figure 9: Geometry-guided sampling and implicit decoding are crucial to avoid cracks and discontinuities under novel views.

Support budget ablations indicate $1.5$ million supports achieve a favorable trade-off between computational efficiency and reconstruction quality.

Limitations and Failure Analysis

InfiniSplat’s main limitations arise from monocular depth ambiguity and reliance on pretrained depth priors, which can propagate errors to support placement and ultimately affect Gaussian layout. Failure modes include incomplete geometry under heavy disocclusion, artifacts where depth estimation is unreliable, and structural distortions under extreme viewpoint extrapolations (see (Figure 10)). Further, regions involving reflective, transparent, or textureless surfaces remain challenging due to the limitations of monocular geometric prediction.

Figure 10: Qualitative failure cases — incomplete geometry, inherited depth artifacts, and distortion under extreme novel views.

Implications and Future Directions

InfiniSplat demonstrates that surface-aligned Gaussian representations, constructed via geometry-aware sampling and implicit decoding, can robustly advance the state of feed-forward monocular NVS. Practically, this architecture enables high-quality, real-time NVS for AR/VR, spatial imaging applications, and interactive navigation using single input images or RGB-plus-sensor inputs.

Theoretically, this work motivates deeper investigation into:

• Geometry-aware sampling paradigms, especially when fusing with generative priors or non-local cues.
• Query-conditioned neural decoders in explicit scene representation contexts beyond 3DGS.
• Structured regularization for explicit primitive sets to prevent degeneration during unconstrained feed-forward prediction.

Future developments may incorporate generative priors (e.g., guided diffusion), uncertainty-aware geometry bootstrapping, or extend to sparse multi-view settings while preserving the efficiency and robustness of the presented approach.

Conclusion

InfiniSplat articulates and validates a surface-aligned alternative to pixel-based feed-forward 3DGS for monocular NVS. Through explicit depth-guided support sampling and query-conditioned parameter decoding, it achieves quantitatively and qualitatively stronger, structurally stable novel-view synthesis. The work delineates a clear path for future geometry-aware, feed-forward 3D scene generation systems in general-purpose computer vision and graphics.