---
title: Feed-Forward Gaussian Splatting
url: https://www.emergentmind.com/topics/feed-forward-gaussian-splatting
type: topic
---

# Feed-Forward Gaussian Splatting

Feed-forward Gaussian splatting denotes a class of generalizable reconstruction pipelines in which a pretrained network predicts a scene-specific set of Gaussian primitives from one or more input images in a single forward pass, rather than optimizing Gaussian parameters per scene. Across recent formulations, the predicted scene is represented as a set such as
\[
\mathcal{G}=\{(\boldsymbol{\mu}_j,\boldsymbol{\Sigma}_j,\mathbf{h}_j,\alpha_j)\}_{j=1}^{N_g},
\]
or an equivalent parameterization by position, covariance, rotation, scale, opacity, and spherical-harmonic color, and a differentiable splatting renderer produces target images by projecting these Gaussians into the image plane and compositing them along each ray [2605.26629][2505.23716].

## 1. Definition and representational core

In the feed-forward setting, the central operation is a learned mapping from sparse observations to a renderable Gaussian scene. This contrasts with classic 3DGS, where each new scene requires per-scene gradient descent. Recent feed-forward systems therefore replace scene-specific optimization with a reusable predictor that outputs Gaussian parameters directly from images, often together with depth, camera parameters, or other auxiliary scene variables [2505.23716].

The representational core remains close to standard 3DGS. Each Gaussian typically carries a 3D center \(\boldsymbol{\mu}\), a positive-definite covariance \(\boldsymbol{\Sigma}\) or its scale-rotation reparameterization, an opacity \(\alpha \in (0,1)\), and view-dependent color encoded by spherical harmonics. Rendering is performed by projecting each Gaussian into the target camera, transforming \(\boldsymbol{\Sigma}\) into a 2D elliptical footprint, and using front-to-back alpha compositing,
\[
C_{\text{out}} = \sum_j w_j c_j,\qquad
w_j = \alpha_j \prod_{k<j}(1-\alpha_k),
\]
or an equivalent rasterized form [2605.26629][2412.12096].

A major axis of variation is the camera model assumed at inference. Some methods are posed and depth-centric, taking calibrated sparse views and estimating geometry by multi-view reasoning. Others are pose-free or pose-light: AnySplat predicts camera intrinsics and extrinsics jointly with the Gaussian scene from uncalibrated image collections, WildSplatter relies on per-pixel ray origins and directions from a Depth Anything 3 backbone, and DrivingForward uses a pose network only during training and discards it at inference [2505.23716][2604.21182][2409.12753]. This establishes feed-forward Gaussian splatting not as a single architecture, but as a family of explicit radiance-field predictors with different assumptions about geometry, calibration, and appearance.

## 2. Architectural lineages

One dominant lineage is cost-volume-based multi-view inference. DelowlightSplat, DenoiseSplat, PanSplat, and ProSplat all follow an MVSNet-style paradigm: per-view features are extracted, warped across depth hypotheses, aggregated into a 3D cost volume, regularized by 3D CNNs or attention, and decoded into depth and Gaussian attributes [2605.26629][2603.09291][2412.12096][2506.07670]. Within this family, the Gaussian center is often obtained by back-projecting a pixel-depth hypothesis,
\[
\boldsymbol{\mu}(\mathbf{u},d)=\mathbf{o}+d\,\mathbf{r}(\mathbf{u};\mathbf{K},\mathbf{T}),
\]
after which scale, rotation, opacity, and SH coefficients are predicted by lightweight heads [2605.26629].

A second lineage replaces explicit cost volumes with transformer token reasoning over views. AnySplat uses a VGGT-style geometry transformer with camera, depth, and Gaussian heads, and then applies differentiable voxelization to merge per-pixel Gaussians into a compact renderable set [2505.23716]. The "Z-Order Transformer for Feed-Forward Gaussian Splatting" serializes candidate Gaussians by Morton order, applies sparse attention over spatially coherent sequences, and uses Z-order pooling to suppress redundancy before Gaussian prediction [2605.13465]. AnyCity introduces an observation-supported geometry latent \(Z_{\mathrm{geo}}\) and a separate completion branch that predicts a gated residual \(\Delta Z\) before Gaussian decoding, explicitly separating reliable scaffold geometry from prior-driven completion in sparse aerial scenes [2605.19949].

A third lineage emphasizes independent per-view geometry with later aggregation. DrivingForward jointly trains pose, depth, and Gaussian networks for surround-view driving scenes, predicting one Gaussian per pixel from each image and aggregating them without a global multi-view transformer at inference [2409.12753]. Splat-SAP uses scale-aware point maps as an intermediate representation robust to large sparsity, then refines target-view geometry via stereo matching before anchoring Gaussian primitives on the refined plane [2511.22704]. These designs treat feed-forward Gaussian splatting less as direct regression from fused latent space and more as a structured lift from image-wise geometry to explicit splats.

## 3. Conditioning, robustness, and generative refinement

A significant recent trend is condition-aware adaptation. DelowlightSplat targets the setting of lowlight context views and clean target views. Its Lowlight Adapter applies residual enhancement,
\[
\tilde I_i=\mathrm{clip}\big(I_i+\lambda\cdot\tanh(\Delta(I_i))\big),
\]
before any cost-volume construction, with the explicit goal of improving feature matchability rather than standalone image restoration [2605.26629]. DenoiseSplat addresses Gaussian, Poisson, speckle, and salt-and-pepper noise by training a MVSplat-style backbone end-to-end from noisy inputs to clean renderings, and adds a dual-branch Gaussian head that decouples geometry from appearance, plus a Cross-Branch Boundary-guided Correction module for uncertain depth boundaries [2603.09291]. In both cases, the feed-forward scene predictor is optimized to recover a clean Gaussian radiance field from degraded observations rather than reproduce the corruption.

A related strand separates geometry from appearance under severe viewpoint or illumination variation. WildSplatter explicitly disentangles geometry and appearance by predicting geometry from context images and a global appearance embedding from a reference image, then modulating Gaussian SH colors with that embedding; it reconstructs 3D Gaussians from sparse input views in under one second and enables appearance control under diverse lighting conditions [2604.21182]. ProSplat uses a two-stage design for wide-baseline sparse views: a DepthSplat-based Gaussian generator first renders target views, then a one-step diffusion improvement model refines those renderings using Maximum Overlap Reference view Injection and Distance-Weighted Epipolar Attention [2506.07670]. AnyCity extends this logic into a controlled generative regime by letting completion tokens act only as scaffold-conditioned residuals on weakly constrained content [2605.19949]. This suggests a broader shift from purely deterministic Gaussian regression toward reconstruction-first systems with explicit condition adaptation and tightly bounded generative completion.

## 4. Task-specific expansions

Feed-forward Gaussian splatting has rapidly specialized to particular sensing regimes. PanSplat adapts the paradigm to wide-baseline panorama synthesis by replacing pixel-aligned splats with a spherical 3D Gaussian pyramid on a Fibonacci lattice, combining a hierarchical spherical cost volume with local Gaussian heads and two-step deferred backpropagation to support resolutions up to 4K [2412.12096]. The representational change is geometric as much as computational: spherical sampling avoids the non-uniform density induced by equirectangular pixel grids.

Sparse aerial reconstruction introduces a different failure mode, namely evidence imbalance between repeatedly observed roofs and weakly supported facades or occluded structures. AnyCity addresses this by first predicting an observation-grounded scaffold latent and only then using scaffold-conditioned completion tokens and a video diffusion prior to update weak regions before Gaussian decoding [2605.19949]. In driving scenes, DrivingForward replaces strong overlap assumptions with scale-aware self-supervised depth and pose learning from flexible surround-view input, allowing one or multiple timesteps and different camera combinations without depth ground truth or per-frame extrinsics during training [2409.12753]. In human-centered wide-baseline binocular settings, Splat-SAP uses scale-aware point map reconstruction, iterative affinity learning, and a target-view Gaussian plane to maintain stability under extreme sparsity [2511.22704].

The same expansion has occurred in pose-free and semantic directions. AnySplat handles uncalibrated image collections and predicts both camera parameters and Gaussians in one pass, from single view up to hundreds of views [2505.23716]. UFV-Splatter adapts pose-free pretrained models to unfavorable views by recentering inputs, inserting LoRA layers into the backbone, and refining predicted Gaussians with a dedicated adapter and alignment stage [2507.22342]. FLEG extends the output space itself: each semantic Gaussian carries a language-aligned feature vector in addition to geometry and color, enabling open-vocabulary querying and segmentation from arbitrary unposed, uncalibrated multi-view images without 3D annotations [2512.17541]. Feed-forward Gaussian splatting therefore increasingly serves not only novel-view synthesis, but also camera estimation, semantic lifting, and controllable scene interaction.

## 5. Resolution, redundancy, and compression

A central systems challenge is that many feed-forward models are pixel-aligned, so primitive count grows with image resolution. PanSplat addresses this for panoramas through spherical lattices, local Gaussian heads, tiled processing, and deferred backpropagation on a single A100 GPU [2412.12096]. LGTM attacks the same scaling law more directly by decoupling geometry from rendering resolution: it predicts a compact grid of 2D Gaussian primitives and attaches per-primitive color and alpha textures, allowing high-fidelity 4K synthesis with significantly fewer Gaussian primitives than pixel-aligned feed-forward methods [2603.25745]. The Z-Order Transformer further treats redundancy as a sequence-modeling problem: candidate Gaussians are Morton-ordered, attended over with sparse group and top-\(k\) attention, and pooled to produce fewer Gaussian primitives while preserving critical structural details [2605.13465].

Compression has become a distinct subfield. CodecSplat moves the coding bottleneck from the irregular 3D Gaussian set back into the structured intermediate 2D Gaussian-generation feature, entropy-codes that latent, and reconstructs depth and Gaussian parameters on the decoder side. On DL3DV and RealEstate10K, it achieves \(23.56\!-\!26.36\) dB and \(24.76\!-\!27.05\) dB PSNR with only \(20.00\!-\!107.77\) KiB and \(3.37\!-\!12.51\) KiB per scene, respectively [2605.25563]. A complementary feed-forward codec based on long-context modeling uses Morton serialization, attention-based transforms, and a space-channel auto-regressive entropy model over windows of thousands of Gaussians, and yields a \(20\times\) compression ratio for 3DGS in a feed-forward inference [2512.00877]. These developments suggest that feed-forward Gaussian splatting is increasingly being optimized as a deployable scene representation, not merely as a synthesis pipeline.

## 6. Supervision, evaluation, and open problems

Training signals vary widely but remain overwhelmingly image-driven. Many systems use pixel and perceptual losses between rendered and target views, sometimes with SSIM or LPIPS terms, and often without any 3D ground truth [2605.26629][2603.09291]. Others supplement rendering loss with geometry-side supervision from pseudo-depth or pseudo-camera estimates: AnySplat distills pose and depth from VGGT while enforcing rendering and depth consistency [2505.23716]; FLEG combines photometric novel-view supervision with geometry distillation and instance-guided contrastive feature learning [2512.17541]; DrivingForward uses self-supervised temporal, spatial, and spatial-temporal photometric consistency to train depth and pose before Gaussian prediction [2409.12753]. Evaluation is correspondingly dominated by PSNR, SSIM, and LPIPS, with depth metrics, pose metrics, or semantic mIoU added when the task requires them.

Several limitations recur across the literature. Under extremely low SNR, severe artifacts, strong motion blur, or pose inaccuracies, feature matchability and cost-volume alignment can still fail [2605.26629]. Sparse aerial reconstruction remains vulnerable to extreme occlusions in dense skyscraper clusters and to domain shift in the aerial video prior [2605.19949]. PanSplat remains limited by dynamic scenes, where moving content induces misaligned Gaussians and ghosting [2412.12096]. LGTM explicitly depends on geometry quality and still requires manual texture-resolution selection [2603.25745]. More generally, many feed-forward Gaussian models remain tied to static-scene assumptions, fixed training distributions, or pixel-aligned primitive growth.

Current work points in several directions. One is richer condition-aware adaptation: lowlight, noise, appearance variation, and unfavorable views are already being handled by adapters, residual branches, and explicit appearance latents [2604.21182][2507.22342]. Another is controlled generative augmentation that preserves observation-supported geometry, as in AnyCity and ProSplat [2605.19949][2506.07670]. A third is semantic enrichment, where language-aligned Gaussian fields support open-vocabulary interaction rather than only rendering [2512.17541]. A plausible implication is that feed-forward Gaussian splatting is evolving from a fast approximation to optimization-based 3DGS into a broader family of explicit scene predictors that unify reconstruction, rendering, compression, and semantic control. A nearby line of work even replaces Gaussians with directly predicted triangle primitives when explicit manifold geometry is required for simulation, underscoring that the feed-forward principle now extends beyond Gaussians themselves [2603.05932].

Source: https://www.emergentmind.com/topics/feed-forward-gaussian-splatting