GIFSplat: Iterative Feed-forward 3D Reconstruction

• GIFSplat is a feed-forward 3D reconstruction framework that employs iterative residual refinement to enhance sparse-view novel-view synthesis.
• It integrates a frozen diffusion enhancer into Gaussian splatting to distill generative prior cues, boosting detail without expensive test-time optimization.
• The method achieves improved PSNR, SSIM, and LPIPS metrics across benchmarks while maintaining second-scale inference speed.

Searching arXiv for GIFSplat and a few directly related feed-forward 3DGS baselines to ground the article with current paper metadata and arXiv IDs. arxiv_search(query="GIFSplat Generative Prior-Guided Iterative Feed-Forward 3D Gaussian Splatting from Sparse Views", max_results=5) GIFSplat is a feed-forward 3D reconstruction framework for sparse-view novel-view synthesis that combines iterative residual refinement with Gaussian-level distillation of a frozen diffusion prior. It operates on unposed input views, predicts an initial 3D Gaussian Splatting representation and camera parameters in a single pass, and then applies a small fixed number of forward-only update steps to refine the scene using rendering discrepancies and, optionally, generative prior cues. The method is presented as an alternative to both per-scene optimization, which is slow at inference and fragile under very sparse views, and one-shot feed-forward pipelines, whose single-pass capacity limits fidelity and scene-specific adaptation (Chen et al., 26 Feb 2026).

1. Problem setting and motivation

GIFSplat is situated in the context of sparse-view 3D reconstruction and novel-view synthesis, particularly the trade-off between per-scene optimization and feed-forward inference. Traditional per-scene 3D reconstruction methods, including NeRF and 3DGS-style optimization, require thousands of gradient steps at test time. According to the paper, this can yield high quality but remains slow and fragile under very sparse views. By contrast, recent feed-forward schemes such as AnySplat, MVSplat, and FLARE predict a scene in one shot and run in milliseconds to seconds, but their prediction quality is bounded by the representational and inference capacity of a single forward pass (Chen et al., 26 Feb 2026).

The paper attributes three limitations to the one-shot prediction paradigm. First, the model is strictly bounded by capacity. Second, there is no inference-time refinement mechanism. Third, the architecture is ill-suited for continuously injecting generative priors. GIFSplat addresses these limitations by replacing one-shot prediction with iterative feed-forward refinement while preserving second-scale runtime. This suggests a design point between classical test-time optimization and fixed single-pass prediction: the model remains forward-only at inference, yet is no longer restricted to a single estimate of the scene.

A related motivation concerns generative priors. The paper states that 2D diffusion priors can fill in missing high-frequency detail under sparse coverage, but earlier approaches often incorporate them through an expensive loop involving optimization and view-set expansion. GIFSplat instead distills a frozen diffusion prior into Gaussian-level cues without gradient backpropagation through the prior and without ever-increasing view-set expansion (Chen et al., 26 Feb 2026).

2. Iterative feed-forward refinement architecture

The architecture is organized into an initialization stage followed by an iterative residual update stage. The input is a sparse uncalibrated view set $V=\{I_m\}_{m=1}^M$. A Gaussian initializer $F_\phi$, adapted from AnySplat but with no voxelization, predicts both camera parameters $\{\Pi_m\}$ and an initial 3D Gaussian Splatting representation $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$ from a single forward pass. Each Gaussian is represented as $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$, where the state includes position, scale, rotation, color, and opacity (Chen et al., 26 Feb 2026).

The core refinement mechanism unrolls a small fixed number $T$ of applications of a weight-shared residual update module $U_\theta$. At iteration $t$, the current scene $G^{(t)}$ is rendered into each reference view through a differentiable Gaussian splat rasterizer:

$R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$

The method then forms frozen-feature discrepancies, termed observation cues, by comparing the input image and the current rendering in the feature space of a frozen extractor $F_\phi$0:

$F_\phi$1

These pixel-level discrepancies are pooled to each Gaussian using soft assignment weights $F_\phi$2:

$F_\phi$3

Given the current Gaussian state, observation cues, and optional prior cues, the residual head predicts an additive update:

$F_\phi$4

$F_\phi$5

This formulation makes the refinement process explicitly feed-forward: each step uses the current rendered evidence to update the scene state, but no test-time gradient optimization is performed. The paper emphasizes that the update module is lightweight and weight-shared across iterations, which is central to keeping inference in the second-scale regime (Chen et al., 26 Feb 2026).

3. Diffusion-prior distillation into Gaussian-level cues

A distinguishing feature of GIFSplat is its use of a frozen diffusion enhancer to provide generative prior information during refinement. At each iteration, the current rendering can be passed through a frozen single-step diffusion enhancer $F_\phi$6, exemplified in the paper by Difix3D+:

$F_\phi$7

Prior cues are then defined in feature space as the difference between the enhanced rendering and the original rendering:

$F_\phi$8

As with observation cues, these pixel-level prior cues are pooled to the Gaussian level using the same soft assignment weights, producing $F_\phi$9 for each Gaussian. The prior cues are concatenated to the input of the residual head, but the method does not backpropagate through $\{\Pi_m\}$0 (Chen et al., 26 Feb 2026).

This design is technically important because it separates generative enhancement from optimization. The diffusion model is frozen, no gradients flow through it, and the scene representation is still updated through forward-only residual prediction. The paper explicitly contrasts this with pipelines that integrate generative priors through optimization loops and growing view sets. A plausible implication is that the prior acts as a structured hint for scene completion and detail enhancement rather than as an optimization target in its own right.

The paper also reports a variant without the generative prior, termed IFSplat. This ablation isolates the effect of iterative feed-forward refinement alone from the additional effect of diffusion-prior cue injection.

4. Training objectives, inference procedure, and implementation

Training is divided into two stages. In Stage I, the initializer $\{\Pi_m\}$1 is trained with a photometric reconstruction term and a geometric distillation term from AnySplat. The reconstruction loss is

$\{\Pi_m\}$2

and the stage objective is

$\{\Pi_m\}$3

In Stage II, the iterative refinement module is trained over unrolled steps. The stage objective is

$\{\Pi_m\}$4

where $\{\Pi_m\}$5 are weighting factors. The training pseudocode in the paper specifies that $\{\Pi_m\}$6 is partially pretrained on geometric distillation, Stage I updates $\{\Pi_m\}$7 via AdamW, and Stage II updates $\{\Pi_m\}$8 and optionally $\{\Pi_m\}$9 via AdamW (Chen et al., 26 Feb 2026).

Inference mirrors the refinement process used during training. For a sparse unposed test set $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$0, the initializer predicts $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$1 and $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$2, then the model repeats $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$3 iterations of rendering, cue computation, optional prior extraction, and residual update. The output may be either the final Gaussian set $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$4 or novel-view renders $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$5. The paper states explicitly that no test-time gradients or pose refinement are required beyond the single forward pass of $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$6 (Chen et al., 26 Feb 2026).

The implementation details reported in the paper specify the following components.

Component	Reported specification
Hardware	4×NVIDIA DGX H200 (141 GB each)
Optimizer	AdamW, weight decay $G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$7
Stage I learning rate	$G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$8 for 30K–50K steps
Stage II learning rate	$G^{(0)}=\{g_i^{(0)}\}_{i=1}^N$9 for 100K–300K steps
Batch size	4 scenes per GPU
Feature extractor $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$0	3-layer CNN, output 256-D per pixel
Diffusion enhancer $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$1	pretrained Difix3D+ single-step denoiser, frozen

The residual head $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$2 takes as input the Gaussian state together with a 256-D observation cue and a 256-D prior cue per Gaussian. It uses a feature MLP with 4 layers, hidden dimension 512, ReLU, plus local window-based self-attention over 3D neighbors implemented as a point transformer. Its output is $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$3. The initializer $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$4 uses a 12-layer ViT backbone for multi-view feature fusion with shared weights and cross-attention to predict camera and Gaussian set; at the 8-view setting it outputs $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$5 Gaussians per scene with downsampled geometry (Chen et al., 26 Feb 2026).

5. Quantitative performance and ablations

The paper reports results on RealEstate10K, DL3DV, and DTU, with comparisons to both pose-required and pose-free baselines. On RealEstate10K for 2-view novel-view synthesis at $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$6, GIFSplat with $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$7 is reported to achieve the best PSNR, SSIM, and LPIPS across Small, Medium, and Large overlap settings. The average PSNR is reported as $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$8, compared with the best baseline at $g_i=(x_i, s_i, r_i, c_i, \alpha_i)$9, a gain of $T$0. The paper further states that IFSplat, without the prior, already outperforms all baselines with average PSNR $T$1, and that GIFSplat adds $T$2 over IFSplat while improving boundary fidelity (Chen et al., 26 Feb 2026).

On DL3DV for 8-view novel-view synthesis at $T$3, the reported numbers are: IFSplat with PSNR $T$4, SSIM $T$5, LPIPS $T$6; GIFSplat with PSNR $T$7, SSIM $T$8, LPIPS $T$9; and the best baseline, AnySplat, at $U_\theta$0. On cross-domain evaluation on DTU with 2 views and a model trained on RealEstate10K, AnySplat is reported at $U_\theta$1, IFSplat at $U_\theta$2, and GIFSplat at $U_\theta$3, corresponding to a $U_\theta$4 gain over AnySplat (Chen et al., 26 Feb 2026).

Dataset / setting	Key reported result
RealEstate10K, 2-view, $U_\theta$5	GIFSplat avg PSNR $U_\theta$6; best baseline $U_\theta$7
DL3DV, 8-view, $U_\theta$8	GIFSplat: PSNR $U_\theta$9, SSIM $t$0, LPIPS $t$1
DTU cross-domain, 2-view	GIFSplat: $t$2; AnySplat: $t$3

The ablation study on RealEstate10K 2-view quantifies the contributions of refinement, attention, and the generative prior. Without refinement ($t$4), the reported metrics are PSNR $t$5, SSIM $t$6, LPIPS $t$7. Removing window-attention in $t$8 yields PSNR $t$9, SSIM $G^{(t)}$0, LPIPS $G^{(t)}$1. IFSplat, which omits the generative prior, yields PSNR $G^{(t)}$2, SSIM $G^{(t)}$3, LPIPS $G^{(t)}$4. Full GIFSplat reaches PSNR $G^{(t)}$5, SSIM $G^{(t)}$6, LPIPS $G^{(t)}$7 (Chen et al., 26 Feb 2026).

The effect of iteration count is also explicitly reported. For IFSplat, PSNR increases from $G^{(t)}$8 at $G^{(t)}$9 to $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$0 at $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$1, $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$2 at $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$3, $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$4 at $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$5, and $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$6 at $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$7. For GIFSplat, the corresponding values are $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$8, $R_m^{(t)}=\mathcal{R}(G^{(t)};\Pi_m).$9, $F_\phi$00, $F_\phi$01, and $F_\phi$02. The paper concludes that improvements saturate around $F_\phi$03 (Chen et al., 26 Feb 2026).

6. Runtime characteristics, scope, and interpretation

The runtime evaluation is reported on NVIDIA DGX H200 hardware with 141 GB memory. Inference time grows roughly linearly with the number of refinement steps $F_\phi$04. For IFSplat, the reported timing is $F_\phi$05 at $F_\phi$06 and $F_\phi$07 at $F_\phi$08. For GIFSplat, the reported timing is $F_\phi$09 at $F_\phi$10 and $F_\phi$11 at $F_\phi$12, with a stated $F_\phi$13 overhead for the diffusion prior. These numbers support the paper’s claim that the method maintains second-scale inference despite iterative refinement and optional prior injection (Chen et al., 26 Feb 2026).

A common misconception would be to interpret GIFSplat as a form of test-time optimization because it performs multiple refinement steps. The paper is explicit that the method is purely feed-forward: refinement is implemented by repeated applications of a learned residual update module, not by gradient descent on a scene-specific objective. Another potential misunderstanding is to view the diffusion prior as a jointly optimized component. In fact, the diffusion enhancer is frozen, used in a single-step mode, and excluded from backpropagation.

The broader significance of GIFSplat lies in the architectural claim that feed-forward 3D reconstruction need not be one-shot. The method retains the deployment advantages of feed-forward inference while adding scene-specific adaptation through iterative residual correction. Its reported gains on RealEstate10K, DL3DV, and especially cross-domain DTU suggest that the combination of rendering-based observation cues and frozen-prior cues is particularly effective when sparse views and domain shift expose the limitations of single-pass prediction. A plausible implication is that the framework defines a reusable pattern for future sparse-view 3DGS systems: forward-only iterative updates can serve as an intermediate regime between static amortization and full per-scene optimization (Chen et al., 26 Feb 2026).