- The paper introduces a minimalist Frequency-Preserving Adapter that injects high-frequency priors into a feed-forward 3D Gaussian Splatting pipeline.
- It leverages 2D Discrete Wavelet Transform to decompose features into multi-scale frequency bands, enhancing geometric sharpness and structural detail.
- Quantitative results demonstrate improved PSNR, SSIM, and robust zero-shot generalization across benchmarks like RealEstate10K and MipNeRF360.
Minimalist Adaptation for 3D Gaussian Splatting via Frequency-Preserving Adapters
Introduction
The paper "AdaptSplat: Adapting Vision Foundation Models for Feed-Forward 3D Gaussian Splatting" (2605.10239) proposes a new paradigm for generalizable novel-view synthesis using feed-forward 3D Gaussian Splatting (3DGS). The central thesis is that high reconstruction fidelity and cross-domain generalization can be achieved without substantial architectural complexity. By introducing a lightweight Frequency-Preserving Adapter (FPA) of just 1.5M parameters within a generic feature extraction–multi-view interaction–feature decoding pipeline, the authors demonstrate significant improvements in geometry reconstruction, specifically in high-frequency fidelity and geometric sharpness.
Background and Motivation
Conventional feed-forward 3DGS methods have increasingly adopted complex, architecture-specific module designs and heuristic training strategies. Despite this, three key limitations persist: 1) heavy reliance on handcrafted inductive biases and over-engineering; 2) generalization bottlenecks from treating vision foundation models (VFMs) as frozen feature extractors; 3) the spectral bias of deep networks, which results in the over-smoothing and loss of high-frequency geometric details. These deficiencies hinder both fidelity—particularly in the recovery of sharp edges and surfaces—and cross-domain robustness.
Prior attempts to mitigate these issues often involved decoupled or manually patched solutions, such as fusing monocular depth [37], dynamic pose estimation [40], or adding geometric inductive biases via attention mechanisms [39]. Nevertheless, such strategies have failed to fully close the domain generalization gap or to recover high-frequency structures in ill-posed inverse problems.
Methodology
Pipeline Abstraction
AdaptSplat adheres to the generic 3DGS pipeline: (1) image feature extraction (using DINOv3-ConvNeXt), (2) multi-view Transformer interaction, and (3) DPT-based decoding to regress 3D Gaussian parameters. No component is re-architectured or over-specialized. Instead, a single FPA module is inserted to bridge 2D VFM priors and 3D geometric constraints.
Frequency-Preserving Adapter (FPA)
The FPA exploits 2D Discrete Wavelet Transform (DWT) to decompose shallow backbone features into multi-scale, direction-aware frequency bands (LL, LH, HL, HH). The LH and HL sub-bands encode high-frequency energy along canonical axes, yielding explicit structural priors about local geometry boundaries.
Two mechanisms facilitate the integration of these priors:
- High-Frequency Guided Attention Positional Encoding: FPA-extracted frequency signals are injected as positional encodings in the Query-Key space of the Transformer attention mechanism. This modulates attention to be sensitive to local structural changes, improving the correspondence of high-frequency features across views.
- Multi-scale Adaptive Residual Modulation: During feature upsampling in the decoder, high-frequency priors are used to spatially gate residual connections. This preserves detail in high-frequency regions despite low-pass losses from interpolation.
Together, these mechanisms guide Gaussian parameter regression toward anisotropic, directionally-aligned representations, reducing the isotropic degeneration that arises in standard deep pipelines.
Training Objective
End-to-end optimization leverages a composite loss: pixel-wise MSE, perceptual LPIPS, focal frequency loss (FFL) to emphasize high-frequency detail, and opacity regularization. This explicitly enforces both low-level fidelity and high-frequency alignment in the reconstructed views.
Experimental Results
On the RealEstate10K benchmark with 6 input and 8 target views, AdaptSplat outperforms all prior baselines, increasing PSNR to 33.86, with simultaneous improvements in SSIM (0.956) and perceptual error (LPIPS 0.062) over MVP (PSNR 32.89, SSIM 0.948, LPIPS 0.067). The lightweight "tiny" variant maintains comparable reconstruction quality with a significantly reduced parameter count.
On DL3DV at higher resolutions (960×540, up to 128 input views), AdaptSplat consistently surpasses MVP, Long-LRM, and iLRM across all metrics, with the advantage widening as the scene scale increases.
Zero-Shot Generalization
Without any retraining, AdaptSplat demonstrates robust zero-shot performance on diverse datasets (Tanks&Temples and MipNeRF360), maintaining superiority over MVP and other baselines in both PSNR and visual sharpness—especially as the number of input views scales. Notably, on MipNeRF360, AdaptSplat attains PSNR improvements of up to +0.84 dB over MVP.
Ablation and Component Analysis
Ablations confirm that each architectural innovation—unfreezing the VFM, high-frequency guided attention, FPA, and FFL—contributes to the final performance. The combinations are synergistic: multi-scale FPA leads to the largest fidelity gains (PSNR increases from 21.12 to 22.10 on DL3DV ablation subset).
FPA's DWT-based extraction outperforms alternatives (Fourier, hand-crafted convolutional, Sobel) for high-frequency prior construction. Fractional Anisotropy (FA) measurements demonstrate that FPA significantly mitigates isotropic degeneration in Gaussian parameterization (FA increases from 0.8015 to 0.8423).
AdaptSplat also achieves favorable efficiency: the "tiny" model is both more parameter-efficient and faster than YoNoSplat and MVP, supporting superior scalability for practical deployment.
Implications and Future Directions
The results affirm that over-complicating the 3DGS pipeline is neither necessary nor efficient. Careful adaptation and injection of 2D structural priors—particularly high-frequency content—provide domain robustness and geometric accuracy without incurring the costs of extensive retraining or elaborate architecture design.
From a practical perspective, AdaptSplat's parameter efficiency and generalization establish a new baseline for scalable 3D view synthesis that is suitable for deployment across varied datasets and visual domains. Theoretically, the findings advocate for a shift in focus: rather than pipeline customization, future efforts may yield greater returns by advancing frequency-aware, plug-and-play adapters that unlock VFM priors.
Potential future research avenues include:
- Extending adapter-based high-frequency preservation methods to other 3D generative tasks or representations (e.g., point clouds, meshes)
- Co-adapting different VFMs (e.g., vision-LLMs) for multi-modal 3D reasoning
- Investigating more advanced frequency injection and gating schemes, possibly leveraging adaptive band selection or cross-attention with multi-level priors
- Exploring real-time rendering constraints and further parameter reductions for deployment on edge devices
Conclusion
AdaptSplat demonstrates that a minimalist, frequency-aware adaptation of vision foundation models suffices to achieve state-of-the-art feed-forward 3D Gaussian Splatting. A single 1.5M-parameter Frequency-Preserving Adapter effectively unlocks the geometric and generalization capacity of pre-trained backbones, resolving the limitations of isotropic degeneration and over-smoothing. The presented results establish AdaptSplat as a new baseline for 3DGS, refocusing the research community on adapter engineering over architectural proliferation for generalizable, efficient 3D reconstruction (2605.10239).