--- title: 'ViewSplat: Feed-forward 3D Gaussian Splatting' url: https://www.emergentmind.com/topics/viewsplat type: topic --- # ViewSplat: Feed-forward 3D Gaussian Splatting ViewSplat denotes feed-forward 3D Gaussian splatting architectures for novel-view synthesis, and the name has been used in two distinct but related senses in the recent literature. In one consolidated description, “ViewSplat” refers to the same three-stage, coarse-to-fine architecture as HiSplat, a hierarchical 3D Gaussian splatting framework for generalizable sparse-view reconstruction that performs novel view synthesis using only two-view reference images [2410.06245]. In a later paper, ViewSplat denotes a view-adaptive 3D Gaussian splatting network for novel view synthesis from unposed images, in which canonical Gaussians are refined on-the-fly through target-view-conditioned residual updates produced by pixel-wise dynamic MLPs [2603.25265]. In both usages, the underlying representation is a set of anisotropic 3D Gaussians rendered by a 3DGS rasterizer, but the mechanisms for improving fidelity differ: one emphasizes hierarchical coarse-to-fine correction, whereas the other emphasizes view-adaptive dynamic splatting. ## 1. Scope of the designation A consolidated description tied to HiSplat states that “ViewSplat” refers to the same three-stage, coarse-to-fine architecture, with “ViewSplat” used as the name of a hierarchical 3D-Gaussian-splatting pipeline for generalizable two-view reconstruction. The 2026 paper titled “ViewSplat: View-Adaptive Dynamic Gaussian Splatting for Feed-Forward Synthesis” uses the same term for a different pipeline centered on dynamic, target-view-conditioned residual refinement [2410.06245]. | Designation | Input regime | Distinguishing mechanism | |---|---|---| | ViewSplat as HiSplat | two-view reference images | hierarchical 3D Gaussians via a coarse-to-fine strategy | | ViewSplat (2026) | \(N\) unposed images | view-adaptive dynamic splatting with pixel-wise dynamic MLPs | This dual usage is significant because it prevents a common conflation. The hierarchical variant is organized around multi-stage Gaussian generation and inter-scale correction. The dynamic variant is organized around canonical Gaussians plus target-view residual offsets for 3D position, scale, rotation, opacity, and color. A plausible implication is that “ViewSplat” should be interpreted by publication context rather than by name alone. ## 2. Gaussian formulation and image formation In the hierarchical formulation, each Gaussian \(j\) is parameterized by \(\mu_j \in \mathbb{R}^3\), \(\Sigma_j \in \mathbb{R}^{3\times 3}\), \(c_j \in \mathbb{R}^3\), and \(\alpha_j \in [0,1]\). When projecting to an image, the contribution of Gaussian \(j\) to pixel \(p\) is $$ G_j(p) = \alpha_j \cdot c_j \cdot N_2(u_p; \Pi(\mu_j), \Pi \Sigma_j \Pi^T + \sigma_{\text{image}}^2 I_2), $$ where \(\Pi\) is the \(3\text{D}\to 2\text{D}\) projection and \(u_p\) is the pixel coordinate. Feed-forward prediction proceeds from a pixel-aligned feature \(F_i^{GS}(u)\), from which a small CNN head regresses the center via un-projecting the predicted depth \(D_i(u)\) along the camera ray \(r(u)\), opacity through a sigmoid head, covariance through predicted positive axis-scales, and color as a small vector or SH coefficients [2410.06245]. In the dynamic formulation, a single anisotropic 3D Gaussian primitive is $$ G(x) = w \exp\!\bigl(-\tfrac12 (x-\mu)^T \Sigma^{-1}(x-\mu)\bigr), $$ with \(\Sigma = R\,\mathrm{diag}(s \odot s)\,R^T\), where \(s \in \mathbb{R}^3\) are scale factors and \(R\) is derived from a rotation quaternion \(r \in \mathbb{R}^4\). In practice the representation stores \(\mu\), \(s\), \(r\), opacity \(\alpha\), and color as \(L\) spherical-harmonic coefficients \(c \in \mathbb{R}^L\). Rendering is expressed as $$ \hat I^t = \mathcal{R}(P^{t\to 1}, \{\hat{\mathcal{G}}^{v\to 1|t}\}_{v,j}), $$ where a 3DGS rasterizer projects each anisotropic Gaussian into the target view, splats its elliptical footprint into screen space, and accumulates color with alpha compositing in front-to-back order [2603.25265]. The shared formalism is the use of anisotropic Gaussians as explicit scene primitives. The divergence lies in how those primitives are predicted and corrected: multi-scale staged prediction in the hierarchical system, and target-view-conditioned residual adaptation in the dynamic system. ## 3. Hierarchical coarse-to-fine reconstruction The hierarchical ViewSplat builds three stages \(i=1,2,3\), ordered low-to-high resolution and coarse-to-fine. Stage 1 operates at input resolution \(H/4 \times W/4\) and uses an MVS-style cost-volume stereo block. Given cross-view features \(F_1\) from a U-Net+Transformer backbone, the method sweeps \(R\) depth planes, warps other views into view \(i\), builds a cost volume, predicts depth by softmax over the cost-volume slice, and then forms \(F_1^{GS}(u)\) from a small CNN on \([F_1, F_{cv}]\). These Gaussians capture the scene “skeleton,” including large-scale structure such as blocks, walls, and major surfaces [2410.06245]. Stages 2 and 3 operate at higher spatial resolutions \(H/2 \times W/2\) and \(H \times W\). Rather than predicting independently, each stage \(i>1\) re-uses stage \(i-1\): it renders the previous-stage Gaussians back into the input views, computes an error map $$ E_{i-1}(u) = |\hat I_{i-1}^{ref}(u) - I_{in}^{ref}(u)|, $$ and applies an Error Aware Module (EAM) that ingests \([E_{i-1}, F_i]\) to predict a small residual depth offset \(\Delta D_i(u)\) and \(F_i^{GS}(u)\). The fractional offset is $$ \Delta D_i(u) = \bigl(2\alpha_i(u)-1\bigr)\,\eta\,\mathrm{Interp}\bigl(D_{i-1}\bigr)(u), $$ with \(\eta \ll 1\) and the updated depth given by $$ D_i(u)=\mathrm{Interp}(D_{i-1})(u)+\Delta D_i(u). $$ This design localizes regions where coarse Gaussians fail and adds corrective, smaller Gaussians. Fusion across stages is handled by the Modulating Fusion Module (MFM). For each previous stage \(k