---
title: 'DeformSplat: Deformable Gaussian Splatting'
url: https://www.emergentmind.com/topics/deformsplat
type: topic
---

# DeformSplat: Deformable Gaussian Splatting

Searching arXiv for the cited DeformSplat-related papers and adjacent work.
DeformSplat denotes a class of deformable splatting formulations in which a canonical set of splats is transformed to represent new geometry, viewpoints, or time, and it also names a specific rigidity-aware framework for reconstructing deformation of a pre-reconstructed 3D Gaussian scene from only a single RGB image [2509.22222]. In the single-image setting, the canonical scene is a 3D Gaussian field \(\mathcal{G}=\{\mu_i,q_i,s_i,\alpha_i,sh_i\}\), the target is an unposed image \(\mathcal{I}_{\text{target}}\), and the deformed output is \(\mathcal{G}'=\{\mu_i',q_i',s_i,\alpha_i,sh_i\}\), with deformation applied to Gaussian centers and rotations while appearance parameters remain fixed [2509.22222]. In adjacent literature, the same conceptual space includes view-dependent deformation fields for 2D-guided editing [2504.05544], canonical-field dynamic Gaussian models [2404.03613, 2404.06270, 2603.20857, 2506.09836], mesh- and cage-guided non-rigid deformation [2411.12168, 2507.07000, 2509.18501], proxy-free Laplacian deformation [2511.19542], and a general theory in which splats are affine deformations of a mother density and Gaussian Splatting is a special case [2511.14042].

## 1. Formal notion of deformable splats

In the most general formulation, a splat is a parametrically transformed bump function obtained from a fixed mother density \(\rho \in P_{ac}(\mathbb{R}^d)\) by an affine map. For parameters \(A \in \mathbb{R}^{d\times d}\) and \(b \in \mathbb{R}^d\), the transformed density is
\[
\rho_{A,b} \coloneqq (A(\cdot)+b)_\#\rho,
\qquad
\rho_{A,b}(x)=\rho(A^{-1}(x-b))\,|\det A^{-1}|.
\]
The location parameter \(b\) controls the center, while \(A\) controls scale and orientation, which is where anisotropy and deformation reside [2511.14042].

The associated function class is defined through a mixing measure \(\mu \in P(\mathbb{R}^p \times {}_\rho(\mathbb{R}^d))\), with model
\[
f_\mu(x)=\mathbb{E}_{(v,\rho_{A,b})\sim\mu}\!\left[v\,\rho_{A,b}(x)\right].
\]
For a finite \(k\)-splat measure this becomes a finite mixture of anisotropically deformed bumps weighted by output vectors \(v_i\), with parameters \(\{v_i\}\) encoding local output values, \(\{b_i\}\) encoding positions, and \(\{A_i\}\) encoding local scales and directions [2511.14042]. This provides the cleanest mathematical reading of “deformable splats”: splats are explicit geometric entities whose shape and placement are optimized rather than fixed.

This formalism also subsumes Gaussian Splatting. If the mother splat is the standard Gaussian \(\rho(x)=N(x;0,I)\), then
\[
\rho_{A,b}(x)=N(x;b,AA^T),
\qquad
f(x)=\sum_{i=1}^k v_i\,N(x;b_i,A_iA_i^T),
\]
so Gaussian splats appear as a special case of splat regression [2511.14042]. The paper further states that this viewpoint “disambiguates the inverse problem, the model, and the optimization algorithm,” separating the forward operator, the splat representation, and the fitting rule [2511.14042]. A plausible implication is that the name DeformSplat is best understood not as a single algorithmic recipe, but as a design space built around affine-deformable splat primitives.

## 2. Rigidity-aware DeformSplat from a single image

The paper explicitly titled “Rigidity-Aware 3D Gaussian Deformation from a Single Image” introduces DeformSplat as a framework for deforming a canonical 3DGS model using one target RGB image of a new configuration [2509.22222]. The method assumes a pre-reconstructed canonical Gaussian scene and an unknown target-camera pose. Only Gaussian centers and rotations are optimized; scales, opacity, and spherical-harmonic appearance remain fixed to avoid color overfitting [2509.22222].

Its first core component is Gaussian-to-Pixel Matching. The canonical Gaussians are rendered from all known reconstruction cameras, each rendered view is matched to the target image with RoMA, and the view with the highest spatial overlap is selected. Dense 2D correspondences are then lifted to Gaussian-to-Pixel pairs \((\mu_i,x_p')\), where \(\mu_i\) is a visible Gaussian center and \(x_p'\) is the matched target pixel [2509.22222]. This yields the deformation loss
\[
\mathcal{L}_{\text{deform}}=\sum_i \|\mu_i^{2D}-x_p'\|^2,
\]
which provides explicit 3D-to-2D geometric supervision rather than relying only on image reconstruction [2509.22222].

The second core component is Rigid Part Segmentation, consisting of initialization and refinement. Initialization combines PnP-RANSAC with a region-growing strategy to form spatially coherent rigid groups from Gaussian-to-Pixel correspondences. Refinement uses a rigidity score
\[
S_{\text{rigid}}(\mu_i,G)=\frac{1}{|G|}\sum_{\mu_j\in G}
\left\|
R_i^{-1}(\mu_i-\mu_j)-{R_i'}^{-1}(\mu_i'-\mu_j')
\right\|^2,
\]
and adds or removes Gaussians from rigid groups according to thresholds \(\tau_{\text{low}}\) and \(\tau_{\text{high}}\) [2509.22222]. This explicitly separates regions that should preserve internal structure from regions where non-rigid deformation is admissible.

The deformation model itself is anchor-based. Sparse anchors are obtained by voxelizing space, and each anchor carries a rotation and translation. Gaussian positions and rotations are interpolated from neighboring anchor transforms:
\[
\mu_i'=\sum_{k\in\mathcal{N}} w_{ik}\left(R_k^a(\mu_i-a_k)+a_k+T_k\right),
\qquad
q_i' = q_i \otimes \sum_{k\in\mathcal{N}} w_{ik} q_k^a.
\]
Optimization minimizes the total objective
\[
\mathcal{L}_{\text{total}}
=
\lambda_{\text{deform}}\mathcal{L}_{\text{deform}}
+
\lambda_{\text{group}}\mathcal{L}_{\text{group}}
+
\lambda_{\text{arap}}\mathcal{L}_{\text{arap}}
+
\lambda_{\text{rgb}}\mathcal{L}_{\text{rgb}},
\]
where \(\mathcal{L}_{\text{group}}\) preserves relative geometry inside rigid groups and \(\mathcal{L}_{\text{arap}}\) regularizes local deformation between anchors [2509.22222]. The same framework naturally supports frame interpolation and interactive manipulation via target correspondences supplied by either additional images or user interaction [2509.22222].

Quantitatively, the reported results on Diva360 are PSNR \(26.84\), SSIM \(0.955\), and LPIPS \(0.050\), and on DFA are PSNR \(21.81\), SSIM \(0.897\), and LPIPS \(0.091\) [2509.22222]. The paper reports that removing deformation loss and group loss reduces Diva360 PSNR to \(21.73\), while removing group loss only yields \(24.36\), indicating that rigidity-aware geometric supervision is central rather than auxiliary [2509.22222].

## 3. View-conditioned and sketch-driven deformation

A distinct branch of DeformSplat-like work treats deformation as a function of viewpoint rather than time. “View-Dependent Deformation Fields for 2D Editing of 3D Models” defines a view-dependent deformation field
\[
f_v:\mathbb{R}^3\times S(2)\to\mathbb{R}^3,
\]
with the congruence condition
\[
\pi_i(f_{v_i}(p))=\phi_i(\pi_i(p)),
\]
so that the deformed 3D object matches a user-authored 2D deformation \(\phi_i\) at key view \(v_i\) [2504.05544]. Each 2D warp is lifted to 3D by preserving depth in camera coordinates, and multiple deformations are blended compositionally through recursive view-dependent interpolation
\[
D_k(p,v)=B_k(v)\,\Phi_k(D_{k-1}(p,v)) + (1-B_k(v))\,D_{k-1}(p,v),
\qquad
f_v(p)=D_n(p,v),
\]
where \(B_k(v)\) are localized basis functions on the viewing sphere [2504.05544].

For 3D Gaussian splats, the method uses a first-order Taylor approximation of the deformation around each Gaussian center \(\mu\),
\[
L(p)=f_v(\mu)+J(p-\mu),
\]
which preserves Gaussian form under deformation by updating
\[
\tilde{\mu}=f_v(\mu),
\qquad
\tilde{\Sigma}=J^T\Sigma J.
\]
This yields view-dependent deformation of both centers and covariances while remaining compatible with standard 3DGS rendering [2504.05544]. The same paper reports interactive performance of approximately \(25\) FPS on an NVIDIA RTX 4090 for models with \(\approx 750\,\text{k}\) Gaussians, with BBW preprocessing around \(100\) ms and triangulation around \(33\) ms [2504.05544].

A second editing-oriented line is “Sketch-guided Cage-based 3D Gaussian Splatting Deformation,” which couples cage-based deformation with a variant of Neural Jacobian Fields [2411.12168]. A deformation cage \(S\) encloses the splat set, the cage interpolation map \(f_{S'}\) moves Gaussian means,
\[
\mu_i' = f_{S'}(\mu_i),
\]
and the covariance update is given by
\[
\Sigma_i' = J_f(\mu_i)\, R_i S_i S_i^\top R_i^\top J_f(\mu_i)^\top.
\]
The cage is controlled through per-face Jacobians decomposed into rotations and symmetric stretches, and optimization combines a silhouette term with 3D-aware SDS:
\[
\nabla \mathcal{L}_{total} = \alpha \nabla \mathcal{L}_{sil} + \nabla \mathcal{L}_{SDS},
\qquad \alpha = 10000.
\]
This formulation turns silhouette sketches into geometrically consistent deformation of 3D Gaussian models, and the paper presents animation as a direct extension obtained by interpolating keyframe cages over time [2411.12168].

## 4. Dynamic-scene deformation architectures

In dynamic view synthesis, DeformSplat usually refers to a canonical Gaussian field plus a learned deformation model. “Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting” argues that coordinate-based deformation fields are mismatched to Gaussian splatting and replaces them with a deformation function of per-Gaussian embeddings \(z_i\) and temporal embeddings \(z_t\):
\[
\mathcal{F}_\theta:(z_i,z_t)\rightarrow(\Delta \mathbf{x}_i,\Delta r_i,\Delta s_i,\Delta \sigma_i,\Delta Y_i).
\]
It further decomposes motion into coarse and fine branches,
\[
(\Delta \mathbf{x}_i,\Delta r_i,\Delta s_i,\Delta \sigma_i,\Delta Y_i)
=
\mathcal{F}_\theta^{\text{coarse}}(z_i,z_t^{\text{coarse}})
+
\mathcal{F}_\theta^{\text{fine}}(z_i,z_t^{\text{fine}}),
\]
to separate slow, large motion from fast, localized motion [2404.03613]. The reported averages are PSNR \(31.42\), SSIM \(0.945\), and LPIPS \(0.037\) on Neural 3D Video; PSNR \(33.38\), SSIM \(0.907\), and LPIPS \(0.103\) on Technicolor; and PSNR \(25.53\), SSIM \(0.694\), and LPIPS \(0.242\) on HyperNeRF [2404.03613].

“3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis” instead conditions deformation on explicit 3D geometry features extracted from canonical Gaussians by a sparse 3D U-Net [2404.06270]. Time-dependent position, scale, and rotation are predicted as
\[
\Delta \mathbf{x}_t,\Delta \mathbf{r}_t,\Delta \mathbf{s}_t
=
\mathcal{G}_\Phi(F_{\text{fuse}},\gamma(\mathbf{x}),\gamma(t)),
\]
with updates
\[
\mathbf{x}_t=\mathbf{x}+\Delta \mathbf{x}_t,
\qquad
\mathbf{s}_t=\mathbf{s}+\Delta \mathbf{s}_t,
\qquad
\mathbf{r}_t=f_{\text{V2M}}(\Delta \mathbf{r}_t)\times f_{\text{V2M}}(\mathbf{r}).
\]
A motion penalty \(L_{\text{motion}}=\|\Delta \mathbf{x}_t\|_1\) is added to the photometric objective [2404.06270]. On D-NeRF, the method reports PSNR \(32.27\), SSIM \(0.9835\), and LPIPS \(0.0164\) for Hell Warrior, and PSNR \(41.43\), SSIM \(0.9969\), and LPIPS \(0.0029\) for Mutant [2404.06270].

Recent work emphasizes speed and robustness. FRoG keeps canonical Gaussians \(G_i=\{\mu_i,s_i,q_i,\alpha_i,sh_i,e_i\}\), uses per-Gaussian embeddings \(e_i\), and fuses coarse and fine temporal embeddings by Hadamard product,
\[
t_{\text{fused}} = t_c \odot t_f,
\]
so that one MLP pass predicts all attribute deltas [2603.20857]. It augments this with depth- and error-guided canonical sampling and opacity modulation
\[
\alpha_{\text{final}}
=
\phi(\phi^{-1}(\alpha)+\Delta \alpha)\cdot \phi(k\Delta \alpha),
\]
to mitigate local optima in dim scenes [2603.20857]. DynaSplat, by contrast, explicitly separates static and dynamic Gaussians using deformation variance and 2D motion flow consistency, then decomposes motion into coarse neighbor-averaged dynamics and fine residuals, while also using physically-based opacity and temporal importance filtering [2506.09836]. This suggests that “dynamic DeformSplat” has evolved from a single deformation field into a family of structured motion models with explicit partitioning, hierarchy, and visibility reasoning.

## 5. Physics, anatomy, and transient decomposition

Another interpretation of DeformSplat replaces purely kinematic deformation with domain-specific constraints. Physics-Informed Deformable Gaussian Splatting treats each Gaussian as a Lagrangian material point and couples a deformation network with a time-evolving material field predicting velocity \(\boldsymbol{v}(x,t)\) and stress \(\boldsymbol{\sigma}(x,t)\) [2511.06299]. The governing residual is the Cauchy momentum residual
\[
\mathbf{r}(x,t)=
\rho\Bigl(\frac{\partial \boldsymbol{v}}{\partial t}+(\boldsymbol{v}\cdot\nabla)\boldsymbol{v}\Bigr)
-
\nabla\cdot\boldsymbol{\sigma},
\]
with training term
\[
\mathcal{L}_{\rm CMR}=\frac{1}{M}\sum_{i=1}^{M}\|\mathbf{r}(x_i,t_i)\|_2^2.
\]
The method adds Lagrangian particle flow matching against camera-compensated optical flow,
\[
\mathcal{L}_{\rm LPFM}
=
\lambda_g \|flow_g-flow_{gt}\|_1
+
\lambda_v \|flow_v-flow_{gt}\|_1,
\]
to accelerate convergence and improve generalization [2511.06299]. The stated goal is a unified constitutive framework for time-evolving material fields rather than unconstrained deformation.

BridgeSplat imports anatomical priors by rigging Gaussians to a CT-derived mesh for deformable surgical navigation [2509.18501]. Each Gaussian mean is parametrized barycentrically on a parent mesh triangle,
\[
\boldsymbol{\mu}_t^i=\sum_{k=1}^{3} b_t^{i,k}\,\mathbf{v}_t^{i,k},
\qquad
\sum_{k=1}^{3} b_t^{i,k}=1,\quad b_t^{i,k}\ge 0,
\]
and mesh vertices are moved by a sparse anchor deformation field \(\theta_t\) [2509.18501]. Photometric supervision is combined with ARAP regularization on the mesh, and deformations are propagated back to the preoperative CT. On synthetic liver simulations, the reported mean Euclidean distances are \(0.11\pm0.19\) mm for SimIn, \(0.14\pm0.27\) mm for SimDown, \(0.09\pm0.16\) mm for SimLeft, \(0.10\pm0.19\) mm for SimCircular, and \(0.12\pm0.24\) mm for SimUp, with maximum Euclidean distance below \(5\) mm in all cases [2509.18501].

ForestSplats uses deformation in a different sense: a deformable transient field explains per-view transient content in unconstrained image collections [2503.06179]. The rendered image is decomposed as
\[
\hat{I}=M\odot \hat{I}_d + (1-M)\odot \hat{I}_s,
\]
where \(\hat{I}_s\) is the static field, \(\hat{I}_d\) is the transient field, and \(M\) is a learned superpixel-aware mask [2503.06179]. The transient field is driven by per-view embeddings and deformation offsets,
\[
(\Delta \mu,\Delta s,\Delta r)=f_d(\gamma(\operatorname{sg}(\mu)),\gamma(t)),
\qquad
\hat{c}=f_c(\gamma(t)),
\]
and is combined with uncertainty-aware densification to avoid placing static Gaussians inside transient regions [2503.06179]. This broadens DeformSplat from surface motion to view-specific scene decomposition.

## 6. Surface priors, proxies, limitations, and outlook

Surface structure is a central fault line in the DeformSplat literature. One response is to add explicit topology. “Enhancing non-Rigid 3D Model Deformations Using Mesh-based Gaussian Splatting” anchors Gaussians to explicit mesh surfaces reconstructed from a 3DGS scene, uses XPBD for non-rigid deformation, and updates Gaussian covariance under the deformation Jacobian as
\[
\Sigma' = J\,\Sigma\,J^\top.
\]
The paper positions the mesh as a carrier of topological, geometric, and physical priors, allowing moving, scaling, rotating, bending, and stretching of components while maintaining real-time rendering [2507.07000]. A closely related perspective appears in sketch-guided cage deformation, where the cage is not itself the visible surface but a low-dimensional control domain for Gaussian motion [2411.12168].

The opposite response is proxy-free surface inference. SpLap introduces a surface-aware splat graph in which neighboring splats are defined by intersection of occupancy regions rather than by center distance [2511.19542]. For splat \(i\), the occupancy region is
\[
\Omega_i = \left\{ \mathbf{x}\in\mathbb{R}^3 \;\Big|\; \mathcal{G}_i(\mathbf{x}) > \max\left\{\bar{V}_i^p,\frac{c}{\alpha_i}\right\} \right\},
\]
and graph edges are created if the normal-wise offset \(\delta_{ij}\) between occupancy regions satisfies \(\delta_{ij}\le \epsilon\) [2511.19542]. Classical ARAP and BBW are then applied directly on the splat graph, and kernel adaptation is performed by transporting a maximum-area inscribed triangle through the deformation and reconstructing the Gaussian via the Steiner circumellipse [2511.19542]. The paper reports superior performance over both proxy-based and proxy-free baselines on \(50\) objects from ShapeNet, Objaverse, Sketchfab, and NeRF-Synthetic [2511.19542].

Several recurrent limitations emerge across these formulations. Single-image DeformSplat depends strongly on matching quality and assumes meaningful rigid components; highly non-rigid objects and appearance change remain difficult [2509.22222]. View-dependent editing can produce artifacts for large Gaussians because affine approximation is local [2504.05544]. Physics-informed approaches remain optimization-heavy and use simplified constitutive assumptions [2511.06299]. Mesh-guided methods inherit the constraints of fixed topology, making tearing or topological change poorly supported [2507.07000]. Proxy-free Laplacian methods rely on well-aligned surface splats and do not themselves repair poor reconstruction [2511.19542]. A plausible synthesis is that DeformSplat is not converging toward a single canonical architecture; instead, it is stratifying into multiple regimes distinguished by the source of deformation evidence—single-image correspondences, view-conditioned edits, temporal motion, physics, anatomy, or surface graph structure—and by the strength of the prior used to keep Gaussian deformation geometrically coherent.

Source: https://www.emergentmind.com/topics/deformsplat