Dynamic 3D Gaussian Representation (Dyna3DGR)

• Dyna3DGR is a self-supervised framework for 4D cardiac motion tracking that uses explicit 3D Gaussian primitives paired with a neural motion field to represent dynamic anatomy.
• The framework preserves image details by leveraging differentiable volumetric rendering to align reconstructed cardiac volumes with observed images without dense supervision.
• Evaluations on the ACDC dataset demonstrate significant improvements in Dice and SSIM metrics compared to traditional registration methods, highlighting its accuracy and efficiency.

Dynamic 3D Gaussian Representation (Dyna3DGR) denotes a dynamic-scene modeling paradigm in which a time-varying scene is represented through explicit 3D Gaussian primitives, and more specifically the name of a self-supervised framework for 4D cardiac motion tracking that combines an explicit 3D Gaussian representation of cardiac anatomy with an implicit neural motion field optimized through differentiable volumetric rendering (Fu et al., 22 Jul 2025). In this formulation, structure and motion are optimized jointly, image-space alignment is retained through rendering, and dense point-to-point correspondences or large supervised training sets are not required. Within the broader dynamic Gaussian literature, Dyna3DGR is closely related to dynamic 3D Gaussian methods that extend static 3D Gaussian Splatting into time-varying settings, while differing from purely explicit basis-driven models such as Gaussian-Flow and from dynamic view-synthesis systems designed primarily for general nonrigid scenes rather than cardiac motion analysis (Lin et al., 2023).

1. Historical placement and problem setting

Dyna3DGR was introduced for 4D cardiac motion tracking in dynamic cardiac magnetic resonance imaging, where fine-grained tissue motion analysis is difficult because myocardial tissue is homogeneous and lacks distinctive features (Fu et al., 22 Jul 2025). The framework is motivated by a contrast between two prior families. Image-based approaches, including traditional non-parametric registration and deep-learning registration, preserve image-space information but may struggle with topological consistency or depend heavily on extensive training data. Representation-based approaches reduce dependence on direct voxel correspondence, but may lose image-level detail.

The framework addresses that tension by combining explicit Gaussian primitives with an implicit neural motion field. The explicit component is intended to preserve image detail and contours; the implicit component provides continuous deformation without point-to-point supervision. This makes Dyna3DGR distinct from methods that operate only in image space and from methods that rely only on landmarks, meshes, or implicit shape fields.

In the broader taxonomy of dynamic 3D Gaussian methods, this design corresponds to the structure-guided paradigm, in which a canonical representation is paired with an auxiliary deformation model rather than encoding all temporal behavior directly inside the Gaussian primitives themselves (Ramlal et al., 30 May 2026). That placement is significant because structure-guided methods are described as using auxiliary structures such as deformation fields, canonical spaces, sparse control points, or grids to model motion separately from appearance.

2. Canonical Gaussian formulation of cardiac anatomy

Dyna3DGR constructs an explicit 3D Gaussian representation of the cardiac anatomy in canonical space. Each Gaussian $G_i$ is parameterized as

$\phi_i = \{xyz_i, \Sigma_i, I_i\},$

where $xyz_i \in \mathbb{R}^3$ is the Gaussian center, $\Sigma_i \in \mathbb{R}^{3\times3}$ is the covariance, and $I_i \in \mathbb{R}$ is the intensity (Fu et al., 22 Jul 2025). The covariance is decomposed into learnable rotation and scaling factors:

$\Sigma = RSS^TR^T.$

The contribution of each Gaussian at spatial location $X\in\mathbb{R}^3$ is

$$G_i(X|\phi_i) = I_i \cdot e^{-\frac{1}{2}(X-xyz_i)^T \Sigma_i^{-1}(X-xyz_i)},$$

and the volumetric value at a point is defined as a local sum over nearby Gaussian contributions:

$$V(X|\theta_i)=\sum_{i : ||X-xyz_i||\leq d_i} G_i(X|\phi_i).$$

This representation places the myocardium into a continuous volumetric field assembled from explicit local primitives. The paper frames this as a way to preserve image detail and contours while remaining compatible with differentiable rendering. A plausible implication is that the Gaussian representation provides a middle ground between voxel-wise registration, which remains fully image-driven, and more abstract representation-based models that may discard local intensity structure.

The representation also differs from standard static 3D Gaussian Splatting only in purpose, not in primitive type. Like other dynamic Gaussian methods, it retains explicit localized anisotropic kernels, but uses them as a canonical anatomical substrate to be deformed over time rather than as a fixed static scene.

3. Implicit neural motion field and sparse-controlled deformation

The second component of Dyna3DGR is an implicit neural motion representation based on control nodes, following the sparse-controlled deformation idea (Fu et al., 22 Jul 2025). The model defines control nodes as

$$C = \{C_i \in \mathbb{R}^3, o_i \in \mathbb{R}^+\},$$

where $\phi_i = \{xyz_i, \Sigma_i, I_i\},$0 are learnable canonical-space positions and $\phi_i = \{xyz_i, \Sigma_i, I_i\},$1 is a learnable RBF radius. For each node, the deformation network predicts translation and scale change as

$\phi_i = \{xyz_i, \Sigma_i, I_i\},$2

with positional encoding

$\phi_i = \{xyz_i, \Sigma_i, I_i\},$3

The network takes an $\phi_i = \{xyz_i, \Sigma_i, I_i\},$4 set of control-point coordinates and an $\phi_i = \{xyz_i, \Sigma_i, I_i\},$5 time vector, and outputs $\phi_i = \{xyz_i, \Sigma_i, I_i\},$6 translations and $\phi_i = \{xyz_i, \Sigma_i, I_i\},$7 scales. Dense deformation for each Gaussian is then obtained through KNN search and linear blend skinning:

$\phi_i = \{xyz_i, \Sigma_i, I_i\},$8

with RBF-style weighting based on

$\phi_i = \{xyz_i, \Sigma_i, I_i\},$9

This architecture links a sparse, learned deformation scaffold to a dense Gaussian anatomical model. The paper emphasizes that motion is continuous in time because the deformation field is time-conditioned, while topological and temporal consistency are encouraged through smooth interpolation over control nodes and linear blend skinning. This is one of the core distinctions between Dyna3DGR and explicitly parameterized dynamic Gaussian models that avoid neural motion fields altogether. Gaussian-Flow, for example, models attribute residuals with a polynomial in the time domain and a Fourier series in the frequency domain and explicitly avoids introducing an additional implicit neural field to model 3D dynamics (Lin et al., 2023).

4. Differentiable rendering, self-supervision, and optimization

Dyna3DGR is optimized instance-wise and self-supervised, with no pre-training and no external training data (Fu et al., 22 Jul 2025). Supervision is supplied by reconstruction of the observed cardiac volumes through differentiable volumetric rendering. The framework is optimized using an L1 loss between rendered predictions and ground-truth cardiac volumes, so the canonical Gaussians, control-node motion field, and deformation network are jointly learned from image reconstruction rather than dense displacement labels.

This rendering-based coupling is central to the method’s formulation. The Gaussian field defines a continuous 3D volume in canonical space, and the deformation network warps that representation over time by changing Gaussian centers and scales. The rendered volume is then compared directly to the target cardiac frame. The paper presents this as the bridge between motion space and image space: if the deformed Gaussian volume matches the target cardiac image, then the underlying motion field is treated as correct.

The implementation uses instance-wise test-time optimization in PyTorch. Coordinates are normalized to $xyz_i \in \mathbb{R}^3$0. Gaussian primitives are initialized by uniform sampling from the end-diastolic segmentation mask, and control nodes are initialized from those Gaussian positions, with the maximum number equal to the initial Gaussian count. Optimization runs for 20,000 iterations, roughly 11 minutes, with a two-stage schedule: the first 1,000 iterations optimize only the canonical 3D Gaussians to stabilize positions and shapes, after which joint optimization of Gaussians, control nodes, and the deformation network begins. Adam is used with learning rates of $xyz_i \in \mathbb{R}^3$1 for Gaussian positions, $xyz_i \in \mathbb{R}^3$2 for intensity, $xyz_i \in \mathbb{R}^3$3 for rotation and scale, $xyz_i \in \mathbb{R}^3$4 for control points, updated at iteration 5,000, and $xyz_i \in \mathbb{R}^3$5 for the deformation network. Learning rates decay exponentially from $xyz_i \in \mathbb{R}^3$6 to $xyz_i \in \mathbb{R}^3$7, with $xyz_i \in \mathbb{R}^3$8 and $xyz_i \in \mathbb{R}^3$9. Gaussian densification occurs every 500 iterations starting from iteration 500, and all experiments run on a single NVIDIA RTX 3090 GPU.

The paper further evaluates deformation regularity through the Jacobian determinant of the predicted Gaussian displacement field. It reports the number of locations where $\Sigma_i \in \mathbb{R}^{3\times3}$0, corresponding to folding or tearing, and the mean absolute deviation $\Sigma_i \in \mathbb{R}^{3\times3}$1 as a measure of volume preservation. The paper states that these metrics indicate a diffeomorphic-like deformation behavior even though the method is not formulated as a classical diffeomorphic registration model.

5. Evaluation on ACDC and reported performance

Dyna3DGR is evaluated on the ACDC dataset, which contains 100 clinical 4D cardiac MRI cases across five categories: normal (NOR), myocardial infarction with systolic heart failure (MINF), dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy (HCM), and abnormal right ventricle (ARV) (Fu et al., 22 Jul 2025). Each case includes expert segmentations of the left ventricle, right ventricle, and myocardium at end-diastole and end-systole. The preprocessing protocol follows prior work: resampling to $\Sigma_i \in \mathbb{R}^{3\times3}$2 mm, center-cropping to $\Sigma_i \in \mathbb{R}^{3\times3}$3, and intensity normalization to $\Sigma_i \in \mathbb{R}^{3\times3}$4.

Evaluation includes Dice score for anatomical tracking accuracy, PSNR and SSIM for image reconstruction quality, Hausdorff distance, Jacobian-based topology and volume-preservation measures, and parameter count for efficiency. Baselines include classical registration methods such as LDDMM, RDMM, and ANTs/SyN, as well as deep learning-based registration methods including VoxelMorph, DiffuseMorph, CorrMLP, DeepTag, Transmatch, FSDiffReg, and GPTrack.

Metric	Dyna3DGR	GPTrack
RV Dice	97.61%	82.91%
LV Dice	97.10%	81.23%
Myo Dice	95.16%	72.86%
Average Dice	96.62%	82.65%

The paper highlights a 17.73% improvement in Dice and a 12.63% gain in SSIM relative to prior art, with paired $\Sigma_i \in \mathbb{R}^{3\times3}$5-test significance $\Sigma_i \in \mathbb{R}^{3\times3}$6. Although its PSNR is not the highest in the comparison table, it reports a strong SSIM of 97.08% and markedly higher anatomical overlap. On topology-related metrics, it reports a Jacobian deviation of 0.002, with $\Sigma_i \in \mathbb{R}^{3\times3}$7 locations violating the determinant constraint in the table. The model is also reported as extremely lightweight, with parameter count 0.002M.

An ablation study varies the number of control nodes from 2048 to 10240. Dice increases from 92.51% to 96.62%, PSNR from 31.65 dB to 34.66 dB, and SSIM from 94.78% to 97.08%. The paper interprets this as evidence that a richer control lattice improves deformation fidelity. It does not provide a detailed failure-case section, but it implicitly suggests that sufficient control-node density is important for best performance and that the method relies on instance-wise optimization at test time.

6. Relation to adjacent dynamic Gaussian methods and scope of the term

Dyna3DGR belongs to a larger family of dynamic 3D Gaussian methods, but that family is architecturally heterogeneous. Some systems remain close to explicit time-parameterized Gaussian primitives. The compact dynamic 3D Gaussian representation of Luiten and colleagues models position with a Fourier series and rotation with a linear function of time while keeping scale, color, and opacity fixed, thereby avoiding per-frame storage and enabling real-time dynamic view synthesis from monocular or few-view videos (Katsumata et al., 2023). Gaussian-Flow similarly treats a 4D scene as deformable 3D Gaussian particles, but models residual attribute motion explicitly with a hybrid polynomial-plus-Fourier formulation rather than with a neural deformation field (Lin et al., 2023).

Other systems stay within the canonical-space-plus-deformation template but emphasize different inductive biases. “3D Geometry-aware Deformable Gaussian Splatting” injects local 3D geometric context through sparse 3D convolutional features, uses a continuous 6D rotation representation, and performs time-aware density control for dynamic view synthesis (Lu et al., 2024). “A Refined 3D Gaussian Representation for High-Quality Dynamic Scene Reconstruction” instead combines a deformable MLP with hash-encoded appearance, a learnable denoising mask, and motion regularization to reduce storage and suppress motion noise (Zhang et al., 2024).

The acronym Dyna3DGR has also been used in a different application domain for head avatars. In “3D Gaussian Head Avatars with Expressive Dynamic Appearances by Compact Tensorial Representations,” Dyna3DGR denotes a mesh-bound 3D Gaussian head avatar representation that stores neutral appearance in tri-planes and expression-dependent detail in 1D feature lines decoded into opacity offsets (Wang et al., 21 Apr 2025). That usage does not refer to cardiac tracking, but it shares the central idea of making Gaussian attributes dynamic while preserving the efficiency of Gaussian rasterization.

A common misconception is that dynamic 3D Gaussian representations are uniformly “gaussian-centric” and therefore avoid auxiliary deformation networks. Dyna3DGR provides a counterexample: it is explicit in its anatomical representation but hybrid in its dynamics, coupling Gaussian primitives to an implicit neural motion field (Fu et al., 22 Jul 2025). Another misconception is that topology preservation in such methods necessarily requires a classical diffeomorphic registration formulation. Dyna3DGR instead seeks diffeomorphic-like behavior through the structure of the deformation field, interpolation scheme, and Jacobian-based evaluation rather than through an explicitly diffeomorphic objective.

Taken together, these works suggest that “Dynamic 3D Gaussian Representation” names both a specific cardiac motion-tracking framework and a broader research direction. In the specific sense of the cardiac method, Dyna3DGR reformulates 4D cardiac motion tracking as joint optimization over a canonical Gaussian anatomy model and a continuous neural motion field. In the broader sense, it sits within an expanding class of methods that use explicit Gaussian primitives as the substrate for time-varying geometry, appearance, semantics, or motion, with design choices ranging from explicit temporal bases to hybrid deformation fields and compact tensorial parameterizations.