Gaussian Splat Representations in Ultrasound

• Gaussian splat representations in ultrasound are an explicit, parametric method that models volumetric data with anisotropic Gaussian functions, ensuring accurate image geometry.
• The technique embeds 2D ultrasound frames into a 3D volume via known probe trajectories, achieving noise-robust reconstructions and temporal consistency.
• Real-time novel view synthesis and mesh extraction are enabled by direct optimization of splat parameters alongside ultrasound-specific rendering and physics.

Gaussian splat representations are an explicit, parametric approach for modeling volumetric ultrasound data by approximating image formation and appearance with collections of anisotropic Gaussian functions ("splats") in three-dimensional space. Unlike neural implicit representations, which fit intensity fields through MLPs or radiance fields, Gaussian splatting provides direct interpretability, rapid optimization, and efficient rendering that conforms to ultrasound image geometry and physical principles. This methodology has yielded high-fidelity, noise-robust reconstructions and real-time synthesis of novel views, as demonstrated in recent frameworks including MedGS (Marzol et al., 20 Sep 2025), UltraGS (Yang et al., 11 Nov 2025), and UltraGauss (Eid et al., 8 May 2025).

1. Mathematical Formulation of Gaussian Splats in Ultrasound

The foundational element of Gaussian splatting is the explicit parameterization of each splat as an anisotropic Gaussian function in three-dimensional space. In both MedGS and UltraGauss, a splat $i$ encodes:

• Mean position $\mu_i \in \mathbb{R}^3$ (center in world coordinates)
• Covariance matrix $\Sigma_i \in \mathbb{R}^{3 \times 3}$ (shape, anisotropy, orientation)
• Opacity/amplitude parameter $\rho_i$ or $\alpha_i \in (0,1)$
• Appearance coefficient $c_i$ (grayscale or multi-channel intensity)
• For MedGS, temporal modulation via functions $a_i(t)$ (scale) and $f_i(t)$ (shift)

The density for each splat is:

$$N(x; \mu_i, \Sigma_i) = (2\pi)^{-3/2} |\Sigma_i|^{-1/2} \exp\left(-\frac{1}{2} (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right)$$

MedGS introduces a "Folded-Gaussian" or spatiotemporal splat over $(s, t)$ via:

$$FN_i(x) = N(s; \mu_{s|t}, \Sigma_{s|t}) \cdot N(t; \mu_t, \sigma_t^2)$$

where $\mu_{s|t} = \mu_s + f(t-\mu_t)$, $\Sigma_{s|t} = a(t) \cdot \Sigma_s$.

UltraGS models each splat as a planar anisotropic disk in local tangent coordinates ($t_u$, $t_v$), with world-space center $p_k$ and spreads $s_u, s_v$. Spherical harmonics encode directional intensity, and ultrasound-specific physics (attenuation, reflection, scattering) modulate appearance (Yang et al., 11 Nov 2025).

UltraGauss employs a numerically stable covariance parametrization, factorizing $\Sigma_i^{-1}$ as $L_i L_i^\top$ with $L_i$ lower-triangular and positive on diagonal, ensuring robust optimization (Eid et al., 8 May 2025).

2. Embedding 2D Ultrasound Frames as 3D Gaussian Primitives

Conversion of sequential 2D ultrasound frames into a dense 3D volume is performed by embedding pixel-wise splats along known probe trajectories (6-DOF poses). For each pixel $(u,v)$ in frame $t$ with pose $(R_t, T_t)$, the corresponding 3D point is defined as:

$s = R_t [u, v, 0]^\top + T_t$

which becomes the mean $\mu_s$ for the splat. Covariance $\Sigma_s$ is initialized to encode in-plane ellipsoidal uncertainty and minimal out-of-plane thickness:

$$\Sigma_s = \mathrm{diag}(\sigma_u^2, \sigma_v^2, \epsilon^2),\quad \epsilon \ll \sigma_u, \sigma_v$$

Temporal embedding allows splats to span multiple frames ($\mu_t$, $\sigma_t$), essential for volumetric consistency in freehand, sparse acquisition (Marzol et al., 20 Sep 2025).

In UltraGS and UltraGauss, point-based splats are organized to respect the ultrasound probe-plane geometry: splats are rasterized only within active imaging planes or wavefront intersections, with boundary rejection and parallel computation yielding real-time performance (Eid et al., 8 May 2025).

3. Ultrasound-Adapted Rendering and View Synthesis

Gaussian splatting in ultrasound deviates from classic photometric ray-marching used in optical rendering—here, image formation is governed by intersection of 3D Gaussians with the probe plane, reflecting acoustic pulse propagation. For each rendered slice, the accumulated intensity is:

$c_\text{us}(x) = \sum_i \alpha_i(x) c_i$

where $\alpha_i(x)$ is the Gaussian weight at slice location $x$, computed via Mahalanobis distance from splat parameters.

UltraGS augments this with explicit modeling of ultrasound physics:

• Depth attenuation: $I_\text{att}(z) = I_0 - \alpha z$
• Specular reflection: $I_\text{refl} = \beta (c \odot c)$
• Volumetric scattering: $I_\text{scat} = (\Gamma c) \odot c$

The total rendered intensity is a learnable weighted sum of these components and a spherical harmonics base illumination model. Adaptive field-of-view (FoV) parameters $\theta_x, \theta_y$ modulate intrinsic camera matrices per splat, accounting for the non-uniform beam spread and depth-dependent acquisition footprint in ultrasound (Yang et al., 11 Nov 2025).

4. Optimization, Training, and Efficiency

Gaussian splat frameworks optimize primitive parameters ($\mu_i$, $\Sigma_i$, $c_i$, $\rho_i$) directly via end-to-end gradient descent (typically Adam). The training losses integrate photometric fidelity (L2 or SSIM), interpolation regularization, and geometric priors:

• Interpolation loss (MedGS): synthesis of "in-between" frames with $\alpha$-weighted blending, regularizing temporal profiles and suppressing frame-specific noise (Marzol et al., 20 Sep 2025)
• Covariance regularization: keeps splat shapes well-conditioned and in-domain (Eid et al., 8 May 2025)
• No explicit MLP: the collection of splats themselves serve as the latent representation, enabling direct editability and facilitating pruned or densified reconstructions

Efficiency is a hallmark, with UltraGauss reconstructing 3D US volumes at 0.995 SSIM within 20 minutes and rendering at >30 fps on typical GPUs (Eid et al., 8 May 2025), and UltraGS supporting real-time novel view synthesis at 64.69 fps (Yang et al., 11 Nov 2025). MedGS optimization requires ~20 minutes for a typical volume; mesh extraction post-optimization completes in seconds (Marzol et al., 20 Sep 2025).

5. Noise Robustness and Volumetric Accuracy

Explicit Gaussian aggregation confers inherent low-pass smoothing: local averaging suppresses high-frequency speckle noise and mitigates missing data in sparsely sampled trajectories. Regularization via interpolation (MedGS's IBFR) penalizes splats fitting only individual slices, further limiting overfitting to noisy observations (Marzol et al., 20 Sep 2025).

Mesh reconstruction from binary segmentation masks uses the same splat representation, with dense interpolation yielding watertight, artifact-resistant surface meshes via marching cubes. Quantitative results (Prostate US dataset, MedGS) demonstrate:

• Chamfer distance: 0.203 mm (MedGS) vs. 0.211 mm (Poisson)
• Hausdorff distance: 0.827 mm vs. 0.920 mm
• HD95: 0.365 mm vs. 0.374 mm

UltraGS delivers state-of-the-art PSNR (up to 29.55), SSIM (up to 0.89), and MSE (as low as 0.002), outperforming alternatives such as NeRF, TensoRF, and 3DGS by margins of 2–4 dB in PSNR and 0.05–0.10 in SSIM (Yang et al., 11 Nov 2025).

6. Practical and Clinical Applications

Gaussian splat representations have demonstrable utility for a range of ultrasound tasks:

• Volumetric reconstruction: Standardized 3D views from arbitrary freehand sweeps, reducing operator dependence and cognitive load (Eid et al., 8 May 2025)
• Novel view synthesis: Real-time reslicing of anatomy from sparse inputs, supporting interventional guidance and measurement (Yang et al., 11 Nov 2025)
• Segmentation and mesh modeling: Direct surface extraction for anatomical registration and downstream quantitative analysis (Marzol et al., 20 Sep 2025)
• Interactive editing: Explicit primitives allow clinical users to adapt splat parameters (centers, eigenvectors) for local correction or annotation

Clinician survey results confirm strong preference for splat-based reconstructions (UltraGauss), with >70% rating images "more realistic than ground truth" after only minutes of training (Eid et al., 8 May 2025).

7. Limitations and Future Directions

Current frameworks assume static volumes, known probe-to-world poses, and employ geometric rather than full acoustic modeling (e.g., no frequency-dependent attenuation or nonlinear effects). Future extensions include:

• Spatiotemporal (4D) splatting for dynamic imaging
• Sensorless probe tracking; joint optimization of pose and intensity
• Enhanced acoustic priors, including speckle noise and complex tissue properties
• Adaptive multi-scale splat distributions for depth-varying resolution (Eid et al., 8 May 2025)

A plausible implication is that Gaussian splatting, with further integration of direction-dependent reflectivity and acoustic attenuation, may become the leading explicit volumetric representation for clinical ultrasound, supporting standardized interpretation, real-time analysis, and efficient 3D imaging pipelines.