UltraGS: Real-Time Ultrasound Synthesis

• UltraGS is a family of frameworks for ultrasound imaging that model anatomy with anisotropic 3D Gaussians and learnable physical parameters.
• It employs ultrasound-specific rendering methods like SH-DARS and Mahalanobis slicing to capture depth attenuation, specular reflection, and scattering.
• Optimization via MSE minimization and regularized Gaussian parameters yields state-of-the-art metrics (PSNR 25.45, SSIM 0.797) and rapid inference at 64.69 fps.

UltraGS is a designation for a family of Gaussian Splatting frameworks specifically designed for ultrasound imaging. These approaches address the challenges of novel view synthesis and volumetric reconstruction in B-mode ultrasound, which are complicated by limited field of view (FoV), physics-specific signal formation, and operator-dependent probe motion. Two principal methods bear the UltraGS name: the 3D reconstruction approach described in "UltraGauss: Ultrafast Gaussian Reconstruction of 3D Ultrasound Volumes" (Eid et al., 8 May 2025) and the novel-view synthesis system presented in "UltraGS: Gaussian Splatting for Ultrasound Novel View Synthesis" (Yang et al., 11 Nov 2025). Both diverge from classical, optically motivated splatting and rendering—integrating specific models for acoustic image formation and explicit learnable physical parameterization—to achieve real-time, accurate visualization of clinical anatomy.

1. Gaussian Splat Representations in Ultrasound

UltraGS models represent the underlying anatomy using explicit collections of oriented anisotropic Gaussians embedded in 3D. In both (Eid et al., 8 May 2025) and (Yang et al., 11 Nov 2025), each Gaussian is parameterized by a center $\mathbf{p}_k \in \mathbb{R}^3$, anisotropic scales (along local axes), orientation (orthonormal frame), and amplitude. The data pipeline involves:

• Probe-plane lifting: 2D ultrasound image pixels $(u,v)$ from a probe at pose $W_k$ are mapped into 3D probe or world coordinates as $x_{k|0} = (u, v, 0)$.
• Local Gaussian Parameterization: Means are initialized in the data volume, with amplitudes and signal intensities as learnable variables.
• Perspective-aware intersection: In "UltraGS" (Yang et al., 11 Nov 2025), each 2D splat is assigned a depth-aware, learnable FoV parameter defining non-uniform, perspective-correct projection onto the local imaging plane. The field of view $\mathrm{FOV}_x, \mathrm{FOV}_y$ is parameterized via $\theta_x,\theta_y$ (log-scale, initialized from COLMAP but refined during training), influencing the rendering intrinsics:

$$f_x = \frac{W_{\rm img}}{2 \tan(\mathrm{FOV}_x / 2)}, \quad f_y = \frac{H_{\rm img}}{2 \tan(\mathrm{FOV}_y / 2)}.$$

• Covariance parameterization: UltraGS (Eid et al., 8 May 2025) employs a lower-triangular matrix $L_i$ to enforce positive-definite covariances:

$\Sigma_i^{-1} = L_i L_i^T,$

with $\beta>0$ to avoid degeneracy and support efficient inversion. This avoids normalization overhead and instabilities associated with alternative schemes (e.g., quaternion-based).

The result is a lightweight, scalable scene representation supporting both sweeping “slice” imaging and freehand investigator motion.

2. Ultrasound-Specific Rendering and SH-DARS

Conventional splatting frameworks accumulate color using transmittance-dependent compositing, unsuitable for ultrasound due to fundamentally different wave-formation physics. Both UltraGS implementations replace standard models with ultrasound-specific rendering:

Model	Rendering Principals
UltraGS (Yang et al., 11 Nov 2025)	SH-DARS: Spherical Harmonic, Depth-Aware Reflectance & Scattering; low-order SHs for base intensity; models for attenuation, specular reflection, and volumetric scattering (cross-channel); learns combination weights.
UltraGS (Eid et al., 8 May 2025)	Mahalanobis “slice” through each 3D Gaussian (no transmittance); explicit sum-based intensity equation for each pixel; probe-plane intersection only.

The SH-DARS model in (Yang et al., 11 Nov 2025) is given by:

$$\mathbf{I}_{\rm final} = \mathbf{c} + w_{\rm att}\mathbf{I}_{\rm att} + w_{\rm refl}\mathbf{I}_{\rm refl} + w_{\rm scat}\mathbf{I}_{\rm scat}$$

where $\mathbf{c}$ is the low-order spherical harmonics signal, with augmentation terms for Beer–Lambert depth attenuation, tissue boundary specular reflection, and cross-channel volumetric scattering. All weights are learnable.

In both variants, the rendering equation is differentiable, supporting end-to-end gradient-based optimization.

3. Training Objectives, Optimization, and Regularization

UltraGS frameworks formulate the learning process as direct minimization of discrepancy between rendered and measured B-mode frames—primarily pixelwise mean squared error (MSE):

$$\mathcal{L}_{\rm rec} = \sum_{i} \|\mathbf{I}_{\rm final}(\mathbf{x}_i) - \mathbf{I}_{\rm gt}(\mathbf{x}_i)\|^2$$

Additional regularization includes:

• Stabilization of Gaussian scale/covariance to suppress collapse or over-spread.
• $\ell_2$ constraints on physically-meaningful parameters (e.g., SH coefficients, FoV deviations).
• Adam optimizer with tuned learning rates—higher for amplitude/color/covariance, lower for mean position variables (Eid et al., 8 May 2025).

For the 3D reconstruction UltraGS (Eid et al., 8 May 2025), periodic resampling of Gaussian positions ensures uniform spatial coverage.

4. Clinical Datasets and Benchmarks

UltraGS (Yang et al., 11 Nov 2025) contributes the Clinical Ultrasound Examination Dataset, designed for realistic evaluation:

• Six unique cases (three wrist, three kidney), each with dozens of frames.
• Data acquired on Canon i900, B-mode, 30 FPS, following clinical sweep patterns with annotated target anatomy (e.g., median nerve, carpal tunnel, kidney sections).
• Intrinsic/extrinsic parameters initialized by COLMAP and further refined during training.

Ground-truth is provided at the level of probe pose and intrinsic mappings, enabling benchmark-grade evaluation on standardized real-world clinical acquisition.

5. Performance Evaluation and Ablations

UltraGS frameworks were compared quantitatively and qualitatively to prior neural and explicit approaches, including NeRF, TensoRF, Ultra-NeRF, 3DGS, and SuGaR (Yang et al., 11 Nov 2025), as well as ImplicitVol and RapidVol (Eid et al., 8 May 2025).

Key results:

Dataset	Model	PSNR	SSIM	MSE	FPS
Wild (Yang et al., 11 Nov 2025)	NeRF	20.18	0.683	0.0066	0.28
	TensoRF	24.06	0.753	0.0051	1.61
	3DGS	22.33	0.775	0.0057	52.56
	SuGaR	21.39	0.629	0.0159	9.81
	UltraGS	25.45	0.797	0.0043	64.69

For clinical freehand "cinesweep" tasks (Eid et al., 8 May 2025), UltraGS-300K achieves SSIM $0.914 \pm 0.012$ after 5 minutes, compared to $0.768 \pm 0.028$ for RapidVol (runtime 2 hours).

Ablations demonstrate that:

• Removal of learnable FoV or SH-DARS in (Yang et al., 11 Nov 2025) leads to $\sim$2 dB and $\sim$1.6 dB PSNR drops, respectively.
• Covariance parameterization (triangular factorization) in (Eid et al., 8 May 2025) offers the best performance–efficiency trade-off, outperforming quaternion or M-matrix schemes.
• 95% chi-square ellipsoid culling boosts speed by 30% versus naïve evaluation.

Expert studies: Ten sonographers (avg. 18 years' experience) preferred UltraGS reconstructions to competing methods after 5 and 20 minutes of optimization; 80% rated UltraGS as more realistic than real scans by 15 minutes (Eid et al., 8 May 2025).

6. Implementation and Real-Time Considerations

UltraGS frameworks are implemented for GPU acceleration via a two-phase CUDA rasterization pipeline:

• Phase 1: Culling of off-plane Gaussians using chi-square ellipsoid bounds in depth ($z$) for each probe position.
• Phase 2: Parallel per-Gaussian rasterization over the probe-plane bounding box, atomic addition for intensity accumulation.

Memory is managed by binning Gaussians into spatial tiles for efficient culling and buffer compaction, with the core representation stored in an $N \times 11$ float buffer (mean, color, amplitude, 6 covariance parameters).

UltraGS (Yang et al., 11 Nov 2025) achieves real-time synthesis (1024 × 1024, 64.69 fps, RTX 3090), supporting both fast inference for live clinical applications and practical optimization within tens of thousands of iterations. Open-source code and datasets are provided, facilitating reproducibility and extensibility.

7. Significance and Distinctions from Prior Art

UltraGS advances the field of ultrasound computational imaging by integrating physically-motivated Gaussian splatting with explicit ultrasound physics and real-time GPU rendering. This results in:

• Clinically meaningful image synthesis: rapid, accurate novel view and 3D visualization in diagnostic protocols.
• Physically accurate rendering: SH-DARS provides decoupled modeling of attenuation, specular, and scattering phenomena encountered in real tissues.
• Robustness to variable field of view and probe motion: learnable intrinsics adjust to the inherent variability of handheld clinical acquisition.
• Numerical and implementation stability through carefully chosen covariance parametrization and efficient culling strategies.

A plausible implication is that these representations, by explicitly modeling ultrasound data formation and clinical probe usage, overcome many limitations of both classical volumetric interpolation and existing neural-field methods—setting new standards for SSIM, PSNR, and expert perceptual realism in clinical ultrasound visualization tasks (Yang et al., 11 Nov 2025, Eid et al., 8 May 2025).