MedGS: Neural Gaussian Surface Reconstruction

• MedGS is a semi-supervised framework that represents sparse 2D medical slices in 3D using time-conditioned folded-Gaussians.
• It employs a Gaussian splatting interpolation mechanism with in-between frame regularization to enhance continuity and reduce noise artifacts.
• The method jointly optimizes Gaussian primitives for both image reconstruction and explicit mesh extraction, yielding faster and more accurate 3D surface modeling.

MedGS is a semi-supervised neural implicit surface reconstruction framework for multi-modal three-dimensional medical imaging that uses a Gaussian Splatting-based interpolation mechanism to unify frame-to-frame interpolation of sparsely acquired two-dimensional slices and high-fidelity surface reconstruction from segmentation masks. It represents medical imaging data as consecutive two-dimensional frames embedded in three-dimensional space and modeled using Gaussian-based distributions, with the stated goals of synthesizing original and intermediate slices with high fidelity and enabling extraction of smooth, topologically correct meshes via marching cubes (Marzol et al., 20 Sep 2025).

1. Conceptual scope and problem setting

MedGS is formulated for multi-modal 3D medical imaging data derived from ultrasound, magnetic resonance imaging, and potentially computed tomography. The framework is designed around a setting in which the input consists of a sequence of grayscale cross-sectional images $\{I_t\}$ acquired at normalized times $t\in[0,1]$, free-hand 3D poses for each frame that place each slice as a plane in $\mathbb{R}^3$, and, optionally, binary segmentation masks for a subset of slices (Marzol et al., 20 Sep 2025).

The central technical claim is that MedGS unifies two tasks within a single pipeline. The first is interpolation between sparsely acquired 2D frames; the second is surface reconstruction from segmentation masks. In the formulation given for the method, these tasks are not handled by separate models. Instead, both are expressed through a shared explicit representation $G=\{G_i\}$ of Folded-Gaussians. This shared representation is optimized so that it can render image slices at observed or intermediate positions and also support mesh extraction.

Traditional methods are described as being limited by image noise and incomplete information between frames. MedGS addresses this by replacing purely frame-local reasoning with a time-conditioned Gaussian representation that spans space and acquisition order. A plausible implication is that the framework is intended to regularize across sparse slice sequences without requiring dense volumetric supervision.

2. Gaussian representation and rendering model

Each primitive in MedGS is a 3D Folded-Gaussian conditioned on time $t$, with an associated opacity $\rho$ and grayscale coefficient $c$. A static Gaussian splat is defined as

$G_i=(\mu_i,\Sigma_i,\rho_i,c_i),$

where $\mu_i\in\mathbb{R}^3$ is the mean, $\Sigma_i\in\mathbb{R}^{3\times 3}$ is the anisotropic covariance, $t\in[0,1]$0 is the opacity, and $t\in[0,1]$1 is the grayscale color (Marzol et al., 20 Sep 2025).

The corresponding 3D Gaussian density is

$t\in[0,1]$2

Because MedGS operates on sequences rather than static volumes, it adopts the VeGaS Folded-Gaussian extension. In this construction, each Gaussian’s mean and covariance are allowed to shift and rescale over time through learnable functions:

$t\in[0,1]$3

The full space-time density of the $t\in[0,1]$4th component is then written as

$t\in[0,1]$5

with $t\in[0,1]$6 (Marzol et al., 20 Sep 2025).

Slice rendering at time $t\in[0,1]$7 is described as projection of each Gaussian primitive onto the imaging plane followed by summation of weighted contributions. The simplified radiance model is

$t\in[0,1]$8

where $t\in[0,1]$9 denotes projection of the mean and covariance onto the plane. In practice, MedGS uses a full differentiable Gaussian-splatting renderer to account for occlusion, spherical harmonics color, and alpha-compositing (Marzol et al., 20 Sep 2025).

All parameters $\mathbb{R}^3$0 are jointly optimized by gradient descent with PyTorch/CUDA. Covariances are often parameterized through an eigendecomposition $\mathbb{R}^3$1, with diagonal $\mathbb{R}^3$2, and the logarithm of the eigenvalues is optimized for stability. This explicit representation is presented as a distinguishing feature relative to traditional neural implicit methods.

3. Interpolation mechanism and temporal regularization

MedGS performs frame interpolation implicitly through its time-conditioned Folded-Gaussians, but it also introduces In-Between Frame Regularization (IBFR) to reduce overfitting to noisy slices and to encourage continuity between adjacent real frames (Marzol et al., 20 Sep 2025).

For every pair $\mathbb{R}^3$3, the framework samples $\mathbb{R}^3$4 and constructs a synthetic target:

$\mathbb{R}^3$5

The model then renders a slice at time $\mathbb{R}^3$6 and adds the interpolation loss

$\mathbb{R}^3$7

According to the formulation in the paper, this term prevents the Folded-Gaussians from drifting or collapsing between real slices.

A second temporal regularizer constrains each Gaussian’s temporal width $\mathbb{R}^3$8 to the interval $\mathbb{R}^3$9:

$G=\{G_i\}$0

This penalty acts directly on the temporal support of the primitives.

The method’s interpretation of interpolation is therefore not merely geometric averaging of slices. It is a learned space-time representation regularized by synthetic in-between supervision. This clarifies a potential misconception: in MedGS, interpolation is not a post hoc operation applied after reconstruction, but part of the training objective that shapes the representation itself.

4. Surface reconstruction and semi-supervised learning

For mesh reconstruction, MedGS is trained on binary masks $G=\{G_i\}$1 rather than grayscale images. After convergence, the method renders a dense sequence of masks at fine time steps, stacks them into a 3D volume $G=\{G_i\}$2, and applies marching cubes at the iso-value $G=\{G_i\}$3 (Marzol et al., 20 Sep 2025).

An equivalent occupancy-field view is given by

$G=\{G_i\}$4

with surface extraction from the level set

$G=\{G_i\}$5

A critical design point is that no signed distance function is learned explicitly; mesh quality is stated to emerge from the smooth tails of the Gaussian mixture. This distinguishes MedGS from SDF-based reconstruction pipelines and is one of the clearest architectural differences emphasized in the source.

The paper characterizes the framework as semi-supervised because unlabeled frames contribute through image reconstruction objectives, while sparsely labeled slices contribute direct supervision for mesh extraction. Between-slice unlabeled data also contribute indirectly through the interpolation regularizer. This suggests that MedGS is aimed at settings in which segmentation masks are sparse or incomplete rather than fully dense across the stack.

5. Optimization pipeline and loss design

The end-to-end pipeline begins by initializing approximately $G=\{G_i\}$6 folded-Gaussian primitives $G=\{G_i\}$7. Training alternates gradient steps on image reconstruction losses for grayscale images and masks with background densification and pruning operations that add or remove Gaussians. IBFR is applied on synthesized intermediate frames, and, for mesh reconstruction, training is conducted only on binary masks before dense interpolation and marching cubes are used for mesh extraction (Marzol et al., 20 Sep 2025).

Two task configurations are defined. For interpolation with grayscale frames and no masks, the common losses are

$G=\{G_i\}$8

and the total interpolation loss is

$G=\{G_i\}$9

For mesh reconstruction, where masks are less noisy and IBFR is not used, the objective is

$t$0

The hyperparameters reported are $t$1, $t$2, $t$3, and $t$4 (Marzol et al., 20 Sep 2025).

The efficiency claims are quantitative. On an NVIDIA A100, each object trains in approximately 20 minutes, starting from 100k Gaussians with adaptive densification, and final rendering or marching-cubes mesh extraction requires only a few seconds. Typical neural implicit SDF methods are described as requiring several hours of training and minutes to extract an SDF mesh. The stated computational profile for MedGS is that each optimization step processes $t$5 rays times $t$6 Gaussians, with pruning stabilizing $t$7 around 50k–150k and $t$8, yielding an empirical $t$9–$\rho$0 speed-up over neural implicit baselines.

6. Empirical performance, ablations, and reported applications

The interpolation experiments on MRI use leave-one-out testing with every 2nd, 3rd, and 5th slice held out, evaluated with PSNR, SSIM, Dice, and IoU. For the every-2nd-frame setting, the reported results are as follows (Marzol et al., 20 Sep 2025):

Method	PSNR	Dice	IoU	SSIM
Linear interp	32.70±1.90	0.57±0.27	0.44±0.22	0.91±0.02
Optical flow	29.07±2.27	0.50±0.25	0.37±0.19	0.86±0.04
MedGS	33.52±1.87	0.67±0.18	0.53±0.16	0.91±0.01

The paper states that MedGS also outperforms baselines on every-3rd and every-5th-slice settings by similar margins and qualitatively produces sharper anatomical boundaries with fewer ghosting artifacts than linear and optical-flow interpolation.

The ablation study for the every-2nd-frame setting isolates the contribution of the regularizers. Without $\rho$1, the reported values are PSNR $\rho$2 and Dice $\rho$3; without $\rho$4, PSNR $\rho$5 and Dice $\rho$6; and with the full model, PSNR $\rho$7 and Dice $\rho$8. These values tie the interpolation gains directly to IBFR and temporal-width regularization rather than to Gaussian parameterization alone.

For mesh reconstruction on prostate ultrasound, the validation metrics are Chamfer Distance, Hausdorff Distance, and HD95. The reported averages across specimens 65–72 are:

Method	CD	HD	HD95
ISO-Surface	0.228	0.755	0.390
FUNSR	0.228	0.835	0.400
Poisson	0.211	0.920	0.374
MedGS	0.203	0.827	0.365

The visual assessment reported for these experiments is that MedGS meshes preserve fine anatomical sulci and gland boundaries that other methods either smooth over or render with staircase artifacts.

The paper also outlines several extensions and applications: interactive deformable registration for ultrasound-guided surgery through manual editing of Gaussian centers or covariances, modeling of dynamic organs by conditioning on physiological phase variables beyond uniform time, jointly trained multi-modal fusion across co-registered CT/MRI/US volumes, extension to 4D volume reconstruction through a second temporal dimension, and real-time deployment via further C++/CUDA optimization targeting under 100 ms per slice (Marzol et al., 20 Sep 2025). These are presented as prospective directions rather than validated benchmarks.

Overall, MedGS occupies the intersection of explicit Gaussian scene representations and medical image reconstruction. Within the reported formulation, its distinctive properties are the use of time-conditioned Folded-Gaussians, IBFR for between-slice regularization, mesh extraction without an explicit SDF, and a semi-supervised optimization regime intended for sparse or noisy annotation settings.