VidFuncta: Implicit Neural Ultrasound Videos

• VidFuncta is a framework for ultrasound video analysis that uses a two-level latent decomposition to separate static and dynamic features for efficient reconstruction.
• It encodes each video as a compact combination of a static video-specific latent vector and a sequence of time-dependent modulation vectors conditioned through an implicit neural decoder.
• The approach improves reconstruction quality and reduces memory usage, offering competitive performance on downstream tasks like cardiac prediction, lung B-line detection, and breast lesion classification.

Searching arXiv for the cited VidFuncta paper and closely related Functa/MedFuncta work. Search query: arXiv ids (Wolleb et al., 29 Jul 2025, Dupont et al., 2022), and keyword MedFuncta. VidFuncta is a framework for ultrasound video analysis that extends the Functa paradigm of treating data points as implicit neural representations to the temporal setting of medical video. Instead of processing a video as a dense spatiotemporal tensor with a conventional 2D or 3D network, it represents each variable-length ultrasound sequence by a compact combination of a static video-specific latent and a sequence of time-dependent modulation vectors that condition a shared implicit neural decoder. The framework is motivated by two features of ultrasound data: substantial redundancy across frames and studies, and the difficulty standard deep models face under non-standardized acquisition, operator dependence, and highly variable image quality. In that sense, VidFuncta is both a video extension of Functa and a domain-specific attempt to obtain compact, time-resolved, and reusable representations for reconstruction and downstream prediction in ultrasound (Wolleb et al., 29 Jul 2025, Dupont et al., 2022).

1. Clinical and methodological setting

Ultrasound is widely used in clinical care, but full-video analysis is difficult because acquisition is non-standardized, operator-dependent, and highly variable in quality. VidFuncta is introduced from the observation that ultrasound videos contain substantial redundancy across frames and across studies, and that this redundancy is not naturally exploited by standard CNN- or video-based pipelines. The framework therefore shifts the representational target: rather than learning directly on the raw spatiotemporal tensor, it compresses each video into learned latent modulations that can reconstruct the signal and also serve as a feature space for downstream tasks (Wolleb et al., 29 Jul 2025).

This design is rooted in the broader Functa framework, in which a datapoint is represented as an implicit neural representation and compressed into a modulation vector that conditions a shared neural function (Dupont et al., 2022). VidFuncta adapts that principle from still images to ultrasound videos. A plausible implication is that the method inherits two goals simultaneously: continuous reconstruction via an INR decoder, and representation learning through low-dimensional modulations that can be consumed by separate task-specific models.

The target signal is a grayscale ultrasound video

$\mathcal{V}\in\mathbb{R}^{T\times h\times w},$

and the model predicts grayscale intensity $z$ at spatial coordinate $(x,y)$ for each frame. This formulation places video reconstruction and representation learning inside a coordinate-based function approximation setting rather than a frame-grid processing pipeline.

2. Core representation and architectural design

VidFuncta uses a shared meta-network $M_\theta$, implemented as a multilayer perceptron with sinusoidal activations in the SIREN-style INR family. In the paper’s notation, the meta-model consists of $K$ linear layers $\{L_k\}_{k=1}^K$ of hidden width $l$, with learnable parameters $\theta$. This shared decoder is intended to capture dataset-level structure such as typical anatomy and general ultrasound appearance (Wolleb et al., 29 Jul 2025).

The central architectural modification relative to Functa is a two-level latent decomposition. The first latent is a static, video-specific modulation vector

$v\in\mathbb{R}^s,$

which captures information consistent across the entire video, such as anatomy, scanner settings, depth, and gain. The second latent is a sequence of frame-dependent modulations

$$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$

which encode time-varying changes across frames. The model is explicitly described as disentangling each video into “a static video-specific vector and a sequence of time-dependent modulation vectors,” separating stable content from motion and temporal dynamics.

Both latent types condition the same shared INR through linear projections into per-layer shift modulations. For the video-level latent, a layer-specific linear operator $z$0 maps $z$1 to

$z$2

which is added to the output of layer $z$3. For the temporal latent sequence, $z$4 maps frame codes to

$z$5

which are likewise added to the layer outputs. Conceptually, this is a FiLM-like conditioning mechanism: the decoder remains shared across the dataset, while the modulations specialize it to one video and one frame. The intended semantic split is that $z$6 carries time-invariant properties and $z$7 carries temporal dynamics.

This two-scale factorization is a defining property of VidFuncta. It differs from a single global latent by making temporal variation explicit and ordered, while also avoiding the need to duplicate static information across all frames.

3. Training objective and inference procedure

The reconstruction objective is mean-squared error over sampled coordinates. For a frame at time $z$8, the loss is

$z$9

where $(x,y)$0 is the number of sampled spatial coordinates and $(x,y)$1 is the true grayscale intensity at coordinate $(x,y)$2 (Wolleb et al., 29 Jul 2025).

Training follows a meta-learning style similar to MedFuncta. In each iteration, one video is loaded, a batch $(x,y)$3 of $(x,y)$4 frames is sampled, and the latents are zero-initialized. Then, for $(x,y)$5 inner steps, the frame latents are updated by

$(x,y)$6

and the video latent is updated by

$(x,y)$7

After latent adaptation, the shared meta-model parameters are updated by

$(x,y)$8

The operational logic is that the meta-network learns cross-video regularities, while the compact latent codes absorb video- and frame-specific detail.

Inference on long videos uses an autoregressive reconstruction scheme to remain within memory limits. The first batch $(x,y)$9 contains the first $M_\theta$0 frames. The shared network $M_\theta$1 is frozen, and both $M_\theta$2 and $M_\theta$3 are optimized for $M_\theta$4 steps. After that, $M_\theta$5 is frozen for the remainder of the video, and only the next frame-specific latents are optimized batch by batch. The stated rationale is that once $M_\theta$6 captures persistent appearance, subsequent batches need only explain temporal changes. Reconstruction of a batch is obtained by querying the INR at all spatial coordinates:

$M_\theta$7

and reconstructed batches are concatenated along time to form the final video $M_\theta$8.

4. Experimental configuration and reconstruction behavior

VidFuncta is evaluated on three public ultrasound video datasets spanning distinct clinical tasks: EchoNet-Dynamic with 10,030 cardiac videos labeled by ejection fraction, BEDLUS with 2,026 lung videos annotated for B-lines, and a breast dataset with 188 videos labeled benign versus malignant. All videos are resized to $M_\theta$9 and normalized to $K$0. The study also defines a mixed dataset containing 188 breast, 190 lung, and 190 cardiac videos to test whether a single model can handle multiple ultrasound modalities. Evaluation uses a 10% test split and 5-fold cross-validation on the remainder (Wolleb et al., 29 Jul 2025).

Dataset	Size	Downstream task
EchoNet-Dynamic	10,030 videos	Ejection fraction prediction
BEDLUS	2,026 videos	B-line detection
Breast dataset	188 videos	Breast lesion classification

The architecture uses $K$1 layers with hidden dimension $K$2, video latent dimension $K$3, frame latent dimension $K$4, about 24× compression, $K$5 inner-loop steps, $K$6, and $K$7. Training runs for 100,000 iterations on a 24GB RTX A5000 GPU and takes about 20 hours per model.

For reconstruction, VidFuncta is compared against MedFuncta 2D, MedFuncta 3D, and Spatial Functa, using PSNR and 3D SSIM. On the cardiac dataset, VidFuncta reaches SSIM3D $K$8 and PSNR $K$9, compared with MedFuncta 2D at $\{L_k\}_{k=1}^K$0 and $\{L_k\}_{k=1}^K$1, MedFuncta 3D at $\{L_k\}_{k=1}^K$2 and $\{L_k\}_{k=1}^K$3, and Spatial Functa at $\{L_k\}_{k=1}^K$4 and $\{L_k\}_{k=1}^K$5. On lung, it achieves $\{L_k\}_{k=1}^K$6 and $\{L_k\}_{k=1}^K$7, exceeding MedFuncta 2D at $\{L_k\}_{k=1}^K$8 and $\{L_k\}_{k=1}^K$9 and Spatial Functa at $l$0 and $l$1. On breast, it obtains $l$2 and $l$3, compared with MedFuncta 2D at $l$4 and $l$5 and MedFuncta 3D at $l$6 and $l$7.

The mixed-dataset model remains strong, with cardiac, lung, and breast SSIM3D values of $l$8, $l$9, and $\theta$0, respectively, indicating that a single VidFuncta model can generalize across ultrasound modalities without major degradation. Out-of-distribution performance drops, as expected, but remains visually plausible and still decent, for example lung SSIM3D $\theta$1 and PSNR $\theta$2. The paper also reports that reconstructing a natural image produces an ultrasound-like texture while preserving coarse structure, suggesting a strong learned ultrasound prior and a style-transfer-like effect.

5. Downstream task performance and computational efficiency

A major part of the VidFuncta evaluation is whether the learned modulations are useful beyond reconstruction. The downstream tasks are ejection fraction prediction on cardiac videos, B-line detection on lung videos, and breast lesion classification. Three VidFuncta input configurations are tested: only the temporal sequence $\theta$3, only the static vector $\theta$4, and both combined. The downstream models are a transformer encoder for the $\theta$5 sequence, an MLP for $\theta$6, and a combined model in which an embedding of $\theta$7 is appended to the $\theta$8 sequence. Comparisons include MedFuncta 2D, Spatial Functa, and the convolutional baselines R(2+1)D and PocovidNet; for PocovidNet, performance is also measured on reconstructed videos $\theta$9 (Wolleb et al., 29 Jul 2025).

For cardiac ejection fraction prediction, the temporal modulations $v\in\mathbb{R}^s,$0 are the most informative among the VidFuncta variants. Using only $v\in\mathbb{R}^s,$1 gives MAE $v\in\mathbb{R}^s,$2, RMSE $v\in\mathbb{R}^s,$3, and $v\in\mathbb{R}^s,$4, slightly better than combining $v\in\mathbb{R}^s,$5 with MAE $v\in\mathbb{R}^s,$6, RMSE $v\in\mathbb{R}^s,$7, and $v\in\mathbb{R}^s,$8. The static vector alone performs poorly for cardiac regression, with MAE $v\in\mathbb{R}^s,$9 and $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$0. However, the convolutional baselines remain stronger: PocovidNet on original videos reaches MAE $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$1, RMSE $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$2, and $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$3, while R(2+1)D reaches MAE $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$4, RMSE $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$5, and $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$6. A notable result is that PocovidNet on reconstructed videos performs nearly as well as on raw videos, with MAE $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$7 versus $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$8, indicating that VidFuncta preserves task-relevant information even when the learned latent-space predictor is not yet optimal.

For breast lesion classification, the best VidFuncta setting is the combination $$\phi := \{\phi_1,\ldots,\phi_T\}, \qquad \phi_t\in\mathbb{R}^r,$$9, with accuracy $z$00, F1 $z$01, and AUROC $z$02. These numbers are comparable to PocovidNet on raw videos, which achieves $z$03, $z$04, and $z$05, and better than MedFuncta 2D at $z$06, $z$07, and $z$08. On lung B-line detection, by contrast, all Functa-based approaches remain limited, with accuracies around $z$09 and AUROC around $z$10, whereas PocovidNet reaches $z$11 accuracy and $z$12 AUROC on raw videos. The authors interpret this gap as evidence that the current downstream heads are not yet extracting the latent information effectively, rather than as a failure of the representation itself.

The learned 1D modulation vectors are also associated with substantial efficiency gains. For 30 epochs with batch size 10, PocovidNet requires 8.0 GB of memory and 4.5 hours, whereas the VidFuncta downstream setup uses only 0.35 GB and 11 minutes, roughly a 25× reduction in time and memory. This efficiency derives from replacing raw video tensors with compact latent sequences and a static summary vector.

6. Interpretation, ablations, limitations, and significance

The paper argues that the modulation vectors are useful because they provide a compact, ordered, and temporally resolved representation of the ultrasound sequence. The static latent $z$13 is optimized only once and then frozen during autoregressive inference, so it functions as a summary of persistent appearance, while the sequence $z$14 captures frame-wise variation. Qualitative analysis supports this interpretation: in reconstructed cardiac sequences, setting $z$15 while keeping $z$16 yields a coherent video-level summary, reinforcing the view that $z$17 contains static appearance information (Wolleb et al., 29 Jul 2025).

Several ablations clarify the representational choice. Spatial Functa reshapes the latent into a $z$18 grid at a similar compression rate, but performs worse than VidFuncta on reconstruction and does not improve downstream performance. This suggests that a simple 1D temporal latent sequence is more suitable for ultrasound video than a spatially reshaped latent at comparable compression. In the mixed-dataset setting, t-SNE visualizations of $z$19 reveal clear modality clusters, indicating that the static latent captures dataset-specific and anatomical factors.

The study also highlights limitations. High-frequency details remain difficult to reconstruct, and the authors note that alternative activations such as WIRE or FINER might improve fidelity in future work. Reconstruction quality and downstream performance are not strongly correlated on the cardiac dataset, showing that better reconstruction alone does not guarantee better task features. A common misconception is therefore that an INR with high PSNR or SSIM necessarily yields the strongest latent representation for prediction; the reported cardiac results contradict that assumption.

The broader significance of VidFuncta lies in moving Functa from still images to ultrasound videos by splitting the latent code into a static video vector $z$20 and a time-indexed sequence $z$21 trained with a meta-learned INR decoder. The resulting framework yields strong reconstruction, compact representations, and large efficiency gains, while downstream performance is competitive in breast lesion classification, promising in cardiac regression, and still limited in lung B-line detection. Taken together, these results position VidFuncta as a generalizable representation framework for ultrasound videos in which temporal structure is encoded explicitly rather than left implicit inside a dense spatiotemporal backbone.