Multimodal Diffeomorphic Registration

• Multimodal diffeomorphic registration is a computational approach that uses smooth, invertible transformations to align images from different modalities while preserving anatomical topology.
• It employs both stationary velocity fields and time-dependent neural ODE models to capture large deformations, ensuring structural correspondence and robust registration across imaging techniques.
• Probabilistic and contrastive feature learning extensions, combined with modality-agnostic similarity measures, enhance performance and facilitate uncertainty quantification in clinical studies.

A multimodal diffeomorphic registration method is a computational framework designed to align images from different acquisition modalities (e.g., T1- and T2-weighted MRI, CT and MRI) through smooth, invertible (diffeomorphic) spatial transformations that account for substantial anatomical and intensity differences. These methods aim to maximize structural correspondence across modalities while guaranteeing topology preservation and invertibility of the deformation, which are essential in quantitative imaging tasks and computational anatomy.

1. Mathematical Foundations: Variational and Probabilistic Formulations

Multimodal diffeomorphic registration originates from the large-deformation diffeomorphic metric mapping (LDDMM) paradigm, seeking a diffeomorphism $\varphi$ that minimizes an objective functional: $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$ subject to the ODE

$$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$

where $I_f$ is the fixed image, $I_m$ the moving image, $v(t, \cdot)$ a time-dependent velocity field in an RKHS $V$ defined by the smoothing operator $L^{-1}$, $\mathcal{S}(\cdot,\cdot)$ a modality-agnostic dissimilarity, and $\lambda > 0$ a regularization parameter. By penalizing the $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$0-norm of $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$1, one ensures $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$2 is diffeomorphic, i.e., invertible and smooth. The deformation map is typically recovered via the group exponential of $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$3 (Rodriguez-Sanz et al., 27 Dec 2025).

Probabilistic extensions, as proposed in (Dalca et al., 2019) and (Ouderaa et al., 2020), employ generative models wherein the observed images and surfaces are generated by warping a template under a stochastic diffeomorphic transformation drawn from a Gaussian process prior, enabling unsupervised training and uncertainty estimation. The evidence lower bound (ELBO) framework fuses data-fit and regularization, permitting variational inference via deep networks.

2. Diffeomorphic Transformation Models and Parameterizations

Two principal techniques exist for parameterizing diffeomorphisms:

• Stationary Velocity Field (SVF): $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$4, with $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$5 fixed in time. Integration employs the scaling-and-squaring method, ensuring efficient and invertible flows (Dalca et al., 2019, Sideri-Lampretsa et al., 2022).
• Time-dependent Velocity/Neural ODEs: The velocity $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$6 evolves continuously, with $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$7 parameterized by a neural network integrated over time by an ODE solver. In (Rodriguez-Sanz et al., 27 Dec 2025), a convolutional Neural ODE architecture computes $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$8, imposing smoothness at all points along the flow and post-smoothing with $$E(v) = \mathcal{S}(I_f, I_m \circ \varphi(1)) + \lambda \int_0^1 \|L v(t)\|^2 dt$$9. This avoids discretization artifacts and enables continuous-depth, per-instance optimization.

Regularization is enforced via the $$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$0-norm and explicit penalties on the Jacobian determinant, e.g., $$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$1, driving topological correctness (Rodriguez-Sanz et al., 27 Dec 2025, Liu et al., 2020).

3. Modality-Agnostic Similarity Metrics and Structural Descriptors

Since multimodal images often lack simple intensity correspondences, robust similarity terms are vital. Three prevalent approaches are:

Descriptor Type	Description	Example Methods
Structural	Modality-independent descriptors exploiting self-similarity	MIND (Rodriguez-Sanz et al., 27 Dec 2025, Liu et al., 2020)
Feature-based	Learned features, e.g., U-Net with contrastive objectives	Neural ODE + contrastive (Rodriguez-Sanz et al., 27 Dec 2025)
Information-theoretic	Entropy-based patch-wise local mutual information	Local MI (Rodriguez-Sanz et al., 27 Dec 2025), Groupwise NMI (Ouderaa et al., 2020)

• MIND (Modality Independent Neighborhood Descriptor): Encodes local patch self-similarity, with descriptors $$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$2. MIND has proven most effective for MRI T1-T2 alignment, yielding highest structure overlap and topology preservation (Rodriguez-Sanz et al., 27 Dec 2025, Liu et al., 2020).
• Contrastive Feature Embedding: U-Net backbones subjected to monotone intensity transforms, trained with voxel-level contrastive losses, provide dense representations that are robust to modality shift (Rodriguez-Sanz et al., 27 Dec 2025).
• Local Mutual Information and NMI: Patch-wise or group-wise normalized mutual information measures are integrated into the loss to drive registration in the absence of intensity homology (Rodriguez-Sanz et al., 27 Dec 2025, Ouderaa et al., 2020).

Alternative approaches include edge-map driven losses that are largely invariant to modality and require no manual labeling (Sideri-Lampretsa et al., 2022).

4. Algorithmic Variants and Network Architectures

Modern methods implement the core registration model via deep networks, varying in modality-handling and diffeomorphic implementation:

• Instance-specific Neural ODEs: Solved per pair, not relying on extensive training datasets, adapt to previously unseen modalities at inference (Rodriguez-Sanz et al., 27 Dec 2025).
• UNet-based Parameterizations: Encoder–decoder architectures predict velocity fields or their parameter distributions, serving as amortized inference within a probabilistic pipeline (Dalca et al., 2019, Ouderaa et al., 2020, Sideri-Lampretsa et al., 2022).
• Groupwise Extensions: Multiple modalities or time-points are registered simultaneously via a shared template, with diffeomorphic velocity fields for each instance and joint optimization of template and transforms (Ouderaa et al., 2020).
• Coarse-to-fine Bilevel Strategies: Multi-scale feature pyramids and iterative refinement secure convergence to globally optimal diffeomorphic maps, with bilevel tuning for hyperparameter optimization (Liu et al., 2020).
• Edge-Driven Unsupervised Learning: Auxiliary edge information extracted via gradient magnitude is processed in a two-branch U-Net; recombination in the decoder aids geometry-aware, fast registration (Sideri-Lampretsa et al., 2022).

5. Evaluation, Benchmarking, and Results

Methods are assessed on a suite of public neuroimaging datasets (e.g., OASIS-3, IXI, CamCAN, BraTS18), using metrics such as structure-wise Dice similarity, fraction of voxels with non-positive Jacobian determinant, and runtime. Consistent findings include:

• State-of-the-art Performance: Neural ODE + MIND variants surpass classical and prior deep-learning baselines (Wilcoxon $$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$3 in most settings) especially in large deformation settings and Alzheimer’s cohorts (Rodriguez-Sanz et al., 27 Dec 2025).
• Negligible Folding: Negative-Jacobian ratios can be tuned to $$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$4–$$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$5; foldings can be nearly eliminated at minimal loss in Dice by explicit penalization (Rodriguez-Sanz et al., 27 Dec 2025, Liu et al., 2020, Sideri-Lampretsa et al., 2022).
• Run-time Efficiency: Deep-learning approaches (instance-based or amortized) are up to $$\frac{d \varphi(t)}{dt} = v(t, \varphi(t)), \quad \varphi(0) = \mathrm{Id}$$6 faster than traditional iterative tools (e.g., ANTs/SyN, NiftyReg), with per-pair times of seconds to minutes (Rodriguez-Sanz et al., 27 Dec 2025, Sideri-Lampretsa et al., 2022, Ouderaa et al., 2020).
• Robust Modality Transfer: SDR-based, MI, and edge-driven pipelines generalize to novel modality combinations (e.g., MRI–CT, MRI–PET) without additional retraining (Rodriguez-Sanz et al., 27 Dec 2025, Ouderaa et al., 2020, Liu et al., 2020, Brudfors et al., 2020).
• Uncertainty Quantification: Probabilistic methods output per-voxel variance or uncertainty maps, facilitating quality control and downstream analysis (Dalca et al., 2019, Ouderaa et al., 2020).

6. Extensions, Limitations, and Prospects

Multimodal diffeomorphic registration methods have extended to:

• Discrete Varifold Models: Generalizing to vector-valued or directionally encoded data (e.g., diffusion MRI peaks), using varifold fidelity terms in the RKHS to match orientation distributions (Hsieh et al., 2018).
• Groupwise and Atlas Construction: Bayesian frameworks accommodate simultaneous registration of image cohorts and allow for joint estimation of anatomical templates, bias fields, and transform parameters (Brudfors et al., 2020, Ouderaa et al., 2020).
• Scale and Domain Adaptation: Instance-wise optimization and edge-based losses allow adaptation to different anatomical regions, scales, and image dimensions (Rodriguez-Sanz et al., 27 Dec 2025, Sideri-Lampretsa et al., 2022).

Identified limitations include scaling to ultra-high-resolution domains, the need for more expressive descriptors for cross-modality alignment beyond MRI, and potential approximation errors in ODE integration. Templates for uncertainty guidance and iterative post-hoc refinement are active research areas (Ouderaa et al., 2020, Brudfors et al., 2020, Dalca et al., 2019).

7. Summary Table of Multimodal Diffeomorphic Registration Approaches

Approach	Diffeo Model	Modality Handling	Notable Feature	Reference
Neural ODE + MIND (instance)	Time-dependent Neural ODE	Self-similarity (MIND)	Robust, pairwise, no retraining	(Rodriguez-Sanz et al., 27 Dec 2025)
VoxelMorph-diff (amortized)	Stationary v, scaling^2	MI, LNCC, learned MI	Prob. inference, fast, uncertainty	(Dalca et al., 2019)
GroupMorph (groupwise VAE)	Stationary v, scaling^2	Groupwise NMI	Multiple images, groupwise avg.	(Ouderaa et al., 2020)
Edge-Map Two-Branch U-Net	Stationary v, scaling^2	Edge LNCC, MI, NGF	No labels, fast, 2-branch U-Net	(Sideri-Lampretsa et al., 2022)
LDDMM-Varifold	v(t), geodesic shooting	Varifold (orientation)	Multi-direction, orient. inv.	(Hsieh et al., 2018)
Bayesian (SPM) Groupwise	Stationary v, shooting	Mixture-of-Gaussian, INU	Joint template/reg., no pre-align	(Brudfors et al., 2020)
Multi-scale/bilevel deep	Coarse2fine, scaling^2	MIND, MI (modality-indep)	Bilevel tuning, robust diffeo.	(Liu et al., 2020)

This table reflects the diversity of models and strategies employed to realize accurate, efficient, and robust multimodal diffeomorphic registration in contemporary computational anatomy and medical imaging research.