TempDiffReg: Temporal Diffusion Model for Non-Rigid 2D-3D Vascular Registration

Abstract: Transarterial chemoembolization (TACE) is a preferred treatment option for hepatocellular carcinoma and other liver malignancies, yet it remains a highly challenging procedure due to complex intra-operative vascular navigation and anatomical variability. Accurate and robust 2D-3D vessel registration is essential to guide microcatheter and instruments during TACE, enabling precise localization of vascular structures and optimal therapeutic targeting. To tackle this issue, we develop a coarse-to-fine registration strategy. First, we introduce a global alignment module, structure-aware perspective n-point (SA-PnP), to establish correspondence between 2D and 3D vessel structures. Second, we propose TempDiffReg, a temporal diffusion model that performs vessel deformation iteratively by leveraging temporal context to capture complex anatomical variations and local structural changes. We collected data from 23 patients and constructed 626 paired multi-frame samples for comprehensive evaluation. Experimental results demonstrate that the proposed method consistently outperforms state-of-the-art (SOTA) methods in both accuracy and anatomical plausibility. Specifically, our method achieves a mean squared error (MSE) of 0.63 mm and a mean absolute error (MAE) of 0.51 mm in registration accuracy, representing 66.7\% lower MSE and 17.7\% lower MAE compared to the most competitive existing approaches. It has the potential to assist less-experienced clinicians in safely and efficiently performing complex TACE procedures, ultimately enhancing both surgical outcomes and patient care. Code and data are available at: \textcolor{blue}{https://github.com/LZH970328/TempDiffReg.git}

• The paper introduces TempDiffReg, which combines a structure-aware global initialization and a temporal diffusion model for branch-wise non-rigid deformation, improving vascular registration accuracy to a mean squared error (MSE) of 0.63 mm.
• The model outperforms state-of-the-art baselines consistently, with significant improvements in MSE, MAE, and curvature error, demonstrating the efficacy of temporal modeling and anatomical priors in vascular registration.
• TempDiffReg offers uncertainty quantification through its stochastic sampling, which provides a practical guidance map for clinicians but further validation is needed for clinical decision-making used due to this uncertainty measures don't statistically count due being tightly concentrated in low regions.

Overview

TempDiffReg addresses non-rigid 2D–3D vascular registration for transarterial chemoembolization (TACE) navigation, where pre-operative CTA-derived 3D hepatic centerlines must be aligned to intra-operative 2D DSA frames. The paper proposes a coarse-to-fine pipeline: a structure-aware perspective-n-point (SA-PnP) module for global rigid initialization, followed by a conditional denoising diffusion model that performs branch-wise non-rigid deformation across multiple frames. Evaluation on a curated clinical dataset of 626 multi-frame samples from 23 hepatocellular carcinoma (HCC) cases shows consistent superiority over four state-of-the-art baselines across five metrics, with a mean squared error of 0.63 mm and mean absolute error of 0.51 mm.

Clinical motivation and problem setting

During TACE, operators advance a microcatheter into tumor-supplying arterial branches under fluoroscopic guidance, but the lack of accurate 3D visualization of vasculature complicates super-selective catheterization, leading to repeated contrast injections, prolonged procedure time, increased ionizing radiation exposure, and risks such as vessel wall irritation. The registration task is difficult because of complex hepatic branching, substantial anatomical variability between patients, and modality discrepancies between pre-operative CTA and intra-operative DSA.

The paper positions itself against two methodological families. Classical iterative optimization approaches (diffeomorphic demons, ICP variants, MR-RSGD) are theoretically grounded but prone to local minima under poor initialization. Deep learning registrars—TransMorph, uniGradICON, ViT-V-Net, SIRU-Net—improve accuracy and speed but operate on single frames and lack explicit anatomical shape modeling. Diffusion-based generative models have shown capacity for structured geometric transformations in domains such as human body modeling and stereo correspondence, but existing medical applications are largely static or unconditional; TempDiffReg extends them to temporally coherent, conditioned shape prediction for registration.

Method: coarse-to-fine registration

Global alignment via SA-PnP

The global stage establishes an initial rigid pose per frame using anatomy-aware correspondences rather than raw point matching alone. Bifurcation points extracted from both 2D and 3D centerlines serve as high-weight landmarks; branch mappings are established automatically by comparing branch length and curvature, while endpoints and outliers are down-weighted or excluded. The camera pose $(R, t)$ is estimated by minimizing a weighted reprojection error against the projection $\pi(K[R \mid t]\mathbf{X}_i)$ with intrinsics $K$. This module also serves dataset construction: it produces multi-frame samples with reliable pose parameters, 2D branch annotations, and projected 3D structures.

Local refinement via temporal diffusion

After fixing camera parameters, the vascular tree is decomposed into anatomically meaningful branches, and only projected 3D points are allowed to deform. Both projected points and 2D annotations are resampled to a fixed number of points to enforce point-wise correspondence.

An encoding block processes each branch pair over $N$ consecutive frames. Four feature types per frame—the 2D centerline, projected 3D points, camera pose parameters ($K$, $R$, $t$), and the frame index—are embedded separately (convolutional layers plus branch embedding for centerlines, MLPs for poses, learnable positional embeddings for indices), concatenated, and passed through a two-layer Transformer encoder. Mean pooling over frames yields a global latent vector $\mathbf{y}$ conditioning the decoder.

Shape restoration uses a conditional DDPM. The forward process adds Gaussian noise to ground-truth 2D coordinates according to the standard cumulative noise schedule $\bar{\alpha}_t$; the reverse process is a Markov chain parameterized by conditional mean and covariance functions taking both the noisy shape and $\mathbf{y}$. Notably, the decoder directly predicts the denoised sample $\pi(K[R \mid t]\mathbf{X}_i)$0 rather than the noise, following evidence from Ho et al. and Ramesh et al. that this parameterization performs better. The decoder is deliberately lightweight—an MLP—since the heavy lifting of spatiotemporal context extraction is done by the encoder. Intermediate predictions can be re-noised during training to supervise multiple diffusion steps.

Training optimizes a composite loss combining point-wise MSE, curvature consistency (comparing discrete curvature profiles computed from adjacent point triplets), and a diffusion prediction term evaluated over sampled steps $\pi(K[R \mid t]\mathbf{X}_i)$1. At inference, sampling begins from pure Gaussian noise and iterates the reverse process; stochasticity yields diverse plausible shapes under identical conditions, which the authors exploit for uncertainty quantification.

Dataset construction

A single-center dataset was collected from 23 HCC cases with paired intra-operative DSA sequences and pre-operative CTA reconstructions (centerlines extracted with VMTK). Only clearly visible, consistently annotated branches and frames were retained. Branch annotations from groups of 4 consecutive frames form multi-frame samples, yielding 626 samples for training TempDiffReg. For fair comparison with single-frame baselines, a separate single-frame dataset of 1322 samples was constructed with identical splits, normalization, and evaluation protocols. A limitation worth noting is that 2D centerlines on DSA were manually annotated, so the reported accuracies presuppose annotation quality that may not be available in fully automated deployment.

Quantitative results

On the single-frame comparison dataset, TempDiffReg outperforms all baselines on every metric:

Method	MSE (mm)	MAE (mm)	MaxErr (mm)	LenErr (mm)	CurvErr
TransMorph	2.25	0.88	3.14	1.75	0.303
uniGradICON	1.87	0.64	3.19	2.22	0.239
ViT-VNet	5.55	1.17	5.14	17.21	0.116
SIRU-Net	1.89	0.62	2.58	1.47	0.124
Ours	0.63	0.51	2.01	1.145	0.095

Relative to the strongest baseline (SIRU-Net on most metrics), this corresponds to reductions of 66.7% in MSE, 17.7% in MAE, 22.1% in MaxErr, and 22.0% in LenErr, and an 18.1% CurvErr reduction relative to ViT-VNet. Qualitative comparisons show smoother trajectories and better alignment near bifurcations; uniGradICON introduces curve-shape distortions, and ViT-VNet exhibits discontinuities and topological inconsistencies. An implication of these numbers is that sub-millimeter average alignment error is achievable on this task even when competing methods trained identically remain above 1.8 mm MSE, suggesting temporal context and shape-aware supervision—not architecture scale alone—drive the gain.

Ablation study

Single-module ablations remove temporal modeling (single-frame input), structural priors (length and curvature losses), the Transformer encoder (replaced by an MLP), or the diversity loss $\pi(K[R \mid t]\mathbf{X}_i)$2:

Variant	MSE (mm)	MAE (mm)	MaxErr (mm)	LenErr (mm)	CurvErr
w/o Temporal Modeling	1.12	0.77	2.89	1.653	0.143
w/o Structural Prior	1.06	0.74	2.71	1.632	0.145
w/o Transformer Encoder	0.91	0.68	2.45	1.410	0.121
w/o Diversity Loss	0.75	0.56	2.33	1.192	0.104
w/o Temporal + Transformer	1.39	0.88	3.11	1.799	0.162
w/o Temporal + Structural Prior	1.33	0.85	2.98	1.765	0.158
w/o Structural Prior + Diversity	1.19	0.79	2.81	1.701	0.150
w/o Transformer + Diversity	1.04	0.72	2.56	1.480	0.125
Full Model	0.63	0.51	2.01	1.145	0.095

Removing temporal modeling causes the largest single-component degradation, confirming multi-frame input as the dominant contributor. Structural priors disproportionately affect geometric consistency metrics (CurvErr, LenErr), as expected. Among dual ablations, removing both temporal modeling and the Transformer is worst (MSE 1.39 mm), indicating complementary roles in spatiotemporal representation. Every component contributes measurably, though the diversity loss has the smallest marginal effect.

Uncertainty analysis

With $\pi(K[R \mid t]\mathbf{X}_i)$3 independent stochastic samplings per test branch, predictions are tightly concentrated in well-constrained regions and exhibit broader point-wise standard deviation at bifurcations, occlusions, and high-curvature segments. This behavior provides an interpretable confidence measure that could flag ambiguous regions clinically, although the paper does not quantitatively validate whether these uncertainty bands correlate with actual error (e.g., via calibration analysis), leaving this as an open question.

Limitations and open questions

The paper concedes two principal limitations. First, dataset size is constrained by the difficulty of acquiring paired DSA–CTA data; all results derive from 23 single-center cases, so generalization across centers, scanners, and patient populations remains unverified. Second, metal-induced artifacts from stents or clips degrade image quality and registration accuracy; the proposed remedy—artifact-robust segmentation—is proposed but not implemented or evaluated here. Additional open questions include whether the manual 2D centerline annotation requirement can be relaxed, whether the uncertainty estimates are calibrated enough for clinical decision support, and how the method behaves on vessel pathologies (e.g., tumor blush, embolized vessels) excluded from the curated dataset.

Conclusion

TempDiffReg combines anatomy-aware global initialization (SA-PnP), branch-wise decomposition, temporal Transformer conditioning, and direct-sample-prediction diffusion into a unified 2D–3D vascular registration framework. It achieves state-of-the-art results on a dedicated clinical dataset—0.63 mm MSE and 0.51 mm MAE, with large margins over strong baselines—and its ablations substantiate each design choice, particularly temporal modeling. The framework's stochastic sampling further enables uncertainty visualization, though validation of calibration and robustness to artifacts on larger, multi-center data remains outstanding.