FractMorph: Dual-Parallel Transformer for DIR

• FractMorph is a 3D dual-parallel transformer architecture for deformable image registration that integrates multi-domain fractional Fourier transforms to capture various deformation scales.
• It employs a novel Fractional Cross-Attention module that fuses local spatial and global spectral features for enhanced inter-image correspondence matching.
• The model achieves high registration accuracy on cardiac MRI by predicting a dense deformation field via a lightweight U-Net, balancing precision and computational efficiency.

FractMorph is a 3D dual-parallel transformer architecture for deformable image registration (DIR) that uses multi-domain fractional Fourier transform (FrFT) branches inside transformer cross-attention to capture fine local, semi-global, and large global deformations within a single end-to-end framework (Kebriti et al., 17 Aug 2025). It operates on paired fixed and moving medical volumes, enriches cross-image feature matching through Fractional Cross-Attention (FCA), and predicts a dense deformation field with a lightweight U-Net style network. On the ACDC cardiac MRI dataset, it is reported to achieve an overall Dice Similarity Coefficient (DSC) of $86.45\%$, an average per-structure DSC of $75.15\%$, and a $95$th-percentile Hausdorff distance (HD95) of $1.54$ mm on the authors’ data split (Kebriti et al., 17 Aug 2025).

1. Concept and problem setting

FractMorph addresses deformable image registration, in which a moving image $I_m$ is aligned to a fixed image $I_f$ by predicting a dense deformation field $\phi$ such that the warped moving image $I_m \circ \phi$ matches $I_f$ (Kebriti et al., 17 Aug 2025). The method is motivated by a specific limitation of prior learned DIR systems: CNN-based models are effective at capturing fine local deformations but often miss long-range context, whereas transformer-based methods model global correspondences more effectively but can lose local precision and do not necessarily represent different deformation scales within one unified design (Kebriti et al., 17 Aug 2025).

The model is formulated as an unsupervised registration system. During training, it does not use segmentation supervision; ground-truth segmentations are reserved for evaluation only (Kebriti et al., 17 Aug 2025). Its stated objective is to model local, semi-global, and global deformation cues simultaneously by combining spatial, spectral, and intermediate fractional-domain information inside the feature-matching mechanism itself, rather than by relying on scenario-specific tuning or hierarchical multi-scale registration pipelines (Kebriti et al., 17 Aug 2025).

A central design choice is that FractMorph processes the fixed and moving volumes in parallel, rather than concatenating them into a single stream at the outset. This makes inter-image correspondence an explicit architectural concern throughout the network. The resulting formulation is therefore neither a purely convolutional registration model nor a standard single-stream vision transformer applied to 3D medical images; it is a dual-stream correspondence-learning system whose defining operation is Fractional Cross-Attention (Kebriti et al., 17 Aug 2025).

2. Network architecture and feature flow

FractMorph begins by dividing each 3D image into non-overlapping cubic patches. For an image $I \in \mathbb{R}^{H \times W \times D}$ with patch size $75.15\%$0, the number of patches is

$75.15\%$1

The patch set is written as

$75.15\%$2

and each patch is projected to a token by

$75.15\%$3

This yields token sequences $75.15\%$4 and $75.15\%$5 for the fixed and moving images, respectively (Kebriti et al., 17 Aug 2025).

The core backbone is a hierarchical dual-parallel transformer. At level $75.15\%$6, the fixed and moving features are denoted

$75.15\%$7

The encoder applies several transformer blocks per level and downsamples by a factor of $75.15\%$8 in each spatial dimension while doubling the channel count:

$75.15\%$9

A symmetric decoder mirrors this structure, upsamples the coarse representations, fuses them with corresponding encoder skip connections, and refines both streams with additional transformer blocks (Kebriti et al., 17 Aug 2025).

Interaction between the streams occurs throughout the transformer, not only at the end. At each stage, fixed and moving features are first enriched through FrFT-based parallel branches and then exchanged through cross-attention in both directions. After decoding, the final fixed and moving features are concatenated, normalized, and passed through reverse patch embedding. A lightweight 3D U-Net style network—composed of three convolutional downsampling layers, a bottleneck, and three deconvolutional upsampling layers—then predicts the dense deformation field $95$0 from the transformer-enriched representation (Kebriti et al., 17 Aug 2025).

This architecture places transformers and convolutions in different roles. The transformer layers perform cross-image matching under multi-domain feature representations, whereas the CNN head converts those features into a dense, locally smooth displacement field at high spatial resolution. The paper presents this division as a way to preserve both correspondence modeling and displacement-field regularity within one network (Kebriti et al., 17 Aug 2025).

3. Fractional Cross-Attention and multi-domain FrFT branches

The defining module of FractMorph is the Fractional Cross-Attention block. FCA combines two operations: multi-domain feature extraction using FrFT branches, followed by cross-attention between the enriched fixed and moving streams (Kebriti et al., 17 Aug 2025).

The paper defines the 1D $95$1-th order FrFT of a signal $95$2 as

$95$3

with kernel, for $95$4,

$95$5

and normalization factor

$95$6

The transform interpolates between the spatial and Fourier domains. In the paper’s framing, $95$7 corresponds to the spatial/original domain, $95$8 to the standard Fourier transform, and $95$9 to an intermediate fractional domain (Kebriti et al., 17 Aug 2025).

FractMorph implements 3D FrFT by separability, applying 1D FrFT successively along the $1.54$0, $1.54$1, and $1.54$2 axes. The discrete 3D transform is written as

$1.54$3

with inverse

$1.54$4

Inside each FCA block, the normalized feature tensor is passed through four parallel branches: $1.54$5, $1.54$6, $1.54$7, and a $1.54$8 branch (Kebriti et al., 17 Aug 2025). The paper interprets these branches as capturing local spatial, semi-global mixed spatial-frequency, global spectral, and fine spectral-magnitude information, respectively. For the $1.54$9 branch, the log-magnitude transform is given by

$I_m$0

with inverse

$I_m$1

Each branch applies FrFT, convolution, ReLU, and inverse FrFT. The spatial branch uses kernel size $I_m$2, while the spectral branches use kernel size $I_m$3, consistent with the paper’s use of the Fourier convolution theorem,

$I_m$4

to motivate minimal spectral-domain kernels (Kebriti et al., 17 Aug 2025).

After branch processing, the outputs and the skip connection are concatenated along channels, normalized, and fused by pointwise convolution. The enriched moving and fixed features are then matched by cross-attention. For moving-to-fixed attention, the paper gives

$I_m$5

where $I_m$6, $I_m$7, and $I_m$8 with $I_m$9 attention heads (Kebriti et al., 17 Aug 2025). The reverse fixed-to-moving attention is computed in parallel. These outputs are reshaped back to volumetric tensors, combined with residual connections, layer normalization, and feed-forward MLP blocks.

The FCA design is therefore the point at which FractMorph departs most sharply from standard transformer registration models. Cross-image attention is not applied to purely spatial tokens; it is applied to features that have already been expanded across multiple fractional Fourier domains (Kebriti et al., 17 Aug 2025).

4. Training objective, warping model, and implementation

FractMorph predicts a dense deformation field $I_f$0 and uses a spatial transformer to warp the moving image. At voxel $I_f$1, the deformation is

$I_f$2

and the warped image is

$I_f$3

The paper describes trilinear interpolation by summing over the neighborhood $I_f$4 of the deformed position $I_f$5 (Kebriti et al., 17 Aug 2025).

Training is unsupervised. The total objective is the sum of an image-similarity term and a deformation smoothness term,

$I_f$6

The similarity term is based on local cross-correlation. With local means $I_f$7 and $I_f$8 over an $I_f$9 neighborhood, the cross-correlation is

$\phi$0

The similarity loss is

$\phi$1

and the smoothness penalty is

$\phi$2

The smoothness weight is fixed to $\phi$3 for all learning-based methods in the reported experiments (Kebriti et al., 17 Aug 2025).

The ACDC cardiac MRI volumes are normalized to $\phi$4 and resized to $\phi$5, corresponding to voxel size $\phi$6 mm (Kebriti et al., 17 Aug 2025). Patch size is set to

$\phi$7

and the token embedding dimension is

$\phi$8

All learning-based models are trained for $\phi$9 epochs with Adam and batch size $I_m \circ \phi$0 (Kebriti et al., 17 Aug 2025). The paper specifies NVIDIA GeForce RTX 3080 hardware with $I_m \circ \phi$1 GB VRAM, CUDA 12.2, and PyTorch 2.2.1 for the learned methods. It does not report data augmentation for FractMorph (Kebriti et al., 17 Aug 2025).

5. Empirical evaluation on ACDC and architectural ablations

The experimental evaluation uses the ACDC dataset, which contains $I_m \circ \phi$2 patients with cine cardiac MRI at end-diastole and end-systole, with manual segmentations for the left ventricular cavity, myocardium, and right ventricular cavity (Kebriti et al., 17 Aug 2025). The authors split the data by the dataset’s five groups, selecting from each group $I_m \circ \phi$3 cases for training, $I_m \circ \phi$4 for validation, and $I_m \circ \phi$5 for testing, for totals of $I_m \circ \phi$6, $I_m \circ \phi$7, and $I_m \circ \phi$8, respectively (Kebriti et al., 17 Aug 2025).

Registration is evaluated by warping moving labels and comparing them against fixed labels using overall DSC, average DSC across structures, HD95, the percentage of voxels with non-positive Jacobian determinant, and the standard deviation of the Jacobian determinant (Kebriti et al., 17 Aug 2025). The paper defines

$I_m \circ \phi$9

and reports HD95 as the $I_f$0th-percentile Hausdorff distance.

The main quantitative results place FractMorph at the top of the authors’ comparison on their data split (Kebriti et al., 17 Aug 2025).

Method	Registration metrics	Efficiency metrics
FractMorph	Overall DSC $I_f$1, Avg. DSC $I_f$2, HD95 $I_f$3 mm	$I_f$4 params, $I_f$5 s, $I_f$6 MB
FractMorph-Light	Overall DSC $I_f$7, Avg. DSC $I_f$8, HD95 $I_f$9 mm	$I \in \mathbb{R}^{H \times W \times D}$0 params, $I \in \mathbb{R}^{H \times W \times D}$1 s, $I \in \mathbb{R}^{H \times W \times D}$2 MB
VoxelMorph	Overall DSC $I \in \mathbb{R}^{H \times W \times D}$3, Avg. DSC $I \in \mathbb{R}^{H \times W \times D}$4, HD95 $I \in \mathbb{R}^{H \times W \times D}$5 mm	$I \in \mathbb{R}^{H \times W \times D}$6 params, $I \in \mathbb{R}^{H \times W \times D}$7 s, $I \in \mathbb{R}^{H \times W \times D}$8 MB
XMorpher	Overall DSC $I \in \mathbb{R}^{H \times W \times D}$9, Avg. DSC $75.15\%$00, HD95 $75.15\%$01 mm	$75.15\%$02 params, $75.15\%$03 s, $75.15\%$04 MB
TransMorph	Overall DSC $75.15\%$05, Avg. DSC $75.15\%$06, HD95 $75.15\%$07 mm	$75.15\%$08 params, $75.15\%$09 s, $75.15\%$10 MB

The topology-related metrics show that FractMorph does not achieve the lowest reported folding rate or Jacobian variability. Fourier-Net reports $75.15\%$11 for the fraction of non-positive Jacobian determinant and $75.15\%$12 for the Jacobian standard deviation, while FractMorph reports $75.15\%$13 and $75.15\%$14, respectively (Kebriti et al., 17 Aug 2025). The paper nevertheless argues that FractMorph improves alignment while keeping deformation plausibility at a competitive level.

The architectural ablations isolate the role of the FrFT branches and the FCA skip connection. The full model with all four branches reports overall DSC $75.15\%$15, average DSC $75.15\%$16, and HD95 $75.15\%$17 mm (Kebriti et al., 17 Aug 2025). Removing $75.15\%$18 lowers performance to overall DSC $75.15\%$19, average DSC $75.15\%$20, and HD95 $75.15\%$21 mm, while removing $75.15\%$22 and keeping the log branch yields overall DSC $75.15\%$23, average DSC $75.15\%$24, and HD95 $75.15\%$25 mm (Kebriti et al., 17 Aug 2025). The paper states that removing the $75.15\%$26 or $75.15\%$27 branch causes more than $75.15\%$28 drop in overall DSC and more than $75.15\%$29 mm increase in HD95.

The skip connection inside FCA also has a measurable effect. With the skip connection, the model reports overall DSC $75.15\%$30, average DSC $75.15\%$31, and HD95 $75.15\%$32 mm; without it, the values drop to $75.15\%$33, $75.15\%$34, and $75.15\%$35 mm (Kebriti et al., 17 Aug 2025). This indicates that the residual path is not merely conventional transformer scaffolding but materially supports the registration accuracy of the FCA block.

6. FractMorph-Light, interpretation, and limitations

FractMorph-Light is a reduced-cost variant of the model that modifies the channel allocation inside the FrFT feature-extractor branches (Kebriti et al., 17 Aug 2025). The paper defines a channel coefficient $75.15\%$36: the main model uses $75.15\%$37, whereas FractMorph-Light uses $75.15\%$38. For branch processing with input shape $75.15\%$39, the branch parameter count is

$75.15\%$40

and the branch convolution FLOPs are

$75.15\%$41

Thus, with $75.15\%$42, the branch parameter and convolution FLOP cost become $75.15\%$43 of those of the main model (Kebriti et al., 17 Aug 2025). The empirical result is a substantial memory reduction—from $75.15\%$44 MB to $75.15\%$45 MB—and a reduction in parameters from $75.15\%$46 to $75.15\%$47, while maintaining nearly identical registration accuracy (Kebriti et al., 17 Aug 2025).

The paper’s interpretation of the model is that FrFT creates a continuous family of representational domains between image space and Fourier space, so FractMorph is not limited to a simple local/global dichotomy. Instead, it explicitly incorporates local, intermediate, and global deformation cues in each FCA block (Kebriti et al., 17 Aug 2025). This suggests that its performance gain comes less from raw model size than from the structured way in which different deformation scales are exposed to cross-image attention.

The reported limitations are primarily computational. The paper identifies runtime overhead from FrFT operations as the main drawback and states that future work will focus on accelerating FrFT (Kebriti et al., 17 Aug 2025). In the reported benchmarks, FractMorph is markedly slower than lightweight learned baselines such as VoxelMorph and Fourier-Net, though still much faster than traditional SyN and Demons implementations on the reported hardware (Kebriti et al., 17 Aug 2025). The published evaluation is on ACDC cardiac MRI, and the paper does not present multi-modal or broader cross-dataset studies (Kebriti et al., 17 Aug 2025).

In the literature of learned deformable registration, FractMorph is therefore best understood as a transformer-based registration model whose novelty lies in relocating spectral-domain reasoning into the correspondence module itself. Rather than using frequency features as a peripheral augmentation, it makes fractional-domain processing a first-class component of cross-attention. Its encyclopedic significance derives from that architectural move: FractMorph is an attempt to formalize multi-scale, multi-domain registration as a single transformer problem, rather than as a sequence of separate local and global alignment stages (Kebriti et al., 17 Aug 2025).