BiRegNet: Cross-Modality Image Registration

• BiRegNet is a deep learning framework that simultaneously predicts forward and backward dense deformation fields to enable efficient cross-modality registration.
• The network architecture features four encoder-decoder streams with attention gates and multi-scale aggregation to capture and refine spatial details.
• Integrated with similarity-based label fusion, BiRegNet enhances segmentation accuracy in clinical tasks while reducing computational demands.

BiRegNet is a bi-directional registration network designed for efficient cross-modality medical image alignment within multi-atlas segmentation (MAS) frameworks. Developed in the context of deep learning–based label propagation, BiRegNet enables accurate and computationally efficient dense deformation field (DDF) estimation between pairs of volumetric images drawn from different imaging modalities, such as MRI and CT. This architecture addresses key challenges in cross-modality MAS, particularly the limited availability of same-modality atlases and the computational burden of conventional registration procedures (Ding et al., 2022).

1. Role in Cross-Modality Multi-Atlas Segmentation

BiRegNet functions as the registration module in a cross-modality MAS pipeline, where the aim is to transfer anatomical labels from a set of annotated atlas images to a target image potentially acquired with a different imaging modality. Unlike traditional MAS, which registers each atlas to the target using computationally expensive non-learning-based registration, BiRegNet leverages deep neural networks for efficient and simultaneous prediction of forward and backward deformation fields (atlas→target and target→atlas).

At inference, given a target image $I_t$ and an atlas image $I_a$ (possibly from distinct modalities), BiRegNet produces:

• A forward DDF $\phi_{a \rightarrow t}$ to map the atlas into the target’s space (for label propagation).
• A backward DDF $\phi_{t \rightarrow a}$ to map the target into the atlas space (enforcing bi-directional consistency).

The set of warped atlas labels can subsequently be fused—via traditional methods (such as majority voting) or using a dedicated learned label fusion network—to produce the final segmentation on the target (Ding et al., 2022).

2. Network Architecture

BiRegNet receives as input a pair of 3D volumes $(I_a, I_t)$. The architecture is characterized by:

• Four Encoder Streams: Each encoder comprises blocks of convolution, normalization, and nonlinearity, recursively downsampling the input and extracting multi-scale features at four resolution levels.
• Four Decoder Streams: Each decoder operates at a matching scale and includes:
  • Attention gates that reweight skip-connection features,
  • ResNet-style residual blocks,
  • Upsampling operations to progress to finer resolutions.
• Skip Connections: Gated feature connections bridge corresponding encoder and decoder stages, facilitating information flow and spatial detail preservation.
• Multi-Scale Aggregation: At the decoder output, features from all resolution levels are upsampled and concatenated to form a comprehensive multi-scale representation.
• Output Heads: A dual-output convolutional head simultaneously predicts the 3D DDFs for both registration directions: $\phi_{a \rightarrow t}$ and $\phi_{t \rightarrow a}$.

This design supports the learning of deformation fields suitable for high-fidelity cross-modality alignment and facilitates downstream label fusion (Ding et al., 2022).

3. Integration with Label Fusion

Within the proposed MAS framework, BiRegNet operates in tandem with a similarity estimation network (SimNet). The typical workflow is as follows:

1. Registration: For each atlas–target pair, BiRegNet computes $\phi_{a \rightarrow t}$ to spatially align atlas labels to the target domain.
2. Similarity-Based Label Fusion: SimNet assesses the similarity between the warped atlas and target images to determine fusion weights reflecting anatomical correspondence.
3. Segmentation Output: Warped (and weighted) atlas labels are fused (using, for example, learned fusion rules) to yield the final segmentation for the target image.

This approach improves robustness to modality gap and anatomical variation while greatly reducing runtime compared to classical MAS strategies (Ding et al., 2022).

4. Theoretical Rationale for Bi-Directionality

The bi-directional design of BiRegNet is motivated by the need for geometric inverse-consistency, which is particularly crucial in cross-modality registration. Predicting both forward and backward deformation fields allows the architecture to impose an implicit consistency constraint analogous to the inverse-consistency prior well-known in medical image registration. This reduces the likelihood of non-physical registration artifacts such as foldings and ensures robustness under drastic intensity differences between modalities. A plausible implication is improved anatomical plausibility of spatial alignment, which is especially critical when the image intensity relationships between trajectories are not simple or consistent (Ding et al., 2022).

5. Pipeline Summary

The workflow in which BiRegNet is embedded can be summarized as:

• Input: Pair of 3D volumes ($I_a$, $I_t$), potentially from different modalities.
• Encoding: Multi-level feature extraction via four parallel encoders.
• Decoding: Progressive refinement, attention-gated skip connections, and multi-scale aggregation.
• Output: Joint prediction of bi-directional dense deformation fields ($\phi_{a \rightarrow t}$, $\phi_{t \rightarrow a}$).
• Post-processing: Warped atlas labels are fused using SimNet-based similarity estimation to generate accurate segmentations in the target domain.

6. Application Domains and Evaluation

BiRegNet has been evaluated as part of a cross-modality MAS framework on clinically significant segmentation tasks, including left ventricle segmentation (MM-WHS dataset) and liver segmentation (CHAOS dataset). Empirical results cited in the principal paper demonstrate the effectiveness of the approach in both image registration and label fusion for heterogeneous modality settings (Ding et al., 2022).

Dataset	Application	Tasks Evaluated
MM-WHS	Cardiac segmentation	Left ventricle
CHAOS	Abdominal organ segmentation	Liver

Table: Example datasets and tasks evaluated with BiRegNet within the cross-modality MAS framework (Ding et al., 2022).

7. Limitations and Unavailable Technical Details

The provided data does not include:

• Explicit mathematical formulation of the registration objective function.
• Definition of loss terms (such as similarity, regularization, or inverse-consistency losses).
• Training strategies, optimization parameters, or regimen specifics.
• Ablation studies or detailed quantitative results comparing BiRegNet variants.
• Pseudocode or algorithmic description of the end-to-end MAS pipeline.

For these aspects—including precise equations, loss functions, hyperparameter settings, and quantitative comparisons—the main text of "Cross-Modality Multi-Atlas Segmentation via Deep Registration and Label Fusion" should be consulted directly (Ding et al., 2022).