Revisiting registration-based synthesis: A focus on unsupervised MR image synthesis

Abstract: Deep learning (DL) has led to significant improvements in medical image synthesis, enabling advanced image-to-image translation to generate synthetic images. However, DL methods face challenges such as domain shift and high demands for training data, limiting their generalizability and applicability. Historically, image synthesis was also carried out using deformable image registration (DIR), a method that warps moving images of a desired modality to match the anatomy of a fixed image. However, concerns about its speed and accuracy led to its decline in popularity. With the recent advances of DL-based DIR, we now revisit and reinvigorate this line of research. In this paper, we propose a fast and accurate synthesis method based on DIR. We use the task of synthesizing a rare magnetic resonance (MR) sequence, white matter nulled (WMn) T1-weighted (T1-w) images, to demonstrate the potential of our approach. During training, our method learns a DIR model based on the widely available MPRAGE sequence, which is a cerebrospinal fluid nulled (CSFn) T1-w inversion recovery gradient echo pulse sequence. During testing, the trained DIR model is first applied to estimate the deformation between moving and fixed CSFn images. Subsequently, this estimated deformation is applied to align the paired WMn counterpart of the moving CSFn image, yielding a synthetic WMn image for the fixed CSFn image. Our experiments demonstrate promising results for unsupervised image synthesis using DIR. These findings highlight the potential of our technique in contexts where supervised synthesis methods are constrained by limited training data.

• The paper introduces a DIR-based synthesis method that leverages deformation warping to generate white matter nulled images from CSFn MR scans.
• It achieves superior performance metrics, including higher SSIM and PSNR, compared to traditional DIR and I2I-based approaches.
• The findings underscore the method's generalizability and potential for cost-efficient cross-modal imaging in scenarios with limited data.

Unsupervised MR Image Synthesis via Deformable Image Registration

Introduction

In the domain of medical imaging, particularly magnetic resonance imaging (MRI), the synthesis of image modalities that are not readily available due to rarity or operational constraints has been a subject of extensive study. Traditional approaches have leaned heavily on deep learning (DL) methodologies for image-to-image (I2I) translation, delivering promising results but facing critical challenges in terms of data demand and generalizability. This paper by Hays et al. revisits an alternative strategy: deformable image registration (DIR), a technique that has seen reduced application due to perceived drawbacks in speed and accuracy but is here re-evaluated for its potential in unsupervised MR image synthesis, specifically targeting the synthesis of white matter nulled (WMn) T1-weighted (T1-w) images from cerebrospinal fluid nulled (CSFn) T1-w images, using the magnetization-prepared rapid acquisition with gradient echo (MPRAGE) sequence.

Synthesis Approaches

The research encompasses both supervised and unsupervised approaches in the context of image synthesis:

• Supervised I2I-based Methods: These rely on having a dataset of paired images across the source and target modalities. While such models can learn direct mappings to generate synthetic T1-w MR images, their efficacy is constrained by the availability of sufficiently large paired datasets.
• Unsupervised I2I-based Methods: Without the need for paired data, these methods aim to maintain the geometric integrity of anatomical structures during synthesis, yet they struggle with fidelity to a significant degree.
• Deformable Image Registration (DIR)-based Techniques: Here, Hays et al. propose leveraging DIR for image synthesis by warping images from a source modality to match the anatomy of a fixed image in the target modality, suggesting an incredibly promising direction for unsupervised synthesis without the need for matched anatomical data.

Methodology and Results

The methodology introduced by Hays et al. entails a novel application of DIR for synthesizing WMn images from CSFn images, leveraging a deep learning-based DIR algorithm, im2grid. The process first involves estimating the deformation between two CSFn images (moving and fixed) and then applying this deformation to warp the WMn counterpart of the moving CSFn image to produce a synthetic WMn image for the fixed CSFn subject. This approach demonstrates superior results compared to traditional DIR methods and showcases enhanced generalizability in comparison to other I2I-based synthesis methods.

The technical achievements spotlighted in the paper include:

• Unsupervised MR Image Synthesis: Showcasing an effective method for the synthesis of WMn images in an unsupervised fashion.
• Superior Performance Metrics: The proposed DIR-based method outperforms existing DIR techniques, achieving higher structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) scores.
• Advantages in Data Limitation Scenarios: By leveraging widely available CSFn images for training, the method effectively addresses the challenges posed by the scarce availability of certain MR sequences, like WMn.

Implications and Future Directions

The implications of this research are multifaceted. Practically, it presents a viable method for generating rare MR image sequences, thereby potentially expanding the scope of available diagnostic imagery without the need for additional, costly scans. Theoretically, it reaffirms the value of DIR in modern imaging synthesis tasks, challenging the prevailing dominance of direct DL methods.

Looking ahead, the technique's adaptability suggests promising avenues for cross-modal synthesis beyond MR imaging, addressing the perennial challenge of data scarcity in medical imaging. Further, the comparative effectiveness of unsupervised versus supervised DIR-based methods invites more nuanced investigations into loss functions and model training strategies specific to imaging contexts. Overall, Hays et al.'s work repositions DIR-based synthesis as a key contender in the ongoing evolution of medical image generation, meriting continued exploration and refinement for wider clinical application.