Deep CT to MR Synthesis using Paired and Unpaired Data (1805.10790v2)

Abstract: MR imaging will play a very important role in radiotherapy treatment planning for segmentation of tumor volumes and organs. However, the use of MR-based radiotherapy is limited because of the high cost and the increased use of metal implants such as cardiac pacemakers and artificial joints in aging society. To improve the accuracy of CT-based radiotherapy planning, we propose a synthetic approach that translates a CT image into an MR image using paired and unpaired training data. In contrast to the current synthetic methods for medical images, which depend on sparse pairwise-aligned data or plentiful unpaired data, the proposed approach alleviates the rigid registration challenge of paired training and overcomes the context-misalignment problem of the unpaired training. A generative adversarial network was trained to transform 2D brain CT image slices into 2D brain MR image slices, combining adversarial loss, dual cycle-consistent loss, and voxel-wise loss. The experiments were analyzed using CT and MR images of 202 patients. Qualitative and quantitative comparisons against independent paired training and unpaired training methods demonstrate the superiority of our approach.

Deep CT to MR Synthesis using Paired and Unpaired Data

The paper "Deep CT to MR Synthesis using Paired and Unpaired Data" presents a novel approach to enhance the synthesis of magnetic resonance (MR) images from computed tomography (CT) images using both paired and unpaired data within a generative adversarial network (GAN) framework. The authors address a significant challenge in the context of radiotherapy treatment planning, where MR imaging's superior soft tissue contrast is pivotal but often faces barriers due to cost and contraindications related to metal implants. Their solution involves translating CT images into MR images, potentially mitigating such barriers while leveraging the widespread availability of CT data.

Methodology

The authors propose a unique MR synthetic GAN (MR-GAN) that integrates adversarial loss, dual cycle-consistent loss, and voxel-wise loss. This architecture supports both paired and unpaired data training, a dual approach intended to reconcile the limitations intrinsic to each data type when used in isolation. The dual cycle-consistent component provides four distinct cycles—forward and backward paths for both paired and unpaired data—enabling the network to learn efficient mapping between CT and MR domains and vice versa.

Paired Data: This mode benefits from direct voxel-wise loss that ensures synthesized images closely approximate their real counterparts, countering blurriness but limited by the scarcity of well-aligned datasets.

Unpaired Data: Here, a CycleGAN model is employed, enforcing cycle-consistency to maintain meaningful contextual translation without leveraging voxel-wise loss, thus capable of utilizing abundant unpaired datasets.

Results

Quantitative comparisons using metrics such as MAE (Mean Absolute Error) and PSNR (Peak-Signal-to-Noise Ratio) demonstrate the proposed method's superiority, with synthesized images achieving an MAE of $19.36 \pm 2.73$ and a PSNR of $65.35 \pm 0.86$. These figures represent clear improvements over independent paired and unpaired training methodologies. Qualitatively, synthesized MR images exhibit high anatomical fidelity, capturing complex structures like gyri and soft brain tissues more effectively than baseline models.

Implications

The implications of this research are profound in the medical imaging and radiotherapy sectors. The ability to generate high-fidelity MR images from CT data expands radiotherapy planning capabilities, especially enhancing accessibility for patients contraindicated for MRI scans. Moreover, the dual data training approach optimizes the use of available medical imaging data, paving the way for broader applications such as MR-CT and CT-PET synthesis.

Future Directions

Future research could enhance the model by incorporating 3D context and temporal information from sequential brain imaging, potentially refining the synthesis of complex anatomical features further. Additionally, perceptual studies involving radiology experts could provide deeper insights into the clinical validity and usability of synthesized images, moving beyond mere quantitative assessments.

In summary, this paper offers a sophisticated approach to MR image synthesis that effectively utilizes paired and unpaired data, pushing the boundaries of synthetic medical imaging in contexts where traditional methods face significant obstacles. As the artificial intelligence field evolves, methodologies like MR-GAN will likely see increased adoption across diverse applications where high-quality synthetic imagery is a necessity.