X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks (1905.06902v1)

Abstract: Computed tomography (CT) can provide a 3D view of the patient's internal organs, facilitating disease diagnosis, but it incurs more radiation dose to a patient and a CT scanner is much more cost prohibitive than an X-ray machine too. Traditional CT reconstruction methods require hundreds of X-ray projections through a full rotational scan of the body, which cannot be performed on a typical X-ray machine. In this work, we propose to reconstruct CT from two orthogonal X-rays using the generative adversarial network (GAN) framework. A specially designed generator network is exploited to increase data dimension from 2D (X-rays) to 3D (CT), which is not addressed in previous research of GAN. A novel feature fusion method is proposed to combine information from two X-rays.The mean squared error (MSE) loss and adversarial loss are combined to train the generator, resulting in a high-quality CT volume both visually and quantitatively. Extensive experiments on a publicly available chest CT dataset demonstrate the effectiveness of the proposed method. It could be a nice enhancement of a low-cost X-ray machine to provide physicians a CT-like 3D volume in several niche applications.

• The paper introduces X2CT-GAN, a GAN framework that reconstructs 3D CT images from two orthogonal X-rays.
• It employs dual encoder-decoder networks with a novel projection loss to optimize the 2D-to-3D mapping and maintain anatomical consistency.
• Quantitative metrics like PSNR and SSIM demonstrate significant improvements over existing models, highlighting its clinical potential.

Overview of "X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks"

The paper "X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks" presents a novel approach to computational imaging, particularly focusing on generating 3D computed tomography (CT) images from biplanar 2D X-ray inputs using a generative adversarial network (GAN) framework. Developed by Xingde Ying et al., this method effectively proposes to bridge the dimensional gap between 2D X-ray data and 3D CT outputs, offering a potential solution for environments where CT scanning is less accessible or prohibitively expensive.

Introduction and Motivation

The reconstruction of 3D CT images from a limited number of 2D X-ray projections is a challenging problem primarily due to the loss of volumetric information inherent in 2D projections. Traditional CT reconstruction methods require extensive imaging data acquired during a comprehensive rotational scan, which is not feasible on standard X-ray machines. This paper leverages advancements in deep learning—specifically GANs—to perform volumetric reconstruction with minimal radiation exposure and at a significantly reduced cost.

Methodology

The authors introduce X2CT-GAN, a system designed to reconstruct CT images from two orthogonal X-ray views. The method uniquely employs a deep neural network architecture within the GAN framework to facilitate the high-dimensional mapping from 2D input to 3D output. Critical to the system's efficacy is the following:

• Network Architecture: The generator in the X2CT-GAN consists of two parallel encoder-decoder networks that process the biplanar X-rays independently, and a feature fusion network that synthesizes these inputs into a coherent 3D volume. Novel connection schemes are used to bridge 2D and 3D feature maps.
• Loss Functions: The training process utilizes a combination of mean squared error (MSE), adversarial, and a novel projection loss. The projection loss enforces consistency between generated 3D shapes and projections derived from orthogonal planes, enhancing the anatomical and structural fidelity of the reconstructed volumes.
• Training with Synthesized Data: Owing to the scarcity of real paired X-ray and CT datasets, the system is trained using DRR-generated X-rays from public CT datasets. The authors also employ CycleGAN to adapt real-world X-ray images into the synthetic domain for real-data applicability.

Results and Implications

Quantitative evaluations using metrics such as the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) demonstrate that the X2CT-GAN significantly outperforms existing models in generating visually and structurally accurate CT volumes from limited X-ray data. Noteworthy is the improvement in reconstruction quality with biplanar inputs, which implies enhanced clinical differentiation capability and suggests substantial practical implications in settings with limited access to full CT infrastructure.

The paper highlights potential niche applications, such as precise organ size measurements and planning in radiotherapy, emphasizing the clinical value of low-cost X-ray systems enhanced by their proposed method. Future research directions might focus on optimizing the model for higher resolution outputs and exploring additional medical imaging scenarios.

Conclusion

X2CT-GAN represents a significant advancement in enabling CT-like reconstructions from minimal X-ray data using GAN methodologies. While not intended to replace traditional CT for every use case, the approach is poised to enhance diagnostic capabilities in resource-constrained environments significantly. Further development and clinical validation will elucidate its utility in broader medical imaging applications.