GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis (2404.03126v1)

Abstract: We present GaSpCT, a novel view synthesis and 3D scene representation method used to generate novel projection views for Computer Tomography (CT) scans. We adapt the Gaussian Splatting framework to enable novel view synthesis in CT based on limited sets of 2D image projections and without the need for Structure from Motion (SfM) methodologies. Therefore, we reduce the total scanning duration and the amount of radiation dose the patient receives during the scan. We adapted the loss function to our use-case by encouraging a stronger background and foreground distinction using two sparsity promoting regularizers: a beta loss and a total variation (TV) loss. Finally, we initialize the Gaussian locations across the 3D space using a uniform prior distribution of where the brain's positioning would be expected to be within the field of view. We evaluate the performance of our model using brain CT scans from the Parkinson's Progression Markers Initiative (PPMI) dataset and demonstrate that the rendered novel views closely match the original projection views of the simulated scan, and have better performance than other implicit 3D scene representations methodologies. Furthermore, we empirically observe reduced training time compared to neural network based image synthesis for sparse-view CT image reconstruction. Finally, the memory requirements of the Gaussian Splatting representations are reduced by 17% compared to the equivalent voxel grid image representations.

• The paper introduces a novel CT imaging method that adapts Gaussian Splatting to synthesize high-quality brain scans from limited projections.
• It employs an enhanced loss function with Total Variation and Beta distribution losses to achieve superior foreground-background distinction.
• The method reduces training time and memory usage by extracting CT camera parameters from DICOM metadata, bypassing SfM challenges.

Advancing CT Imaging with GaSpCT: A Novel Approach for CT Projection View Synthesis

Overview of GaSpCT

Recent advancements in 3D scene representation and novel view synthesis have led to the development of GaSpCT, a cutting-edge methodology tailored for Computer Tomography (CT) imaging, particularly brain scans. Building upon the foundation of Gaussian Splatting, GaSpCT introduces a series of modifications and enhancements geared towards optimizing CT imaging processes. These optimizations not only promise to refine the quality of synthesized CT images but also aim to reduce the radiation dose patients are exposed to, by minimizing the number of required projections.

Key Contributions of GaSpCT

The paper outlines several key contributions that collectively enhance the utility and efficiency of CT imaging through the application of GaSpCT:

• Novel Use of Gaussian Splatting for CT: Unlike previous methods, GaSpCT leverages Gaussian Splatting specifically for CT imaging, focusing on brain scans. This adaptation proves essential in capturing the nuances of CT images, which are often rich in anisotropic radiodensities.
• Enhanced Loss Function: By incorporating a Total Variation (TV) loss and a Beta distribution negative log likelihood loss into the loss function, GaSpCT significantly improves background and foreground distinction. This refinement is crucial for medical imaging, where clarity and detail are paramount.
• Initialization and SfM Adaptation: GaSpCT introduces a practical approach to initializing Gaussian locations across the 3D space, considering the expected positioning of the brain within the scan. Additionally, it innovates by extracting CT camera parameters directly from DICOM metadata, bypassing the challenges associated with Structure from Motion (SfM) methodologies in CT images.
• Empirical Results: The empirical validation of GaSpCT demonstrates its superiority over existing methods. Specifically, it showcases reduced training times, a smaller memory footprint, and improved performance in rendering novel views of brain CT scans.

Methodological Insights

The methodology section explores the technical aspects of adapting Gaussian Splatting for CT imaging. It highlights two key augmentations: the introduction of a TV regularizer to enhance image smoothness and the adaptation of a Beta distribution regularizer to promote sparsity. These modifications contribute to the final loss function, which is optimized to generate high-fidelity CT images from a limited number of projections.

Furthermore, the paper outlines a novel approach to overcome the limitations of SfM in CT imaging. By extracting camera parameters from DICOM metadata and utilizing prior knowledge of scanning parameters, GaSpCT effectively simulates accurate 3D scenes without relying on traditional SfM outputs.

Experimental Validation and Results

The experimental section offers a comprehensive evaluation of GaSpCT. Utilizing brain CT scans from the Parkinson's Progression Markers Initiative (PPMI) dataset, the paper compares GaSpCT against traditional Gaussian Splatting and other state-of-the-art methodologies. The results underscore GaSpCT's effectiveness in rendering realistic and accurate CT images from sparse views. Notably, GaSpCT achieves remarkable performance in terms of Peak Signal-to-Noise Ratio (PSNR), Structure Similarity Index Measure (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) metrics.

Implications and Future Directions

The successful implementation and validation of GaSpCT mark a significant step forward in the domain of CT imaging. By enhancing image quality and reducing scanning requirements, GaSpCT has the potential to revolutionize patient care, enabling faster, safer, and more efficient diagnostic processes.

Looking forward, the paper proposes several avenues for further research, including the development of a new camera model better suited to CT imaging and the adaptation of SfM techniques for improved initial point cloud definition. These future directions promise to unlock even greater potential in the field of medical imaging, leveraging the power of GaSpCT and Gaussian Splatting methodologies.

Acknowledgments

The paper acknowledges the contributions of various datasets and software that facilitated the research, highlighting the collaborative effort involved in advancing medical imaging technology.