Deep Convolutional Neural Network for Inverse Problems in Imaging (1611.03679v1)

Abstract: In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyper parameter selection. The starting point of our work is the observation that unrolled iterative methods have the form of a CNN (filtering followed by point-wise non-linearity) when the normal operator (H*H, the adjoint of H times H) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill-posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a 512 x 512 image on GPU.

• The paper presents FBPConvNet, combining classical filtered back projection with a deep CNN to efficiently solve inverse imaging problems.
• It leverages a U-net inspired architecture with multi-level decomposition, multichannel filtering, and residual learning to enhance reconstruction quality.
• Experimental results show that FBPConvNet outperforms traditional TV methods in biomedical and experimental datasets, offering faster inference times.

Deep Convolutional Neural Network for Inverse Problems in Imaging

The paper "Deep Convolutional Neural Network for Inverse Problems in Imaging" authored by Kyong Hwan Jin, Michael T. McCann, Emmanuel Froustey, and Michael Unser introduces a novel deep learning-based approach for solving inverse imaging problems. Traditional methods for these problems, such as iterative algorithms incorporating regularization, although effective, often face challenges such as high computational costs and intricate hyperparameter tuning. The authors, leveraging convolutional neural networks (CNNs), propose a method that offers robustness and computational efficiency.

Overview of Proposed Method: FBPConvNet

The authors propose FBPConvNet, a hybrid approach combining classical filtered back projection (FBP) with advanced CNN techniques. The paper's core idea is to first apply an FBP to reduce the complexity of the inverse problem by incorporating domain-specific physical model information and then use the CNN to refine this preliminary reconstruction. The CNN is designed to remove artifacts induced by the ill-posed nature of the problem, thereby improving image quality.

Key Components and Methods

Filtered Back Projection (FBP):

The FBP is used as an initial step to leverage knowledge about the underlying physical system. This approach reduces the problem's complexity and ensures the CNN works with data closer to the desired solution. The FBP applies a fast, approximate inversion to the input resulting in a preliminary reconstruction.

Convolutional Neural Network Design:

The CNN architecture follows a design inspired by U-net with modifications to suit the inverse problems in imaging:

• Multilevel Decomposition: This is achieved through max-pooling, allowing the network to capture features at multiple scales, addressing the issue of non-compact support in filters such as in CT imaging.
• Multichannel Filtering: Multiple feature maps at each layer enhance the network's expressive power, analogous to using multi-channel wavelet transforms in traditional methods.
• Residual Learning: A skip connection is incorporated, allowing the network to learn the residual between the FBP result and the ground truth. This bypass connection helps in mitigating the vanishing gradient problem.

Experimental Results and Comparisons

The authors evaluate FBPConvNet on three distinct datasets:

1. Ellipsoidal Dataset: Synthetic images of ellipsoids with varying attributes.
2. Biomedical Dataset: Real in-vivo CT images from a grand challenge dataset.
3. Experimental Dataset: Real sinograms collected from a rat brain experiment.

For performance metrics, they employ signal-to-noise ratio (SNR). Comparisons are drawn between FBPConvNet, traditional FBP, and a state-of-the-art Total Variation-regularized iterative reconstruction method (TV method).

Results:

• Ellipsoidal Dataset: FBPConvNet performs comparably to the TV method in the seven-fold subsampling case but lags in the twenty-fold subsampling case. Given the piecewise constant nature of the ellipsoidal images, the TV regularization aligns well with this dataset.
• Biomedical and Experimental Datasets: FBPConvNet outperforms the TV method in these more realistic scenarios. The improvements are significant in retaining fine structural details and exhibiting fewer artifacts compared to TV-based methods, which often produce cartoon-like artifacts.

Implications and Future Directions

The paper demonstrates that FBPConvNet not only offers a viable solution to inverse imaging problems, especially in the biomedical domain, but also provides a computationally efficient alternative to traditional iterative methods, with inference times under a second for $512 \times 512$ images. This efficiency is a compelling advantage for real-time applications such as medical diagnostics where quick turnarounds are essential.

However, one limitation noted is the need for retraining the network when transferring across different datasets or imaging modalities. Future work could focus on improving the generalization capabilities of the network, potentially through techniques like domain adaptation or meta-learning.

Moreover, while this paper focuses on CT reconstruction, the underlying principles extend to other imaging modalities like MRI and diffraction tomography. Future research could further explore and adapt the methodology for such applications, potentially addressing challenges such as handling complex-valued data and large-scale data processing.

In summary, the FBPConvNet presents a promising advancement for inverse problems in imaging, marrying classical techniques with contemporary deep learning methodologies to achieve robust and efficient reconstruction performance.