Imaging through glass diffusers using densely connected convolutional networks (1711.06810v1)

Abstract: Computational imaging through scatter generally is accomplished by first characterizing the scattering medium so that its forward operator is obtained; and then imposing additional priors in the form of regularizers on the reconstruction functional so as to improve the condition of the originally ill-posed inverse problem. In the functional, the forward operator and regularizer must be entered explicitly or parametrically (e.g. scattering matrices and dictionaries, respectively.) However, the process of determining these representations is often incomplete, prone to errors, or infeasible. Recently, deep learning architectures have been proposed to instead learn both the forward operator and regularizer through examples. Here, we propose for the first time, to our knowledge, a convolutional neural network architecture called "IDiffNet" for the problem of imaging through diffuse media and demonstrate that IDiffNet has superior generalization capability through extensive tests with well-calibrated diffusers. We found that the Negative Pearson Correlation Coefficient loss function for training is more appropriate for spatially sparse objects and strong scattering conditions. Our results show that the convolutional architecture is robust to the choice of prior, as demonstrated by the use of multiple training and testing object databases, and capable of achieving higher space-bandwidth product reconstructions than previously reported.

• The paper presents IDiffNet, a novel densely connected CNN that simultaneously learns the scattering operator and regularizer using NPCC loss.
• It achieves superior space-bandwidth product reconstructions under varied scattering conditions, validated on datasets like Faces-LFW, ImageNet, and MNIST.
• The study emphasizes the trade-off between generalization and overfitting, paving the way for practical applications in computational imaging.

Imaging through Glass Diffusers Using Densely Connected Convolutional Networks

The paper "Imaging through glass diffusers using densely connected convolutional networks" presents a novel approach to computational imaging through scattering media, leveraging advances in convolutional neural networks (CNNs). This method addresses the inherent challenges posed by scattering phenomena in optics, which complicate the extraction of meaningful information from scattered light. Traditional methods typically require explicit or parametric representations of the scattering media's forward operators, combined with priors in a Tikhonov-Wiener optimization framework. However, acquiring accurate representations of scattering operators is often impractical. The authors propose an alternative approach, using a CNN architecture, termed IDiffNet, to learn the forward operator and regularizer simultaneously through training data. This approach not only simplifies the characterization process but also enhances the imaging capability through dense convolutional connectivity.

Prominent Contributions and Findings

1. CNN Architecture - IDiffNet: The paper introduces IDiffNet, a convolutional neural network specifically designed for imaging through diffusers. Distinguished by its densely connected architecture, IDiffNet facilitates extensive feature sharing across layers, enabling it to capture complex mappings required to counteract scattering effects. The architecture effectively generalizes across different object classes, as validated by comprehensive testing using databases like Faces-LFW, ImageNet, and MNIST.
2. Use of NPCC Loss Function: A significant innovation in the training process is the utilization of the Negative Pearson Correlation Coefficient (NPCC) loss function. This choice is justified by its superior performance in handling spatially sparse objects and under strong scattering conditions. The authors demonstrate that NPCC provides better guidance for the network to leverage inherent sparsity priors present in certain data types, leading to improved reconstruction quality compared to the commonly used Mean Absolute Error (MAE).
3. Optimal Reconstruction Capabilities: IDiffNet achieves higher space-bandwidth product (SBP) reconstructions, outperforming previous methodologies. The reconstruction quality is notably better for the 600-grit diffuser than for the more challenging 220-grit diffuser, showcasing its robustness under various scattering levels. The results include achieving recognizable reconstructions even for datasets not encountered during training, affirming its generalization strength.
4. Training Set Influence and Overfitting: The CNN's performance is contingent upon the characteristics of the training dataset. Generic datasets like ImageNet enhance generalization, whereas more specialized datasets like MNIST impose stronger feature-specific priors. Interestingly, during significant scattering, constrained datasets aid reconstruction, highlighting a trade-off between generalization and overfitting, particularly under strong sparsity conditions.

Implications and Future Directions

The implications of this research are profound for the computational imaging field. By automating the learning of scattering operators and priors, IDiffNet significantly simplifies the process and expands the range of scenarios where effective imaging is possible. Practical applications can be envisioned in medical imaging, surveillance, and any domain where imaging through turbid media is required.

Further research may explore the adaptation of IDiffNet to different scattering conditions or media. There is also potential in examining more complex and diverse datasets to further enhance generalization capabilities. Future innovations could integrate IDiffNet with other neural network paradigms or hybrid models to improve reconstruction fidelity and broaden applicability. Given the momentum in AI advancements, this paper paves the way for leveraging deep learning architectures beyond their current application scope, suggesting transformative impacts in computational optics and imaging.