NETT: Solving Inverse Problems with Deep Neural Networks (1803.00092v3)

Abstract: Recovering a function or high-dimensional parameter vector from indirect measurements is a central task in various scientific areas. Several methods for solving such inverse problems are well developed and well understood. Recently, novel algorithms using deep learning and neural networks for inverse problems appeared. While still in their infancy, these techniques show astonishing performance for applications like low-dose CT or various sparse data problems. However, there are few theoretical results for deep learning in inverse problems. In this paper, we establish a complete convergence analysis for the proposed NETT (Network Tikhonov) approach to inverse problems. NETT considers data consistent solutions having small value of a regularizer defined by a trained neural network. We derive well-posedness results and quantitative error estimates, and propose a possible strategy for training the regularizer. Our theoretical results and framework are different from any previous work using neural networks for solving inverse problems. A possible data driven regularizer is proposed. Numerical results are presented for a tomographic sparse data problem, which demonstrate good performance of NETT even for unknowns of different type from the training data. To derive the convergence and convergence rates results we introduce a new framework based on the absolute Bregman distance generalizing the standard Bregman distance from the convex to the non-convex case.

• The paper introduces the NETT framework, integrating a learned neural regularizer within classical Tikhonov regularization to solve inverse problems.
• It rigorously establishes well-posedness and convergence through theoretical analysis using concepts like absolute Bregman distances in non-convex settings.
• Numerical experiments demonstrate NETT’s ability to reduce undersampling artifacts and maintain data consistency in applications like CT and photoacoustic tomography.

Inverse Problems and the Network Tikhonov Approach

The paper "NETT: Solving Inverse Problems with Deep Neural Networks" by Li et al., investigates using deep neural networks for addressing inverse problems, a task central to numerous scientific domains such as biomedical imaging, geophysics, and engineering sciences. It primarily introduces the Network Tikhonov (NETT) approach, which aims to solve inverse problems by leveraging a neural network-based regularization method. This work is timely, considering recent advances in machine learning and the prospect of utilizing deep learning frameworks for enhanced performance across varying applications such as low-dose computed tomography (CT) and in scenarios with sparse data.

NETT Framework

The NETT framework is presented as an adaptation of classical Tikhonov regularization within a neural network architecture. Unlike traditional methods, NETT uses a regularizer that is defined by a trained neural network. Specifically, it considers data-consistent solutions that minimize a regularization functional expressed in terms of deep networks. The problem is approached via a minimization of a functional typically consisting of a data consistency term and a learned regularization term.

The authors provide an exhaustive convergence analysis for the NETT approach. Significant theoretical contributions of this paper include demonstrating the well-posedness of NETT, analyzing convergence in both weak and strong topologies, and deriving quantitative error estimates. The paper also proposes strategies for deriving and training the regularizer using networks capable of representing complex relationships.

Theoretical Contributions and Implications

The authors rigorously address the theoretical underpinnings of the NETT approach by utilizing concepts like total non-linearity and introducing absolute Bregman distances for non-convex functional settings. These theoretical innovations are crucial in extending traditional concepts like Bregman distances to work with the non-convex scenarios arising from neural network-based regularizers. This approach ensures that NETT can provide stable and consistent solution schemes for ill-posed inverse problems.

Establishing well-posedness guarantees that, for any sequence of data converging to some measurement, the NETT solutions converge to a true solution in a certain sense. Furthermore, by demonstrating strong convergence, the authors ensure that solutions generated by NETT will be stable even in the normed spaces, a desirable property for inverse problem solvers.

Numerical Experiments and Practical Implications

The paper also includes numerical experiments to validate the proposed framework, specifically in the context of reconstruction from sparse data in photoacoustic tomography (PAT). Undersampling is a recurring challenge in practical settings across many imaging techniques. Utilizing a trained neural network regularizer demonstrates potential in effectively removing undersampling artifacts while preserving high-resolution information.

The results underscore the efficacy of NETT in maintaining data consistency even with limited training data and indicate the framework's versatility for tasks beyond the immediate class of training samples. This aspect is particularly valuable in real-world applications where access to expansive training data is limited.

Future Directions

In conclusion, this paper provides a comprehensive initial exploration into the integration of deep learning methodologies with traditional inverse problem frameworks. It opens several avenues for future research, including the exploration of alternative neural architectures, more extensive datasets for different applications, and development of enhanced optimization algorithms for functional minimization. The theoretical foundation laid by this paper sets a precedent for the broader application of neural network-based regularizations in solving inverse problems effectively and reliably.