Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learned reconstruction methods for inverse problems: sample error estimates (2312.14078v1)

Published 21 Dec 2023 in stat.ML, cs.LG, cs.NA, and math.NA

Abstract: Learning-based and data-driven techniques have recently become a subject of primary interest in the field of reconstruction and regularization of inverse problems. Besides the development of novel methods, yielding excellent results in several applications, their theoretical investigation has attracted growing interest, e.g., on the topics of reliability, stability, and interpretability. In this work, a general framework is described, allowing us to interpret many of these techniques in the context of statistical learning. This is not intended to provide a complete survey of existing methods, but rather to put them in a working perspective, which naturally allows their theoretical treatment. The main goal of this dissertation is thereby to address the generalization properties of learned reconstruction methods, and specifically to perform their sample error analysis. This task, well-developed in statistical learning, consists in estimating the dependence of the learned operators with respect to the data employed for their training. A rather general strategy is proposed, whose assumptions are met for a large class of inverse problems and learned methods, as depicted via a selection of examples.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (72)
  1. J. Adler and O. Öktem. Solving ill-posed inverse problems using iterative deep neural networks. Inverse Problems, 33(12):124007, 2017.
  2. J. Adler and O. Öktem. Learned primal-dual reconstruction. IEEE transactions on medical imaging, 37(6):1322–1332, 2018.
  3. On one-sided lipschitz stability of set-valued contractions. Numerical Functional Analysis and Optimization, 35(7-9):837–850, 2014.
  4. Learning the optimal tikhonov regularizer for inverse problems. Advances in Neural Information Processing Systems, 34, 2021.
  5. Continuous generative neural networks. arXiv preprint arXiv:2205.14627, 2022.
  6. Input convex neural networks. In International Conference on Machine Learning, pages 146–155. PMLR, 2017.
  7. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proceedings of the National Academy of Sciences, 117(48):30088–30095, 2020.
  8. Solving inverse problems using data-driven models. Acta Numerica, 28:1–174, 2019.
  9. Data driven regularization by projection. Inverse Problems, 36(12):125009, 2020.
  10. Deep equilibrium models. Advances in Neural Information Processing Systems, 32, 2019.
  11. M. Benning and M. Burger. Modern regularization methods for inverse problems. Acta numerica, 27:1–111, 2018.
  12. Uniformly convex neural networks and non-stationary iterated network tikhonov (inett) method. Inverse Problems, 39(5):055002, 2023.
  13. G. Blanchard and N. Mücke. Optimal rates for regularization of statistical inverse learning problems. Foundations of Computational Mathematics, 18(4):971–1013, 2018.
  14. Convex regularization in statistical inverse learning problems. Inverse Problems and Imaging, 2023.
  15. Deep neural networks for inverse problems with pseudodifferential operators: An application to limited-angle tomography. SIAM Journal on Imaging Sciences, 14(2):470–505, 2021.
  16. Learning the invisible: A hybrid deep learning-shearlet framework for limited angle computed tomography. Inverse Problems, 35(6):064002, 2019.
  17. Large noise in variational regularization. Transactions of Mathematics and its Applications, 2(1):tny002, 2018.
  18. D. Calvetti and E. Somersalo. Inverse problems: From regularization to bayesian inference. Wiley Interdisciplinary Reviews: Computational Statistics, 10(3):e1427, 2018.
  19. B. Carl and I. Stephani. Entropy, compactness and the approximation of operators, volume 98 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, 1990.
  20. F. Cucker and S. Smale. On the mathematical foundations of learning. Bulletin of the American Mathematical Society, 39(1):1–49, 2002.
  21. M. Dashti and A. M. Stuart. The bayesian approach to inverse problems. arXiv preprint arXiv:1302.6989, 2013.
  22. Deep learning architectures for nonlinear operator functions and nonlinear inverse problems. Mathematical Statistics and Learning, 4(1):1–86, 2022.
  23. Elastic-net regularization in learning theory. Journal of Complexity, 25(2):201–230, 2009.
  24. R. M. Dudley. The sizes of compact subsets of hilbert space and continuity of gaussian processes. Journal of Functional Analysis, 1(3):290–330, 1967.
  25. H. Egger. Regularization of inverse problems with large noise. In Journal of Physics: Conference Series, volume 124, page 012022. IOP Publishing, 2008.
  26. Regularization of inverse problems, volume 375. Springer Science & Business Media, 1996.
  27. Inverse problems as statistics. Inverse problems, 18(4):R55, 2002.
  28. Deep equilibrium architectures for inverse problems in imaging. arXiv preprint arXiv:2102.07944, 2021.
  29. K. Gregor and Y. LeCun. Learning fast approximations of sparse coding. In Proceedings of the 27th international conference on international conference on machine learning, pages 399–406, 2010.
  30. R. Gribonval and M. Nikolova. A characterization of proximity operators. Journal of Mathematical Imaging and Vision, 62(6-7):773–789, 2020.
  31. E. Haber and L. Tenorio. Learning regularization functionals—a supervised training approach. Inverse Problems, 19(3):611, 2003.
  32. A. Habring and M. Holler. A generative variational model for inverse problems in imaging. SIAM Journal on Mathematics of Data Science, 4(1):306–335, 2022.
  33. Learning a variational network for reconstruction of accelerated mri data. Magnetic resonance in medicine, 79(6):3055–3071, 2018.
  34. Multi-scale learned iterative reconstruction. IEEE Transactions on Computational Imaging, 2020.
  35. T. Hohage and F. Werner. Inverse problems with poisson data: statistical regularization theory, applications and algorithms. Inverse Problems, 32(9):093001, 2016.
  36. Bayesian imaging with data-driven priors encoded by neural networks. SIAM Journal on Imaging Sciences, 15(2):892–924, 2022.
  37. Gradient step denoiser for convergent plug-and-play. arXiv preprint arXiv:2110.03220, 2021.
  38. Deep convolutional neural network for inverse problems in imaging. IEEE transactions on image processing, 26(9):4509–4522, 2017.
  39. Convergent data-driven regularizations for ct reconstruction. arXiv preprint arXiv:2212.07786, 2022.
  40. J. Kaipio and E. Somersalo. Statistical and computational inverse problems, volume 160. Springer Science & Business Media, 2006.
  41. Iterative regularization methods for nonlinear ill-posed problems. Walter de Gruyter, 2008.
  42. 2DeteCT - A large 2D expandable, trainable, experimental Computed Tomography dataset for machine learning: Slices OOD (reference reconstructions and segmentations), 2023.
  43. Total deep variation for linear inverse problems. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 7549–7558, 2020.
  44. Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481, 2021.
  45. Bayesian imaging using plug & play priors: when langevin meets tweedie. SIAM Journal on Imaging Sciences, 15(2):701–737, 2022.
  46. E. L. Lehmann and G. Casella. Theory of point estimation. Springer Science & Business Media, 2006.
  47. Nett: Solving inverse problems with deep neural networks. Inverse Problems, 36(6):065005, 2020.
  48. T.-C. Lim. On fixed point stability for set-valued contractive mappings with applications to generalized differential equations. Journal of Mathematical Analysis and Applications, 110(2):436–441, 1985.
  49. Using deep neural networks for inverse problems in imaging: beyond analytical methods. IEEE Signal Processing Magazine, 35(1):20–36, 2018.
  50. Adversarial regularizers in inverse problems. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 8516–8525, 2018.
  51. P. Mathé and U. Tautenhahn. Regularization under general noise assumptions. Inverse Problems, 27(3):035016, 2011.
  52. Convolutional neural networks for inverse problems in imaging: A review. IEEE Signal Processing Magazine, 34(6):85–95, 2017.
  53. Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem. arXiv preprint arXiv:2211.08875, 2022.
  54. End-to-end reconstruction meets data-driven regularization for inverse problems. Advances in Neural Information Processing Systems, 34:21413–21425, 2021.
  55. Learned convex regularizers for inverse problems. arXiv preprint arXiv:2008.02839, 2020.
  56. Learned reconstruction methods with convergence guarantees. arXiv preprint arXiv:2206.05431, 2022.
  57. Augmented nett regularization of inverse problems. Journal of Physics Communications, 5(10):105002, 2021.
  58. Convergence analysis of tikhonov regularization for non-linear statistical inverse problems. Electronic Journal of Statistics, 14:2798–2841, 2020.
  59. One network to solve them all–solving linear inverse problems using deep projection models. In Proceedings of the IEEE International Conference on Computer Vision, pages 5888–5897, 2017.
  60. Plug-and-play methods provably converge with properly trained denoisers. In International Conference on Machine Learning, pages 5546–5557. PMLR, 2019.
  61. Regularization methods in Banach spaces, volume 10. Walter de Gruyter, 2012.
  62. Deep null space learning for inverse problems: convergence analysis and rates. Inverse Problems, 35(2):025008, 2019.
  63. S. Shalev-Shwartz and S. Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
  64. I. Steinwart and A. Christmann. Support vector machines. Springer Science & Business Media, 2008.
  65. A. M. Stuart. Inverse problems: a Bayesian perspective. Acta Num., 19:451–559, 2010.
  66. M. Talagrand. The generic chaining: upper and lower bounds of stochastic processes. Springer Science & Business Media, 2005.
  67. Plug-and-play priors for model based reconstruction. In 2013 IEEE global conference on signal and information processing, pages 945–948. IEEE, 2013.
  68. R. Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
  69. M. J. Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge university press, 2019.
  70. G. Wang. A perspective on deep imaging. IEEE access, 4:8914–8924, 2016.
  71. Deep convolutional framelets: A general deep learning framework for inverse problems. SIAM Journal on Imaging Sciences, 11(2):991–1048, 2018.
  72. H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):301–320, 2005.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now