Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Scalable iterative data-adaptive RKHS regularization (2401.00656v1)

Published 1 Jan 2024 in math.NA and cs.NA

Abstract: We present iDARR, a scalable iterative Data-Adaptive RKHS Regularization method, for solving ill-posed linear inverse problems. The method searches for solutions in subspaces where the true solution can be identified, with the data-adaptive RKHS penalizing the spaces of small singular values. At the core of the method is a new generalized Golub-Kahan bidiagonalization procedure that recursively constructs orthonormal bases for a sequence of RKHS-restricted Krylov subspaces. The method is scalable with a complexity of $O(kmn)$ for $m$-by-$n$ matrices with $k$ denoting the iteration numbers. Numerical tests on the Fredholm integral equation and 2D image deblurring show that it outperforms the widely used $L2$ and $l2$ norms, producing stable accurate solutions consistently converging when the noise level decays.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Iterated preconditioned LSQR method for inverse problems on unstructured grids. Inverse Probl., 30(7):075009, 2014.
  2. A novel modified TRSVD method for large-scale linear discrete ill-posed problems. Appl. Numer. Math., 164:72–88, 2021.
  3. On regularization algorithms in learning theory. Journal of complexity, 23(1):52–72, 2007.
  4. Span of regularization for solution of inverse problems with application to magnetic resonance relaxometry of the brain. Scientific Reports, 12(1):20194, 2022.
  5. Åke Björck. A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT Numer. Math., 28(3):659–670, 1988.
  6. Convergence analysis of LSQR for compact operator equations. Linear Algebra and its Applications, 583:146–164, 2019.
  7. Stochastic convergence of regularized solutions and their finite element approximations to inverse source problems. SIAM Journal on Numerical Analysis, 60(2):751–780, 2022.
  8. A weighted-GCV method for Lanczos-hybrid regularization. Electr. Trans. Numer. Anal., 28(29):149–167, 2008.
  9. Learning theory: an approximation theory viewpoint, volume 24. Cambridge University Press, Cambridge, 2007.
  10. Lars Eldén. A weighted pseudoinverse, generalized singular values, and constrained least squares problems. BIT Numer. Math., 22:487–502, 1982.
  11. Regularization of inverse problems, volume 375. Springer Science & Business Media, 1996.
  12. Ir tools: a matlab package of iterative regularization methods and large-scale test problems. Numerical Algorithms, 81(3):773–811, 2019.
  13. Matrix Computations. The Johns Hopkins University Press, Baltimore, 4th edition, 2013.
  14. Jacques Salomon Hadamard. Lectures on Cauchy’s problem in linear partial differential equations, volume 18. Yale University Press, 1923.
  15. Per Christian Hansen. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. SIAM, 1998.
  16. Per Christian Hansen. The L-curve and its use in the numerical treatment of inverse problems. In in Computational Inverse Problems in Electrocardiology, ed. P. Johnston, Advances in Computational Bioengineering, pages 119–142. WIT Press, 2000.
  17. Tikhonov regularization with MTRSVD method for solving large-scale discrete ill-posed problems. J. Comput. Appl. Math., 405:113969, 2022.
  18. A joint bidiagonalization based iterative algorithm for large scale general-form Tikhonov regularization. Appl. Numer. Math., 157:159–177, 2020.
  19. A projection–based approach to general-form Tikhonov regularization. SIAM J. Sci. Comput., 29(1):315–330, 2007.
  20. Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl., 22(4):1204–1221, 2001.
  21. Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra Appl., 436(8):2845–2865, 2012.
  22. Small noise analysis for Tikhonov and RKHS regularizations. arXiv preprint arXiv:2305.11055, 2023.
  23. A modified Tikhonov regularization for linear operator equations. Numerical functional analysis and optimization, 26(4-5):543–563, 2005.
  24. Haibo Li. A preconditioned Krylov subspace method for linear inverse problems with general-form Tikhonov regularization. arXiv:2308.06577v1, 2023.
  25. Nonparametric learning of kernels in nonlocal operators. Journal of Peridynamics and Nonlocal Modeling, pages 1–24, 2023.
  26. Data adaptive RKHS Tikhonov regularization for learning kernels in operators. Proceedings of Mathematical and Scientific Machine Learning, PMLR 190:158-172, 2022.
  27. An adaptive RKHS regularization for Fredholm integral equations. arXiv preprint arXiv:2303.13737, 2023.
  28. Generalized inverses in reproducing kernel spaces: an approach to regularization of linear operator equations. SIAM Journal on Mathematical Analysis, 5(6):974–987, 1974.
  29. A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Statist. Comput., 2(4):474–489, 1981.
  30. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Software, 8:43–71, 1982.
  31. Tikhonov regularization based on generalized Krylov subspace methods. Appl. Numer. Math., 62(9):1215–1228, 2012.
  32. Hybrid and iteratively reweighted regularization by unbiased predictive risk and weighted GCV for projected systems. SIAM J. Sci. Statist. Comput., 39(2):B221–B243, 2017.
  33. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena, 60(1-4):259–268, 1992.
  34. Robert Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996.
  35. Andrei Nikolajevits Tikhonov. Solution of incorrectly formulated problems and the regularization method. Soviet Math., 4:1035–1038, 1963.
  36. Grace Wahba. Convergence rates of certain approximate solutions to fredholm integral equations of the first kind. Journal of Approximation Theory, 7(2):167–185, 1973.
  37. Grace Wahba. Practical approximate solutions to linear operator equations when the data are noisy. SIAM journal on numerical analysis, 14(4):651–667, 1977.
  38. Tikhonov regularization and randomized GSVD. SIAM J. Matrix Anal. Appl., 37(2):649–675, 2016.
  39. Regularization with randomized svd for large-scale discrete inverse problems. Inverse Problems, 29(8):085008, 2013.
  40. Randomized algorithms for large-scale inverse problems with general Tikhonov regularizations. Inverse Probl., 31(8):085008, 2015.
  41. Ye Zhang and Chuchu Chen. Stochastic asymptotical regularization for linear inverse problems. Inverse Problems, 39(1):015007, 2022.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now