Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unique reconstruction for discretized inverse problems: a random sketching approach via subsampling (2403.05935v2)

Published 9 Mar 2024 in math.NA, cs.NA, and math.OC

Abstract: Theoretical inverse problems are often studied in an ideal infinite-dimensional setting. The well-posedness theory provides a unique reconstruction of the parameter function, when an infinite amount of data is given. Through the lens of PDE-constrained optimization, this means one attains the zero-loss property of the mismatch function in this setting. This is no longer true in computations when we are limited to finite amount of measurements due to experimental or economical reasons. Consequently, one must compromise the goal, from inferring a function, to a discrete approximation. What is the reconstruction power of a fixed number of data observations? How many parameters can one reconstruct? Here we describe a probabilistic approach, and spell out the interplay of the observation size $(r)$ and the number of parameters to be uniquely identified $(m)$. The technical pillar here is the random sketching strategy, in which the matrix concentration inequality and sampling theory are largely employed. By analyzing a randomly subsampled Hessian matrix, we attain a well-conditioned reconstruction problem with high probability. Our main theory is validated in numerical experiments, using an elliptic inverse problem as an example.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. The linearized inverse problem in multifrequency electrical impedance tomography. SIAM journal on imaging sciences, 9(4):1525–1551, 2016.
  2. Alen Alexanderian. Optimal experimental design for infinite-dimensional bayesian inverse problems governed by pdes: A review. Inverse Problems, 37(4):043001, 2021.
  3. A fast and scalable method for a-optimal design of experiments for infinite-dimensional bayesian nonlinear inverse problems. SIAM Journal on Scientific Computing, 38(1):A243–A272, 2016.
  4. Simon R Arridge. Optical tomography in medical imaging. Inverse problems, 15(2):R41, 1999.
  5. Goal-oriented optimal design of experiments for large-scale bayesian linear inverse problems. Inverse Problems, 34(9):095009, 2018.
  6. Experimental design and inverse problems in plant biological modeling. Journal of Inverse and Ill-posed Problems, 20(2):169–191, 2012.
  7. Experimental design for mri by greedy policy search. Advances in Neural Information Processing Systems, 33:18954–18966, 2020.
  8. A preconditioning technique for a class of pde-constrained optimization problems. Advances in Computational Mathematics, 35:149–173, 2011.
  9. Machine learning for data-driven discovery in solid earth geoscience. Science, 363(6433):eaau0323, 2019.
  10. Rajendra Bhatia. Linear algebra to quantum cohomology: the story of alfred horn’s inequalities. The American Mathematical Monthly, 108(4):289–318, 2001.
  11. Waveform inversion via reduced order modeling. Geophysics, 88(2):R175–R191, 2023.
  12. Optimizing multiplexed imaging experimental design through tissue spatial segregation estimation. Nature Methods, 20(3):418–423, 2023.
  13. A computational framework for infinite-dimensional bayesian inverse problems part i: The linearized case, with application to global seismic inversion. SIAM Journal on Scientific Computing, 35(6):A2494–A2523, 2013.
  14. Bridging and improving theoretical and computational electrical impedance tomography via data completion. SIAM Journal on Scientific Computing, 44(3):B668–B693, 2022.
  15. Insensitive Functionals, Inconsistent Gradients, Spurious Minima, and Regularized Functionals in Flow Optimization Problems. International Journal of Computational Fluid Dynamics, 16(3):171–185, jan 2002.
  16. Adjoint DSMC for nonlinear boltzmann equation constrained optimization. Journal of Computational Physics, 439:110404, 2021.
  17. Consensus-based sampling. Studies in Applied Mathematics, 148(3):1069–1140, 2022.
  18. Consensus-based optimization and ensemble kalman inversion for global optimization problems with constraints. In Modeling and Simulation for Collective Dynamics, pages 195–230. World Scientific, 2023.
  19. Fast full waveform inversion with source encoding and second-order optimization methods. Geophysical Journal International, 200(2):720–744, 2015.
  20. Parameterizations for ensemble kalman inversion. Inverse Problems, 34(5):055009, 2018.
  21. Ke Chen and Ruhui Jin. Tensor-structured sketching for constrained least squares. SIAM Journal on Matrix Analysis and Applications, 42(4):1703–1731, 2021.
  22. Structured random sketching for pde inverse problems. SIAM Journal on Matrix Analysis and Applications, 41(4):1742–1770, 2020.
  23. Hessian-based sampling for high-dimensional model reduction. International Journal for Uncertainty Quantification, 9(2), 2019.
  24. Hessian-based adaptive sparse quadrature for infinite-dimensional bayesian inverse problems. Computer Methods in Applied Mechanics and Engineering, 327:147–172, 2017.
  25. Matrix probing and its conditioning. SIAM Journal on Numerical Analysis, 50(1):171–193, 2012.
  26. Convergence rates for learning linear operators from noisy data. SIAM/ASA Journal on Uncertainty Quantification, 11(2):480–513, 2023.
  27. Matrix probing: A randomized preconditioner for the wave-equation Hessian. Applied and Computational Harmonic Analysis, 32(2):155–168, 2012.
  28. Fast wave computation via Fourier integral operators. Mathematics of Computation, 81(279):1455–1486, 2012.
  29. Optimal transport based seismic inversion: Beyond cycle skipping. Communications on Pure and Applied Mathematics, 75(10):2201–2244, 2022.
  30. A Newton-CG method for large-scale three-dimensional elastic full-waveform seismic inversion. Inverse Problems, 24(3):034015, 2008.
  31. Total variation regularization strategies in full-waveform inversion. SIAM Journal on Imaging Sciences, 11(1):376–406, 2018.
  32. Learning particle swarming models from data with gaussian processes. arXiv preprint arXiv:2106.02735, 2021.
  33. Adaptive A-optimal experimental design for linear dynamical systems. SIAM/ASA Journal on Uncertainty Quantification, 4(1):1138–1159, 2016.
  34. Randomized Nyström preconditioning. SIAM Journal on Matrix Analysis and Applications, 44(2):718–752, 2023.
  35. Interacting Langevin diffusions: Gradient structure and ensemble Kalman sampler. SIAM Journal on Applied Dynamical Systems, 19(1):412–441, 2020.
  36. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, 53(2):217–288, 2011.
  37. Kinetic methods for inverse problems. Kinetic & Related Models, 12(5), 2019.
  38. Optimization with PDE constraints, volume 23. Springer Science & Business Media, 2008.
  39. An operator learning perspective on parameter-to-observable maps. arXiv preprint arXiv:2402.06031, 2024.
  40. Inverse Problems: Tikhonov Theory and Algorithms, volume 22. World Scientific, 2014.
  41. Batch greedy maximization of non-submodular functions: Guarantees and applications to experimental design. The Journal of Machine Learning Research, 22(1):11397–11458, 2021.
  42. Reinforced inverse scattering. arXiv preprint arXiv:2206.04186, 2022.
  43. A reconstruction algorithm for electrical impedance tomography based on sparsity regularization. International Journal for Numerical Methods in Engineering, 89(3):337–353, 2012.
  44. Extensions of Lipschitz maps into a hilbert space. Contemporary Mathematics, 26:189–206, 01 1984.
  45. A modified particle swarm optimization algorithm for global optimizations of inverse problems. IEEE Transactions on Magnetics, 52(3):1–4, 2015.
  46. Switchnet: a neural network model for forward and inverse scattering problems. SIAM Journal on Scientific Computing, 41(5):A3182–A3201, 2019.
  47. J. Kiefer. General Equivalence Theory for Optimum Designs (Approximate Theory). The Annals of Statistics, 2(5):849 – 879, 1974.
  48. Monte Carlo gradient in optimization constrained by radiative transport equation. SIAM Journal on Numerical Analysis, 61(6):2744–2774, 2023.
  49. Swarm-based gradient descent method for non-convex optimization. arXiv preprint arXiv:2211.17157, 2022.
  50. A deconvolution-based objective function for wave-equation inversion. In SEG International Exposition and Annual Meeting, pages SEG–2011. SEG, 2011.
  51. Wave-equation reflection traveltime inversion with dynamic warping and hybrid waveform inversion, pages 871–876. 2013.
  52. Michael W Mahoney et al. Randomized algorithms for matrices and data. Foundations and Trends® in Machine Learning, 3(2):123–224, 2011.
  53. Randomized numerical linear algebra: Foundations and algorithms. Acta Numerica, 29:403–572, 2020.
  54. Optimal transport for mitigating cycle skipping in full-waveform inversion: A graph-space transform approach. Geophysics, 83(5):R515–R540, 2018.
  55. Full waveform inversion and the truncated newton method: quantitative imaging of complex subsurface structures. Geophysical Prospecting, 62(6):1353–1375, 2014.
  56. Full waveform inversion and the truncated newton method. SIAM Journal on Scientific Computing, 35(2):B401–B437, 2013.
  57. Melanie Mitchell. An Introduction to Genetic Algorithms. MIT Press, Cambridge, MA, USA, 1998.
  58. The SEISCOPE optimization toolbox: A large-scale nonlinear optimization library based on reverse communication. GEOPHYSICS, 81(2):F1–F15, 2016.
  59. Three dimensional inversion of multisource time domain electromagnetic data. Geophysics, 78(1):E47–E57, 2013.
  60. On cycle-skipping and misfit function modification for full-wave inversion: Comparison of five recent approaches. Geophysics, 86(4):R563–R587, 2021.
  61. Fokker–Planck particle systems for bayesian inference: Computational approaches. SIAM/ASA Journal on Uncertainty Quantification, 9(2):446–482, 2021.
  62. Mark Rudelson. Random vectors in the isotropic position. Journal of Functional Analysis, 164(1):60–72, 1999.
  63. Sampling from large matrices: An approach through geometric functional analysis. Journal of the ACM (JACM), 54(4):21–es, 2007.
  64. The Fourier reconstruction of a head section. IEEE Transactions on Nuclear Science, 21(3):21–43, 1974.
  65. William W Symes. Approximate linearized inversion by optimal scaling of prestack depth migration. Geophysics, 73(2):R23–R35, 2008.
  66. Improving accuracy and computational efficiency of optimal design of experiments via greedy backward approach. International Journal for Uncertainty Quantification, 14(1), 2024.
  67. Joel A Tropp. On the conditioning of random subdictionaries. Applied and Computational Harmonic Analysis, 25(1):1–24, 2008.
  68. Joel A Tropp. User-friendly tail bounds for sums of random matrices. Foundations of computational mathematics, 12:389–434, 2012.
  69. Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47. Cambridge university press, 2018.
  70. Hermann Von Weyl. Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung). Mathematische Annalen, 71:441–479, 1912.
  71. David P Woodruff et al. Sketching as a tool for numerical linear algebra. Foundations and Trends® in Theoretical Computer Science, 10(1-2):1–157, 2014.
  72. Large-scale bayesian optimal experimental design with derivative-informed projected neural network. Journal of Scientific Computing, 95(1):30, 2023.
  73. Application of optimal transport and the quadratic wasserstein metric to full-waveform inversion. Geophysics, 83(1):R43–R62, 2018.
  74. Compositional heterogeneity near the base of the mantle transition zone beneath hawaii. Nature Communications, 9(1):1266, 2018.
  75. A scalable design of experiments framework for optimal sensor placement. Journal of Process Control, 67:44–55, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now