Papers
Topics
Authors
Recent
Search
2000 character limit reached

Spatiotemporal Besov Priors for Bayesian Inverse Problems

Published 28 Jun 2023 in stat.ME and stat.ML | (2306.16378v2)

Abstract: Fast development in science and technology has driven the need for proper statistical tools to capture special data features such as abrupt changes or sharp contrast. Many inverse problems in data science require spatiotemporal solutions derived from a sequence of time-dependent objects with these spatial features, e.g., dynamic reconstruction of computerized tomography (CT) images with edges. Conventional methods based on Gaussian processes (GP) often fall short in providing satisfactory solutions since they tend to offer over-smooth priors. Recently, the Besov process (BP), defined by wavelet expansions with random coefficients, has emerged as a more suitable prior for Bayesian inverse problems of this nature. While BP excels in handling spatial inhomogeneity, it does not automatically incorporate temporal correlation inherited in the dynamically changing objects. In this paper, we generalize BP to a novel spatiotemporal Besov process (STBP) by replacing the random coefficients in the series expansion with stochastic time functions as Q-exponential process (Q-EP) which governs the temporal correlation structure. We thoroughly investigate the mathematical and statistical properties of STBP. A white-noise representation of STBP is also proposed to facilitate the inference. Simulations, two limited-angle CT reconstruction examples and a highly non-linear inverse problem involving Navier-Stokes equation are used to demonstrate the advantage of the proposed STBP in preserving spatial features while accounting for temporal changes compared with the classic STGP and a time-uncorrelated approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Rates of contraction of posterior distributions based on p-exponential priors. Bernoulli, 27(3):1616 – 1642, 2021.
  2. Yves A. Atchadaé. On the contraction properties of some high-dimensional quasi-posterior distributions. The Annals of Statistics, 45(5), October 2017.
  3. An approach to large-scale quasi-bayesian inference with spike-and-slab priors. arXiv:1803.10282, 03 2018.
  4. Frank Aurzada. On the lower tail probabilities of some random sequences in lp. Journal of Theoretical Probability, 20(4):843–858, Dec 2007.
  5. DEHB: Evolutionary hyberband for scalable, robust and efficient hyperparameter optimization. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, aug 2021.
  6. Hybrid Monte-Carlo on Hilbert spaces. Stochastic Processes and their Applications, 121:2201–2230, 2011.
  7. Alexandros Beskos. A stable manifold MCMC method for high dimensions. Statistics & Probability Letters, 90:46–52, 2014.
  8. Geometric MCMC for infinite-dimensional inverse problems. Journal of Computational Physics, 335, 2017.
  9. Sparse reconstructions from few noisy data: analysis of hierarchical Bayesian models with generalized gamma hyperpriors. Inverse Problems, 36(2):025010, 2020.
  10. On the theory of elliptically contoured distributions. Journal of Multivariate Analysis, 11(3):368–385, 1981.
  11. The horseshoe estimator for sparse signals. Biometrika, 97(2):465–480, 2010.
  12. Penalized regression, standard errors, and Bayesian Lassos. Bayesian analysis, 5(2):369–411, 2010.
  13. Dimension-robust mcmc in bayesian inverse problems. arXiv preprint arXiv:1803.03344, 2018.
  14. Besov priors for bayesian inverse problems. Inverse Problems and Imaging, 6(2):183–200, may 2012.
  15. The Bayesian Approach to Inverse Problems, pages 311–428. Springer International Publishing, Cham, 2017.
  16. Map estimators for piecewise continuous inversion. Inverse Problems, 32(10):105003, 2016.
  17. Regularization of inverse problems, volume 375. Springer Science & Business Media, 1996.
  18. BOHB: Robust and efficient hyperparameter optimization at scale. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1437–1446. PMLR, 10–15 Jul 2018.
  19. K. Fang and Y.T. Zhang. Generalized Multivariate Analysis. Science Press, 1990.
  20. A general framework for bayes structured linear models. The Annals of Statistics, 48(5), October 2020.
  21. S. Ghosal and A.W. van der Vaart. Convergence rates of posterior distributions for non-i.i.d. observations. Annals of Statistics, 35(1):192–223, 2007. MR2332274.
  22. Subhashis Ghosal and Aad van der Vaart. Fundamentals of nonparametric bayesian inference. 2017.
  23. Rates of contraction for posterior distributions in Lrsuperscript𝐿𝑟L^{r}italic_L start_POSTSUPERSCRIPT italic_r end_POSTSUPERSCRIPT-metrics, 1≤r≤∞1𝑟1\leq r\leq\infty1 ≤ italic_r ≤ ∞. The Annals of Statistics, 39(6), December 2011.
  24. Matteo Giordano. Besov-laplace priors in density estimation: optimal posterior contraction rates and adaptation. Electronic Journal of Statistics, 17(2), January 2023.
  25. Analysis of spdes arising in path sampling part ii: The nonlinear case. The Annals of Applied Probability, 17(5–6), October 2007.
  26. Deep Residual Learning for Image Recognition. 2015.
  27. Tommi Heikkilä. Stempo–dynamic X-ray tomography phantom. arXiv preprint arXiv:2209.12471, 2022.
  28. Young M. Hermena S. Ct-scan image production procedures. StatPearls Publishing, August 2023.
  29. Geoffrey Hinton. Imagenet classification with deep convolutional neural networks. NIPS’12: Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, 1:1097–1105, 2012.
  30. A bayesian level set method for geometric inverse problems. Interfaces and free boundaries, 18(2):181–217, 2016.
  31. Spike and slab variable selection: Frequentist and bayesian strategies. The Annals of Statistics, 33(2), April 2005.
  32. Valentin Konstantinovich Ivanov. On linear problems which are not well-posed. In Doklady akademii nauk, volume 145, pages 270–272. Russian Academy of Sciences, 1962.
  33. Bayesian approach to inverse problems for functions with a variable-index Besov prior. Inverse Problems, 32(8):085006, 2016.
  34. Mark E. Johnson. Multivariate Statistical Simulation, chapter 6 Elliptically Contoured Distributions, pages 106–124. Probability and Statistics. John Wiley & Sons, Ltd, 1987.
  35. Y. Kano. Consistency property of elliptic probability density functions. Journal of Multivariate Analysis, 51(1):139–147, 1994.
  36. W Clem Karl. Regularization in image restoration and reconstruction. In Handbook of Image and Video Processing, pages 183–V. Elsevier, 2005.
  37. Bayesian inverse problems with gaussian priors. The Annals of Statistics, 39(5):2626–2657, 2011.
  38. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1–97, 2023.
  39. Multivariate generalized laplace distribution and related random fields. Journal of Multivariate Analysis, 113:59–72, 2013. Special Issue on Multivariate Distribution Theory in Memory of Samuel Kotz.
  40. Shiwei Lan. Adaptive dimension reduction to accelerate infinite-dimensional geometric markov chain monte carlo. Journal of Computational Physics, 392:71 – 95, September 2019.
  41. Shiwei Lan. Learning temporal evolution of spatial dependence with generalized spatiotemporal gaussian process models. Journal of Machine Learning Research, 23(259):1–53, 2022.
  42. Discretization-invariant bayesian inversion and besov space priors. Inverse Problems and Imaging, 3(1):87–122, 2009.
  43. Can one use total variation prior for edge-preserving Bayesian inversion? Inverse Problems, 20(5):1537, 2004.
  44. L. LeCam. Convergence of estimates under dimensionality restrictions. The Annals of Statistics, 1(1):38–53, Jan 1973.
  45. J Lee and PK Kitanidis. Bayesian inversion with total variation prior for discrete geologic structure identification. Water Resources Research, 49(11):7658–7669, 2013.
  46. Hyperband: A novel bandit-based approach to hyperparameter optimization. J. Mach. Learn. Res., 18(1):6765–6816, jan 2017.
  47. Bayesian learning via q-exponential process. In Proceedings of the 37th Conference on Neural Information Processing Systems. NeurIPS, 12 2023.
  48. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021.
  49. Model selection and accounting for model uncertainty in graphical models using occam’s window. Journal of the American Statistical Association, 89(428):1535–1546, December 1994.
  50. Cauchy difference priors for edge-preserving Bayesian inversion. Journal of Inverse and Ill-posed Problems, 27(2):225–240, 2019.
  51. Tomographic x-ray data of 3d emoji. arXiv preprint arXiv:1802.09397, 2018.
  52. Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83(404):1023–1032, December 1988.
  53. R. M. Neal. MCMC using Hamiltonian dynamics. In S. Brooks, A. Gelman, G. Jones, and X. L. Meng, editors, Handbook of Markov Chain Monte Carlo. Chapman and Hall/CRC, 2010.
  54. Bernt Øksendal. Stochastic Differential Equations. Springer Berlin Heidelberg, 2003.
  55. The Bayesian Lasso. Journal of the American Statistical Association, 103(482):681–686, 2008.
  56. A computational framework for edge-preserving regularization in dynamic inverse problems. Electron. Trans. Numer. Anal., 58:486–516, 2023.
  57. Sparsity information and regularization in the horseshoe and other shrinkage priors. Electronic Journal of Statistics, 11(2), January 2017.
  58. Estimation for stochastic models driven by laplace motion. Communications in Statistics - Theory and Methods, 40(18):3281–3302, sep 2011.
  59. The Bayesian Bridge. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(4):713–733, 11 2013.
  60. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  61. Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2005.
  62. Posterior concentration rates for infinite dimensional exponential families. Bayesian Analysis, 7(2), June 2012.
  63. Bayesian deblurring with integrated noise estimation. In CVPR 2011, pages 2625–2632. IEEE, 2011.
  64. I. J. Schoenberg. Metric spaces and completely monotone functions. Annals of Mathematics, 39:811–841, 1938.
  65. The fourier reconstruction of a head section. IEEE Transactions on Nuclear Science, 21(3):21–43, 1974.
  66. Inverse problems in geophysics. In Wavefield inversion, pages 119–190. Springer, 1999.
  67. Andrew M Stuart. Inverse problems: a Bayesian perspective. Acta Numerica, 19:451–559, 2010.
  68. Cauchy markov random field priors for bayesian inversion. Statistics and Computing, 32(2):33, Mar 2022.
  69. Andrey Nikolayevich Tikhonov. On the stability of inverse problems. In Dokl. Akad. Nauk SSSR, volume 39, pages 195–198, 1943.
  70. Hans Triebel. Theory of Function Spaces. Springer Basel, 1983.
  71. The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography. Ultramicroscopy, 157:35–47, 2015.
  72. A. W. van der Vaart and J. H. van Zanten. Rates of contraction of posterior distributions based on gaussian process priors. The Annals of Statistics, 36(3):1435–1463, 2008.
  73. A. W. van der Vaart and J. H. van Zanten. Adaptive bayesian estimation using a gaussian random field with inverse gamma bandwidth. Ann. Statist., 37(5B):2655–2675, 10 2009.
  74. Aad van der Vaart and Harry van Zanten. Information rates of nonparametric gaussian process methods. J. Mach. Learn. Res., 12:2095–2119, July 2011.
  75. Convolutional Neural Networks in Visual Computing: A Concise Guide. CRC Press, 2017.
  76. Sebastian J Vollmer. Posterior consistency for bayesian inverse problems through stability and regression results. Inverse Problems, 29(12):125011, November 2013.
  77. Image quality assessment: From error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  78. A tv-gaussian prior for infinite-dimensional bayesian inverse problems and its numerical implementations. Inverse Problems, 32(7):075006, May 2016.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 1 like about this paper.

Sign Up for Free