Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Probabilistic Exponential Integrators (2305.14978v2)

Published 24 May 2023 in math.NA, cs.LG, cs.NA, and stat.ML

Abstract: Probabilistic solvers provide a flexible and efficient framework for simulation, uncertainty quantification, and inference in dynamical systems. However, like standard solvers, they suffer performance penalties for certain stiff systems, where small steps are required not for reasons of numerical accuracy but for the sake of stability. This issue is greatly alleviated in semi-linear problems by the probabilistic exponential integrators developed in this paper. By including the fast, linear dynamics in the prior, we arrive at a class of probabilistic integrators with favorable properties. Namely, they are proven to be L-stable, and in a certain case reduce to a classic exponential integrator -- with the added benefit of providing a probabilistic account of the numerical error. The method is also generalized to arbitrary non-linear systems by imposing piece-wise semi-linearity on the prior via Jacobians of the vector field at the previous estimates, resulting in probabilistic exponential Rosenbrock methods. We evaluate the proposed methods on multiple stiff differential equations and demonstrate their improved stability and efficiency over established probabilistic solvers. The present contribution thus expands the range of problems that can be effectively tackled within probabilistic numerics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Julia: A fresh approach to numerical computing. SIAM review, 59(1):65–98, 2017.
  2. Calibrated adaptive probabilistic ODE solvers. In International Conference on Artificial Intelligence and Statistics. PMLR, 2021.
  3. Pick-and-mix information operators for probabilistic ODE solvers. In International Conference on Artificial Intelligence and Statistics. PMLR, 2022.
  4. Parallel-in-time probabilistic numerical ODE solvers, 2023.
  5. Neural ordinary differential equations. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2018.
  6. Bayesian probabilistic numerical methods. SIAM Review, 61:756–789, 2019.
  7. Exponential time differencing for stiff systems. Journal of Computational Physics, 176(2):430–455, 2002.
  8. W. E. A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 5(1):1–11, Mar 2017.
  9. A. Griewank and A. Walther. Evaluating Derivatives. Society for Industrial and Applied Mathematics, second edition, 2008.
  10. E. Hairer and G. Wanner. Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems. Springer series in computational mathematics. Springer-Verlag, 1991.
  11. Probabilistic numerics and uncertainty in computations. Proceedings. Mathematical, physical, and engineering sciences, 471(2179):20150142–20150142, Jul 2015.
  12. Probabilistic Numerics: Computation as Machine Learning. Cambridge University Press, 2022.
  13. M. Hochbruck and C. Lubich. On Krylov subspace approximations to the matrix exponential operator. SIAM Journal on Numerical Analysis, 34(5):1911–1925, Oct 1997.
  14. M. Hochbruck and A. Ostermann. Explicit integrators of Rosenbrock-type. Oberwolfach Reports, 3(2):1107–1110, 2006.
  15. M. Hochbruck and A. Ostermann. Exponential integrators. Acta Numerica, 19:209–286, 2010.
  16. Exponential integrators for large systems of differential equations. SIAM Journal on Scientific Computing, 19(5):1552–1574, 1998.
  17. Exponential Rosenbrock-type methods. SIAM Journal on Numerical Analysis, 47(1):786–803, 2009.
  18. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, May 2021.
  19. H. Kersting and P. Hennig. Active uncertainty calibration in Bayesian ode solvers. In Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI), pages 309–318, June 2016.
  20. Differentiable likelihoods for fast inversion of ’Likelihood-free’ dynamical systems. In International Conference on Machine Learning. PMLR, 2020a.
  21. Convergence rates of Gaussian ODE filters. Statistics and computing, 30(6):1791–1816, 2020b.
  22. A. N. Kolmogorov. A study of the equation of diffusion with increase in the quantity of matter, and its application to a biological problem. Moscow University Bulletin of Mathematics, 1:1–25, 1937.
  23. N. Krämer and P. Hennig. Linear-time probabilistic solution of boundary value problems. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2021.
  24. Probabilistic ODE solutions in millions of dimensions. In International Conference on Machine Learning. PMLR, 2022.
  25. Probabilistic numerical method of lines for time-dependent partial differential equations. In International Conference on Artificial Intelligence and Statistics. PMLR, 2022.
  26. N. Krämer and P. Hennig. Stable implementation of probabilistic ode solvers, 2020.
  27. J. D. Lambert. Computational Methods in Ordinary Differential Equations. Introductory Mathematics for Scientists And Engineers. Wiley, 1973.
  28. V. T. Luan and A. Ostermann. Exponential Rosenbrock methods of order five — construction, analysis and numerical comparisons. Journal of Computational and Applied Mathematics, 255:417–431, 2014.
  29. N. K. Madsen. The method of lines for the numerical solution of partial differential equations. Proceedings of the SIGNUM meeting on Software for partial differential equations, 1975.
  30. Bayesian filtering for ODEs with bounded derivatives, 2017.
  31. A review of exponential integrators for first order semi-linear problems. Technical report, Norges Teknisk-Naturvetenskaplige Universitet, 2005.
  32. C. Moler and C. Van Loan. Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Review, 45(1):3–49, Jan 2003.
  33. A modern retrospective on probabilistic numerics. Statistics and Computing, 29(6):1335–1351, 2019.
  34. C. Rackauckas and Q. Nie. DifferentialEquations.jl – a performant and feature-rich ecosystem for solving differential equations in julia. Journal of Open Research Software, 5(1), 2017.
  35. Universal differential equations for scientific machine learning, 2021.
  36. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, Feb 2019.
  37. S. Särkkä. Bayesian Filtering and Smoothing, volume 3 of Institute of Mathematical Statistics textbooks. Cambridge University Press, 2013.
  38. W. E. Schiesser. The numerical method of lines: integration of partial differential equations. Elsevier, 2012.
  39. A probabilistic state space model for joint inference from differential equations and data. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2021.
  40. A probabilistic model for the numerical solution of initial value problems. Statistics and Computing, 29(1):99–122, Jan 2019.
  41. R. B. Sidje. Expokit: A software package for computing matrix exponentials. ACM Transactions on Mathematical Software, 24(1):130–156, mar 1998.
  42. T. Stillfjord. Low-rank second-order splitting of large-scale differential Riccati equations. IEEE Transactions on Automatic Control, 60(10):2791–2796, 2015.
  43. T. Stillfjord. Adaptive high-order splitting schemes for large-scale differential Riccati equations. Numerical Algorithms, 78(4):1129–1151, Sep 2017.
  44. S. Särkkä and A. Solin. Applied Stochastic Differential Equations. Institute of Mathematical Statistics Textbooks. Cambridge University Press, 2019.
  45. Probabilistic solutions to ordinary differential equations as nonlinear Bayesian filtering: a new perspective. Statistics and Computing, 29(6):1297–1315, 2019.
  46. Bayesian ODE solvers: The maximum a posteriori estimate. Statistics and Computing, 31(3):1–18, 2021.
  47. Fenrir: Physics-enhanced regression for initial value problems. In International Conference on Machine Learning. PMLR, 2022.
  48. C. Van Loan. Computing integrals involving the matrix exponential. IEEE Transactions on Automatic Control, 23(3):395–404, 1978.
Citations (5)

Summary

We haven't generated a summary for this paper yet.