Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Semilinear Neural Operators : A Unified Recursive Framework For Prediction And Data Assimilation (2402.15656v2)

Published 24 Feb 2024 in cs.LG, cs.AI, cs.NA, and math.NA

Abstract: Recent advances in the theory of Neural Operators (NOs) have enabled fast and accurate computation of the solutions to complex systems described by partial differential equations (PDEs). Despite their great success, current NO-based solutions face important challenges when dealing with spatio-temporal PDEs over long time scales. Specifically, the current theory of NOs does not present a systematic framework to perform data assimilation and efficiently correct the evolution of PDE solutions over time based on sparsely sampled noisy measurements. In this paper, we propose a learning-based state-space approach to compute the solution operators to infinite-dimensional semilinear PDEs. Exploiting the structure of semilinear PDEs and the theory of nonlinear observers in function spaces, we develop a flexible recursive method that allows for both prediction and data assimilation by combining prediction and correction operations. The proposed framework is capable of producing fast and accurate predictions over long time horizons, dealing with irregularly sampled noisy measurements to correct the solution, and benefits from the decoupling between the spatial and temporal dynamics of this class of PDEs. We show through experiments on the Kuramoto-Sivashinsky, Navier-Stokes and Korteweg-de Vries equations that the proposed model is robust to noise and can leverage arbitrary amounts of measurements to correct its prediction over a long time horizon with little computational overhead.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Extended Kalman filter based observer design for semilinear infinite-dimensional systems. IEEE Transactions on Automatic Control, 2023.
  2. Data assimilation: methods, algorithms, and applications. SIAM, 2016.
  3. Spectral variability in hyperspectral data unmixing: A comprehensive review. IEEE geoscience and remote sensing magazine, 9(4):223–270, 2021.
  4. Clifford neural layers for PDE modeling. arXiv preprint arXiv:2209.04934, 2022a.
  5. Lie point symmetry data augmentation for neural PDE solvers. In International Conference on Machine Learning, pp. 2241–2256. PMLR, 2022b.
  6. Richard Buxton. The physics of functional magnetic resonance imaging (fMRI). Reports on Progress in Physics, 76(9):096601, 2013.
  7. Laplace neural operator for complex geometries. arXiv preprint arXiv:2302.08166, 2023.
  8. Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review. IEEE/CAA Journal of Automatica Sinica, 10(6):1361–1387, 2023.
  9. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  10. Introduction to infinite-dimensional systems theory: a state-space approach, volume 71. Springer Nature, 2020.
  11. Lawrence Evans. Partial differential equations, volume 19. American Mathematical Society, 2022.
  12. Using machine learning to correct model error in data assimilation and forecast applications. Quarterly Journal of the Royal Meteorological Society, 147(739):3067–3084, 2021.
  13. Neural Koopman prior for data assimilation. arXiv preprint arXiv:2309.05317, 2023.
  14. Moritz Gerlach. Semigroups of kernel operators. PhD thesis, Universität Ulm, 2014.
  15. Physics-informed neural operators. arXiv preprint arXiv:2207.05748, 2022.
  16. Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  11474–11484, 2020.
  17. Adaptive Fourier neural operators: Efficient token mixers for transformers. arXiv preprint arXiv:2111.13587, 2021.
  18. Multiwavelet-based operator learning for differential equations. Advances in neural information processing systems, 34:24048–24062, 2021.
  19. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pp. 12556–12569. PMLR, 2023.
  20. Digital twin earth–coasts: Developing a fast and physics-informed surrogate model for coastal floods via neural operators. arXiv preprint arXiv:2110.07100, 2021.
  21. Efficient super-resolution of near-surface climate modeling using the Fourier neural operator. Journal of Advances in Modeling Earth Systems, 15(7):e2023MS003800, 2023.
  22. Semi-supervised invertible neural operators for Bayesian inverse problems. Computational Mechanics, pp.  1–20, 2023.
  23. Fourth-order time-stepping for stiff pdes. SIAM Journal on Scientific Computing, 26(4):1214–1233, 2005.
  24. Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems, 33:6696–6707, 2020.
  25. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  26. Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481, 2021.
  27. Boundary control of PDEs: A course on backstepping designs. SIAM, 2008.
  28. Dynamic mode decomposition: data-driven modeling of complex systems. SIAM, 2016.
  29. The finite element method: theory, implementation, and applications, volume 10. Springer Science & Business Media, 2013.
  30. A framework for machine learning of model error in dynamical systems. Communications of the American Mathematical Society, 2(07):283–344, 2022.
  31. Matthew Emanuel Levine. Machine Learning and Data Assimilation for Blending Incomplete Models and Noisy Data. PhD thesis, California Institute of Technology, 2023.
  32. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020a.
  33. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020b.
  34. Learning chaotic dynamics in dissipative systems. Advances in Neural Information Processing Systems, 35:16768–16781, 2022.
  35. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019.
  36. Approximate Bayesian neural operators: Uncertainty quantification for parametric PDEs. arXiv preprint arXiv:2208.01565, 2022.
  37. Neural rough differential equations for long time series. In International Conference on Machine Learning, pp. 7829–7838. PMLR, 2021.
  38. Fourcastnet: A global data-driven high-resolution weather model using adaptive Fourier neural operators. arXiv preprint arXiv:2202.11214, 2022.
  39. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=j3oQF9coJd.
  40. Samuel M Rankin. Semilinear evolution equations in Banach spaces with application to parabolic partial differential equations. Transactions of the American Mathematical Society, 336(2):523–535, 1993.
  41. KalmanNet: Neural network aided Kalman filtering for partially known dynamics. IEEE Transactions on Signal Processing, 70:1532–1547, 2022.
  42. Semi-supervised learning of partial differential operators and dynamical flows. In Uncertainty in Artificial Intelligence, pp.  1785–1794. PMLR, 2023.
  43. Neural stochastic PDEs: Resolution-invariant learning of continuous spatiotemporal dynamics. Advances in Neural Information Processing Systems, 35:1333–1344, 2022.
  44. Bayesian filtering and smoothing, volume 17. Cambridge university press, 2023.
  45. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
  46. Variation is the norm: Brain state dynamics evoked by emotional video clips. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp.  6003–6007. IEEE, 2021.
  47. Application of statistical filter theory to the optimal estimation of position and velocity on board a circumlunar vehicle, volume 135. National Aeronautics and Space Administration, 1962.
  48. Stefanie Sonner. An introduction to partial differential equations. Radboud University, Nijmegen, IMAPP - Mathematics, 2022.
  49. Roger Temam. Behaviour at time t=0 of the solutions of semi-linear evolution equations. Journal of Differential Equations, 43(1):73–92, 1982.
  50. Fourier neural operator for learning solutions to macroscopic traffic flow models: Application to the forward and inverse problems. arXiv preprint arXiv:2308.07051, 2023.
  51. Abdul-Majid Wazwaz. Partial differential equations and solitary waves theory. Springer Science & Business Media, 2010.
  52. Timesen2crop: A million labeled samples dataset of Sentinel 2 image time series for crop-type classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:4699–4708, 2021.
  53. Fred Weissler. Semilinear evolution equations in Banach spaces. Journal of Functional Analysis, 32(3):277–296, 1979.
  54. U-FNO—An enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now