Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Fast Simulation of Dynamical System with Neural Vector Enhanced Numerical Solver (2208.03680v3)

Published 7 Aug 2022 in cs.CE, cs.AI, cs.LG, cs.NA, and math.NA

Abstract: The large-scale simulation of dynamical systems is critical in numerous scientific and engineering disciplines. However, traditional numerical solvers are limited by the choice of step sizes when estimating integration, resulting in a trade-off between accuracy and computational efficiency. To address this challenge, we introduce a deep learning-based corrector called Neural Vector (NeurVec), which can compensate for integration errors and enable larger time step sizes in simulations. Our extensive experiments on a variety of complex dynamical system benchmarks demonstrate that NeurVec exhibits remarkable generalization capability on a continuous phase space, even when trained using limited and discrete data. NeurVec significantly accelerates traditional solvers, achieving speeds tens to hundreds of times faster while maintaining high levels of accuracy and stability. Moreover, NeurVec's simple-yet-effective design, combined with its ease of implementation, has the potential to establish a new paradigm for fast-solving differential equations based on deep learning.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. AI Pontryagin or how artificial neural networks learn to control dynamical systems. \JournalTitleNature Communications 13, 1–9 (2022).
  2. Dynamical systems and numerical analysis, vol. 2 (Cambridge University Press, 1998).
  3. Machine learning for prediction with missing dynamics. \JournalTitleJournal of Computational Physics 428, 109922 (2021).
  4. Nitrogen-fixing trees could exacerbate climate change under elevated nitrogen deposition. \JournalTitleNature Communications 10, 1–8 (2019).
  5. A general theory of glacier surges. \JournalTitleJournal of Glaciology 65, 701–716 (2019).
  6. Chemnode: A neural ordinary differential equations framework for efficient chemical kinetic solvers. \JournalTitleEnergy and AI 7, 100118 (2022).
  7. Chemical mechanism solvers in air quality models. \JournalTitleAtmosphere 2, 510–532 (2011).
  8. Delpini, D. et al. Evolution of controllability in interbank networks. \JournalTitleScientific Reports 3, 1–5 (2013).
  9. The impact of damping in second-order dynamical systems with applications to power grid stability. \JournalTitleSIAM Journal on Applied Dynamical Systems 21, 405–437 (2022).
  10. Dynamically induced cascading failures in power grids. \JournalTitleNature Communications 9, 1–13 (2018).
  11. Saberi, M. et al. A simple contagion process describes spreading of traffic jams in urban networks. \JournalTitleNature Communications 11, 1–9 (2020).
  12. A network percolation-based contagion model of flood propagation and recession in urban road networks. \JournalTitleScientific Reports 10, 1–12 (2020).
  13. Design principles of collateral sensitivity-based dosing strategies. \JournalTitleNature Communications 12, 1–14 (2021).
  14. Butner, J. D. et al. A mathematical model for the quantification of a patient’s sensitivity to checkpoint inhibitors and long-term tumour burden. \JournalTitleNature Biomedical Engineering 5, 297–308 (2021).
  15. A general pharmacodynamic interaction model identifies perpetrators and victims in drug interactions. \JournalTitleNature communications 8, 1–11 (2017).
  16. Butcher, J. C. Numerical methods for ordinary differential equations (John Wiley and Sons, 2016).
  17. Ames, W. F. Numerical methods for partial differential equations (Academic press, 2014).
  18. Shampine, L. F. Numerical solution of ordinary differential equations (Routledge, 2018).
  19. QMBlender: Particle-based visualization of 3D quantum wave function dynamics. \JournalTitleJournal of Computational Science 35, 44–56 (2019).
  20. Xi, R. et al. Survey on smoothed particle hydrodynamics and the particle systems. \JournalTitleIEEE Access 8, 3087–3105 (2019).
  21. Cumuliform cloud animation control based on natural images. In 2020 International Conference on Virtual Reality and Visualization (ICVRV), 218–224 (IEEE, 2020).
  22. Ensemble methods for neural network-based weather forecasts. \JournalTitleJournal of Advances in Modeling Earth Systems 13 (2021).
  23. Characteristics of SARS-CoV-2 and COVID-19. \JournalTitleNature Reviews Microbiology 19, 141–154 (2021).
  24. Deep learning forecasting using time-varying parameters of the SIRD model for Covid-19. \JournalTitleScientific Reports 12, 1–13 (2022).
  25. A differential equations model-fitting analysis of COVID-19 epidemiological data to explain multi-wave dynamics. \JournalTitleScientific Reports 11, 1–13 (2021).
  26. Optimal strategies for social distancing and testing to control COVID-19. \JournalTitleJournal of Theoretical Biology 512, 110568 (2021).
  27. Hsiang, S. et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. \JournalTitleNature 584, 262–267 (2020).
  28. Progress of the COVID-19 vaccine effort: viruses, vaccines and variants versus efficacy, effectiveness and escape. \JournalTitleNature Reviews Immunology 21, 626–636 (2021).
  29. A wave-shaped deep neural network for smoke density estimation. \JournalTitleIEEE Transactions on Image Processing 29, 2301–2313 (2019).
  30. Data-driven particle-based liquid simulation with deep learning utilizing sub-pixel convolution. \JournalTitleProceedings of the ACM on Computer Graphics and Interactive Techniques 4, 1–16 (2021).
  31. Hardware-based simulation and collision detection for large particle systems. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware, 123–131 (2004).
  32. Using virtual reality technology to construct computer-aided animation material development. \JournalTitleComputer-Aided Design & Applications 19, 155–166 (2022).
  33. Towards reliable extreme weather and climate event attribution. \JournalTitleNature Communications 10, 1–7 (2019).
  34. Human-driven greenhouse gas and aerosol emissions cause distinct regional impacts on extreme fire weather. \JournalTitleNature Communications 12, 1–8 (2021).
  35. Singular vectors, predictability and ensemble forecasting for weather and climate. \JournalTitleJournal of Physics A: Mathematical and Theoretical 46, 254018 (2013).
  36. The ensemble approach to forecasting: a review and synthesis. \JournalTitleTransportation Research Part C: Emerging Technologies 132, 103357 (2021).
  37. A multifidelity ensemble kalman filter with reduced order control variates. \JournalTitleSIAM Journal on Scientific Computing 43, A1134–A1162 (2021).
  38. Graphics processing unit (GPU) programming strategies and trends in GPU computing. \JournalTitleJournal of Parallel and Distributed Computing 73, 4–13 (2013).
  39. Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th annual international symposium on computer architecture, 1–12 (2017).
  40. FastVA: deep learning video analytics through edge processing and NPU in mobile. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications, 1947–1956 (IEEE, 2020).
  41. A high-performance and low-power 32-bit multiply-accumulate unit with single-instruction-multiple-data (SIMD) feature. \JournalTitleIEEE Journal of Solid-State Circuits 37, 926–931 (2002).
  42. Fehlberg, E. Low-order classical Runge–Kutta formulas with stepsize control and their application to some heat transfer problems, vol. 315 (National Aeronautics and Space Administration, 1969).
  43. Neural ordinary differential equations. \JournalTitleAdvances in Neural Information Processing Systems 31 (2018).
  44. Stiffness-aware neural network for learning Hamiltonian systems. In International Conference on Learning Representations (2022).
  45. Li, Z. et al. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations (2021).
  46. Long-time predictive modeling of nonlinear dynamical systems using neural networks. \JournalTitleComplexity 2018, 1–26 (2018).
  47. Predicting parametric spatiotemporal dynamics by multi-resolution pde structure-preserved deep learning. \JournalTitlearXiv preprint arXiv:2205.03990 (2022).
  48. On generalized residual network for deep learning of unknown dynamical systems. \JournalTitleJournal of Computational Physics 438, 110362 (2021).
  49. Dresdner, G. et al. Learning to correct spectral methods for simulating turbulent flows. \JournalTitlearXiv preprint arXiv:2207.00556 (2022).
  50. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. \JournalTitleNature Machine Intelligence 3, 218–229 (2021).
  51. Choudhary, A. et al. Physics-enhanced neural networks learn order and chaos. \JournalTitlePhysical Review E 101, 062207 (2020).
  52. Adaptable Hamiltonian neural networks. \JournalTitlePhysical Review Research 3, 023156 (2021).
  53. Hamiltonian neural networks. \JournalTitleAdvances in Neural Information Processing Systems 32 (2019).
  54. The adaptive Verlet method. \JournalTitleSIAM Journal on Scientific Computing 18, 239–256 (1997).
  55. Stable climate simulations using a realistic general circulation model with neural network parameterizations for atmospheric moist physics and radiation processes. \JournalTitleGeoscientific Model Development 15, 3923–3940 (2022).
  56. Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. \JournalTitleAdvances in neural information processing systems 32 (2019).
  57. Abadi, M. et al. Tensorflow: A system for large-scale machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16), 265–283 (2016).
  58. Rational neural networks. \JournalTitleAdvances in Neural Information Processing Systems 33, 14243–14253 (2020).
  59. Symplectic recurrent neural networks. In International Conference on Learning Representations (2020).
  60. Wave packet dynamics and chaos in the hénnon–heiles system. \JournalTitleThe Journal of Chemical Physics 80, 2578–2584 (1984).
  61. Sparse symplectically integrated neural networks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, 6074–6085 (Curran Associates, Inc., 2020).
  62. The elastic pendulum: a nonlinear paradigm. \JournalTitleJournal of Mathematical Physics 22, 1196–1210 (1981).
  63. Dynamics of the N-link pendulum: a fractional perspective. \JournalTitleInternational Journal of Control 90, 1192–1200 (2017).
  64. Numerical solution of ordinary differential equations (John Wiley and Sons, 2011).
  65. Gradient descent provably optimizes over-parameterized neural networks. In International Conference on Learning Representations (2019).
  66. Neural tangent kernel: Convergence and generalization in neural networks. \JournalTitleAdvances in Neural Information Processing Systems 31 (2018).
  67. On lazy training in differentiable programming. \JournalTitleAdvances in Neural Information Processing Systems 32 (2019).
  68. The expressive power of neural networks: A view from the width. \JournalTitleAdvances in neural information processing systems 30 (2017).
  69. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7132–7141 (2018).
  70. Dianet: Dense-and-implicit attention network. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, 4206–4214 (2020).
  71. Instance enhancement batch normalization: An adaptive regulator of batch noise. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, 4819–4827 (2020).
  72. Blending pruning criteria for convolutional neural networks. In International Conference on Artificial Neural Networks, 3–15 (Springer, 2021).
  73. Darts: Differentiable architecture search. In International Conference on Learning Representations (2018).
  74. Rethinking the pruning criteria for convolutional neural network. \JournalTitleAdvances in Neural Information Processing Systems 34, 16305–16318 (2021).
  75. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, 4171–4186 (2019).
  76. Wu, Y. et al. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 374–382 (2019).
  77. On robust numerical solver for ode via self-attention mechanism. \JournalTitlearXiv preprint arXiv:2302.10184 (2023).
  78. An introduction to numerical analysis (Cambridge University Press, 2003).
  79. Trench, W. F. Elementary differential equations with boundary value problems (Brooks Cole Thomson Learning, 2013).
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now