Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring Physical Latent Spaces for High-Resolution Flow Restoration (2211.11298v2)

Published 21 Nov 2022 in cs.LG

Abstract: We explore training deep neural network models in conjunction with physics simulations via partial differential equations (PDEs), using the simulated degrees of freedom as latent space for a neural network. In contrast to previous work, this paper treats the degrees of freedom of the simulated space purely as tools to be used by the neural network. We demonstrate this concept for learning reduced representations, as it is extremely challenging to faithfully preserve correct solutions over long time-spans with traditional reduced representations, particularly for solutions with large amounts of small scale features. This work focuses on the use of such physical, reduced latent space for the restoration of fine simulations, by training models that can modify the content of the reduced physical states as much as needed to best satisfy the learning objective. This autonomy allows the neural networks to discover alternate dynamics that significantly improve the performance in the given tasks. We demonstrate this concept for various fluid flows ranging from different turbulence scenarios to rising smoke plumes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. “OptNet: Differentiable Optimization as a Layer in Neural Networks” In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17 Sydney, NSW, Australia: JMLR.org, 2017, pp. 136–145
  2. “Learning data-driven discretizations for partial differential equations” In Proceedings of the National Academy of Sciences 116.31 National Acad Sciences, 2019, pp. 15344–15349
  3. “Model Reduction and Neural Networks for Parametric PDEs” In The SMAI journal of computational mathematics 7, 2021, pp. 121–157 DOI: 10.5802/smai-jcm.74
  4. “Dynamic Upsampling of Smoke through Dictionary-Based Learning” In ACM Transactions on Graphics 40.1, 2020, pp. 4:1–4:19 DOI: 10.1145/3412360
  5. Steven L Brunton, Joshua L Proctor and J Nathan Kutz “Discovering governing equations from data by sparse identification of nonlinear dynamical systems” In Proceedings of the National Academy of Sciences 113.15 National Acad Sciences, 2016, pp. 3932–3937
  6. Johannes Brandstetter, Daniel E. Worrall and Max Welling “Message Passing Neural PDE Solvers” In International Conference on Learning Representations, 2022 URL: https://openreview.net/forum?id=vSix3HPYKSU
  7. Alexandre Joel Chorin “The numerical solution of the Navier-Stokes equations for an incompressible fluid” In Bulletin of the American Mathematical Society 73.6, 1967, pp. 928–931
  8. “Data-Driven Discovery of Coordinates and Governing Equations” In Proceedings of the National Academy of Sciences 116.45 National Academy of Sciences, 2019, pp. 22445–22451 DOI: 10.1073/pnas.1906995116
  9. James P Crutchfield and Bruce S McNamara “Equations of motion from a data series” In Complex systems 1.417-452, 1987, pp. 121
  10. “Neural ordinary differential equations” In Advances in neural information processing systems, 2018, pp. 6571–6583
  11. “Data-Driven Synthesis of Smoke Flows with CNN-Based Feature Descriptors” In ACM Trans. Graph. 36.4, 2017, pp. 69:1–69:14 DOI: 10.1145/3072959.3073643
  12. “End-to-End Differentiable Physics for Learning and Control” In Advances in Neural Information Processing Systems 31 Curran Associates, Inc., 2018 URL: https://proceedings.neurips.cc/paper/2018/file/842424a1d0595b76ec4fa03c46e8d755-Paper.pdf
  13. Kai Fukami, Koji Fukagata and Kunihiko Taira “Super-resolution reconstruction of turbulent flows with machine learning” In Journal of Fluid Mechanics 870 Cambridge University Press, 2019, pp. 106–120
  14. “Sparse Identification of Nonlinear Dynamics with Low-Dimensionalized Flow Representations” In Journal of Fluid Mechanics 926 Cambridge University Press, 2021 DOI: 10.1017/jfm.2021.697
  15. “DiffTaichi: Differentiable Programming for Physical Simulation” In International Conference on Learning Representations (ICLR), 2020
  16. Philipp Holl, Vladlen Koltun and Nils Thuerey “Learning to Control PDEs with Differentiable Physics” In International Conference on Learning Representations (ICLR), 2020
  17. “A differentiable programming system to bridge machine learning and scientific computing” In arXiv 1907.07587, 2019
  18. “Deep Fluids: A Generative Network for Parameterized Fluid Simulations” In Computer Graphics Forum The Eurographics AssociationJohn Wiley & Sons Ltd., 2019 DOI: 10.1111/cgf.13619
  19. “Adam: A Method for Stochastic Optimization” In arXiv:1412.6980 [cs], 2014 arXiv:1412.6980 [cs]
  20. “Equation-free, coarse-grained multiscale computation: Enabling mocroscopic simulators to perform system-level analysis” In Communications in Mathematical Sciences 1.4 International Press of Boston, 2003, pp. 715–762
  21. “Machine Learning–Accelerated Computational Fluid Dynamics” In Proceedings of the National Academy of Sciences 118.21 National Academy of Sciences, 2021 DOI: 10.1073/pnas.2101784118
  22. “Data-Driven Fluid Simulations Using Regression Forests” In ACM Trans. Graph. 34.6, 2015, pp. 199:1–199:9 DOI: 10.1145/2816795.2818129
  23. “Fourier Neural Operator for Parametric Partial Differential Equations” In International Conference on Learning Representations, 2021 URL: https://openreview.net/forum?id=c8P9NQVtmnO
  24. Bethany Lusch, J.Nathan Kutz and Steven L. Brunton “Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics” In Nature Communications 9.1 Nature Publishing Group, 2018, pp. 4950 DOI: 10.1038/s41467-018-07210-0
  25. Junbang Liang, Ming Lin and Vladlen Koltun “Differentiable Cloth Simulation for Inverse Problems” In Advances in Neural Information Processing Systems, 2019, pp. 771–780
  26. “Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence” In arXiv:1903.00033, 2019
  27. “Deep dynamical modeling and control of unsteady fluid flows” In Advances in Neural Information Processing Systems, 2018
  28. Romit Maulik, Bethany Lusch and Prasanna Balaprakash “Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders” In Physics of Fluids 33.3 American Institute of Physics, 2021, pp. 037106 DOI: 10.1063/5.0039986
  29. Young Jin Oh and In-Kwon Lee “Two-step Temporal Interpolation Network Using Forward Advection for Efficient Smoke Simulation” In Computer Graphics Forum 40.2, 2021, pp. 355–365 DOI: 10.1111/cgf.142638
  30. Samuel S Schoenholz and Ekin D Cubuk “JAX, MD: End-to-End Differentiable, Hardware Accelerated, Molecular Dynamics in Pure Python” In arXiv:1912.04232, 2019
  31. “SPNets: Differentiable Fluid Dynamics for Deep Neural Networks” In Conference on Robot Learning, 2018, pp. 317–335
  32. “Learned Simulators for Turbulence” In International Conference on Learning Representations, 2022 URL: https://openreview.net/forum?id=msRBojTz-Nh
  33. Justin Sirignano, Jonathan F. MacArt and Jonathan B. Freund “DPM: A Deep Learning PDE Augmentation Method with Application to Large-Eddy Simulation” In Journal of Computational Physics 423, 2020, pp. 109811 DOI: 10.1016/j.jcp.2020.109811
  34. Jos Stam “Stable Fluids” In SIGGRAPH ’99 ACM, 1999, pp. 121–128 DOI: 10.1145/311535.311548
  35. “Differentiable physics and stable modes for tool-use and manipulation planning” In Robotics: Science and Systems, 2018
  36. “Physics-based Deep Learning” In arXiv:2109.05237 [physics], 2021 arXiv:2109.05237 [physics]
  37. “Accelerating Eulerian Fluid Simulation With Convolutional Networks” In Proceedings of Machine Learning Research, 2017, pp. 3424–3433
  38. “Solver-in-the-loop: learning from differentiable physics to interact with iterative PDE-solvers” In Advances in Neural Information Processing Systems 33, 2020 arXiv:2007.00016
  39. Kiwon Um, Xiangyu Hu and Nils Thuerey “Liquid Splash Modeling with Neural Networks” In Computer Graphics Forum 37.8, 2018, pp. 171–182 DOI: 10.1111/cgf.13522
  40. Wujie Wang, Simon Axelrod and Rafael Gómez-Bombarelli “Differentiable Molecular Simulations for Control and Learning” In arXiv:2003.00868, 2020
  41. “Latent Space Subdivision: Stable and Controllable Time Predictions for Fluid Flow” In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’20 Goslar, DEU: Eurographics Association, 2020, pp. 1–11 DOI: 10.1111/cgf.14097
  42. “Towards Physics-Informed Deep Learning for Turbulent Flow Prediction” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20 New York, NY, USA: Association for Computing Machinery, 2020, pp. 1457–1466 DOI: 10.1145/3394486.3403198
  43. “tempogan: A temporally coherent, volumetric gan for super-resolution fluid flow” In ACM Transactions on Graphics (TOG) 37.4 ACM, 2018, pp. 1–15
  44. Xiangyun Xiao, Cheng Yang and Xubo Yang “Adaptive Learning-Based Projection Method for Smoke Simulation” In Computer Animation and Virtual Worlds 29.3-4, 2018, pp. e1837 DOI: 10.1002/cav.1837
  45. “Learned Discretizations for Passive Scalar Advection in a Two-Dimensional Turbulent Flow” In Physical Review Fluids 6.6 American Physical Society, 2021, pp. 064605 DOI: 10.1103/PhysRevFluids.6.064605

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now