Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

From Zero to Turbulence: Generative Modeling for 3D Flow Simulation (2306.01776v3)

Published 29 May 2023 in physics.flu-dyn and cs.LG

Abstract: Simulations of turbulent flows in 3D are one of the most expensive simulations in computational fluid dynamics (CFD). Many works have been written on surrogate models to replace numerical solvers for fluid flows with faster, learned, autoregressive models. However, the intricacies of turbulence in three dimensions necessitate training these models with very small time steps, while generating realistic flow states requires either long roll-outs with many steps and significant error accumulation or starting from a known, realistic flow state - something we aimed to avoid in the first place. Instead, we propose to approach turbulent flow simulation as a generative task directly learning the manifold of all possible turbulent flow states without relying on any initial flow state. For our experiments, we introduce a challenging 3D turbulence dataset of high-resolution flows and detailed vortex structures caused by various objects and derive two novel sample evaluation metrics for turbulent flows. On this dataset, we show that our generative model captures the distribution of turbulent flows caused by unseen objects and generates high-quality, realistic samples amenable for downstream applications without access to any initial state.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. Radovan Bast. Numgrid: Numerical integration grid for molecules, January 2021.
  2. Neural Flows: Efficient Alternative to Neural ODEs. In Neural Information Processing Systems, 2021. doi: 10.48550/arXiv.2110.13040.
  3. AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier-Stokes Solutions. In Neural Information Processing Systems, 2022. doi: 10.48550/arXiv.2212.07564.
  4. Message Passing Neural PDE Solvers. In International Conference on Learning Representations, 2022.
  5. John Burkardt. SPHERE_LEBEDEV_RULE: Quadrature rules for the unit sphere, 2010.
  6. Is vortex stretching the main cause of the turbulent energy cascade? Journal of Fluid Mechanics, 883:R2, January 2020. ISSN 0022-1120, 1469-7645. doi: 10.1017/jfm.2019.923.
  7. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. In Sebastien Ourselin, Leo Joskowicz, Mert R. Sabuncu, Gozde Unal, and William Wells (eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016, Lecture Notes in Computer Science, pp.  424–432, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46723-8. doi: 10.1007/978-3-319-46723-8_49.
  8. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. In Neural Information Processing Systems, 2022. doi: 10.48550/arXiv.2205.14135.
  9. Generative Modeling of Turbulence. Physics of Fluids, 34(3):035114, March 2022. ISSN 1070-6631, 1089-7666. doi: 10.1063/5.0082562.
  10. Generalization capabilities of conditional GAN for turbulent flow under changes of geometry, February 2023.
  11. Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data. In Proceedings of the 37th International Conference on Machine Learning, pp.  3165–3176. PMLR, November 2020.
  12. POT: Python optimal transport. Journal of Machine Learning Research, 22(78):1–8, 2021.
  13. B. Galanti and A. Tsinober. Is turbulence ergodic? Physics Letters A, 330(3):173–180, September 2004. ISSN 0375-9601. doi: 10.1016/j.physleta.2004.07.009.
  14. GemNet: Universal Directional Graph Neural Networks for Molecules. In Neural Information Processing Systems, volume 34, pp. 6790–6802. Curran Associates, Inc., 2021.
  15. Enabling rootless Linux Containers in multi-user environments: The udocker tool. Computer Physics Communications, 232:84–97, November 2018. ISSN 00104655. doi: 10.1016/j.cpc.2018.05.021.
  16. Towards Multi-spatiotemporal-scale Generalized PDE Modeling, September 2022.
  17. Array programming with NumPy. Nature, 585(7825):357–362, September 2020. doi: 10.1038/s41586-020-2649-2.
  18. Deep Residual Learning for Image Recognition, December 2015.
  19. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Neural Information Processing Systems, 2017.
  20. Denoising Diffusion Probabilistic Models. In Neural Information Processing Systems, 2020. doi: 10.48550/arXiv.2006.11239.
  21. Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditions. In Neural Information Processing Systems, 2022.
  22. J. D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3):90–95, 2007. doi: 10.1109/MCSE.2007.55.
  23. EAGLE: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformers. In International Conference on Learning Representations, 2023. doi: 10.48550/arXiv.2302.10803.
  24. J. Jiménez. Intermittency in Turbulence. In Jean-Pierre Françoise, Gregory L. Naber, and Tsou Sheung Tsun (eds.), Encyclopedia of Mathematical Physics, pp.  144–151. Academic Press, Oxford, January 2006. ISBN 978-0-12-512666-3. doi: 10.1016/B0-12-512666-2/00368-0.
  25. Perry L. Johnson. On the role of vorticity stretching and strain self-amplification in the turbulence energy cascade. Journal of Fluid Mechanics, 922:A3, September 2021. ISSN 0022-1120, 1469-7645. doi: 10.1017/jfm.2021.490.
  26. Deep Fluids: A Generative Network for Parameterized Fluid Simulations. Computer Graphics Forum, 38(2):59–70, 2019. ISSN 1467-8659. doi: 10.1111/cgf.13619.
  27. Variational Diffusion Models. In Neural Information Processing Systems, 2021.
  28. Jupyter Notebooks - a publishing format for reproducible computational workflows. In Fernando Loizides and Birgit Scmidt (eds.), Positioning and Power in Academic Publishing: Players, Agents and Agendas, pp.  87–90, Netherlands, 2016. IOS Press.
  29. Machine learning–accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21), 2021. ISSN 0027-8424, 1091-6490. doi: 10.1073/pnas.2101784118.
  30. Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting. In Neural Information Processing Systems. arXiv, 2023. doi: 10.48550/arXiv.2307.11494.
  31. V. Lebedev and D. Laikov. A Quadrature Formula for the Sphere of the 131st Algebraic Order of Accuracy. Doklady Mathematics, 1999.
  32. A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence. Journal of Turbulence, 9:N31, January 2008. doi: 10.1080/14685240802376389.
  33. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on Learning Representations. arXiv, 2021. doi: 10.48550/arXiv.2010.08895.
  34. Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks. In International Conference on Learning Representations, 2022.
  35. Time-resolved evolution of coherent structures in turbulent channels: Characterization of eddies and cascades. Journal of Fluid Mechanics, 759:432–471, November 2014. ISSN 0022-1120, 1469-7645. doi: 10.1017/jfm.2014.575.
  36. Add and Thin: Diffusion for Temporal Point Processes. In Neural Information Processing Systems. arXiv, 2023. doi: 10.48550/arXiv.2311.01139.
  37. RePaint: Inpainting using Denoising Diffusion Probabilistic Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  38. An Introduction to Turbulent Flow. Cambridge University Press, Cambridge, 2000. doi: 10.1017/CBO9781316529850.
  39. Supervised convolutional network for three-dimensional fluid data reconstruction from sectional flow fields with adaptive super-resolution assistance, March 2021.
  40. Improved Denoising Diffusion Probabilistic Models. In International Conference on Machine Learning, 2021. doi: 10.48550/arXiv.2102.09672.
  41. CFDNet: A deep learning-based accelerator for fluid simulations. In International Conference on Supercomputing, pp. 1–12, June 2020. doi: 10.1145/3392717.3392772.
  42. An Extensible Benchmark Suite for Learning to Simulate Physical Systems. In Neural Information Processing Systems, 2021. doi: 10.48550/arXiv.2108.07799.
  43. Nicholas T. Ouellette. Turbulence in two dimensions. Physics Today, 65(5):68–69, May 2012. ISSN 0031-9228. doi: 10.1063/PT.3.1570.
  44. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  165–174, 2019.
  45. PyTorch: An Imperative Style, High-Performance Deep Learning Library, December 2019.
  46. Learning Mesh-Based Simulation with Graph Networks. In International Conference on Learning Representations, number arXiv:2010.03409, 2021.
  47. Stephen B. Pope. Turbulent Flows. Cambridge University Press, 2000.
  48. Fr\’echet ChemNet Distance: A metric for generative models for molecules in drug discovery, August 2018.
  49. Universal Differential Equations for Scientific Machine Learning, November 2021.
  50. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019. ISSN 00219991. doi: 10.1016/j.jcp.2018.10.045.
  51. Acceleration of a Full-Scale Industrial CFD Application with OP2. IEEE Transactions on Parallel and Distributed Systems, 27(5):1265–1278, May 2016. ISSN 1558-2183. doi: 10.1109/TPDS.2015.2453972.
  52. Alex Rogozhnikov. Einops: Clear and reliable tensor manipulations with einstein-like notation. In International Conference on Learning Representations, 2022.
  53. High-Resolution Image Synthesis with Latent Diffusion Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  10674–10685, June 2022. doi: 10.1109/CVPR52688.2022.01042.
  54. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (eds.), Medical Image Computing and Computer-Assisted Intervention, Lecture Notes in Computer Science, pp.  234–241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24574-4. doi: 10.1007/978-3-319-24574-4_28.
  55. E(n) Equivariant Graph Neural Networks. In Proceedings of the 38th International Conference on Machine Learning, pp.  9323–9332. PMLR, July 2021.
  56. A physics-informed diffusion model for high-fidelity flow field reconstruction. Journal of Computational Physics, 478, April 2023. ISSN 0021-9991. doi: 10.1016/j.jcp.2023.111972.
  57. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In International Conference on Machine Learning, 2015.
  58. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, January 2021.
  59. Consistency Models, March 2023.
  60. Learned Coarse Models for Efficient Turbulence Simulation. In International Conference on Learning Representations, 2022.
  61. PyVista: 3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK). Journal of Open Source Software, 4(37):1450, May 2019. doi: 10.21105/joss.01450.
  62. PDEBench: An Extensive Benchmark for Scientific Machine Learning. In Neural Information Processing Systems, 2022.
  63. Towards Accurate Generative Models of Video: A New Metric & Challenges, March 2019.
  64. Attention Is All You Need. In Neural Information Processing Systems, 2017.
  65. SciPy 1.0: Fundamental algorithms for scientific computing in python. Nature Methods, 17:261–272, 2020. doi: 10.1038/s41592-019-0686-2.
  66. Towards Physics-informed Deep Learning for Turbulent Flow Prediction. In SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020.
  67. 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
  68. A tensorial approach to computational continuum mechanics using object-oriented techniques. Computer in Physics, 12(6):620–631, November 1998. ISSN 0894-1866. doi: 10.1063/1.168744.
  69. Group Normalization. International Journal of Computer Vision, 128(3):742–755, March 2020. ISSN 1573-1405. doi: 10.1007/s11263-019-01198-w.
  70. tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow. ACM Transactions on Graphics, 36(4):1–14, August 2017. ISSN 0730-0301, 1557-7368. doi: 10.1145/3072959.3073643.
  71. On Layer Normalization in the Transformer Architecture. In International Conference on Machine Learning, 2020. doi: 10.48550/arXiv.2002.04745.
  72. Omry Yadan. Hydra - A framework for elegantly configuring complex applications. Github, 2019.
  73. A Denoising Diffusion Model for Fluid Field Prediction, January 2023.
  74. Learning to Solve PDE-constrained Inverse Problems with Graph Networks. In International Conference on Machine Learning, pp. 26895–26910. PMLR, June 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Marten Lienen (10 papers)
  2. David Lüdke (7 papers)
  3. Jan Hansen-Palmus (2 papers)
  4. Stephan Günnemann (169 papers)
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now