CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics (2310.05963v2)

Abstract: In recent years, applying deep learning to solve physics problems has attracted much attention. Data-driven deep learning methods produce fast numerical operators that can learn approximate solutions to the whole system of partial differential equations (i.e., surrogate modeling). Although these neural networks may have lower accuracy than traditional numerical methods, they, once trained, are orders of magnitude faster at inference. Hence, one crucial feature is that these operators can generalize to unseen PDE parameters without expensive re-training.In this paper, we construct CFDBench, a benchmark tailored for evaluating the generalization ability of neural operators after training in computational fluid dynamics (CFD) problems. It features four classic CFD problems: lid-driven cavity flow, laminar boundary layer flow in circular tubes, dam flows through the steps, and periodic Karman vortex street. The data contains a total of 302K frames of velocity and pressure fields, involving 739 cases with different operating condition parameters, generated with numerical methods. We evaluate the effectiveness of popular neural operators including feed-forward networks, DeepONet, FNO, U-Net, etc. on CFDBnech by predicting flows with non-periodic boundary conditions, fluid properties, and flow domain shapes that are not seen during training. Appropriate modifications were made to apply popular deep neural networks to CFDBench and enable the accommodation of more changing inputs. Empirical results on CFDBench show many baseline models have errors as high as 300% in some problems, and severe error accumulation when performing autoregressive inference. CFDBench facilitates a more comprehensive comparison between different neural operators for CFD compared to existing benchmarks.

• The paper introduces CFDBench, featuring over 302,000 CFD frames from four canonical flow problems to test neural network generalization.
• It evaluates various architectures, including U-Net and FNO, in both autoregressive and non-autoregressive settings to address real-world CFD complexities.
• The study highlights the importance of matching model designs with CFD problem characteristics to achieve faster, more reliable inference.

Analysis of CFDBench: A Benchmark for Evaluating Neural Operators in Fluid Dynamics

The paper "CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics" presents CFDBench, a benchmark designed to evaluate the generalization ability of neural networks in computational fluid dynamics (CFD). The benchmark includes data from four canonical CFD problems: lid-driven cavity flow, laminar boundary layer flow in circular tubes, dam flows over obstacles, and periodic Karman vortex streets. The dataset contains over 302,000 frames of velocity and pressure fields, sampled across different parameters like boundary conditions, fluid properties, and geometries, which are critical for testing a model’s ability to generalize to unseen fluid dynamics scenarios.

Context and Motivation

The motivation behind creating CFDBench stems from the unique challenges posed by traditional numerical methods employed in solving CFD problems, including high computational costs and complex geometry handling requirements. Although deep learning approaches lack certain aspects of accuracy compared to traditional methods, they offer significant time advantages during inference. The construction of CFDBench enables the examination of neural operators' generalization without the need for retraining, a critical step towards practical adoption in industry applications.

Methods Evaluated

The paper evaluates several well-known architectures in the context of CFD:

• Feed-Forward Networks (FFNs) and variants in both autoregressive and non-autoregressive settings
• DeepONet and enhancements such as Auto-EDeepONet
• ResNet and U-Net, well-established architectures in image processing
• FNO (Fourier Neural Operator) known for leveraging frequency domain transformations

The methods varied in terms of how they incorporate operating parameters in problem settings, such as boundary conditions and fluid properties, directly influencing model performance and ease of adaptation.

Key Findings

A notable insight from the experiments was the importance of matching model architectures with the intrinsic nature of the CFD problems. For instance, U-Net showed superior performance on tasks without source terms such as gravity, while FNO excelled in contexts involving periodic eddy currents, owing to its frequency domain capabilities. The results highlighted a general inadequacy of current data-driven methods to completely replace traditional methods, given their limitations in generalizing to complex and dynamically varying conditions.

Implications and Future Work

CFDBench represents a significant stride towards standardized evaluation protocols for applying deep learning to CFD. Its design facilitates comparative analysis across varying conditions, thus advancing the field's understanding of model behaviors in unseen physics-driven scenarios. As future research delves deeper into developing neural architectures that enhance generalization capabilities, CFDBench can serve as a critical resource for benchmarking such advancements.

Practical implications of the paper extend to various industries relying on fluid dynamics simulations. Improved methods of simulation—especially those leveraging faster deep learning inferences—can lead to more efficient design processes in areas like aerodynamics, meteorology, and hydraulics.

Conclusion

The introduction of CFDBench is positioned to have a significant impact on accelerating the adoption of machine learning methods in CFD research. However, the notable discrepancies in error rates across different fluid dynamics scenarios underscore the necessity for continued research and the development of novel architectures capable of capturing the complex dynamics present in real-world applications. The observations made offer foundational insights into which architectures are currently best suited to various fluid dynamics challenges and pave the way for the continuous improvement of machine learning models in physics-based simulations. CFDBench, with its emphasis on evaluating generalization to diverse, unseen conditions, establishes a solid framework for future explorations in this increasingly critical intersection of fluid dynamics and machine learning.