Neural Network Subgrid Stress Models

• Neural network subgrid stress models are data-driven closures for LES turbulence that use deep learning to map resolved variables to unclosed subgrid stresses from high-fidelity simulations.
• They integrate advanced architectures—such as TBNNs, CNNs, and multiscale U-Nets—with invariant and multiscale feature engineering to improve regression accuracy and simulation fidelity.
• Robust training protocols involving data augmentation and careful normalization enhance generalization, while challenges remain in ensuring stability and tackling high-Re turbulent regimes.

Neural network subgrid stress models constitute a significant class of data-driven closures for large eddy simulation (LES) of turbulence and related multiscale flow systems. These models employ machine learning—predominantly deep neural networks—to learn the mapping between resolved-scale variables and unclosed subgrid-scale (SGS) stresses directly from high-fidelity data, typically filtered direct numerical simulation (DNS) or high-resolution LES. This approach departs from traditional algebraic or phenomenological closures by optimizing expressive non-linear function approximators to regress the full tensorial SGS stress or related closures, with increasing attention to embedding invariance, multiscale structure, and generalization capacity. The resulting models have demonstrated promise for both a priori regression accuracy and a posteriori flow fidelity, though their deployment in simulation imposes additional constraints related to stability, robustness, and interpretability.

1. Mathematical Formulation of Neural Network Subgrid Stress Models

The foundational construct is the filtered Navier–Stokes equation, where the explicit spatial filtering of velocity and pressure fields introduces the unclosed subgrid stress tensor

$$\tau_{ij}(x, t) = \overline{u_i u_j}(x, t) - \bar{u}_i(x, t)\,\bar{u}_j(x, t),$$

with $\overline{\cdot}$ denoting convolution with a filter kernel $G_\Delta$ of width $\Delta$. The LES momentum equation, with $\tau_{ij}$ appearing under divergence, requires closure at the grid scale.

Neural SGS models seek to approximate $\tau_{ij}$ via a learned function of resolved variables—typically velocity gradients and tensorial invariants: $$\tau_{ij}^{\mathrm{NN}}(x) = \mathcal{F}_\theta\left(\mathcal{I}\left[\bar{\mathbf{u}}(x), \nabla \bar{\mathbf{u}}(x), \Delta, \ldots \right]\right),$$ where $\mathcal{I}[\cdot]$ denotes an input feature vector of chosen invariants (e.g., strain and rotation rates, principal invariants of $\nabla \bar{\mathbf{u}}$, or physical/coordinate invariants for specialized applications), and $\mathcal{F}_\theta$ is a neural network parameterized by weights $\theta$.

Tensor Basis Neural Networks (TBNN) encode tensor invariance by expressing $\tau_{ij}$ as a contraction over a fixed tensor basis $\{T_{ij}^{(n)}\}$ and NN-predicted scalar coefficients $c_n$: $$\tau_{ij} = \sum_{n=1}^8 c_n(\mathcal{I})\, T_{ij}^{(n)}(S, \Omega),$$ where $S$ and $\Omega$ are the symmetric and antisymmetric parts of the velocity gradient, and the basis spans Cayley–Smith polynomials built from $S$ and $\Omega$ (Wu et al., 21 Nov 2025).

Normalization of both inputs (global, via empirical min–max scaling) and outputs (local, with characteristic velocity/length scales) is crucial to attain grid- and Reynolds-number invariance.

2. Core Network Architectures and Feature Engineering

A diversity of neural architectures has been applied, including fully connected multilayer perceptrons (MLPs), convolutional neural networks (CNNs), hybrid TBNN–GNNs, multiscale U-Nets, and sequence models (S4ND). Key design tradeoffs include input complexity, spatial locality, inductive bias (embedding invariance, multiscale structure), and computational budget.

Typical feature sets:

• Local, invariant features: Scalar invariants of the velocity gradient tensor ($P$, $Q$, $R$, $|S|$, $|\Omega|$) and/or their norms (Wu et al., 21 Nov 2025, Prakash et al., 2021).
• Stencil or nonlocal input: Local neighborhoods in the grid for nonlocal models, as in CNNs applied to 2D/3D slices (Liu et al., 2021).
• Multiscale stacks: Simultaneous inputs of filtered velocity/strain-rate at multiple filter widths to enable scale-aware closure (Jalaali et al., 15 Feb 2025, Hasan et al., 20 Apr 2025).
• Geometric invariants: Singular values of stencil-invariant matrices to encode physical and geometric invariance (Hasan et al., 20 Apr 2025).
• Complete gradient and Hessian features for architectures trained on gradient/Hessian fields, especially in studies prioritizing completeness over invariance (MacArt et al., 2021).

Representative architectures:

• TBNN–MLP hybrids: 4-layer GNN/MLP (width 32) for $c_n$ coefficients, coupled to Cayley–Smith basis (Wu et al., 21 Nov 2025).
• S4ND U-Net: State-space sequence models parameterizing multidimensional convolutions, facilitating grid-size extrapolation and continuous convolution kernels for different grid spacings; S4ND block hidden size $N=8$, four downsampling/upsampling levels (Wu et al., 14 Nov 2025).
• Nonlocal 5-layer CNN (ResNet-style): ELU activations, skip connections, with receptive field tuned via two-point correlation analysis (Liu et al., 2021).
• MSC-SGS: Multiscale convolutional network encoding an energy-cascade-like hierarchy of filtered strain-rate inputs (Jalaali et al., 15 Feb 2025).
• Recursive dual-NN: Two separate networks for normal and shear components, trained and updated recursively through filtered LES data augmentation (Cho et al., 2023).
• Invariant single-layer MLP (20 neurons): Explicit S-frame decomposition with non-dimensionalization, yielding robust minimal-invariant closures for anisotropic and isotropic filters (Prakash et al., 2022, Prakash et al., 2021).

3. Training Protocols, Data Augmentation, and Regularization

The fidelity and robustness of neural SGS models are determined not simply by network expressivity but by representation and coverage of training data, normalization, and loss function design.

Data augmentation:

• Two-filter strategy: Jointly augmenting training data with explicit filtering via two distinct kernels (e.g., spatial box filter and 2/3 dealiasing or discrete spectral cutoff) dramatically improves a posteriori robustness, collapses posterior spectral variance, and ensures insensitivity to architecture tweaks (Wu et al., 21 Nov 2025).
• Multiscale/recursive augmentation: Training recursively with outputs from successively coarser filtered LES, using filtered-LES (“fLES”) data as new training, enables models to span wide ranges of Reynolds numbers and grid spacings without repeated DNS (Cho et al., 2023).
• Weighted sampling: Stratifying/reshaping the PDF of target stresses to accentuate rare but dynamically important large-magnitude events improves regression accuracy at extremes (Miyazaki et al., 2020, Cho et al., 2023).

Normalization:

• Global min–max or standardization for all scalar/tensorial inputs.
• Output normalization with dimensionless groups, e.g. scaling $\tau_{ij}$ by $\Delta^2|V|^2$ or the norm of the gradient model, to achieve scale-awareness and Re-independence (Wu et al., 21 Nov 2025, Prakash et al., 2022, Kang et al., 2022).

Loss function:

• Composite RMSE on SGS stresses and SGS-dissipation $\epsilon=-\tau_{ij} S_{ij}$ to ensure both structural and energetic performance in regression.
• Mean-squared error (MSE) on non-dimensionalized SGS stresses for invariant and physics-aware models.
• Binary cross-entropy (BCE) and ensemble MSE for two-stage architectures embedding classification and regression of signed Smagorinsky coefficients (Hasan et al., 20 Apr 2025).

4. A Priori and A Posteriori Evaluation Methodologies

Rigorous assessment of neural SGS models requires both regression metrics and simulation-based diagnostics:

A priori (offline, regression) metrics:

• Root-mean-square error (RMSE) and Pearson correlation $$\rho(\tau_{ij}^{\mathrm{pred}},\, \tau_{ij}^{\mathrm{DNS}}$$).
• Normalized error in energy dissipation and structural correlation coefficients for SGS-dissipation.
• PDFs and statistics of backscatter, dissipation, and higher-order SGS moments.

A posteriori (online, coupled LES) metrics:

• Energy and spectra: Sum-spectral error

$$\mathrm{SSE} = \sum_k |\log E_{\mathrm{DNS}}(k) - \log E_{\mathrm{LES}}(k)|$$

• Mean velocity profile and RMS fluctuations in wall-bounded flows.
• Stationarity, stability, and robustness across different LES solvers and numerical schemes.
• Generalization across grid spacings, Reynolds numbers, and underlying flow classes without retraining (Wu et al., 14 Nov 2025, Prakash et al., 2022, Cho et al., 2023).
• Visualization of spectral rolloff, pile-up, and inertial-range fidelity.

Critically, a priori performance does not reliably predict a posteriori fidelity; a variety of models with near-identical a priori RMSE can fail in simulation due to high-wavenumber noise amplification, missing invariances, or sensitivity to numerical artifacts (Wu et al., 21 Nov 2025).

5. Invariance, Generalization, and Robustness

Ensuring Galilean, rotational, reflectional, and (for anisotropic grids) filter-form invariance is increasingly seen as essential for generalization and physical robustness:

• Eigenframe-based decomposition encodes rotation/reflection invariance by expressing all inputs/outputs in the local strain-rate basis (Prakash et al., 2021, Prakash et al., 2022).
• Tensor basis expansions using known polynomial bases (Cayley–Smith, Pope–Ling bases) guarantee frame invariance and reduce parameter count (Wu et al., 21 Nov 2025, Hardy et al., 2023).
• Embedding filter anisotropy via parent-space mapping $x = A \xi$ and input normalization preserves invariance under grid stretching (Prakash et al., 2022).
• Multiscale and recursive frameworks facilitate grid-size and Reynolds-number extrapolation without loss of accuracy or numerical instability (Wu et al., 14 Nov 2025, Cho et al., 2023).
• Classification and regression networks leveraging singular values of local-invariant feature matrices further encode geometric invariance and yield sharp discrimination between cascade/backscatter zones (Hasan et al., 20 Apr 2025).

Combined, these advances enable models to remain stable and accurate across $500\,000\times$ their training Re, grid spacings $\Delta_g$ up to $64\,\Delta_\mathrm{DNS}$, and extrapolation to extreme out-of-distribution conditions (Wu et al., 14 Nov 2025, Prakash et al., 2022).

6. Practical Deployment and Design Recommendations

Comprehensive practical guidelines have emerged from recent studies:

1. Explicitly filter training data with at least two qualitatively distinct kernels (Wu et al., 21 Nov 2025).
2. Normalize all inputs and outputs using robust, physically grounded scales; prefer minimal, non-redundant invariants (Wu et al., 21 Nov 2025).
3. Prefer simple input sets (e.g., $\{P,\,|S|,\,|\Omega|\}$) unless additional physics is essential (Wu et al., 21 Nov 2025).
4. Evaluate models with both a priori and a posteriori diagnostics, across at least two LES solver architectures with different numerics.
5. Use data augmentation (multiscale, recursive, or spatial) for generalization and robust high-Re extrapolation (Wu et al., 14 Nov 2025, Cho et al., 2023).
6. Leverage neural architectures explicit in invariance (TBNN, S-frame, geometric invariants) to ensure transferability and avoid symmetry-breaking instabilities (Prakash et al., 2021, Prakash et al., 2022, Hasan et al., 20 Apr 2025).
7. For geophysical and multi-physics contexts, partition output variables by physical process, and enforce exact conservation constraints via architectural constraints or boundary-conditioning (Yuval et al., 2020).

These best practices produce neural SGS models with a posteriori spectral fidelity within $10\%$ of DNS (SSE error) and minimal sensitivity to solver and grid specifics, while avoiding the dramatic failures and spectral pile-up that previously plagued naïve neural closures (Wu et al., 21 Nov 2025).

7. Challenges, Limitations, and Future Directions

Despite substantial progress, open challenges persist:

• Ensuring a priori regression accuracy translates to stable and robust a posteriori performance remains non-trivial; the complex interaction between model, numerics, and flow characteristics can yield unexpected instabilities if invariance or normalization is neglected (Wu et al., 21 Nov 2025).
• Embedding invariance, while beneficial for generalization, may limit model expressivity if physical effects (e.g., near-wall anisotropy, compressibility) are not explicitly encoded; ongoing work aims to extend invariant frameworks to compressible, multiphase, and wall-bounded flows (Prakash et al., 2022, Hardy et al., 2023).
• Efficient integration with established CFD solvers is essential; deployment architectures such as heterogeneous ML–CFD frameworks and client–server coupling mitigate computational overhead and infrastructure constraints (Liu et al., 2021).
• Ongoing research addresses adaptive bandlimiting, in-network invariant computation, strong coupling to data assimilation frameworks, and robust uncertainty quantification (Hasan et al., 20 Apr 2025, Pawar et al., 2020).

Advancements in multiscale network design, recursive generalization, and invariant embedding position neural network subgrid stress models as viable, robust closures for next-generation turbulent flow simulation—conditional on fidelity to invariance, normalization, and multiscale data augmentation protocols.