Fluid Attention Block (FAB)

• FAB is a neural network module that integrates local continuous convolutions with global self-attention for improved fluid simulation fidelity.
• It mitigates error propagation by blending neighborhood-level physical laws with global contextual cues to accurately model long-term fluid behavior.
• FAB, as the core of FluidFormer, demonstrates significant improvements in numerical stability and physical plausibility in complex and turbulent flow simulations.

The Fluid Attention Block (FAB) is a neural network module introduced to address the limitations of strictly local, particle-based fluid simulators. Standard architectures in this domain, following Smoothed Particle Hydrodynamics (SPH), compute interactions via local kernels or continuous convolutions but are prone to error propagation and loss of stability during long-term or complex simulations. FAB combines a local continuous convolution branch with a global self-attention mechanism, fusing their outputs via a learned, gated blend. This architecture enables the suppression of accumulated discretization errors while capturing both short- and long-range fluid phenomena. FAB constitutes the core module within FluidFormer—a Transformer-based architecture specifically designed for continuous, particle-based fluid simulation (Wang et al., 3 Aug 2025).

1. Motivation and Conceptual Foundation

Traditional neural solvers for particle-based fluid simulation are grounded in SPH, which ensures local physical consistency by computing each particle's interactions through neighborhood-limited kernels. However, this strict locality causes incremental numerical errors to propagate, accumulate, and ultimately produce large-scale artifacts—particularly in scenarios involving violent flows or intricate boundary conditions. To overcome these deficiencies, the Fluid Attention Block was designed to hierarchically combine two processing paths:

• A local path, using continuous convolution (CConv), to enforce neighborhood-level physical laws.
• A global path, using multi-head self-attention, to capture context and dependencies spanning the entire particle ensemble.

By fusing these two feature representations, FAB enables stable and accurate long-term simulation, models global pressure phenomena, and effectively controls error accumulation without sacrificing local detail.

2. Continuous Convolution for Local Feature Extraction

The local branch in FAB applies a continuous convolution operator, generalizing the SPH mechanism. Input features for each particle at position $\mathbf{x}$, $f(\mathbf{x})$, are aggregated from their neighbors $\mathcal{N}(\mathbf{x},R)$ (those within radius $R$). The convolution operation is formulated as: $$(f * g)(\mathbf{x}) = \sum_{i \in \mathcal{N}(\mathbf{x},R)} a(\mathbf{x}_i,\mathbf{x})\, f_i\, g\left( \Lambda(\mathbf{x}_i - \mathbf{x}) \right)$$ where $a(\mathbf{x}_i, \mathbf{x})$ is a differentiable, learnable window function, $\Lambda$ is a trainable coordinate transform, and $g$ is the kernel. The local extractor stacks two such CConv layers, each followed by BatchNorm and a ReLU nonlinearity. This architecture ensures that each particle's update remains conditioned on physically meaningful, local neighbor information while adapting dynamically to particle distributions.

3. Multi-Head Self-Attention for Global Dependency Modeling

The global branch employs standard multi-head self-attention augmented with 3D rotary position encoding (3D-RoPE). For a set of $N$ particles with $d$-dimensional features, queries, keys, and values are projected as: $f(\mathbf{x})$0 For each attention head, the 3D-RoPE mechanism rotates $f(\mathbf{x})$1 and $f(\mathbf{x})$2 based on their spatial positions, and the pairwise attention is computed as: $f(\mathbf{x})$3 The per-head outputs are aggregated and projected. This global attention path allows information—such as pressure waves or coherent flow patterns—to be propagated beyond local neighborhoods, thereby capturing the nonlocal interactions essential for accurate long-term physical simulation.

4. Fusion Mechanism: Hierarchical Local–Global Integration

FAB processes two distinct inputs (in FluidFormer, from CConv and ASCC branches), each through local conv and global attention subpaths: $f(\mathbf{x})$4 The two global outputs are concatenated to form a fused representation. A learned sigmoid gate $f(\mathbf{x})$5 and amplifier $f(\mathbf{x})$6 blend the original inputs: $f(\mathbf{x})$7 This soft-gated fusion enables selective passage of local and global information to subsequent layers, preserving the inherent locality of SPH-style convolutions while injecting global context where needed.

5. Implementation and Key Hyperparameters

A schematic pseudocode for a single FAB forward pass is as follows: $\mathcal{N}(\mathbf{x},R)$5 Notable hyperparameters:

• Support radius $f(\mathbf{x})$8 in CConv/ASCC, chosen so each particle has approximately 40 neighbors.
• Embedding dimension $f(\mathbf{x})$9.
• Number of attention heads $\mathcal{N}(\mathbf{x},R)$0.
• Rotary embedding base $\mathcal{N}(\mathbf{x},R)$1.
• Position update scaling $\mathcal{N}(\mathbf{x},R)$2.
• Initial learning rate 0.01, halved at scheduled intervals (Wang et al., 3 Aug 2025).

6. Empirical Evaluation and Ablation of Components

Ablation studies performed to isolate the impact of FAB components report long-term sequence error ($\mathcal{N}(\mathbf{x},R)$3-SE, in mm) and maximum density error (MDE, in g/cm³):

Method	$\mathcal{N}(\mathbf{x},R)$4-SE	MDE
w/o Global Feature Extractor	41.024	0.019
w/o Local Feature Extractor	75.073	0.057
w/o CConv	177.151	0.141
w/o ASCC	93.524	0.073
w/o 3D-RoPE	29.131	0.017
Ours (full FAB + dual‐path)	24.442	0.008

Omitting the global branch nearly doubles the error, while eliminating the local branch or CConv yields collapse in performance. The duality and synergy of local-global fusion is demonstrated to be crucial for both numerical stability and physical plausibility of simulated flows.

7. Impact on Error Suppression and Global Phenomena Modeling

FAB’s principal contribution is the suppression of error accumulation typical in local-only particle networks; incremental discretization drifts, if unchecked, can lead to macroscopic artifacts such as clustering or collapse. The global attention path in FAB enables “broadcast” updates—such as global pressure adjustments or coherent bulk motion corrections—to all particles, providing nonlocal regularization. Empirical results show reduced sequence error, stabilized density, and visual robustness across sloshing and turbulent flow conditions, substantiating the effectiveness of the FAB design in high-fidelity continuous fluid simulation (Wang et al., 3 Aug 2025).