GotFlow3D: End-to-End 3D Fluid Flow Estimation

• GotFlow3D is a novel framework for learning dense 3D fluid motion by addressing key challenges in particle tracking velocimetry using an end-to-end deep neural network.
• It fuses Static–Dynamic Graph Neural Networks with a recurrent adaptive point-voxel transport mechanism to capture both large displacements and fine motion details.
• Empirical evaluations show state-of-the-art accuracy, robust noise handling, and improved performance in both synthetic and real-world fluid dynamics applications.

GotFlow3D is an end-to-end deep neural network framework for learning dense 3D fluid flow motion from double-frame particle sets, designed to address the significant challenges inherent in particle tracking velocimetry (PTV) such as large displacements, dense seeding, complex flow features, and measurement noise. The method synergistically fuses graph neural networks (GNNs) with an entropic-mass regularized optimal transport (OT) formulation and a novel, recurrent, point-voxel-adaptive transport plan. By encoding both geometric and feature-based neighborhoods into learned feature descriptors and leveraging a gated recurrent update mechanism, GotFlow3D achieves state-of-the-art performance in 3D scene flow and particle tracking accuracy, robustness, and generalization across synthetic and real-world fluid dynamics datasets (Liang et al., 2022).

1. Motivation and Problem Setting

Particle tracking velocimetry relies on linking thousands of tracer particles across temporally adjacent frames in turbulent, three-dimensional flows. The associated challenges include large particle displacements (up to several times the inter-particle spacing), dense seeding, highly nonrigid and vortical flow characteristics, noise, and the presence of missing or spurious detections. Classical approaches are reliant on heuristics or global smoothness priors, which struggle particularly under extreme deformation or outlier-heavy regimes. Existing deep learning-based scene flow methods typically assume rigid or near-rigid motion and do not generalize effectively to the scenarios encountered in practical PTV.

GotFlow3D overcomes these limitations by:

• Learning spatially and dynamically adaptive pointwise feature representations with a Static–Dynamic Graph Neural Network (SDGNN).
• Framing the association between particle sets as an optimal transport problem on the learned feature descriptors.
• Employing a recurrent, locally adaptive flow update via the AdaPT (Adaptive Point-Voxel Transport) mechanism, supervised by a GRU module.

The unified, differentiable pipeline enables the capture of both long-range transport and fine-scale deformation phenomena, essential for accurate scene flow estimation in experimental and computational fluid dynamics contexts (Liang et al., 2022).

2. Data Representation and Graph Construction

Let $\mathcal{P} = \{p_i \in \mathbb{R}^3\}_{i=1}^{n_1}$ and $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$ denote the source and target particle sets, respectively, with no additional attributes provided. To capture relevant spatial and structural information, GotFlow3D constructs two graphs for each point set:

• Static Spatial Graph ($\mathcal{G}^s$): Each $p_i$ is connected to its $k$ Euclidean nearest neighbors, with edge features including relative position and spherical coordinates $(r_{i_k}, \theta_{i_k}, \varphi_{i_k})$ to ensure rotational invariance.
• Dynamic Feature Graph ($\mathcal{G}^d$): Edges are reconstructed in a learned feature space, where connections are based on $k$-nearest neighbors under the evolving feature similarity metric, allowing the receptive field to adapt to global scene structure.

This dual-graph approach enables the disentanglement and fusion of intrinsic local geometric structure with extrinsic, learned affinities, providing a rich descriptor space for subsequent correspondence estimation.

3. Static–Dynamic Graph Neural Network Architecture

The fusion of geometric and adaptive feature information is achieved via the Static–Dynamic Graph Neural Network (SDGNN), which alternates between two specialized layers:

• GeoSetConv (Static):

$${}^s g_i^{\beta} = \max_{j\in\mathcal{N}^s(p_i)} \phi^\beta(\tilde f_{ij},\;{}^s g_j^{\beta-1} - {}^s g_i^{\beta-1})$$

where $\tilde f_{ij}$ encodes relative position and spherical coordinates.

• EdgeConv (Dynamic):

$\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$0

Features from multiple SDGNN layers are concatenated and processed by multilayer perceptrons (MLPs) $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$1 yielding 128-dimensional pointwise descriptors $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$2. Additionally, a SetConv-based encoder produces context features $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$3 for recurrent flow reconstruction.

4. Optimal Transport Correspondence Formulation

The association of particles between frames is posed as a differentiable, entropic-mass regularized optimal transport (OT) problem. The sets $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$4 and $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$5 are modeled as discrete probability measures with uniform weights $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$6, $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$7. The classical OT matching seeks

$\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$8

with cost matrix $\mathcal{Q} = \{q_j \in \mathbb{R}^3\}_{j=1}^{n_2}$9 ensuring points with similar features have low matching cost.

To handle noise, outliers, and ensure differentiability, the matching is performed using an entropic–mass-penalized Sinkhorn algorithm:

$\mathcal{G}^s$0

where $\mathcal{G}^s$1 and $\mathcal{G}^s$2 are learned parameters controlling sparsity and adherence to the marginal constraints.

5. Recurrent Refinement and Adaptive Transport Plan (AdaPT)

Rather than direct, global flow prediction, GotFlow3D adopts a recurrent residual flow refinement. At each iteration $\mathcal{G}^s$3:

$\mathcal{G}^s$4

where $\mathcal{G}^s$5 accumulates incremental flows. The AdaPT module retrieves a localized, adaptive transport plan for each warped point, composed of:

• Point-based retrieval:

$\mathcal{G}^s$6

with $\mathcal{G}^s$7 adaptively decreased per iteration.

• Voxel-based retrieval: Multi-scale deformable cuboids are constructed, partitioning neighbors into spatial sub-cells, and averaging $\mathcal{G}^s$8 values over a pyramid of $\mathcal{G}^s$9 levels, encoded with $p_i$0.

The local plan $p_i$1 dynamically adapts to both the magnitude and locality of residual motions and maintains statistical efficiency.

6. Gated Recurrent Flow Update and Supervision

A Gated Recurrent Unit (GRU) module ingests the composite vector $p_i$2 along with the previous hidden state $p_i$3, processing updates via 1D convolutions:

$p_i$4

The overall loss enforces supervision on all intermediate flow predictions $p_i$5 with exponential temporal discounting:

$p_i$6

7. Empirical Evaluation and Impact

Empirical results demonstrate GotFlow3D's efficacy and robustness:

• FluidFlow3D Synthetic Dataset: Benchmarked across 16k training and 1.6k validation samples from regimes such as isotropic/MHD turbulence, channel flow, boundary layer, and analytic Beltrami/uniform fields. Achieves EPE = 0.00487, AccStrict = 93.2%, Outliers = 3.6%, outperforming FlowNet3D, FLOT, PointPWC-Net, and PV-RAFT.
• Robustness: Maintains high accuracy and low EPE under wide ranges of displacement ratio ($p_i$7 as low as 0.33) and noise up to 10%. Recurrent iteration studies find optimal performance at 8–10 steps, with coarse motion captured in early iterations and fine refinement subsequently.
• Integration into PTV pipelines: On CylinderFlow (Re ≈ 3900) Lagrangian CFD data, initializing T-PT or SerialTrack with GotFlow3D predictions raises yield rate $p_i$8 by 15–30% and achieves reliability $p_i$9, confirming sub-voxel accuracy (EPE ≈ 0.0058).
• Experimental Validation: In spherical-indentation hydrogel experiments, GotFlow3D+PTV recovers dense, large-deformation fields inaccessible to standalone T-PT or SerialTrack.

These results establish GotFlow3D as the state-of-the-art method for learning 3D motion in PTV and related scene flow estimation contexts, with demonstrated superiority in accuracy, robustness to noise, generalization across flow regimes, and integrability into existing analysis pipelines (Liang et al., 2022).