ChannelFlow-Tools: 3D CFD Dataset Pipeline

• ChannelFlow-Tools is a standardized framework for generating 3D obstructed channel flow datasets by converting CAD solids into ML-ready CFD data.
• It integrates procedural geometry synthesis, feasibility filtering, SDF voxelization, HPC solver orchestration, and Cartesian resampling via a single Hydra/OmegaConf configuration.
• The system supports deterministic reproduction and parameter sweeps, as demonstrated by a case study with over 10k scenes spanning Re=100–15000.

ChannelFlow-Tools is a configuration-driven, end-to-end dataset creation framework for 3D obstructed channel flows that standardizes the path from programmatic CAD solid generation to ML-ready inputs and targets for CFD surrogate modeling. It integrates procedural geometry synthesis, feasibility filtering, signed distance field voxelization, automated waLBerla LBM solver orchestration on HPC systems, and Cartesian resampling to co-registered multi-resolution tensors, all under a single Hydra/OmegaConf configuration. In the paper introducing the framework, the system is positioned as infrastructure for reproducible CFD data generation rather than as a new solver or a new surrogate architecture, with a case study generating 10k+ scenes spanning $Re=100$–15000 and a minimal 3D U-Net benchmark at $128\times32\times32$ used to illustrate predictable scaling behavior (Kavane et al., 17 Sep 2025).

1. Scope, definition, and naming

ChannelFlow-Tools denotes a standardized toolchain for generating datasets of obstructed channel-flow scenes, with explicit support for programmatic obstacle synthesis, scene feasibility checks, SDF generation, HPC simulation orchestration, co-registered multi-resolution resampling, and metadata/provenance capture (Kavane et al., 17 Sep 2025). The framework converts

Hydra/OmegaConf config $\rightarrow$ CAD solids $\rightarrow$ feasibility-filtered scenes $\rightarrow$ SDF voxel grids $\rightarrow$ waLBerla LBM simulations on HPC $\rightarrow$ Cartesian resampled tensors for ML

into a deterministic and auditable workflow. This standardization is motivated by the observation that CFD surrogate modeling often suffers from inconsistent geometry formats, ad hoc mesh or voxel preprocessing, and non-reproducible solver campaigns.

A recurrent misconception is that the name refers to a general family of “channel flow tools.” In the literature provided here, the exact title “ChannelFlow-Tools” refers specifically to the dataset-creation framework for 3D obstructed channel flows (Kavane et al., 17 Sep 2025). By contrast, in the Channelflow/Openpipeflow paper, “ChannelFlow-Tools” refers not to a separate new algorithmic package, but to the practical use of existing Newton–Krylov steady-state machinery in Channelflow and Openpipeflow to implement Stokes preconditioning by parameter choice alone (Tuckerman et al., 2018). The dataset framework is therefore distinct in both purpose and implementation scope.

The paper places obstructed channel flows in a methodological middle ground: realistic enough to exhibit wake dynamics, separation, and regime diversity, yet controlled enough to support systematic dataset creation. This suggests that the framework is designed less as a domain-general CFD corpus generator than as a reproducible substrate for geometry-conditioned surrogate studies.

2. Problem setting and design rationale

The framework addresses several concrete bottlenecks in CFD ML datasets: geometry variability is hard to manage reproducibly, SDF conventions are often undocumented, simulation campaigns are hard to resume and audit, ML inputs and targets are often not co-registered across resolutions, ablation studies are difficult when preprocessing is not configurable, and storage/runtime trade-offs are usually implicit rather than measured (Kavane et al., 17 Sep 2025). ChannelFlow-Tools makes these choices explicit through a single hierarchical configuration.

The design rationale is configuration centric. Geometry generation, simulation parameters, export choices, provenance settings, and sampling modes are all controlled from one source of truth. The paper emphasizes that this supports deterministic reproduction, controlled ablations, parameter sweeps, exact resumption, and transparent provenance. The resulting workflow is intended to replace one-off preprocessing pipelines with a reproducible data product.

The system is also explicitly staged. Each phase emits inspectable artifacts consumed by downstream stages, so the dataset is not merely a collection of derived tensors but a traceable chain of geometry files, metadata sidecars, solver folders, exported fields, and resampled representations. A plausible implication is that reproducibility is treated as a property of the entire data-generation graph, not only of the final tensor export.

3. End-to-end pipeline structure

The framework is organized into five major stages: programmatic CAD solid generation, feasibility checks and scene fusion, SDF voxelization, HPC solver orchestration with waLBerla LBM, and Cartesian resampling into co-registered multi-resolution tensors (Kavane et al., 17 Sep 2025).

Stage	Main operation	Primary emitted artifacts
1	Programmatic CAD solid generation	binary STL mesh, YAML metadata sidecar
2	Feasibility checks and scene fusion	accepted fused obstacle scene
3	SDF voxelization	dense .npy, optional VTK/VTI, optional slice plots
4	waLBerla LBM orchestration on HPC	per-case run folder, solver inputs, logs, averaged fields
5	Cartesian resampling	co-registered tensor exports, optional masks and SDF channels

At the geometry stage, the channel domain is fixed in lattice units as

$[0,2048]\times[0,512]\times[0,512],$

with obstacle placement restricted to the streamwise region

$x\in[146,1800].$

Obstacles are generated procedurally from six primitive families: cube/cuboid, cone, cylinder, sphere, torus, and wedge. These shapes are implemented in CadQuery on top of Open CASCADE and exported as watertight CAD solids. For each scene, one or more objects are sampled from configured family distributions using either uniform random sampling or Sobol low-discrepancy sampling. Each object receives shape-specific dimensions, a centroid position, an orientation sampled uniformly on $SO(3)$, and optional direction vectors for shapes with preferred axes (Kavane et al., 17 Sep 2025).

Feasibility filtering then rejects invalid candidates unless they satisfy four criteria: in-bounds placement, non-intersection with already accepted geometry, minimum volume, and minimum clearance. The paper gives an example clearance margin,

$128\times32\times32$0

For multi-object scenes, each accepted object updates a fused context, and the final geometry is boolean-fused and exported as one solid. In Sobol mode, rejected samples still advance and record the sequence index so that feasibility filtering does not reorder the low-discrepancy sequence. This preserves reproducibility across restarts.

The downstream stages are coupled to this same artifact chain. SDFs are computed from the fused STL; the solver consumes the geometry and metadata; the resampling stage converts solver outputs to regular Cartesian tensors aligned with the SDF and original domain. The pipeline therefore enforces representational continuity from CAD space to ML tensors.

4. Geometric and physical representations

The SDF stage is one of the framework’s central ML-facing components. For solid obstacle region $128\times32\times32$1 with boundary $128\times32\times32$2, the signed distance field is defined as

$128\times32\times32$3

where

$128\times32\times32$4

and

$128\times32\times32$5

Thus $128\times32\times32$6 inside solids, $128\times32\times32$7 in fluid, and $128\times32\times32$8 on the interface. The paper also notes the standard properties that $128\times32\times32$9 is 1-Lipschitz and that $\rightarrow$0 almost everywhere away from the medial axis and surface (Kavane et al., 17 Sep 2025).

Implementation proceeds by loading the fused STL mesh into OpenVDB via pyopenvdb, constructing a sparse level set, and rasterizing it into a dense NumPy array. The main SDF presets are $\rightarrow$1 with $\rightarrow$2, $\rightarrow$3 with $\rightarrow$4, and $\rightarrow$5 with $\rightarrow$6. These grids are exactly co-registered, since the voxel spacing is chosen so that the domain extent divides exactly. Outputs include dense .npy arrays, optional VTK/VTI exports for QA in ParaView/PyVista, and optional fixed-slice plots.

The solver stage uses waLBerla, described as a high-performance block-structured LBM framework. Simulations use a D3Q27 stencil, a cumulant collision operator, and a Smagorinsky LES closure with

$\rightarrow$7

where $\rightarrow$8 is the base kinematic viscosity, $\rightarrow$9 is the Smagorinsky constant, $\rightarrow$0 is the filter width taken equal to lattice spacing, and $\rightarrow$1 is the strain-rate magnitude. The target Reynolds number is defined as

$\rightarrow$2

with $\rightarrow$3, the channel height, and $\rightarrow$4 the inlet-plane area-averaged streamwise velocity (Kavane et al., 17 Sep 2025).

Boundary conditions follow a fixed dataset policy: velocity inlet at $\rightarrow$5, pressure outlet at $\rightarrow$6, and no-slip walls on the $\rightarrow$7 faces. Periodic boundaries may be enabled only for controlled studies and are recorded in metadata. Geometry is embedded into the solver grid using an octree/SDF-based solid-fluid classification to generate a link-wise wall mask. The pipeline performs online time-averaging after transients and checks stationarity using sliding-window diagnostics, flux balance, and mean-field stability tests; cases failing stationarity checks can be extended or discarded.

The final resampling stage exports solver outputs onto fixed Cartesian grids at $\rightarrow$8, $\rightarrow$9, and $\rightarrow$0, all co-registered with the SDF. The implementation uses a ParaView/pvpython pipeline: read VTU, merge blocks, convert cell data to points, and interpolate onto a target Cartesian grid using PointVolumeInterpolator. The default kernel is Linear with an N-closest footprint of $\rightarrow$1, while Gaussian, Shepard, Voronoi, and Ellipsoidal Gaussian are also supported. The pipeline can additionally export the SDF $\rightarrow$2 and the binary fluid mask $\rightarrow$3, enabling solid-voxel masking during training (Kavane et al., 17 Sep 2025).

5. Configuration, artifacts, and reproducibility

A defining feature of ChannelFlow-Tools is that all stages are governed by one hierarchical configuration using Hydra and OmegaConf (Kavane et al., 17 Sep 2025). The configuration captures global domain bounds, shape-family ranges, number of objects per scene, shape mixture weights, sampling mode, retry budgets, minimum volume and clearance, simulation parameter ranges, Reynolds-number band, inlet velocity policy, periodicity flags, voxel spacing and grid resolution, resampling kernel and footprint, output paths, and provenance settings.

Because this configuration serves as the source of truth, the framework supports deterministic reproduction, controlled ablations, parameter sweeps, exact resumption, and transparent provenance. The paper notes launch-time overrides such as switching to two-object scenes, tightening Reynolds-number ranges, or enabling Sobol sampling, while preserving reproducibility. Provenance capture includes the resolved Hydra config snapshot, RNG seed, Sobol index or counter, config hash, run manifest, and solver and environment identifiers. In the geometry metadata, the YAML sidecar stores per-object parameters, poses, simulation settings, provenance keys, and configuration hash or RNG/Sobol state.

The framework’s artifact model is correspondingly explicit. The geometry stage emits fused .stl, .yaml metadata, and provenance logs/config snapshots. The SDF stage emits dense .npy tensors, optional .vti/.vtk, and optional slice PNGs. The solver stage emits per-case run folders, solver input files, job scripts, output fields, time-averaged fields, logs, and job IDs. The resampling stage emits co-registered tensor exports at multiple resolutions together with origin, spacing, and dimensions metadata, plus optional masks and SDF channels. This artifact structure makes the dataset inspectable at every transformation boundary rather than only at the final tensor stage.

The HPC orchestration layer reinforces this reproducibility objective. It creates per-case working folders, stages geometry and metadata, patches solver parameters from the same configuration, submits jobs to SLURM, preserves execution order through dependency chains, and records solver version, config hash, Git commit, and environment identifiers. The workflow is therefore deterministic and resumable at cluster scale.

6. Storage trade-offs, empirical case study, and limitations

The paper reports explicit storage and runtime trade-offs for dense float32 SDF export (Kavane et al., 17 Sep 2025). Approximate dense storage is about $\rightarrow$4 MB per SDF at $\rightarrow$5, about $\rightarrow$6 MB at $\rightarrow$7, and about $\rightarrow$8 MB at $\rightarrow$9. Per-SDF time on one core is about $\rightarrow$0 s, $\rightarrow$1 s, and $\rightarrow$2 min for these three resolutions, respectively. For 10k SDFs, the single-core extrapolated runtime is about $\rightarrow$3 hours for the 128-grid, $\rightarrow$4 hours for the 256-grid, and $\rightarrow$5 hours for the 512-grid. On 40 cores, the paper reports roughly $\rightarrow$6–$\rightarrow$7 speedup. The default export is therefore chosen as $\rightarrow$8, described as a balance among fidelity, storage, and compute time.

The case study dataset comprises roughly 10k scenes with Reynolds number range about $\rightarrow$9 to $\rightarrow$0, diverse obstacle shapes, diverse poses and placements, single- and multi-object scenes, and laminar, transitional, and turbulent regimes. The discussion highlights laminar flows roughly at $\rightarrow$1, transitional flows for $\rightarrow$2, and turbulent flows at $\rightarrow$3. Geometry diversity explicitly includes obstacle family, size, position, orientation, inflow velocity, periodicity flags, and Reynolds number. Some intentional balancing is reported across primitive types, alongside some overrepresentation of more complex shapes like tori to probe richer wakes.

To demonstrate that the standardized representation supports predictable learning, the authors train a minimal 3D U-Net on a downsampled grid of

$\rightarrow$4

Dataset sizes are $\rightarrow$5, $\rightarrow$6, $\rightarrow$7, and $\rightarrow$8, each with an 80/20 train/validation split, trained with L1 loss for up to $\rightarrow$9–$[0,2048]\times[0,512]\times[0,512],$0 epochs and learning-rate scheduling for larger sets. The reported trend is monotonic improvement with increasing dataset size: at $[0,2048]\times[0,512]\times[0,512],$1 samples, RMSE is approximately $[0,2048]\times[0,512]\times[0,512],$2 and MAE approximately $[0,2048]\times[0,512]\times[0,512],$3; at $[0,2048]\times[0,512]\times[0,512],$4, both drop by more than $[0,2048]\times[0,512]\times[0,512],$5; at $[0,2048]\times[0,512]\times[0,512],$6 and $[0,2048]\times[0,512]\times[0,512],$7, performance continues improving; and at the largest set, RMSE/MAE stabilize near approximately $[0,2048]\times[0,512]\times[0,512],$8. The paper characterizes this as roughly a $[0,2048]\times[0,512]\times[0,512],$9 reduction from the smallest dataset to the largest. The benchmark is explicitly presented as a baseline rather than an attempt at SOTA.

The framework’s limitations are also stated directly. Coarse SDF grids lose thin features and can show staircase artifacts; dense float32 export is storage-heavy at high resolution; SDF quality depends on watertight, correctly oriented meshes; the narrow-band width is partly a code constant and ideally should be fully configurable; the benchmark is intentionally minimal and not a comprehensive model comparison; and the dataset is specialized to obstructed channel flows rather than a universal CFD corpus (Kavane et al., 17 Sep 2025). The paper therefore presents ChannelFlow-Tools as a blueprint for reproducible, ML-ready CFD dataset creation in a specific but technically rich regime, not as a universal solution for all flow configurations.