H2OFlow: Synthetic 3D Affordance Inference

• H2OFlow is a framework that learns comprehensive 3D human–object interaction affordances using synthetic data and a three-stage diffusion pipeline.
• It converts human–object meshes into point clouds, then learns dense flow fields from a canonical human pose to interaction-specific configurations.
• Evaluations on benchmarks like OMOMO and BEHAVE show that H2OFlow outperforms mesh-based methods while efficiently handling partial and noisy data.

Searching arXiv for the H2OFlow paper and closely related works to ground the encyclopedia entry. arXiv search query: "H2OFlow Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flows" H2OFlow is a framework for learning comprehensive three-dimensional human–object interaction affordances from synthetic data, with the stated goal of covering not only contact but also orientation and spatial occupancy around objects (Zhang et al., 17 Oct 2025). It is formulated as a three-stage pipeline that uses a pre-trained text-driven 3D generative model to synthesize human–object interaction meshes, converts these meshes into point-cloud correspondences, learns a dense 3D flow distribution from a canonical human “zero-pose” to interaction-conditioned human configurations, and then derives affordance scores analytically from sampled flows and cross-attention weights (Zhang et al., 17 Oct 2025). In the reported experiments, the method generalizes to real-world point clouds and exceeds prior methods that depend on manual annotations or mesh-based representations on the stated benchmarks (Zhang et al., 17 Oct 2025).

1. Problem setting and representational scope

H2OFlow is positioned within 3D affordance understanding for computer vision, robotics, and AI, where the central task is to infer how humans can interact with surrounding objects (Zhang et al., 17 Oct 2025). The motivating limitation identified for earlier approaches is twofold: dependence on labor-intensive, hand-labeled human–object interaction datasets, and a predominant focus on contact-based analysis that neglects other interaction attributes such as preferential orientation with respect to objects and spatial occupancy around them.

Within this formulation, an affordance is not restricted to pointwise contact likelihood. H2OFlow instead models three modalities jointly: contact, orientational structure, and spatial occupancy. This broader scope matters because many interactions are not exhausted by touch. The examples given in the source material are explicitly non-contact-centric: humans may have a preferential orientation with respect to a TV, and may occupy the front of a microwave rather than its back (Zhang et al., 17 Oct 2025). The framework therefore treats affordance as a distribution over geometrically structured human placements relative to an observed object point cloud.

The method operates directly on point clouds rather than on meshes at inference time. A plausible implication is that the framework is designed to remain usable when only partial or noisy geometric observations are available, which is consistent with the reported evaluation on partial real-world scans and the comparison against mesh-based baselines.

2. Three-stage pipeline and synthetic supervision regime

The H2OFlow pipeline has three stages: Synthetic HOI Generation, Diffuse Dense Flows Learning, and Affordance Inference (Zhang et al., 17 Oct 2025). In the first stage, a pre-trained text-driven 3D generative model, CHOIS, is prompted with normalized natural-language instructions to sample diverse human–object interaction meshes. Both human and object meshes are then subsampled into point clouds by per-vertex sampling via Furthest-Point Sampling.

The second stage fixes a canonical human point cloud $H_0$, described as a “zero-pose” human such as an SMPL T-pose, and computes ground-truth dense flow from each synthesized interaction sample:

$F_{GT} = H - H_0.$

A diffusion transformer is then trained to learn the conditional flow distribution

$$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$

where $O$ is the object point cloud (Zhang et al., 17 Oct 2025).

In the third stage, a test-time object point cloud $O$ is observed, possibly partial. The learned reverse diffusion process samples $F \sim p_\theta(F \mid O)$, reconstructs a human goal point cloud

$\hat H = H_0 + F,$

and uses the collection of sampled $\hat H$ together with learned cross-attention weights to compute contact, orientation, and spatial occupancy affordances (Zhang et al., 17 Oct 2025).

A central property of this pipeline is the absence of manual affordance labels. The source explicitly states that no manual affordance labels are used, and that contact, orientation, and spatial cues derive from the flow and attention during inference (Zhang et al., 17 Oct 2025). This suggests a shift from annotation-supervised affordance learning to structure-supervised inference grounded in synthesized geometry and learned correspondences.

3. Dense 3D flow formulation and diffusion objective

The core representation is a dense flow field on point clouds. Let the canonical human point cloud be

$H_0 = \{\,h_{0,i}\in\mathbb{R}^3\}_{i=1}^{N_H},$

and the synthesized goal human point cloud be

$H = \{\,h_i\in\mathbb{R}^3\}_{i=1}^{N_H},$

with one-to-one correspondence via shared sampling indices. The dense flow field is then

$F_{GT} = H - H_0.$0

Each vector $F_{GT} = H - H_0.$1 describes how the $F_{GT} = H - H_0.$2-th human point moves under interaction (Zhang et al., 17 Oct 2025).

H2OFlow models the conditional distribution of these flows with a discrete-time diffusion process of length $F_{GT} = H - H_0.$3. The forward noising process starts from $F_{GT} = H - H_0.$4 and applies

$F_{GT} = H - H_0.$5

where $F_{GT} = H - H_0.$6 and $F_{GT} = H - H_0.$7 is a fixed noise schedule (Zhang et al., 17 Oct 2025). The reverse process learns

$F_{GT} = H - H_0.$8

following the parameterization described in the source summary.

Training minimizes a hybrid loss consisting of a denoising mean-squared error term and a cumulative KL term:

$F_{GT} = H - H_0.$9

The source also states that no additional supervised losses are introduced for affordances; the three modalities are computed analytically from sampled flows and cross-attention weights (Zhang et al., 17 Oct 2025).

This formulation places dense human motion, rather than explicit contact labels, at the center of the learning problem. A plausible implication is that the method uses the geometry of the entire interacting body as a latent carrier of affordance structure.

4. Affordance inference: contact, orientation, and occupancy

Affordance prediction is defined per human–object point pair. Let $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$0 be object points, let $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$1 denote the conditional distribution over human point locations given object point $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$2, let

$$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$3

and let $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$4 be the diffusion transformer cross-attention weight between $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$5 and $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$6 (Zhang et al., 17 Oct 2025).

The contact affordance is defined as

$$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$7

This score combines relative proximity with learned human–object association weights.

The orientational affordance first computes a relative orientation vector

$$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$8

The sphere $$p_\theta(F \mid O), \qquad O \in \mathbb{R}^{N_O \times 3},$$9 is discretized into $O$0 bins with centers $O$1, and a discrete orientation distribution is formed as

$O$2

With neg-entropy

$O$3

the orientational score is

$O$4

In the reported construction, orientation is therefore not a separate supervised label but a statistic of sampled geometric relations and their concentration (Zhang et al., 17 Oct 2025).

The spatial occupancy affordance voxelizes the space around the object into a grid $O$5, defines an indicator $O$6 when human point $O$7 falls in voxel $O$8, and scores occupancy by

$O$9

The output is a 3D occupancy heatmap around the object (Zhang et al., 17 Oct 2025).

Taken together, these definitions make H2OFlow a multimodal affordance field estimator. Rather than predicting a single contact map, it predicts where a human body may be, how it is oriented, and which object regions are implicated by these distributions.

5. Architecture, datasets, and reported quantitative results

The backbone is a Diffusion Transformer with input encoders implemented as MLPs for flow features $O$0 from $O$1, human features $O$2 from $O$3, and object features $O$4 from $O$5 (Zhang et al., 17 Oct 2025). The model forms a joint token stream $O$6 and applies $O$7 DiT blocks, each comprising self-attention on $O$8, cross-attention with queries from $O$9 and keys/values from $F \sim p_\theta(F \mid O)$0, and a feed-forward MLP. The output per point is a predicted noise term $F \sim p_\theta(F \mid O)$1 and an interpolation vector $F \sim p_\theta(F \mid O)$2. The reported hyperparameters are hidden size 128, 4 heads, 5 blocks, $F \sim p_\theta(F \mid O)$3 diffusion steps, batch size 32, AdamW, learning rate $F \sim p_\theta(F \mid O)$4, and weight decay $F \sim p_\theta(F \mid O)$5 (Zhang et al., 17 Oct 2025).

The main datasets are OMOMO and BEHAVE. OMOMO is used with 12 train categories, 5 test categories, and 100 HOI sequences per training object. BEHAVE is used for zero-shot and fine-tuned generalization (Zhang et al., 17 Oct 2025). Baselines listed in the source are COMA on meshes, COMA-Recon from reconstructed point-cloud meshes, H2OSMPL based on diffusion over SMPL parameters, H2OFlow-NoAttn without cross-attention, and the full H2OFlow model.

The evaluation metrics are contact SIM and MAE, orientation Precision@K over vertex rankings by entropy, and spatial MSE for voxel occupancy (Zhang et al., 17 Oct 2025). The reported OMOMO results are:

Metric	COMA	H2OFlow
SIM-H	41.3%	72.3%
SIM-O	56.9%	81.0%
MAE-H	0.22	0.11
MAE-O	0.14	0.07
Prec@K	42.9%	75.6%
MSE	0.14	0.12

On BEHAVE, the source states that H2OFlow-NoFT is close to COMA in zero-shot evaluation, while H2OFlow-FT exceeds COMA by approximately 20% after fine-tuning (Zhang et al., 17 Oct 2025). The memory and runtime numbers on a V100 GPU are reported as approximately 8.4 GB and 6.7 s per object for H2OFlow, versus approximately 9.8 GB and 65 s for COMA, in addition to mesh reconstruction overhead for the latter.

These results situate H2OFlow as a point-cloud-native method with an explicit computational comparison against mesh-centric baselines. A plausible implication is that the flow representation and direct point-cloud conditioning reduce some of the burdens associated with mesh reconstruction pipelines.

6. Qualitative behavior, ablations, and terminological disambiguation

The reported qualitative outputs proceed from point-cloud inputs to flow samples and then to contact, orientation, and spatial-affordance heatmaps (Zhang et al., 17 Oct 2025). Within OMOMO, the cited examples include monitor interactions with side-torso versus center-torso contact patterns, and tripod interactions with leg versus hand orientational consistency. On real-world iPhone-depth scans, the method is described as robust to partial occlusion and as producing semantically meaningful affordances such as backpack grasp regions and head-torso orientations (Zhang et al., 17 Oct 2025).

The ablation results summarized in the source attribute more symmetric affordances under low-sample regimes to cross-attention, and report that random masking during training yields robustness up to 50% occlusion (Zhang et al., 17 Oct 2025). These observations are consistent with the architectural choice to couple human and object tokens via cross-attention rather than to infer human placement from object geometry alone.

A recurring source of ambiguity is the name itself. H2OFlow is unrelated to physical studies of water flow. In the arXiv record, for example, low-dimensional water research examines hydrogen-bond ordering, modified Poiseuille-like laws, viscosity across confinement regimes, and ionic selectivity in one- and two-dimensional channels (Trushin et al., 2 Jul 2025), while water-bridge research analyzes spontaneous two-way coaxial flow in a millimeter-scale bridge using particle image velocimetry, tracers, and electrohydrodynamic arguments (Tsai et al., 2020). H2OFlow, by contrast, uses “H2O” as shorthand for human–object affordances and addresses 3D generative modeling, diffusion over dense human-point flows, and annotation-free affordance inference (Zhang et al., 17 Oct 2025).

In that sense, H2OFlow belongs to the literature on synthetic-data-driven HOI modeling rather than to hydrodynamics. Its distinctive claim is that a dense diffused flow representation, learned from synthetic human–object interaction geometry, is sufficient to recover multimodal 3D affordance structure without manual affordance labels.