GravityDB: Gravity-Field Diffusion Bridge

• The paper introduces GravityDB, a method that refines coarse hand-object interactions using a gravity-field guided stochastic diffusion bridge.
• It employs hierarchical gravity fields, MANO regularization, and normal alignment to iteratively refine a deformable 3D hand point cloud against a rigid object.
• Empirical evaluations on SHOWMe, GRAB, and HO3D-V3 demonstrate significant improvements in contact accuracy, reduced interpenetrations, and realistic hand-object deformations.

Searching arXiv for the specified GravityDB paper and closely related diffusion-bridge references. Gravity-Field Based Diffusion Bridge (GravityDB) is a method for realistic hand-object interaction (HOI) refinement that models contact formation as a continuous attraction-driven deformation process between a deformable hand surface and a rigid object. Given an initial hand-object state represented primarily by a 3D hand point cloud $P_h$, an object point cloud $P_v$, and optionally a text instruction $t$, the method predicts semantically meaningful contact regions and then evolves the hand under a hierarchical gravity field, MANO regularization, normal alignment, and stochastic noise to obtain a refined interaction state that is intended to be physically plausible, visually realistic, and semantically appropriate (Xu et al., 3 Sep 2025). Although the method is named a diffusion bridge, its operative form is an SDE-based stochastic refinement process bridging an initial coarse interaction state and a refined interaction state, rather than a standard DDPM-style forward/reverse diffusion construction (Xu et al., 3 Sep 2025).

1. Problem setting and representational assumptions

GravityDB is introduced for the refinement of coarse HOI reconstructions or pose estimates that often exhibit three recurrent geometric failures: mesh interpenetration, contact gaps or grasp failure, and rigid-hand artifacts. The method attributes these failures to the fact that many earlier systems emphasize skeletal or coarse MANO pose estimation, use contact only as a weak geometric prior, or retain a rigid or overly simplified hand surface representation, so that surface-level interaction geometry remains underconstrained even when joint pose is accurate (Xu et al., 3 Sep 2025).

The method therefore reframes HOI as a surface-level deformation problem rather than only a pose-estimation problem. The hand is represented as a deformable 3D point cloud during refinement, while anatomical plausibility is tethered to a MANO hand model $\mathcal{M}(\theta,\beta)$. The object is represented as a point cloud $P_v$, with predicted object contact region $P_v^c$, and object surface normals are used in the contact-geometry regularizer. The object is assumed rigid, whereas the hand is allowed local deformation beyond rigid MANO articulation (Xu et al., 3 Sep 2025).

This representational split is central to the method’s scope. A rigid MANO surface can model articulation and identity shape, but not all fine-grained local deformations needed for realistic contact. GravityDB addresses that gap by refining the hand point cloud directly while constraining it toward the MANO manifold. A plausible implication is that the method is designed as a post-estimation refinement layer rather than a full replacement for upstream HOI reconstruction systems.

2. Gravity-field dynamics and stochastic bridge formulation

The core construct is a hierarchical gravity field defined over predicted object contact points. The object contact region $P_v^c$ is treated as a zero-potential manifold, and each hand point $X\in P_h$ experiences a multi-scale Gaussian potential

$$\varphi(X) = \sum_{l=1}^{L} \sum_{p \in P_v^c} w_p^{(l)} \cdot k_l \cdot \exp\left(-\frac{\|X - p\|^2}{\sigma_l^2}\right),$$

where $w_p^{(l)}\in[0,1]$ is the contact-mask confidence at scale $P_v$0, $P_v$1 is the attraction strength, and $P_v$2 controls the attraction radius. The corresponding field term is written as

$P_v$3

The paper explicitly notes that this is not Newtonian gravity; it is a constructed potential field over semantically selected object regions (Xu et al., 3 Sep 2025).

Hand-point evolution is modeled initially by

$P_v$4

where $P_v$5 is a spatially adaptive diffusion coefficient and $P_v$6 is Brownian motion. The drift term implements attraction toward the object contact surface, while the diffusion term provides stochastic exploration to escape poor local minima and smooth the transition path. The manuscript states that the noise is annealed later but does not provide an explicit annealing formula (Xu et al., 3 Sep 2025).

Two regularizers are then added. The first is a MANO prior loss,

$P_v$7

which penalizes deviation from the MANO hand manifold. The second is a surface-normal alignment loss,

$P_v$8

where $P_v$9 denotes predicted hand contact points, $t$0 is the closest object contact point, and $t$1 is the object surface normal at $t$2. In the manuscript, these displayed equations contain typesetting errors; the technical summary identifies the intended forms above. The resulting GravityDB dynamics are

$t$3

This formulation makes the method a stochastic geometry-refinement process rather than a fully specified score-based bridge model (Xu et al., 3 Sep 2025).

Numerical solution uses Euler–Maruyama: $t$4 with stopping conditions given as $t$5, maximum iteration count $t$6, or satisfaction of physical validity criteria described as “no penetrations, all assigned contact regions are occupied” (Xu et al., 3 Sep 2025).

3. Semantic contact inference and the two-stage refinement pipeline

GravityDB couples geometric refinement to semantic intent through an LLM-Based Contact Region Inference Module. In schematic form,

$t$7

where $t$8 is an instruction text and $t$9 are soft contact regions for hand and object. The paper describes this equation as schematic rather than a full mathematical decomposition. Operationally, the module encodes instruction text using a pretrained 7B Vicuna model, encodes hand and object point clouds with PointNet++, fuses semantic and spatial features through a multilayer projection network and cross-attention, and predicts soft segmentation masks over hand and object points (Xu et al., 3 Sep 2025).

These masks directly modulate the gravity field. Predicted object contact points become Gaussian centers in the potential, and confidence weights $\mathcal{M}(\theta,\beta)$0 determine local attraction strength. As a result, language does not directly deform the hand; it shapes the attraction landscape that drives the refinement. The paper’s intended effect is to steer the hand toward semantically appropriate regions, such as a handle rather than an arbitrary nearby surface (Xu et al., 3 Sep 2025).

The refinement itself is explicitly two-stage. Stage 1, Coarse Guidance, uses a low-resolution field with larger $\mathcal{M}(\theta,\beta)$1 and weaker anatomical constraints to move the hand into the correct topological relation with the object. Stage 2, Fine Refinement, uses a high-resolution field with smaller $\mathcal{M}(\theta,\beta)$2 and stronger MANO and normal constraints to obtain accurate finger placement, stable contact, and collision-free local geometry. At test time, the pipeline is: obtain initial hand and object point clouds, let an MLLM inspect the image and generate an instruction, predict $\mathcal{M}(\theta,\beta)$3 and $\mathcal{M}(\theta,\beta)$4, build the hierarchical gravity field, initialize the hand from the coarse reconstruction, run stage-1 Euler–Maruyama updates, switch to stage-2 refinement, and stop according to the gradient, step-count, or validity criteria. The paper emphasizes that manual instructions are not required at test time (Xu et al., 3 Sep 2025).

A common misunderstanding is to equate this pipeline with a standard conditional diffusion model. The method section explicitly does not define a forward noising process $\mathcal{M}(\theta,\beta)$5, a learned reverse process $\mathcal{M}(\theta,\beta)$6, or a score-matching objective. The “bridge” terminology is therefore operational: it denotes a guided stochastic transition between coarse and refined HOI states, not a textbook DDPM or Schrödinger bridge formulation (Xu et al., 3 Sep 2025).

4. Data, supervision, and implementation profile

The method is evaluated on SHOWMe, GRAB, and HO3D-V3. SHOWMe is described as an object-agnostic hand-object 3D reconstruction benchmark with precise 3D scans; GRAB as a MoCap-based dataset with full-body and hand-object interaction; and HO3D-V3 as an RGB-D hand-object interaction dataset with MANO and object pose annotations. The paper also reports an extended dataset for LLM-guided HOI, with contact regions annotated on 3D hand/object point clouds with instructions, and with hand meshes refined to reflect realistic deformation; annotation specifics are said to be in Appendix D (Xu et al., 3 Sep 2025).

Supervision includes contact-region annotations for hand and object point clouds, instruction text associated with interactions, MANO parameters or hand geometry supervision, and object geometry. For evaluation and some comparisons, the paper explicitly uses ground-truth object meshes. This is important because one stated limitation is that performance with noisy or incomplete object reconstruction is less clear (Xu et al., 3 Sep 2025).

Implementation details are split between the contact module and the refinement dynamics. For the contact module, the reported settings are PointNet++ for spatial encoding, 7B Vicuna as the LLM backbone, LoRA fine-tuning, cross-attention for text-geometry alignment, $\mathcal{M}(\theta,\beta)$7 points with $\mathcal{M}(\theta,\beta)$8 dimensions per point, AdamW optimization, learning rate $\mathcal{M}(\theta,\beta)$9, training for 3 epochs, BCE plus Dice losses, and data augmentation by random rotations, translations, and scaling. For GravityDB refinement, the reported settings are Euler–Maruyama integration with adaptive step size, $P_v$0, $P_v$1, $P_v$2, $P_v$3, gravitational step coefficient $P_v$4, stopping threshold $P_v$5, and maximum diffusion steps $P_v$6 (Xu et al., 3 Sep 2025).

Several details remain unspecified in the paper: batch size, runtime, GPU type, explicit number of scales $P_v$7, explicit form of $P_v$8, and the noise schedule. This suggests that the method is algorithmically clear at the level of field construction and iterative refinement, but not fully formalized as a complete stochastic-process specification.

5. Empirical performance and ablation structure

Evaluation uses two groups of metrics. For hand reconstruction, the paper reports MPVPE, PA-PVE, MPJPE, and PA-MPJPE. For hand-object interaction, it reports IV, PD, PE, and HOMC, although the tables shown in the paper do not include HOMC values (Xu et al., 3 Sep 2025).

In HOI denoising comparisons against TOCH, TOCH w/ Mixstyle, TOCH w/ Aug, and GeneOH, GravityDB achieves its strongest improvements on penetration and proximity metrics. On SHOWMe, the input state has MPJPE 25.42, MPVPE 24.93, IV 5.13, PD 6.02, and PE 14.77; GeneOH reports MPJPE 10.10, MPVPE 10.21, IV 1.44, PD 1.95, and PE 2.81; GravityDB reports MPJPE 9.74, MPVPE 8.23, IV 0.62, PD 0.77, and PE 1.53. On GRAB, GravityDB reports MPJPE 8.91, MPVPE 10.05, IV 0.41, PD 0.55, and PE 1.22, outperforming GeneOH on MPJPE and interaction metrics but not on MPVPE, where GeneOH has 9.23. On HO3D, GravityDB reports MPJPE 10.93, MPVPE 12.07, IV 0.83, PD 0.95, and PE 1.70, again leading on MPJPE and interaction metrics while remaining runner-up on MPVPE (Xu et al., 3 Sep 2025).

For HOI reconstruction on SHOWMe, the reported baselines are HoMan, IHOI, DiffHOI, and Hold. Their results are HoMan 25.42/24.93/5.13/6.02/14.77, IHOI 26.13/26.10/5.40/6.30/15.20, DiffHOI 22.35/21.00/3.50/4.80/10.50, Hold 13.21/12.00/1.85/2.10/4.00, and GravityDB 9.74/8.23/0.62/0.77/1.53 for MPJPE/MPVPE/IV/PD/PE. With ground-truth contact regions, the paper reports $P_v$9 at 8.69/7.90/0.58/0.72/1.42, which is described as only a modest gain over predicted semantic contact regions (Xu et al., 3 Sep 2025).

The ablations identify the gravity field as the dominant component. On SHOWMe, the full model gives MPVPE 8.23, IV 0.62, and PE 1.53. Removing the gravity field yields 14.36, 1.65, and 4.30; removing $P_v^c$0 yields 17.43, 2.40, and 6.10; removing $P_v^c$1 yields 12.18, 1.05, and 2.50. A separate contact-source ablation reports Input at 24.93/5.13/14.77, Ours w/ Gene at 18.21/2.90/8.60, Ours w/ Ray at 15.32/1.85/5.10, Ours w/ PLLM at 8.23/0.62/1.53, and Ours w/ GT at 7.90/0.58/1.42 for MPVPE/IV/PE. The paper interprets this as strong evidence that semantic, text-guided contact-region prediction matters substantially (Xu et al., 3 Sep 2025).

The paper claims stable grasping and physically plausible interaction, but it does not provide a formal grasp-stability energy such as force-closure or friction-cone constraints. “Stability” is therefore supported indirectly through reduced gaps and penetrations, more appropriate contact regions, and the reported interaction metrics rather than through an explicit mechanics model (Xu et al., 3 Sep 2025).

6. Conceptual placement within diffusion-bridge research and principal limitations

GravityDB uses the term “diffusion bridge” in a broader and more operational sense than classical bridge simulation or modern score-based bridge models. In classical multivariate diffusion bridge theory, a bridge is the law of a diffusion conditioned on $P_v^c$2 and $P_v^c$3, and exact or approximately exact samplers can be constructed by coupling a forward diffusion with a time-reversed diffusion and correcting path-space bias through MCMC (Bladt et al., 2014). In learned image-translation bridge models such as DDBM, the bridge is defined through a conditional score $P_v^c$4, a tractable Gaussian bridge interpolation, and reverse-time SDE/ODE dynamics between paired endpoint distributions (Zhou et al., 2023). In UniDB, bridge construction is further recast as stochastic optimal control with a finite terminal penalty coefficient $P_v^c$5, explicitly balancing control energy against endpoint fidelity (Zhu et al., 9 Feb 2025). GravityDB does not instantiate any of these formal bridge definitions. Instead, it uses a manually constructed attraction field and SDE-based stochastic refinement on a hand surface point cloud (Xu et al., 3 Sep 2025).

This naming difference is not merely terminological. The GravityDB method section explicitly does not define a standard diffusion forward process $P_v^c$6, a reverse process $P_v^c$7, a bridge-matching loss, a score function $P_v^c$8, a Schrödinger bridge marginal-matching objective, or explicit transition distributions. This suggests that GravityDB is most accurately classified as a field-guided stochastic bridge process in the broad sense of state-to-state stochastic refinement, rather than as a fully specified score-based or Schrödinger bridge model (Xu et al., 3 Sep 2025).

A plausible analogy is to manually constrained bridge constructions in which an additive drift correction is derived from a time-dependent constraint energy $P_v^c$9 via $P_v^c$0 (Naderiparizi et al., 27 Feb 2025). GravityDB’s use of a potential field, normal-alignment energy, and MANO prior suggests a similar energy-shaped perspective, although that analogy is interpretive rather than stated in the GravityDB paper itself.

The principal limitations reported or directly apparent are that evaluation often relies on ground-truth object shapes; the object is assumed rigid; the refinement depends on a good enough initial coarse state; there is no explicit learned collision model or physics simulator; and important details of the noise model $P_v^c$1 and semantic-conditioning internals remain underspecified. The manuscript also does not present explicit failure cases in the provided text. These constraints locate GravityDB as a high-fidelity HOI refinement method with strong geometric and semantic bias, rather than as a complete probabilistic bridge theory for hand-object interaction (Xu et al., 3 Sep 2025).