---
title: 'FIGNet: Face Interaction Graph Network'
url: https://www.emergentmind.com/topics/fignet
type: topic
---

# FIGNet: Face Interaction Graph Network

Searching arXiv for FIGNet-related papers to ground the article in the current literature.
FIGNet, short for Face Interaction Graph Network, is a learned simulator for rigid-body dynamics that represents contact through interactions between triangular mesh faces rather than only between mesh nodes. It was introduced to address a persistent failure mode of node-based and particle-based learned physics models on sparse rigid meshes: collisions often occur on faces or edges far from any pair of nearby vertices, while dense particleizations become expensive on complex geometry. In its original formulation, FIGNet combines mesh vertices, rigid-object nodes, and face-face interaction structure in a graph neural architecture for contact-rich rigid dynamics; later work introduced a memory-efficient FIGNet* variant for real-world scenes with editable NeRF-based perception, and an action-conditioned extension that predicts both motion and reaction force-torque for control and state estimation in manipulation [2212.03574] [2401.11985] [2509.12151].

## 1. Origins and problem formulation

FIGNet was proposed in the context of rigid collisions among arbitrary shapes, a setting described as notoriously difficult because of complex geometry and the strong non-linearity of the interactions. The original work identifies three limitations in prior approaches. First, node-based GNN simulators such as MeshGraphNets connect pairs of vertices within a fixed collision radius; when collisions occur on faces or edges far from any two vertices, they can be missed, especially for sparse meshes with large triangles. Second, particle-based methods avoid this by densely sampling object volume, but this incurs $O(N^2)$ collision tests and rapidly becomes intractable on complex shapes. Third, analytical rigid-body solvers cope poorly with complex geometry and friction because their contact formulations are non-smooth, parameter tuning is tedious, and real-world outcomes can be mis-predicted because of sim-to-real gap [2212.03574].

The central idea of FIGNet is to build a graph whose nodes represent mesh vertices and object centers, while interactions between triangular faces on different objects are represented explicitly. In the original formulation these are face-face hyper-edges, activated whenever the minimum distance between faces is at most a collision radius $d_c$. By computing collision features at the closest points between two faces, FIGNet is designed to capture contact location, normal, and frictional dynamics even on very sparse meshes. The paper reports that this face-interaction representation remains sparse because only faces within a small radius generate interaction edges, yet avoids the collision-missing behavior of node-only formulations; on complex shapes it is reported as around $4\times$ more accurate than learned node- and particle-based methods while also being $8\times$ more computationally efficient on sparse rigid meshes [2212.03574].

This positioning places FIGNet within learned simulation rather than generic graph representation learning. Its stated application domains include robotics, graphics, and mechanical design, with later work making the robotics connection more explicit through model-predictive control for peg-in-hole insertion and through perceptual deployment in real-world scenes [2212.03574] [2509.12151].

## 2. Graph representation and contact geometry

In the original rigid-dynamics model, a scene at time $t$ is represented by a graph with mesh nodes $V^M$, one object node $v^O$ per rigid body, regular directed edges among mesh nodes, object-to-mesh and mesh-to-object edges, and face-face hyper-edges between faces on different objects. Mesh nodes carry finite-difference motion history together with static properties and control status. The detailed node feature for a mesh node is
$$
x_i = [v_i^{(t)}-v_i^{(t-1)},\; v_i^{(t-h+1)}-v_i^{(t-h)},\; p_i,\; k_i,\; f_i^t],
$$
with history length $h=2$, where $p_i$ includes static properties such as mass, friction, and restitution; $k_i$ is a kinematic flag; and for kinematic nodes $f_i^t = k_i\cdot(x_i^{t+1}-x_i^t)$. Object nodes provide long-range communication across each rigid body by connecting to all mesh nodes of that body [2212.03574].

The distinguishing geometric structure lies in the face-face interaction features. For a directed hyper-edge $F_s \to F_r$, the original paper defines closest points $p_s$ and $p_r$ on the sender and receiver faces, the closest-point displacement $d^F_{rs}=p_r-p_s$, three spanning vectors on each face relative to the closest point, face normals $n_s,n_r$, and friction coefficients $\mu_s,\mu_r$. The feature vector is
$$
q^{\mathrm{features}}_{F_s\to F_r}
=
[d^F_{rs},\; \{d^F_{s_j}\}_{j=1..3},\; \{d^F_{r_j}\}_{j=1..3},\; n_s,\; n_r,\; \mu_s,\; \mu_r].
$$
This construction is intended to encode both contact geometry and frictional context directly at the face level rather than indirectly through nearby vertices [2212.03574].

The action-conditioned extension reformulates the graph as a heterogeneous graph
$$
G^{in}=(V^M,V^O,V^W,E^{M\to M},E^{O\leftrightarrow M},E^{W\leftrightarrow M}),
$$
adding a third node type, world nodes $V^W$, with two world nodes per object: a type-$f$ node for force and a type-$\tau$ node for torque. Mesh-mesh edges connect pairs of vertices whose faces fall within a collision sphere, object-mesh edges link each object node to its mesh vertices, and world-mesh edges connect each world node to every mesh node of the corresponding body. The world-mesh edge features explicitly encode the applied wrench: for force edges, $e^{f\to m_i}=[f_t,\|f_t\|]$, and for torque edges, $e^{\tau\to m_i}=[\tau_t,\|\tau_t\|,p^{m_i}-CoM,\|p^{m_i}-CoM\|]$. The paper states that this formulation has no explicit global state variables and is purely node/edge-level [2509.12151].

## 3. Encode–process–decode mechanics

The original FIGNet uses message passing over both regular edges and face-face hyper-edges. At each message-passing iteration $l$, regular edges are updated by combining the current edge latent and the sender and receiver node latents, while each face-face hyper-edge produces three latent vectors, one for each receiver node on the receiving triangle. Mesh-node updates aggregate both regular incoming edge messages and incoming face-interaction messages; object-node updates aggregate messages from mesh nodes. Predicted accelerations are decoded from the final node embeddings, and positions are integrated with a second-order Euler rule,
$$
x_i^{t+1}=a_i+2x_i^t-x_i^{t-1}.
$$
Training uses per-node mean-squared error on accelerations or positions plus $L_2$ regularization on MLP weights. The architecture details reported in the paper are 2 hidden layers of size 128 for encoders and processors, LayerNorm after each hidden layer, ReLU activations, residual connections at each step, and unshared weights across 10 message-passing iterations [2212.03574].

The action-conditioned model adopts an explicit encoder–processor–decoder stack over the heterogeneous graph. For each node type $X\in\{M,O,W\}$ and each edge type $X\to Y$, a 2-layer MLP encoder maps raw features to learned embeddings. The processor repeats for $N=10$ message-passing steps, with edge updates
$$
e^{\ell+1}_{X\to Y,s\to r}
=
\phi^{proc,\ell}_{E^{X\to Y}}
\big([e^\ell_{X\to Y,s\to r},v^{X,\ell}_s,v^{Y,\ell}_r]\big)
$$
and node updates formed by summing incoming processed edges and applying a node MLP. The processor MLPs are stated to use residual connections and LayerNorm. Decoding produces both vertex accelerations and reaction force/torque. Specifically, mesh-node decoding outputs $\hat a_t^{m_i}$, while the reaction wrench is decoded from final mesh-to-world edge states and averaged over tool vertices to obtain $\hat f_t$ and $\hat\tau_t$ [2509.12151].

Post-processing in the action-conditioned formulation proceeds by Euler integration of predicted vertex accelerations,
$$
\hat p_{t+1}^{m_i} = \hat a_t^{m_i}\Delta t^2 + 2p_t^{m_i}-p_{t-1}^{m_i},
$$
followed by an alignment step to recover body pose from the predicted mesh and finite differencing to recover body velocities. The supervised training objective is
$$
L = \lambda_{pos}L_{pos} + \lambda_f L_f + \lambda_\tau L_\tau,
$$
with $\lambda_{pos}=1$ and $\lambda_f=\lambda_\tau=0.1$, where $L_{pos}$ is mean-squared position error and $L_f,L_\tau$ are mean-squared force and torque errors [2509.12151].

## 4. Quantitative performance of the original simulator

The original FIGNet paper evaluates rigid-dynamics prediction on Kubric MOVi-A and MOVi-B, compares against node-based and particle-based baselines, and reports both accuracy and graph sparsity. On MOVi-A after 50 steps, FIGNet attains translation RMSE $0.115\pm0.008$ m and rotation RMSE $14.4\pm0.2^\circ$ with 233 collision edges. On MOVi-B, it attains translation RMSE $0.127\pm0.006$ m and rotation RMSE $14.0\pm0.5^\circ$ with 1386 edges. The same tables show that node-based baselines either miss collisions at small collision radius or require much denser graphs at large radius, while particle-based DPI* uses substantially more edges [2212.03574].

| Dataset | FIGNet | Selected comparison |
|---|---|---|
| MOVi-A | $0.115\pm0.008$ m, $14.4\pm0.2^\circ$, 233 edges | MGN-LargeRadius+: $0.119\pm0.009$ m, $15.1\pm0.6^\circ$, 10637 edges |
| MOVi-B | $0.127\pm0.006$ m, $14.0\pm0.5^\circ$, 1386 edges | MGN*: $0.538\pm0.035$ m, $26.9\pm0.8^\circ$, 34 edges |
| MOVi-B | $0.127\pm0.006$ m, $14.0\pm0.5^\circ$, 1386 edges | DPI*: $0.368\pm0.057$ m, $26.9\pm2.7^\circ$, 2251 edges |

The efficiency results further sharpen the intended tradeoff. On MOVi-A, the paper reports single-step CPU runtime of 0.094 s for FIGNet versus 0.258 s for MGN-LR, which is described as $\times 2.7$ slower. On MOVi-B, FIGNet runtime is 0.342 s, whereas the baselines are listed in the range 0.145–0.218 s, but the paper notes that those baselines use an implicit floor and miss real collisions. The interpretation advanced in the paper is that face-level collision modeling improves accuracy without requiring the quadratic blow-up in node-level collision edges that occurs when large collision radii are used [2212.03574].

Ablation studies emphasize which components are structurally important. Removing face-face edges (“NoFaceCollision”) increases translation error by more than $3\times$ on MOVi-A, and even with large $d_c$ cannot match FIGNet on MOVi-B. Removing the object node doubles translation error on MOVi-B and increases rotation error by approximately $5^\circ$. FIGNet is reported as stable for collision radii $d_c\in[0.05,0.5]$, whereas MGN is not. The same paper also states that on MIT Pushing real data, FIGNet outperforms MuJoCo, PyBullet, and analytic pushing models with as few as 128 trajectories, indicating that the architecture can learn frictional rigid dynamics directly from real-world data rather than only from synthetic supervision [2212.03574].

## 5. FIGNet* and deployment in real-world scenes

“Scaling Face Interaction Graph Networks to Real World Scenes” introduces FIGNet*, a memory-efficient variant intended for scenes with hundreds of objects and complex 3D shapes, and for settings where inputs come from perception rather than full 3D state. The paper identifies mesh-mesh edges $E_{mm}$ as the dominant memory cost in FIGNet, noting that they can exceed 50% of total edges in complex meshes. FIGNet* removes $E_{mm}$ entirely and retains only object-mesh edges $E_{mo}$ and face-face edges $E_{ff}$. Its memory model is summarized by
$$
M_{total}\approx |V|\cdot d_v + |E|\cdot d_e,
$$
with the edge set reduced to $|E|=|E_{mo}|+|E_{ff}|$ rather than including $|E_{mm}|$, and the corresponding memory reduction estimated as $\Delta M\approx |E_{mm}^{orig}|\cdot d_e$ [2401.11985].

Empirically, on Kubric MOVi-B the reported peak GPU memory decreases from $63.4\pm3.3$ GiB for FIGNet to $50.1\pm3.4$ GiB for FIGNet*, while runtime decreases from $26.4\pm0.7$ ms/step to $19.4\pm0.2$ ms and the number of edges drops from 24,514 to 8,630. Accuracy remains comparable: translation RMSE changes from $0.14\pm0.01$ m to $0.13\pm0.01$ m, while rotation RMSE changes from $14.99\pm0.67^\circ$ to $15.96\pm0.87^\circ$. On MOVi-C, the original FIGNet is reported as out of memory, whereas FIGNet* runs at $71.8\pm6.4$ GiB and $20.4\pm0.6$ ms with translation RMSE $0.18\pm0.01$ m and rotation RMSE $19.82\pm0.64^\circ$. The paper states that edge counts drop by approximately 65% by removing mesh-mesh edges [2401.11985].

The same work attaches a perceptual interface that converts real scenes from multi-view RGB images and camera parameters into a structured representation for FIGNet*. The pipeline trains a static NeRF $F_\Phi$ using standard photometric loss, segments the object of interest with XMem on selected images, unprojects masks into 3D to obtain an axis-aligned bounding box, extracts a mesh with Marching Cubes followed by decimation, and builds initial graphs $G^{t-1},G^t$ for rollout. After FIGNet* predicts rigid transformations, the scene is re-rendered by editing the NeRF through ray bending. The paper describes this as a two-way coupling that requires no simulator fine-tuning on real data, and reports qualitative rollouts in 360° captures of a kitchen counter, an outdoor garden, and a “figurines” set. A decimation study further states that FIGNet frequently runs out of memory or yields penetration artifacts, whereas FIGNet* “gracefully degrades and still prevents interpenetration even at 1k faces” [2401.11985].

## 6. Action-conditioned FIGNet for control and force-torque prediction

The 2025 action-conditioned extension generalizes FIGNet from passive next-state prediction to action-conditional dynamics for control and state estimation. Its stated modifications are threefold: the addition of virtual world nodes for force and torque, world-mesh edges carrying control-input embeddings, and a decoder for reaction force/torque. This turns FIGNet into a differentiable physics simulator that predicts both kinematic rollouts and force-torque responses under applied wrench inputs [2509.12151].

The control integration is implemented through model-predictive control using iCEM, with deterministic rollout of the learned FIGNet model. The reported planner settings are planning horizon $H=50$, samples $=20$, iterations $=5$, replanning every 5 steps, and 1 particle. The per-step cost is
$$
r_t = 10\cdot \mathbf{1}_{\|\Delta p_t\|\le \epsilon} + \exp(-\|\Delta p_t\|) + \exp(-\|\Delta \theta_t\|),
\qquad \epsilon = 2\ \mathrm{mm}.
$$
Experiments are conducted on a peg-in-hole task with a UR10e robot using an impedance controller that drives wrench inputs $(f,\tau)$. The tools are triangle, square, and hexagon for seen geometries, circle for unseen geometry against a matching slot, and also circle into square slot as an unseen geometry pairing. Success is defined as insertion depth greater than 2 mm, and the baseline simulator used for comparison is Mujoco XLA (MJX) [2509.12151].

In simulation, the learned model achieves 70% exact success and 90% half-way success, compared with 85% for the MJX baseline. Zero-shot generalization on circle-in-square, which is described as unseen geometry, remains at 70%. The same section reports that FIGNet trajectories converge faster in reward, with comparable variance, and that fine-tuned and augmented-data variants yield only marginal gains, which the paper interprets as robustness to domain shifts. In real-world experiments over 100-step multi-step rollouts and one-step force/torque prediction, the reported metrics are: position RMSE 3.18 mm (22.3% rel.) for FIGNet versus 8.08 mm (56.6%) for MJX; orientation RMSE 0.866 rad (19.9%) versus 1.84 rad (42.0%); one-step force error 0.92 N versus 3.69 N, described as approximately $3\times$ improvement; and one-step torque error 0.038 Nm versus 0.125 Nm. An action-shift test further reports zero-shot transfer to expert peg-in-hole controller actions while still yielding tight trajectory and force/torque tracking and capturing discontinuities at contact events [2509.12151].

## 7. Scope, limitations, and related acronyms

Across the three FIGNet papers, several limitations are stated explicitly. The original rigid-dynamics paper notes that the model uses a deterministic loss and may struggle on highly stochastic contact dynamics, that scalability to millions of faces is untested, and that it requires accurate state information; integrating with learned perception is identified there as future work. The FIGNet* paper addresses the perception gap but states that static physical parameters such as mass, friction, and restitution are set by default rather than inferred from data, that active object selection and segmentation currently requires manual view prompting, that no real-world ground-truth dynamics were available for direct error quantification, and that extension to multi-object active interactions and to articulated or deformable bodies is left to future work [2212.03574] [2401.11985].

A common source of confusion is the existence of acronymically similar but conceptually unrelated graph-learning models. “Feature Interaction-aware Graph Neural Networks” introduces FI-GNN as a plug-and-play framework for personalized second-order feature interactions on feature-sparse attributed graphs, targeting tasks such as node classification and link prediction rather than rigid-body contact simulation [1908.07110]. “FIGNN: Feature-Specific Interpretability for Graph Neural Network Surrogate Models” is a separate architecture for feature-specific interpretability in scientific surrogate modeling on unstructured grids, with feature-wise masks and error-budget regularization; it is not a face-interaction rigid-body simulator [2506.11398].

Taken together, the FIGNet lineage describes a specific research program in learned physical simulation: the original face-interaction simulator for rigid collisions, a memory-reduced and perceptually grounded FIGNet* for real-world scenes, and an action-conditioned extension for control with motion and reaction wrench prediction. A plausible implication is that FIGNet occupies a distinct niche at the intersection of learned contact dynamics, sparse geometric representation, and model-based robotics, rather than within the broader family of feature-interaction or interpretability-oriented GNNs [2212.03574] [2401.11985] [2509.12151].

Source: https://www.emergentmind.com/topics/fignet