Part-Centric Assembly Graph (PAG)
- Part-centric Assembly Graph (PAG) is a graph-based representation that elevates parts and their interfaces as primary units to capture geometry and relational constraints.
- PAG formulations use dynamic message passing and phase-specific connectivity to iteratively refine part poses and ensure coherent assembly.
- Applications of PAG extend to 3D part assembly, object discovery, scene configuration, and CAD retrieval, demonstrating its versatile role in complex assembly tasks.
Searching arXiv for papers on "Part-centric Assembly Graph" and closely related formulations to ground the article in published work. Part-centric Assembly Graph (PAG) is a graph-based representation in which parts, rather than only whole objects, are treated as the primary carriers of geometry, state, and relational structure. Across the literature, the term has been used in several related but non-identical senses: as a dynamic message-passing graph for generative 3D part assembly (Huang et al., 2020), as a graph for single-image-guided 3D part assembly under a related “part graph network” formulation (Li et al., 2020), as an explicit part-and-object relational structure for physically grounded indoor scene configuration (Bai et al., 8 Mar 2026), as a graph for multi-part object discovery in images (Foo et al., 20 Dec 2025), as a multi-state graph for RGB-D tracking during assembly (Li et al., 2024), as an assembly graph for sequence and production-line planning (Hartmann et al., 15 Dec 2025), and as an interface-augmented contact graph for CAD part retrieval (Jignasu et al., 1 Jul 2026). The common principle is that PAG localizes structure at the level where mating interfaces, symmetries, support relations, occlusions, and assembly constraints actually occur, rather than assuming that these are recoverable from object-level abstractions alone.
1. Conceptual scope and formal variants
The most narrowly defined PAG in 3D part assembly is the one introduced in "Generative 3D Part Assembly via Dynamic Graph Learning" (Huang et al., 2020). There, given a set of input parts , each a point cloud , the method forms a fully-connected, self-looped directed graph
with node set and directed edge set (Huang et al., 2020). In that formulation, nodes carry learned part geometry features, edge messages are computed dynamically, and the graph evolves over iterations through learned relation weights and alternating dense and aggregated representations.
A related but earlier formulation appears in "Learning 3D Part Assembly from a Single Image" (Li et al., 2020), where the graph is described as a “part graph network” rather than PAG. Nodes are normalized input part point-clouds, and graph connectivity is phase-dependent: a first graph links geometrically equivalent parts, and a second graph augments this with “top-5 nearest neighbors” under the current assembled-shape metric after an initial pose pass (Li et al., 2020). This suggests that, even within assembly-centric work, PAG-like structures need not be fully connected; they may instead encode task-specific inductive biases such as equivalence-class ambiguity and proximity during refinement.
In "PARSE: Part-Aware Relational Spatial Modeling" (Bai et al., 8 Mar 2026), PAG is broadened into a heterogeneous relational structure over object-level nodes , part-level nodes , object-object edges 0, ownership links 1, and part-part geometric relations 2. The formal definition is
3
where each part-part edge is annotated by a mapping 4 with relation type and optionally named surfaces such as top or bottom (Bai et al., 8 Mar 2026). In this usage, PAG is not merely a message-passing substrate; it is a symbolic-geometric scaffold from which analytic constraints are derived.
Other uses shift the emphasis further. "Multi-Part Object Representations via Graph Structures and Co-Part Discovery" (Foo et al., 20 Dec 2025) constructs a PAG from image segmentations, with nodes as 2D parts and edges between adjacent parts in the segmentation. "GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance" (Li et al., 2024) uses a multi-state PAG in which nodes may denote either single parts or rigid modules formed by previously assembled parts. "A Unified Framework for Automated Assembly Sequence and Production Line Planning using Graph-based Optimization" (Hartmann et al., 15 Dec 2025) defines the part-centric assembly graph as an undirected graph 5 over rigid components and joints. "Linkify: Learning from Interface-Augmented Assembly Graphs" (Jignasu et al., 1 Jul 2026) uses a PAG for mechanical CAD assemblies in which nodes are parts and edges denote physically computed contacts endowed with learned interface embeddings.
Taken together, these formulations indicate that PAG is best understood as a family of part-level graph representations rather than a single canonical data structure. The invariant element is the elevation of parts and their interfaces, relations, or ownership links to first-class graph entities.
2. Node semantics, edge semantics, and feature parameterizations
In the dynamic 3D assembly formulation, node features are produced by a shared PointNet encoder 6, with
7
so each node is a 256-dimensional descriptor of part geometry (Huang et al., 2020). Edge features are not stored explicitly at initialization; instead they are produced on-the-fly by an edge network during message passing (Huang et al., 2020). In the same line of work, RGL-NET also initializes each node by 8 into 9 and uses a fully connected graph including self-loops, with edge messages derived from concatenated node features and scalar relation weights computed from current part transforms (Harish et al., 2021).
The single-image assembly formulation uses substantially richer node features. At 0, each node feature 1 is the concatenation of an image embedding 2 from ResNet-18 over the crop region masked by part 3, a mask embedding 4 from a separate ResNet-18 over the predicted binary mask 5, and a 3D context-aware feature 6 built from PointNet geometry and instance/class context, yielding 7 (Li et al., 2020). Edge features are initialized to 8 on 9, while newly introduced phase-2 edges in 0 receive initial value 1 (Li et al., 2020). This design makes edge state itself a coarse indicator of graph phase.
In the image-based co-part discovery setting, node features are explicitly geometric but two-dimensional. After Felzenszwalb’s algorithm segments an image into 2 disjoint parts 3, each part 4 is represented by its centroid 5 and 6 uniformly sampled boundary pixels 7; the node feature 8 stacks the vectors from centroid to sampled boundary pixels, and may equivalently be flattened into a 9-dimensional vector 0 (Foo et al., 20 Dec 2025). Edges are undirected and exist iff two parts appear in the same image and share a nonzero boundary length; the edge attribute is the centroid displacement
1
In PARSE, node and edge semantics are more explicitly typed. The node sets are divided into objects and parts, while part-part edges carry relation labels such as “on,” “in,” and “against,” and optionally two named surfaces (Bai et al., 8 Mar 2026). The graph therefore couples topology with relational ontology. In GBOT, each node carries a semantic part-type label and a current 6D pose estimate 2, and each undirected edge carries a “ground-truth” relative transform 3 for the fully assembled state (Li et al., 2024). In Linkify, node attributes are part embeddings 4 from a pretrained PointMAE encoder over 5 sampled surface points, and edge attributes are interface embeddings 6 from sampled points on the Boolean-intersection geometry between contacting bodies (Jignasu et al., 1 Jul 2026).
A useful comparison is that some PAGs treat edges primarily as communication channels to be learned online (Huang et al., 2020), some as explicit geometric evidence such as centroid offsets or contact embeddings (Foo et al., 20 Dec 2025, Jignasu et al., 1 Jul 2026), and some as carriers of hard assembly semantics such as joints, support, or containment (Li et al., 2024, Hartmann et al., 15 Dec 2025, Bai et al., 8 Mar 2026).
3. Dynamic graph learning for 6-DoF part assembly
The dynamic PAG backbone in "Generative 3D Part Assembly via Dynamic Graph Learning" (Huang et al., 2020) performs 7 iterations of message passing. At each iteration 8, the model computes edge messages
9
where 0 is an MLP shared across all edges at that iteration (Huang et al., 2020). These messages are modulated by learned scalar relation weights 1 on directed edges 2, and node updates use the normalized weighted average
3
with normalization by 4 to keep scale stable (Huang et al., 2020).
Pose regression is performed after each update:
5
where 6, with 7 a unit quaternion and 8 a translation (Huang et al., 2020). Because the network weights of 9, 0, and 1 differ across iterations, the procedure models a coarse-to-fine refinement of node features and predicted poses (Huang et al., 2020).
A central component is the Part Relation Reasoning Module. For iteration 2, relation scores are computed from the previous iteration’s predicted poses:
3
with 4 so that the first pass is an unweighted average (Huang et al., 2020). This means that geometric state feeds back into graph connectivity strength, making the graph time-varying even when its explicit topology remains fully connected.
The Part Aggregation Module addresses repeated or “geometrically equivalent” parts such as chair legs. Such groups are detected by comparing part PCA sizes and point-cloud Chamfer distances, after which graph learning alternates between a dense node set and a sparse set where each group is merged into one node (Huang et al., 2020). On even iterations, grouped node features are average-pooled,
5
and relation features are pooled analogously; equations (1)–(2) are then run on the sparse graph, followed by unpooling back to the dense graph on the next odd iteration (Huang et al., 2020). The paper states that average-pooling is used for all pooling operations.
The recurrent alternative in RGL-NET preserves the fully connected graph but combines graph reasoning with bidirectional GRUs. At each iteration, raw edge messages are computed as
6
weights 7 are produced from current transforms, and incoming messages are aggregated by a normalized weighted sum (Harish et al., 2021). These are then processed by forward and backward GRUs over an ordered part sequence, with global noise injected into boundary hidden states, and pose updates are regressed recurrently from current features, initial features, and previous transforms (Harish et al., 2021). A plausible implication is that PAGs can be embedded within either purely feedforward iterative refinement (Huang et al., 2020) or recurrent state-space models (Harish et al., 2021), while retaining the same underlying commitment to part-level relational reasoning.
4. Supervision, optimization, and empirical behavior
For 6-DoF assembly, the dynamic PAG model supervises translation, rotation, and global assembled shape. Each pose 8 uses translation loss
9
rotation loss
0
and global shape loss
1
(Huang et al., 2020). To permit multi-modal outputs, the method employs a Min-of-N loss: it samples 2, feeds 3 through the network 4 times, and keeps only the best match to the single ground-truth assembly; in practice 5 (Huang et al., 2020). The network is trained with Adam at learning rate 6 for about 200 epochs, with batch size 32 shapes, 7 iterations, and supervision of 8 at every iteration; equal weights are used on 9, 0, and 1 inside the MoN loss (Huang et al., 2020).
The reported metrics are Shape Chamfer distance, Part accuracy defined as the percentage of parts whose 2, and Connectivity accuracy defined as the percentage of ground-truth adjacent pairs whose predicted contact points are within 3 (Huang et al., 2020). On PartNet Chair/Table/Lamp, the full model outperforms three baselines—a global one-shot regressor, bidirectional LSTM, and component-pooling network—by large margins in all three metrics (Huang et al., 2020). In ablation, removing the GNN backbone drops part accuracy from 4; adding only relation reasoning yields 5, adding only part aggregation yields 6, and the full dynamic graph yields 7 (Huang et al., 2020). Performance peaks at 8; fewer points yield slightly worse performance; and max-pool versus mean-pool has similar effect (Huang et al., 2020).
The single-image assembly model uses a different supervision regime. It first trains a mask segmentation module with a soft-IoU loss under Hungarian matching within each equivalence class, then enables the PAG module for pose prediction (Li et al., 2020). The pose loss combines translation, a symmetric-robust rotation Chamfer term over rotated point clouds, an 9 regularizer on the full point set, and a holistic assembly Chamfer loss,
0
with weights 1, 2, 3, and 4 (Li et al., 2020). Evaluation is on PartNet Chair/Table/Cabinet using Part Accuracy and Shape Chamfer Distance (Li et al., 2020).
RGL-NET also uses a minimum-over-5 loss with translation, rotation, and global shape Chamfer terms (Harish et al., 2021). The abstract reports up to 6 improvement in part accuracy and up to 7 improvement in connectivity accuracy on PartNet relative to the then current state of the art (Harish et al., 2021). This suggests that progressive graph refinement and recurrent memory improve not only pose fidelity but also relational coherence at predicted contacts.
5. Explicit relational and constraint-based PAGs
In PARSE, PAG is a source of analytic geometric constraints rather than only latent messages. Each part-part edge 8 with label 9 induces constraints on rigid-body poses 00 and 01 (Bai et al., 8 Mar 2026). For contact relations such as “on,” the method enforces non-penetration and proximity:
02
For coplanarity and normal alignment, if 03 denotes the unit normal of the named surface in canonical pose and 04, then
05
for antiparallel normals, together with a coplanarity condition involving canonical offsets of named surfaces’ centroids (Bai et al., 8 Mar 2026). For containment (“in”), every vertex 06 of the moving part’s mesh must satisfy half-space inequalities describing the container volume:
07
The associated Part-Aware Spatial Configuration Solver places objects in topological order of a support DAG and solves a feasibility or soft-penalty optimization over pose,
08
where 09 collects residuals for the relation and 10 is signed mesh distance (Bai et al., 8 Mar 2026). In practice, PARSE performs “coarse localization” by projecting to 2D support regions and object-level constraints, then “part-level alignment” by analytically intersecting linearized constraints before sampling a feasible pose and rejecting on collisions; a brief physics-based refinement then settles minor penetrations and ensures stability (Bai et al., 8 Mar 2026). The paper provides the explicit running example of a book on a table, with a PAG edge between the book bottom and table top surface and constraints enforcing exact contact, antiparallel normals, and 2D support-region membership (Bai et al., 8 Mar 2026).
GBOT uses a different explicit-relational PAG. Assembly is organized into discrete states 11, each with a graph 12 whose nodes represent either single parts or rigid modules formed by already assembled parts [(Ye et al., 2024)?] Wait.
In GBOT, each edge encodes a kinematic link or mating constraint active in state 13 and carries a ground-truth relative transform 14 (Li et al., 2024). At runtime, RGB-D observations from YOLOv8Pose + PnP initialize per-part poses, and graph-based tracking refines them while testing candidate edges not yet in 15 by checking whether the observed relative transform is within translational and rotational tolerances 16 and 17 of the stored ground-truth relative transform (Li et al., 2024). If a node is fully occluded, its pose is propagated via a connected partner:
18
Pose refinement minimizes a graph-regularized energy
19
and is solved via Gauss-Newton on 20 (Li et al., 2024).
In PyCAALP, the part-centric assembly graph is the undirected assembly graph
21
where vertices are rigid components and edges are physical joints or mating interfaces (Hartmann et al., 15 Dec 2025). Geometric feasibility is captured through a symmetric relational matrix 22 with labels contact, blocking, or free, and through a 23 degree-of-freedom matrix 24 for contacting or blocking pairs, whose entries indicate whether specific translational or rotational motions are collision-free (Hartmann et al., 15 Dec 2025). During assembly sequence generation, a proposed joint is pruned if all relevant translational degrees of freedom in the mating direction are blocked (Hartmann et al., 15 Dec 2025). Here PAG functions as a constraint-bearing substrate for reverse-assembly enumeration and downstream MIP-based production-line planning rather than as a learned representation.
6. Beyond assembly: discovery, retrieval, and robustness
The PAG notion has also been used outside direct 6-DoF assembly prediction. In "Multi-Part Object Representations via Graph Structures and Co-Part Discovery" (Foo et al., 20 Dec 2025), PAG is built from segmented images and used for explicit clustering and matching rather than gradient-based graph learning. Parts are clustered based on cosine similarity between boundary-vector shape descriptors,
25
and merges also require relational consistency in neighboring edge patterns, namely small 26 and large 27 (Foo et al., 20 Dec 2025). Each discovered object must form a connected subgraph of 28 (Foo et al., 20 Dec 2025). The algorithm iterates over unprocessed parts, finds similar parts above threshold 29, checks incident-edge compatibility, merges connected part-sets into objects, and returns the discovered object set 30; convergence is stated to be guaranteed in 31 time (Foo et al., 20 Dec 2025).
That work also uses PAG for robustness under occlusion and out-of-distribution backgrounds. Under occlusion of up to 32 of each object, the method builds partial graphs of visible parts, matches them against a memory of complete objects using the same cosine-similarity test, and fills in missing parts via the stored full-graph prototype (Foo et al., 20 Dec 2025). On the AbsScene-C benchmark, the method is reported to generalize because it relies on geometry-based part shapes and relative-pose vectors rather than raw appearance (Foo et al., 20 Dec 2025). The reported results include 33 ARI on Tetrominoes, 34 ARI on clean AbsScene, 35 ARI on GSO, and 36 ARI on MS COCO versus 37 for MaskCut; on AbsScene-C, ECO-Net achieves ARI / mDice / mIoU of 38, versus 39 for DINOSAUR (Foo et al., 20 Dec 2025).
In Linkify, PAG is specialized for interface-aware part retrieval in CAD assemblies (Jignasu et al., 1 Jul 2026). Starting from each assembly, the method identifies overlapping part-body pairs via axis-aligned bounding boxes, then computes precise B-Rep contacts with OpenCASCADE’s Boolean common operation. Whenever the common geometry is nonempty and has area 40, an edge 41 is declared (Jignasu et al., 1 Jul 2026). Node and edge embeddings are then propagated through GATv2Conv layers with dynamic attention:
42
43
44
so interface features influence both attention scoring and aggregation (Jignasu et al., 1 Jul 2026). The training objective is masked part prediction over 45 part clusters using class-weighted cross-entropy (Jignasu et al., 1 Jul 2026). Reported performance with PointMAE embeddings is Top-1 46, Top-3 47, Top-5 48, Top-10 49, Top-50 50, and F1 51; a logistic-regression baseline on neighborhood-pooled node features achieves Top-1 52 and F1 53, while k-NN gives Top-1 54 (Jignasu et al., 1 Jul 2026). Ablations show that random edge rewiring drops Top-1 from 55, deleting 56 of edges yields 57, no edge attributes yields 58, random edge attributes yields 59, and replacing GATv2 with standard GAT reduces Top-1 to 60 (Jignasu et al., 1 Jul 2026).
These works collectively indicate that PAG is not tied to any single downstream task. It can support clustering, retrieval, robust recognition, and structural memory, provided that part-level relations are the signal of interest.
7. Common themes, differences, and research significance
A recurring misconception is that PAG denotes a single standardized graph architecture. The literature does not support that interpretation. In some papers, PAG is fully connected and directed with self-loops, and relation structure is learned dynamically through scalar weights over all ordered part pairs (Huang et al., 2020, Harish et al., 2021). In others, edges are sparse and semantically prescribed by adjacency in segmentation (Foo et al., 20 Dec 2025), physical contact in CAD (Jignasu et al., 1 Jul 2026), mating constraints across assembly states (Li et al., 2024), or physical joints in a product graph (Hartmann et al., 15 Dec 2025). In PARSE, PAG is explicitly heterogeneous and typed, spanning both object- and part-level nodes (Bai et al., 8 Mar 2026). The term therefore identifies a modeling granularity and relational philosophy more than a fixed topology.
Another important distinction concerns whether PAG is primarily latent or explicit. Dynamic assembly models use PAG as a learnable structure for message passing and pose refinement, with supervision applied to poses and assembled geometry rather than to graph labels themselves (Huang et al., 2020, Harish et al., 2021). By contrast, PARSE, GBOT, and PyCAALP use PAG edges as explicit carriers of geometric or kinematic constraints that can be checked, optimized, or propagated analytically (Bai et al., 8 Mar 2026, Li et al., 2024, Hartmann et al., 15 Dec 2025). Linkify occupies an intermediate position: edge existence is explicit and physically computed, but its interface geometry is embedded and then consumed by a learned attention mechanism (Jignasu et al., 1 Jul 2026).
The shift from object-centric to part-centric relational structure is a consistent motivation across these works. In 3D assembly, ambiguities arise from repeated parts and long-range inter-part dependencies (Huang et al., 2020, Li et al., 2020). In scene generation, object-level relations are “too coarse to specify which regions actually support, contain, or contact one another,” motivating part-level PAGs with named surfaces and analytic constraints (Bai et al., 8 Mar 2026). In object discovery under occlusion and distribution shift, explicit graph representations are introduced because implicit object-centric methods “fail to recognize these learned objects in occluded or out-of-distribution contexts” (Foo et al., 20 Dec 2025). In CAD retrieval, interface geometry is emphasized because “function is realized” at contacts between parts rather than in isolated part shape (Jignasu et al., 1 Jul 2026).
This suggests a broader interpretation of PAG as a unifying abstraction for situations in which the decisive information lies in how substructures meet. Whether the downstream problem is pose estimation, scene feasibility, assembly tracking, sequence planning, unsupervised discovery, or masked-node retrieval, PAG concentrates the representation on parts, interfaces, and relational constraints at the scale where physical and structural semantics are expressed.