StructureGraph: Graph-Based 3D Modeling

• StructureGraph representation is a graph-based blueprint capturing semantic part labels, existence, and connectivity in 3D point clouds.
• It integrates convolutional feature extraction, graph attention propagation, and a diffusion transformer for robust structure-aware latent encoding.
• The framework outperforms traditional methods on metrics like MMD and JSD, enabling applications in interactive design, CAD, and VR asset synthesis.

StructureGraph representation refers to a paradigm that encodes the presence and connectivity of semantic parts within 3D point cloud shapes, serving as a structural blueprint for controllable generative modeling (Shu et al., 28 Sep 2025). In the StrucADT framework, this representation forms the foundation for conditioning high-fidelity, structure-consistent point cloud generation via graph-based latent encoding, priors over adjacency, and diffusion-based synthesis.

1. StructureGraph Definition, Construction, and Role

A StructureGraph (SG) is formulated for each 3D point cloud shape $X$ with per-point semantic segmentation $S$. The representation is defined as

$SG = \{S, V, E\}$

where:

• $S$ encodes point-wise semantic labels (part segmentation),
• $V$ is a vector indicating part existence ($v_k = 1$ if part $k$ is present, $0$ otherwise),
• $E \in \{0,1\}^{m \times m}$ is an adjacency matrix, with $e_{ij}=1$ denoting connectivity between parts $i$ and $j$.

Adjacency relationships $E$ are manually annotated for each shape in the ShapeNet dataset. Each semantic part is treated as a node with attributes, and edges encode explicit part-to-part connections, constructing a graph that mirrors the high-level assembly topology (e.g., seat–armrest–back relationships in chairs). This blueprint guides both latent feature extraction and conditional generation, ensuring that the synthesized geometry conforms to the specified part composition and adjacency.

2. Encoding via StructureGraphNet

To infuse both local and structural information from $X$ and $SG$, the StructureGraphNet module produces part-conditioned latent codes. The process comprises:

• 1D convolutional feature extraction: $F_\text{conv}(X) \in \mathbb{R}^{n \times d}$ for $n$ points.
• Segmentation-based pooling: $F_\text{conv}(X)^T \cdot S$ isolates features for each part.
• Graph attention propagation: Local features $z_k$ corresponding to part $k$ are propagated over $SG$ using a graph attention network $F_\text{gat}$, so that the final latent $Z = \{z_1, \dots, z_m\}$ is structure-aware.

A key formula for reparameterization in encoding is:

$$Z = \mu_\phi(X,S,V,E) + \sigma_\phi(X,S,V,E) \cdot \epsilon,$$

where $\epsilon \sim \mathcal{N}(0,I)$, enabling stochastic optimization of the latent conditional distribution.

3. cCNF Prior Over Structure-Conditioned Latents

StrucADT uses a conditional continuous normalizing flow (cCNF) to model the structure-conditioned distribution of latent codes. For each part $j$, the mapping

$w_j = P_\psi^{(j)}(z_j, v_j, e_{j\cdot})$

transforms latent features into a standard base distribution. During generation,

$$\hat{z}_j = (P_\psi^{(j)})^{-1}(\hat{w}_j, \hat{v}_j, \hat{e}_{j\cdot}), \quad \hat{w}_j \sim \mathcal{N}(0,I)$$

samples part features consistent with the specified existence and adjacency structure. KL-divergence loss between the encoding and prior distributions ensures proper structural regularization.

4. Structure-Conditioned Diffusion Transformer Generation

A denoising diffusion probabilistic model (DDPM) is leveraged for structure-guided 3D shape synthesis. The transformer network ingests the latent features, part existence, adjacency matrix, and segmentation as conditioning. During each reverse diffusion step,

• Queries ($Q$) are constructed from the noisy cloud and segmentation $(X, S)$,
• Keys and values are from $(Z, V, E)$ and temporal embedding $\text{Emb}(t)$,
• Cross-attention layers perform context fusion:

$$\text{CrossAttention}(Q,K,V) = \text{Softmax}(Q K^T/\sqrt{d}) V$$

Iterative denoising produces a final point cloud $\hat{X}^{(0)}$ consistent with the prescribed StructureGraph $SG$.

5. Quantitative and Qualitative Performance

Experiments on ShapeNet and StructureNet report metrics including MMD, COV, 1-NNA, Chamfer Distance (CD), EMD, and JSD. StrucADT achieves lower MMD and JSD, and higher COV and SCA (Structure Consistency Accuracy—comparing adjacencies in input and output via network prediction), outperforming methods such as PointFlow, DPM, DiffFacto, SPAGHETTI, and SALAD.

Ablation studies confirm that each submodule—StructureGraphNet, cCNF Prior, and Diffusion Transformer—contributes nontrivially to the fidelity and controllability. The framework generalizes well even to rare structural combinations, exhibiting robustness in shape structure adherence.

6. Applications and Research Implications

StructureGraph-centric generative modeling enables interactive design, CAD, VR asset synthesis, and rapid prototyping where user-specified part connectivity is paramount. The graph-based control paradigm allows precise sampling over the combinatorial space of plausible assemblies, facilitating style, functionality, and structural diversity in generated content.

Plausible future directions include automation of adjacency annotation (via self-/weak-supervision), the integration of multimodal user specifications (including text or sketches), and generalized structure-controlled synthesis in other domains where assembly or connectivity constrains the generative process. The fusion of graph encodings and diffusion models is broadly extensible to settings requiring structurally faithful outputs.

7. Summary Table: StructureGraph Representation in StrucADT

Component	Description	Role in Pipeline
StructureGraphNet	Graph attention fused part-wise latent encoding	Extracts structure-aware features
cCNF Prior	Conditional flow for structure-controlled latents	Regularizes latent distributions
Diffusion Transformer	Structure-conditioned denoising generation	Synthesizes point cloud geometry
SCA Metric	Structure consistency accuracy	Quantifies adjacency fidelity

This framework uniquely leverages explicit graph representations to achieve state-of-the-art controllability in generative 3D modeling, demonstrating the efficacy of structure-based guidance in neural point cloud synthesis.