PartCrafter: Compositional Part-Aware Design and Fabrication Systems

PartCrafter refers to computational systems and methodologies for generating, decomposing, assembling, and documenting physical and digital objects via compositional, part-aware abstractions. The term encompasses a range of recent research innovations in generative 3D modeling, fabrication-aware design, robotic assembly, rapid prototyping, workflow exploration, and knowledge documentation. PartCrafter systems are unified by their emphasis on semantic parts, structured representations, and interoperable craft workflows, significantly advancing object-level synthesis, assembly, and collaborative expertise sharing.

1. Compositional Generation and 3D Part Synthesis

PartCrafter’s core innovation is the unified, part-aware 3D mesh generation paradigm that eschews classical monolithic or two-stage (segmentation-then-reconstruction) methods. The architecture, as described in "PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers" (Lin et al., 5 Jun 2025 ), builds on pretrained mesh diffusion transformers but introduces a compositional latent space, where each semantic part is encoded by disentangled latent token sets: $\mathcal{O} = \{\mathbf{p}_i\}_{i=1}^N, \quad \mathbf{p}_i \leftrightarrow \boldsymbol{z}_i \in \mathbb{R}^{K \times C}$ Each part token set is further tied to a learnable part identity embedding, enabling permutation-invariant and semantically labeled representations. A hierarchical attention mechanism alternates between local (intra-part) and global (inter-part) attention steps: $\mathbf{A}_i^{\text{local}} = \text{softmax}\left(\frac{\boldsymbol{z}_i \boldsymbol{z}_i^T}{\sqrt{C}}\right), \quad \mathbf{A}^{\text{global}} = \text{softmax}\left(\frac{\mathcal{Z} \mathcal{Z}^T}{\sqrt{C}}\right)$ where $\mathcal{Z}$ represents the concatenation of all part tokens. The image prompt is encoded and injected into this stack via cross-attention, ensuring global shape and fine part details are jointly optimized.

Unlike prior part-wise or segmentation-dependent pipelines, PartCrafter's end-to-end design allows for simultaneous denoising and mesh construction of all parts, including those occluded or not directly visible in the input image. The output is a set of globally aligned, individually editable 3D meshes, with applications spanning asset creation, simulation, robotic manipulation, and compositional editing.

2. Parameterization, Assembly, and Fabrication-Awareness

Fabrication-aware design principles are foundational within the PartCrafter landscape. The system proposed in "Fabrication-aware Design for Furniture with Planar Pieces" (Yan et al., 2021 ) demonstrates a parameterized abstraction:

• Components are defined via

$\text{Component}(para_1, para_2, ..., para_n)$

permitting library-driven or user-defined classes (e.g., Rectangle, Trapezoid) with constrained dimensions and meta-parameter tying.

• Connections are specified abstractly as

$\text{Connection}\left((C_A, I^i_A), (C_B, I^j_B), P_A, P_O, P_R\right)$

encoding component pairs, interfaces, alignment, offsets, and rotations.

• Hierarchical assemblies are composed recursively, with data structures:

$M^i = (\{Component^i\}, \{Connection^i\}, \{Constraint^i\}, \{Interface^i\})$

supporting unbounded composability (e.g., embedding bunk bed, desk, and shelf hierarchies).

Automatic intersection detection, joint synthesis (using manufacturability-driven formulas for finger, hole, and slot joints), and interference-fit output file generation guarantee that exported models are suitable for digital fabrication (laser-cutting, CNC routing) and user assembly without adhesives or fasteners.

3. Reverse Engineering, Robotic Assembly, and Component Selection

Beyond generation, PartCrafter encompasses techniques for extracting part structure from physical exemplars and selecting assembly components:

• In "Fabrication-Aware Reverse Engineering for Carpentry" (Noeckel et al., 2021 ), a pipeline is described that infers fabricable, CAD-editable assemblies from RGB imagery using multi-view stereo, primitive fitting, energy minimization, and fabrication-constrained assembly inference. The result is a part-decomposed, parametrically editable object ready for re-fabrication or customization.
• "Component Selection for Craft Assembly Tasks" (Isume et al., 19 Jul 2024 ) formalizes the challenge of assembling a target object from available (non-matching) candidates, leveraging vision transformers for part segmentation, template mesh retrieval and pose optimization via differentiable rendering, simplification to primitives, and proportion-aware matching. Quantitative and qualitative benchmarks show the viability of data-driven component selection in human and robotic settings.

4. Procedural and Hierarchical Compositionality

Compositional generation is extended to procedural contexts in "Hierarchically Composing Level Generators for the Creation of Complex Structures" (Beukman et al., 2023 ), where hierarchical module trees recursively compose independent generators (e.g., town, house, garden) for scalable structure synthesis. Each generator operates over a subset of the architectural topology, optimizing localized objectives and allowing interpretable mapping from specification to complex assembly. This recursive, modular approach is directly applicable to part-wise asset generation and can empower PartCrafter systems for both virtual and physical object ecosystems.

5. Workflow Exploration, Documentation, and Knowledge Sharing

Exploration and documentation of craft workflows are central to effective PartCrafter systems. "CAMeleon: Interactively Exploring Craft Workflows in CAD" (Feng et al., 23 Oct 2024 ) presents a modular, extensible CAD tool that allows users to upload designs, preview and compare fabrication workflows (both machine and human-in-the-loop), and iteratively defer workflow selection. Its architecture is functionally defined as: $\text{Workflow}_i: S_{\text{vol}} \rightarrow \langle G_{\text{workflow}}, I_{\text{machine}}, I_{\text{human}} \rangle$ where $S_\text{vol}$ is the input shape and outputs are geometry plus multi-modal instructions.

For knowledge capture and sharing, "(De)composing Craft: An Elementary Grammar for Sharing Expertise in Craft Workflows" (Batra et al., 12 Jun 2025 ) formalizes an elementary grammar supporting patterns such as granularity shifts, reflective loops, branches, and revision cycles. The CraftLink interface operationalizes this grammar using graph-based representations, multimodal LLM analysis of narrated workflow videos, and interactive annotation, facilitating collaborative, non-linear archives of craft expertise.

6. Rapid Prototyping and Part-Level Generation

For design prototyping, targeted part generation is critical. "PartStickers: Generating Parts of Objects for Rapid Prototyping" (Zhou et al., 7 Apr 2025 ) establishes a text-to-image diffusion pipeline (built on Stable Diffusion + LoRA) specifically trained to produce isolated object parts ("stickers") from prompts such as "a wing of a bird". The process uses part segmentation masks, neutral backgrounds, and prompt pairing for data construction, yielding significant fidelity and alignment gains (e.g., FID 39.52 and SSIM 0.74, exceeding baseline models). This part sticker corpus directly supports compositional workflows in digital prototyping, virtual assembly, and customizable asset pipelines.

7. Optimization, Automation, and Design-Fabrication Co-Search

Advanced optimization plays a central role in crafting parts and assemblies:

• "Co-Optimization of Design and Fabrication Plans for Carpentry" (Zhao et al., 2021 ) employs bi-level optimization using equality-graph (e-graph) data structures to simultaneously optimize designs and fabrication plans. A Bag of Parts (BOP) e-graph captures shared substructures for amortized cross-variant optimization, while the ICEE algorithm navigates the exponential search space efficiently to discover Pareto-optimal tradeoffs.
• "Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example" (Dilhara et al., 11 Feb 2024 ) demonstrates analogous principles in the coding domain, where LLM-powered generative variant expansion and filtering yield large, validated rule sets for automated code transformations—a methodological template for extending rule- or example-driven part crafting in digital and hybrid systems.

8. Summary Table: Core Dimensions Across PartCrafter Systems

Conclusion

PartCrafter constitutes a converging line of research on compositional, part-oriented design, generation, assembly, and documentation systems. Through advanced latent representations, fabrication and assembly constraints, automated optimization, hierarchical procedural strategies, and collaborative workflow documentation, PartCrafter methodologies enable the efficient creation, manipulation, prototyping, and sharing of objects and practices—spanning physical fabrication, digital content, and knowledge transfer within and across communities.