Partlets: Implicit 3D Part Representation

• Partlets are implicit 3D part representations that jointly segment shapes and assign natural language labels by integrating geometric, visual, and linguistic data.
• The ALIGN-Parts framework uses transformer-style layers and bi-directional cross-attention to achieve efficient segmentation at approximately 0.05 seconds per shape.
• Partlets support open-vocabulary and confidence-calibrated segmentation, offering complete, non-overlapping part decomposition adaptable to diverse 3D datasets.

A partlet is an implicit 3D part representation that jointly segments a shape into meaningful components and embeds those components into a shared semantic space with natural-language descriptions. Introduced in the context of open-vocabulary 3D part segmentation and naming, partlets are the core units in the ALIGN-Parts framework, which enables efficient, permutation-invariant, and theoretically open-vocabulary decomposition and annotation of 3D shapes. The partlet formalism unifies geometric, visual, and linguistic information in a single, end-to-end trainable model, providing a solution to the problem of inconsistent part ontologies and inefficient pipelines in prior part segmentation systems (Paul et al., 19 Dec 2025).

1. Formal Definition and Mathematical Construction

A partlet is defined as a pair $(\mathbf{m}_k, \mathbf{s}_k)$, where:

• $\mathbf{m}_k \in [0,1]^N$ is a soft mask over the $N$ points of a point cloud $\mathcal{P} = \{\mathbf{x}_i\}_{i=1}^N$, denoting the assignment strength of each point to the $k$-th partlet.
• $\mathbf{s}_k \in \mathbb{R}^{d_t}$ is a prototype embedding in the $d_t$-dimensional space of text embeddings.

The mask $\mathbf{m}_k$ is implicitly defined by: $$m_{ki} = \sigma\left(\frac{1}{\sqrt{d_t}} (\mathbf{W}_q \mathbf{s}_k)^\top (\mathbf{W}_k \mathbf{h}_i)\right), \quad i = 1, \ldots, N,$$ where

• $\mathbf{h}_i \in \mathbb{R}^{d_f}$ is a fused geometric-appearance feature for point $\mathbf{m}_k \in [0,1]^N$0,
• $\mathbf{m}_k \in [0,1]^N$1, $\mathbf{m}_k \in [0,1]^N$2 are learned linear projections,
• $\mathbf{m}_k \in [0,1]^N$3 is the sigmoid function.

Each partlet $\mathbf{m}_k \in [0,1]^N$4 additionally has a scalar partness score $\mathbf{m}_k \in [0,1]^N$5, indicating whether it should be considered as an active part or suppressed as “no-part”.

2. Model Architecture for Partlet Generation

The ALIGN-Parts architecture processes an input point cloud as follows:

1. Geometric 3D Part Field: Each point receives a geometry-first feature $\mathbf{m}_k \in [0,1]^N$6 from a frozen PartField backbone, providing a class-agnostic, continuous encoding of local shape.
2. Multi-View Appearance Features: The shape is rendered from multiple views; image features are extracted by a DINOv2 vision encoder, then lifted to each 3D point to provide appearance features $\mathbf{m}_k \in [0,1]^N$7.
3. Bi-Directional Cross-Attention Fusion: Geometry and appearance features are fused on the $\mathbf{m}_k \in [0,1]^N$8-NN graph ($\mathbf{m}_k \in [0,1]^N$9) via attention mechanisms with Fourier-encoded positional biases. Gated residuals merge these into final fused features $N$0.
4. Partlet Decoder: There are $N$1 learnable, shape-specific partlet embeddings $N$2. These are updated by multiple transformer-style layers that alternately apply self-attention (among partlets), cross-attention (from points into partlets), and MLPs. The final embedding $N$3 is used for both part mask and text alignment.

The overall architecture enables feed-forward inference without iterative clustering or post-processing.

3. Bipartite Assignment and Semantic Alignment

The partlet assignments to ground truth part labels are performed via direct set (bipartite) alignment:

• For a shape with $N$4 annotated parts and corresponding text descriptions, the cost for assigning partlet $N$5 to annotation $N$6 is: $N$7 where $N$8 is the ground-truth mask and $N$9 is the text embedding of its LLM-generated affordance description.
• A soft assignment matrix is computed using Sinkhorn-Knopp, followed by extraction of a hard, permutation-invariant matching $\mathcal{P} = \{\mathbf{x}_i\}_{i=1}^N$0.
• “No-part” assignments allow the set of active partlets to dynamically vary per shape.

The InfoNCE contrastive loss aligns each matched partlet embedding $\mathcal{P} = \{\mathbf{x}_i\}_{i=1}^N$1 with its assigned part’s text embedding, while penalizing similarity to other texts. The full training objective combines mask, partness, coverage, overlap, and global contrastive losses.

4. Inference Regimes: Open-Vocabulary, Zero-Shot, and Confidence

At test time, the unified embedding space allows partlets to operate in:

• Open-vocabulary (zero-shot) mode: For an arbitrary set of textual part descriptions, active partlets select the most similar text embedding for labeling. No retraining is required for new vocabularies.
• Closed-vocabulary (confidence-calibrated) mode: Mahalanobis distances between partlet embeddings and class centroids, together with softmax over text similarity, are fused to provide fine-grained uncertainty estimates. Low-confidence predictions can be routed to human verifiers, facilitating high-precision automated annotation.

5. Key Properties and Theoretical Guarantees

The partlet-centric approach offers several formal and empirical advantages:

• Complete, non-overlapping decomposition: Assignments ensure every point belongs to at most one active part.
• Permutation invariance: Bipartite alignment does not require a canonical ordering of part labels.
• Variable part count: The model activates only as many partlets as needed; surplus partlets are suppressed.
• Feed-forward efficiency: The architecture achieves segmentation and naming in approximately 0.05 seconds per shape, more than $\mathcal{P} = \{\mathbf{x}_i\}_{i=1}^N$2 faster than clustering-based pipelines.
• Open-vocabulary extensibility: Any natural language description, including those generated by LLMs, can serve as a target part label.
• Confidence calibration: Integrated softmax and Mahalanobis heuristics support reliable auto-labeling with human-in-the-loop fallbacks.

6. Applications, Unified Ontology, and Benchmarks

Partlets support several downstream tasks, most notably scalable auto-annotation for large shape collections. The ALIGN-Parts system has been used to create a unified 3D part ontology of 1,794 unique parts across PartNet, 3DCoMPaT++, and Find3D datasets. Examples are presented from Tex-Parts, an additional resource. The partlet framework also introduces two novel metrics tailored for named 3D part segmentation.

A plausible implication is that the partlet paradigm supports further unification of multimodal shape corpora and could serve as a foundation for open-vocabulary 3D understanding at scale. The decoupling of geometric and semantic definitions inherent in partlets addresses longstanding issues with inconsistent part annotation conventions, supporting generalization to novel categories and affordances (Paul et al., 19 Dec 2025).