Duplex Part Latents in 3D Generation

• Duplex Part Latents (DPLs) are modular, part-wise latent units that use separate 3D tokens, 2D tokens, and an identity embedding to maintain semantic independence.
• They enable synchronized geometry–appearance alignment and robust slot persistence across diffusion steps, crucial for stable text-to-3D generation.
• Empirical evidence shows DPLs improve metrics like CD, EMD, and CLIP/ULIP scores, enhancing interpretability and controllability in generated 3D models.

Duplex Part Latents (DPLs) are the part-wise latent units introduced in DreamPartGen for semantically grounded, part-aware text-to-3D generation. In the formulation of DreamPartGen, an object is decomposed into a set of semantic parts, $O=\{p_i\}_{i=1}^N$, and each part is represented not by a single latent code but by a paired 3D-and-2D latent representation together with a persistent part identity embedding. This design is intended to preserve semantic independence, slot persistence, and modularity across denoising timesteps, while enabling geometry–appearance alignment within parts and language-conditioned coordination across parts (Yu et al., 19 Mar 2026).

1. Representational definition

For each semantic part $p_i$, DreamPartGen defines a DPL through three components (Yu et al., 19 Mar 2026):

Component	Definition	Stated role
3D tokens	$$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$	Capture local geometry and spatial structure
2D tokens	$$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$	Capture color, texture, and shading cues
Part identity embedding	$e_i \in \mathbb{R}^d$	Provide persistent slot identity

The 3D tokens are obtained by sampling the part mesh as surface points with normals and passing them through a 3D VAE encoder. The 2D tokens are obtained by rendering the part from multiple views and passing those images through a pretrained image VAE encoder. The identity term $e_i$ is a learnable identifier embedding assigned to each part.

A central clarification is that a DPL is not a monolithic latent. The basic unit is the paired latent triplet

$$\big(\mathbf{L}^{\text{3D}}_i,\mathbf{L}^{\text{2D}}_i,e_i\big),$$

with separate streams for 3D and 2D information and an explicit identifier for persistence across diffusion steps. The paper states that the identity embedding prevents slot swapping and keeps each latent tied to the same part throughout diffusion (Yu et al., 19 Mar 2026).

2. Duplex factorization of geometry and appearance

The term “duplex” refers to the joint representation of two complementary aspects of each part: geometry or spatial structure through $\mathbf{L}^{\text{3D}}_i$, and appearance through $\mathbf{L}^{\text{2D}}_i$ (Yu et al., 19 Mar 2026). The 3D stream captures “local geometry and spatial structure,” while the 2D stream captures “color, texture, and shading cues.”

DreamPartGen therefore maintains separate but synchronized latent streams rather than collapsing these attributes into a single entangled representation. The paper explicitly contrasts this with monolithic latents that entangle everything together. Within the stated design, the separation is intended to preserve local geometric detail, preserve visual appearance detail, align the two modalities within the same part, and keep part identities stable across timesteps (Yu et al., 19 Mar 2026).

This organization also constrains the interpretation of DPLs. They are not solely geometric part embeddings, and they are not merely appearance descriptors attached to mesh tokens. Their definition is explicitly bimodal at the part level. A common misunderstanding is to equate DPLs with a conventional part code enriched by rendering features; the paper instead describes a paired latent structure whose streams remain distinct and are synchronized during denoising.

3. Persistence, modularity, and contrast with prior part-level latents

DreamPartGen positions DPLs against prior structured or part-level latent designs that typically operate on voxel-aligned local features, capture shape and texture, but remain tied to spatial grids rather than semantic components (Yu et al., 19 Mar 2026). According to the paper, such unified structured latents therefore lack explicit part-level disentanglement, modularity across objects, stable identity tracking, and relational reasoning support.

The claimed improvements of DPLs are fourfold. First, they provide permutation robustness, because learnable part identities make the semantics independent of the input ordering of parts. Second, they provide slot persistence across denoising, because the identity embeddings stabilize part tracking and reduce slot swapping. Third, they provide cross-object generalization, because each part is represented as a modular latent triplet that can transfer across objects with shared functional components. Fourth, they are designed for modularity and lightweight integration with diffusion and collaborative denoising (Yu et al., 19 Mar 2026).

The paper also states that DPLs are designed to support language-conditioned relational reasoning, unlike earlier geometry-only part latents. This distinction is important: DPLs themselves encode the geometry and appearance of individual parts, but their full role emerges only when they are coordinated with the semantic system used in DreamPartGen. A related misconception is that DPLs alone provide inter-part semantics. The paper does not state that; instead, inter-part dependencies are captured by Relational Semantic Latents.

Ablation evidence in the paper supports the significance of the identity component. Removing the part identifier worsens CD from $0.771$ to $p_i$0, EMD from $p_i$1 to $p_i$2, IoU from $p_i$3 to $p_i$4, and ULIP-T from $p_i$5 to $p_i$6 (Yu et al., 19 Mar 2026). The reported interpretation is that identity embeddings are crucial for disentanglement and semantic consistency.

4. Interaction with Relational Semantic Latents

DPLs operate together with Relational Semantic Latents (RSLs), the second latent system in DreamPartGen (Yu et al., 19 Mar 2026). The division of labor is explicit: DPLs represent the geometry and appearance of individual parts, whereas RSLs represent language-derived part attributes and inter-part relations.

The paper defines RSLs as “persistent semantic control signals” with two roles. The first is a set of global relational tokens, $p_i$7, described as fixed structural conditions encoding inter-part predicates. The second is a set of local semantic tokens, $p_i$8, which are diffused and denoised alongside the part latents to refine part-level details. Global relational tokens are derived from canonicalized triplets $p_i$9, where $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$0 may be support, attach, symmetry, articulation, and related relations (Yu et al., 19 Mar 2026).

The interaction between DPLs and RSLs occurs at two levels. Intra-part synchronization aligns $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$1 and $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$2 and then injects $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$3 to refine the part. Inter-part synchronization propagates context across all parts and injects $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$4 to enforce object-level relational constraints (Yu et al., 19 Mar 2026). The paper identifies this as the core collaborative latent denoising idea: DPLs provide part-wise structure, while RSLs impose language-grounded relational organization.

This division addresses another potential misconception. DPLs are not presented as a standalone semantic parser of relations among parts. Their contribution is modular part representation with persistent identity and duplex geometry–appearance structure; RSLs supply the explicit language-derived relational control.

5. Synchronized co-denoising

The operational role of DPLs is specified through synchronized co-denoising. At each diffusion timestep $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$5, the model jointly denoises part geometry latents $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$6, part appearance latents $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$7, and local semantic tokens $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$8, while keeping $$\mathbf{L}^{\text{3D}}_i \in \mathbb{R}^{T_{\text{3D}}\times d}$$9 fixed as a persistent planner (Yu et al., 19 Mar 2026).

Within intra-part synchronization, the 3D and 2D streams attend to one another, and both streams then attend to local semantic tokens. The paper’s interpretation is direct: 3D and 2D streams attend to each other to keep geometry and appearance aligned, and local semantic tokens refine each part according to language. The fusion coefficients $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$0, $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$1, $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$2, and $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$3 modulate these interactions (Yu et al., 19 Mar 2026).

Inter-part synchronization then propagates context across parts, injects global relational tokens into both the 3D and 2D streams, and updates the global relational planner through bottom-up grounding from synthesized parts. The paper interprets this process as enabling direct communication among parts, persistent global structure, and grounding of the relational planner in the evolving generated object (Yu et al., 19 Mar 2026).

The denoising losses are defined separately for the 3D and 2D streams and are combined in a full objective

$$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$4

with

$$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$5

The paper states that this SNR-based curriculum shifts emphasis from faithful denoising toward relational alignment over time (Yu et al., 19 Mar 2026).

6. Empirical role, interpretability, and controllability

The paper attributes substantial performance gains to DPLs in combination with the broader DreamPartGen design (Yu et al., 19 Mar 2026). On geometric metrics, DreamPartGen is reported to achieve average reductions of $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$6 CD and $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$7 EMD. The part-level evaluation is also reported to show the best CD, EMD, and F-score on ShapeNet and PartRel3D. The stated interpretation is that this is consistent with DPLs helping geometry and appearance stay aligned at the part level.

For text-to-3D alignment, the paper reports at least $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$8 improvement in CLIP/ULIP at the object level and $$\mathbf{L}^{\text{2D}}_i \in \mathbb{R}^{T_{\text{2D}}\times d}$$9 improvement at the part level (Yu et al., 19 Mar 2026). These gains are attributed to the synchronization of RSLs and DPLs, where DPLs receive persistent semantic guidance while preserving part-level modularity.

The paper also associates DPLs with improved interpretability and controllability. Because each part has a persistent slot and separate geometry and appearance streams, the model can edit individual parts, preserve unedited parts, generate articulated objects, generate mini-scenes, and maintain semantic correspondence across components (Yu et al., 19 Mar 2026). It explicitly highlights that DPLs make parts “trackable throughout diffusion” and support fine-grained control.

A further reported effect concerns geometric independence. The model obtains lower pairwise IoU than baselines, which the paper interprets as indicating less overlap and better disentanglement of generated parts, with stronger ability to generate “non-intersecting yet composable parts” (Yu et al., 19 Mar 2026). A plausible implication is that the persistent part identity and duplex representation jointly reduce interference among part slots during synthesis, although the paper frames this chiefly through the reported IoU behavior and ablation evidence.

7. Conceptual significance and scope

Within DreamPartGen, DPLs formalize a specific view of part-aware 3D generation: semantic parts should be represented as persistent modular units whose geometry and appearance are explicitly separated but jointly coordinated (Yu et al., 19 Mar 2026). The design targets semantic independence, slot persistence, modularity, and compatibility with collaborative diffusion-based denoising.

Two scope boundaries follow directly from the paper. First, DPLs are part-wise latent units, not a full relational model by themselves; inter-part dependencies are handled through RSLs. Second, DPLs are not defined as a single compressed token sequence for an entire object; they are instantiated per part as a triplet of 3D tokens, 2D tokens, and a persistent identity embedding (Yu et al., 19 Mar 2026). These boundaries matter for interpreting both the architecture and the ablation results.

In summary, DPLs are the mechanism by which DreamPartGen assigns each semantic part a stable latent slot with separate 3D and 2D channels, enabling synchronized geometry–appearance alignment within parts and coordinated semantic assembly across parts. In the paper’s formulation, their significance lies in stable part tracking, modularity, and compatibility with language-grounded collaborative denoising rather than in replacing the relational semantic machinery with a single unified latent (Yu et al., 19 Mar 2026).