Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control

Abstract: Reconstructing articulated 3D objects is important for animation, gaming, and robotic simulations. Recent neural networks can estimate the articulated structure of 3D objects, but their generalization remains limited by the scarcity of annotated data for this task. To address this gap, we introduce Instruct-Particulate, a model that takes a 3D mesh together with a target kinematic specification, including part descriptions, connectivity, joint types, and optional point prompts, and predicts the corresponding kinematic part segmentation and joint motion parameters. The kinematic specification disambiguates the task and allows the model to target annotations of different granularity, thereby making it possible to use more abundant heterogeneous training data. At test time, the kinematic specification can be obtained automatically from large-scale vision-LLMs, so the model can be applied to any input mesh. To train our model at scale, we construct a heterogeneous dataset of more than 150,000 articulated 3D objects, extending existing publicly available collections with data obtained by partially labelling other 3D models (monolithic or already decomposed into parts) with kinematic labels by means of vision-LLMs. Experiments show that our model generalizes better across categories and to AI-generated meshes, enabling articulated asset reconstruction from real-world images via image-to-3D models.

• The paper presents a feed-forward model that segments 3D objects into articulated parts using explicit kinematic instructions.
• It leverages a large-scale heterogeneous dataset curated via VLM pseudo-labeling, spatial annotation, and procedural asset generation.
• Experimental results demonstrate superior segmentation, joint estimation, and robust generalization across diverse object types compared to previous methods.

Authoritative Summary of "Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control" (2606.14699)

Motivation and Problem Statement

The paper addresses articulated 3D object reconstruction, a task essential for downstream applications in animation, robotics, and virtual environments. Previous neural models in this domain have been constrained by the scarcity and lack of diversity in annotated datasets. Most existing approaches overfit to limited categories, producing suboptimal generalization. The authors propose a feed-forward model, capable of segmenting 3D objects into articulated parts with fine-grained kinematic control, scalable training, and robust generalization across object types, including AI-generated assets and novel categories.

Figure 1: Articulated 3D objects predicted by Instruct-Particulate from real-world images, supporting kinematic prompting and diverse output.

Scalable Dataset Construction

To overcome the annotated data bottleneck, the paper introduces a heterogeneous dataset exceeding 150,000 assets, constructed by:

• Vision-LLM (VLM) pseudo-labeling: Rendering synthetic 3D assets from multiple views, VLMs extract kinematic structures, produce segmentation masks per a fixed color scheme, and unproject 2D segmentations to 3D, yielding part segmentations and kinematic trees from static assets.
• Captioning and spatial annotation: Assets with existing decompositions receive part captions generated via VLMs, resolved with spatial cues only when canonical orientations exist, avoiding ambiguous labeling.
• Procedural asset generation: A coding agent, Articraft, produces fully supervised articulated 3D assets with explicit joint parameters.

This data curation strategy leverages web-scale 3D asset collections and advances in VLM reasoning, dramatically increasing both scale and diversity compared to previous datasets.

Figure 2: Data pipelines combining VLM pseudo-labeling and spatial captioning for scalable articulated 3D asset annotation.

Model Architecture and Conditioning

The proposed model accepts a 3D mesh and explicit kinematic instructions—part descriptors, connectivity (kinematic tree), joint types, and optionally spatial point prompts. This explicit conditioning disambiguates part granularity and semantics, allowing the model to learn from datasets with diverse annotation schemes.

Key architectural components:

• Encoders: Mesh points, part descriptors (textual+geometric), and query points are embedded. Text prompts utilize CLIP embeddings and spatial point prompts.
• Transformer blocks: Shape, part, and query tokens are processed via custom attention; part tokens cross-attend shape tokens, query tokens cross-attend both shape and part tokens.
• Decoders: Segmentation head assigns part labels to surface query points; joint parameter head predicts motion axes and bounds in an over-parameterized manner, aggregated via geometric fitting.

Unlike methods reliant on test-time optimization or weak conditioning, the proposed feed-forward approach enables efficient inference and precise articulation estimation.

Figure 3: Architectural overview showing shape, part, and query token encoding, custom transformer attention, and decoder heads for segmentation and articulation.

Experimental Results and Comparative Evaluation

Quantitative and qualitative benchmarks demonstrate robust performance across categories and input modalities:

• When applied to Lightwheel—a challenging articulated object dataset—the model surpasses eight baselines, including SINGAPO, PhysX-Anything, PartField, and previous Particulate variants. Key metrics such as part recall, mIoU (up to 0.825), articulated geometry IoU (up to 0.757), and joint axis errors (as low as 11.0 degrees) are strong.
• Ablations show complementary gains from each data source; the procedural coding agent most improves articulation metrics, while VLM-based synthetic and part-segmented data improve segmentation fidelity.
• Conditioning modality ablations reveal both textual and spatial prompting improve accuracy, with explicit kinematic structure instruction necessary to avoid annotation ambiguity.

Practically, the pipeline enables articulated asset generation from real-world images—offloading geometry reconstruction to a 3D generator and inferring kinematic structure via VLMs.

Figure 4: Qualitative comparison (Image Only mode); Instruct-Particulate produces more realistic articulated objects, recovering small parts missed by baselines.

Figure 5: Qualitative comparison in Mesh mode; superior segmentation and generalization to complex or internal parts compared to baselines.

Kinematic Prompting and Test-Time Control

The conditioning formulation enables test-time prompting. Different kinematic structures can be specified for the same mesh, and the model reliably follows these, including highly granular prompts (up to 24 button joints). Spatial reasoning with text descriptors (e.g., "left drawer", "right drawer") is supported, provided canonical orientation exists.

Figure 6: Qualitative results demonstrate faithful articulation according to diverse kinematic prompting instructions.

Limitations and Failure Cases

While the model achieves robust performance, several failure modes persist:

• Segmentation errors may propagate to joint motion estimation due to query-point aggregation.
• Artifacts in AI-generated meshes (e.g., floating or missing parts) degrade predictive accuracy.
• Outputs lack simulation-ready properties (physical constraints, cleaned geometry).

  Figure 7: Selected failure cases including segmentation artifacts and incomplete part separation.

Practical and Theoretical Implications

From a practical standpoint, this approach substantially advances scalable articulated asset creation for simulation environments and embodied agent training. The reliance on vision-LLMs and explicit kinematic prompting addresses semantic ambiguity and annotation inconsistency, permitting learning from heterogeneous sources. Theoretically, the paper positions kinematic prompt conditioning as a crucial axis for feed-forward 3D articulation models, suggesting future research directions:

• Scaling datasets through web-scale pseudo-labeling and procedural asset generation.
• Improving simulation-readiness through multi-modal post-processing.
• Expanding prompting modalities (multimodal, temporal) for broader control in generative assets.

Conclusion

The paper presents Instruct-Particulate, a feed-forward model for articulated 3D object segmentation with kinematic control, trained on a uniquely large and diverse dataset annotated through vision-language modeling and procedural agents. With explicit kinematic prompting, the model achieves strong generalization and practical utility for asset generation from images or meshes, outperforming prior methods across segmentation, articulation, and joint axis metrics. The conditioning paradigm enables flexible test-time control, positioning this line of research as foundational for advancing large-scale articulated asset creation and simulation in AI-driven environments.