UnfoldArt: Zero-Shot Recovery of Full Articulated 3D Objects from Text or Image

Abstract: Articulated 3D objects are essential for interactive environments in embodied AI, robotics, and virtual reality, but reconstructing their structure and motion from sparse observations remains challenging. Existing approaches remain largely constrained by lack of supervised data or lack the priors needed to reliably recover articulation, hidden geometry, and internal object structure. We present the first debate-driven agentic approach to articulated 3D object reconstruction from text or image inputs that both grounds articulation reasoning in concrete motion and exposes the occluded geometry revealed under articulation. High-level agents reason about object semantics and motion using knowledge from vision-language and video models, while low-level agents estimate articulation parameters and interaction points; together, they engage in a two-round structured debate that first exploits global--local disagreement and then grounds the agents in freely generated video. The same video prior, conditioned on the agreed articulation, then drives each part through its motion to expose occluded interiors and geometry that cannot be inferred from a single static view. By combining agentic reasoning with a video generative prior, our approach jointly infers articulation and reconstructs complete 3D articulated objects, producing high-fidelity geometry, internal structure, and motion-consistent states beyond directly observed surfaces.

• The paper introduces a zero-shot method to recover full articulated 3D objects from text or image using debate-driven agentic reasoning and video generative models.
• It employs a two-round debate framework combining global and local perspectives to accurately infer hidden geometry and articulation parameters.
• The approach outperforms supervised baselines, achieving high-fidelity reconstructions with improved joint-type accuracy for diverse applications.

"UnfoldArt: Zero-Shot Recovery of Full Articulated 3D Objects from Text or Image" (2606.30608)

Abstract and Introduction

The paper "UnfoldArt: Zero-Shot Recovery of Full Articulated 3D Objects from Text or Image" presents a novel approach for generating articulated 3D objects from text or image inputs. It addresses the challenge of reconstructing both the structure and motion of these objects, going beyond traditional methods that rely heavily on supervised data or specific priors. The proposed method combines debate-driven agentic reasoning with video generative models to expose occluded geometry and infer articulation parameters, leading to the recovery of complete articulated 3D objects.

Articulated 3D objects are crucial in various domains such as robotics, virtual reality, and embodied AI, but their reconstruction from sparse observations has been limited by the availability of supervised datasets and the inability to capture hidden geometry and articulation. Previous attempts either required articulated datasets like PartNet-Mobility or relied on multi-view observations, precluding single-image reconstruction. UnfoldArt leverages the inherent knowledge of vision-LLMs (VLMs) and video generative models, orchestrating them in a debate framework to predict object semantics and motion articulation without the need for direct supervision.

Methodology

UnfoldArt employs a structured agentic approach, where high-level agents reason about object semantics and motion, while low-level agents estimate articulation parameters. This hierarchy facilitates a two-round debate that first exploits disagreements between global and local perspectives and then grounds decisions in concrete motion evidence from synthesized videos. The debate framework allows agents to refine predictions using rich latent knowledge encoded in VLMs and video models, ultimately driving the generation of motion-consistent articulated states and revealing occluded interior geometry.

Figure 1: Method overview. Given an unposed image or a text prompt as input, we generate a TRELLIS mesh and use a video generative prior for articulation reasoning.

The method begins with obtaining an initial 3D estimate using TRELLIS, which provides a grounded representation for agentic reasoning. The agents work in tandem, with a Decomposer identifying parts and plausible motions, a Grounder implementing local segmentation strategies, and an Articulator predicting joint parameters. The structured debate process ultimately leads to a consensus that drives articulation-conditioned video generation, exposing hidden internal geometry.

Results and Comparisons

The paper presents strong quantitative and qualitative results across multiple datasets, demonstrating the robustness and generality of UnfoldArt. The approach outperforms both supervised and zero-shot baselines in reconstructing high-fidelity articulated objects, excelling particularly in environments where supervised methods struggle due to novel object categories or motion structures. Numerical results highlight significant improvements in joint-type accuracy and articulation prediction, affirming the efficacy of agentic deliberation over single-model predictions.

Figure 2: Part reconstruction via 3D latent inpainting. This process results in interactable URDFs with complete interior geometry, showcasing the capabilities of the reconstruction method.

Figure 3: Qualitative results on Objaverse. Diverse articulated objects are faithfully recovered, highlighting the method's ability to handle complex geometries and motions.

Implications and Future Directions

The implications of this research are wide-ranging. By reducing the dependence on articulated supervision and extending the reconstruction paradigm to include occluded interiors, UnfoldArt opens up new possibilities for generating interactive 3D environments directly from sparse inputs. This can significantly enhance applications in simulation and virtual reality, offering scalability and diversity in object creation. Future developments may focus on refining the debate mechanisms and exploring more efficient use of latent knowledge in VLMs and video models.

The zero-shot nature of UnfoldArt invites further exploration into how generative models can be utilized without domain-specific training, potentially leading to more generalized AI systems capable of understanding and interacting with complex environments intuitively.

Figure 4: Qualitative results on Partnet-Mobility. Demonstrating high fidelity in both geometry and articulation compared to supervised baselines.

Figure 5: Diversity of interior geometry. Our approach recovers detailed, plausible interior structures.

Conclusion

UnfoldArt represents a substantial advancement in the field of articulated 3D object reconstruction. Through the integration of agentic debate-driven reasoning and video generative models, it achieves zero-shot recovery of complete articulated structures, including occluded interiors, from minimal input signals. This methodology enhances the fidelity of 3D object generation, offering new opportunities for creating interactive environments in a scalable manner. As the field progresses, UnfoldArt sets the stage for further innovations in zero-shot generative modeling, pushing the boundaries of what is possible in automated 3D scene synthesis.