UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments

Abstract: Learning a universal manipulation policy encompassing doors with diverse categories, geometries and mechanisms, is crucial for future embodied agents to effectively work in complex and broad real-world scenarios. Due to the limited datasets and unrealistic simulation environments, previous works fail to achieve good performance across various doors. In this work, we build a novel door manipulation environment reflecting different realistic door manipulation mechanisms, and further equip this environment with a large-scale door dataset covering 6 door categories with hundreds of door bodies and handles, making up thousands of different door instances. Additionally, to better emulate real-world scenarios, we introduce a mobile robot as the agent and use the partial and occluded point cloud as the observation, which are not considered in previous works while possessing significance for real-world implementations. To learn a universal policy over diverse doors, we propose a novel framework disentangling the whole manipulation process into three stages, and integrating them by training in the reversed order of inference. Extensive experiments validate the effectiveness of our designs and demonstrate our framework's strong performance. Code, data and videos are avaible on https://unidoormanip.github.io/.

• The paper presents a universal door manipulation policy through a three-stage framework integrating handle grasping, manipulation, and door opening in a large-scale, diverse simulation environment.
• The paper employs a segmentation-based PointNet++ and a conditional VAE to address complex mechanical dynamics and reduce error rates in real-time manipulation.
• The paper demonstrates significant performance improvements over legacy methods, validated through extensive simulation and real-world experiments.

Learning Universal Door Manipulation Policies

"UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments" (2403.02604) presents a structured framework for developing door manipulation policies applicable to a wide range of doors with varying designs, mechanisms, and geometries. The proposed methodology not only involves the creation of a robust simulation environment but also introduces a multi-stage manipulation framework critically tuned for efficient learning.

Large-scale Diverse Door Manipulation Environment

A distinctive feature of this research is the development of a door manipulation environment and dataset that encompass six door categories, from interior to refrigerator doors, comprising 328 bodies and 204 handles. This dataset architecture allows the creation of thousands of distinct door configurations, facilitating comprehensive policy training (Figure 1).

Figure 1: Our pipeline for the framework. We disentangle the entire door manipulation process into three stages: handle grasping, handle manipulation, and door opening.

The environment simulates diverse manipulation mechanisms supported by a realistic simulation that leverages IsaacGym. This includes detailed mechanical properties such as latching forces and resilient forces applied to door joints, facilitating translation from simulation to real-world execution. The methodology employed ensures robust policy learning against various types of realistic occlusions and maneuvering constraints imposed by a robot with a mobile base, offering a significant improvement over legacy simulation environments.

Methods

The methodology segregates door manipulation into three stages: handle grasping, handle manipulation, and door opening (Figure 2). This segmentation aids in isolating the complexities of each phase, thereby simplifying learning and improving generalization across unseen doors with diverse geometries and mechanisms.

Figure 2: Manipulation Sequence Guided by Our Universal Manipulation Policy.

Stage 1: Handle Grasping

The first stage employs point-level visual affordance to predict a score map indicating action feasibility, using a segmentation version of PointNet++. By sampling potential actions and evaluating them through a discriminator model, it ensures the selection of the optimal grasp pose.

Stage 2: Handle Manipulation

Leveraging a generative architecture, a conditional VAE structure is used to manage varied manipulation mechanisms. The model is trained with extensive data covering various handle types to ensure robust performance and reduced error rates in real-time applications.

Stage 3: Door Opening

Third-stage policies are trained in a closed-loop formulation, which monitors ongoing adjustments to reduce cumulative errors. This architecture supports a dynamic, real-time framework compared to the open-loop methodology which lacks the ability to adapt during execution.

Results and Evaluation

The empirical results highlight the efficacy of the framework in scenarios of diverse geometry and category, outperforming previous strategies such as GAPartNet and VAT-MART. The universal manipulation policy demonstrates high success rates across multiple tasks, significantly excelling in environments involving unseen categories and complex shapes (Figure 3).

Figure 3: Comparison of ablations and our approach for various door joint angles. Average success rate metrics are shown for each angle.

A notable highlight is the integration capability, where conditioned training bridges independent policy stages into a seamless manipulation sequence. The research underscored that without such integration or state consideration, the effectiveness significantly drops, particularly in environments with complex dynamics and occlusions (Figure 4, Figure 5).

Figure 4: Failure cases of ablated versions.

Figure 5: Real-World Experiments demonstrate the application of learned policies in practical settings.

Conclusion

This research advances universal door manipulation policy learning, presenting a novel, efficiently structured framework that accommodates diverse types, handles, and integration methods. Through extensive experimentation, both in simulation and real-world settings, the study validates its significant contributions to robotic manipulation domains. Future directions may involve further improving real-time adaptation mechanisms and expanding policy universality to other articulated objects beyond the scope of doors.