Adaptive Mobile Manipulation for Articulated Objects In the Open World (2401.14403v2)

Abstract: Deploying robots in open-ended unstructured environments such as homes has been a long-standing research problem. However, robots are often studied only in closed-off lab settings, and prior mobile manipulation work is restricted to pick-move-place, which is arguably just the tip of the iceberg in this area. In this paper, we introduce Open-World Mobile Manipulation System, a full-stack approach to tackle realistic articulated object operation, e.g. real-world doors, cabinets, drawers, and refrigerators in open-ended unstructured environments. The robot utilizes an adaptive learning framework to initially learns from a small set of data through behavior cloning, followed by learning from online practice on novel objects that fall outside the training distribution. We also develop a low-cost mobile manipulation hardware platform capable of safe and autonomous online adaptation in unstructured environments with a cost of around 20,000 USD. In our experiments we utilize 20 articulate objects across 4 buildings in the CMU campus. With less than an hour of online learning for each object, the system is able to increase success rate from 50% of BC pre-training to 95% using online adaptation. Video results at https://open-world-mobilemanip.github.io/

• The paper introduces an open-world mobile manipulation system that starts with behavior cloning and refines with online reinforcement learning.
• The paper leverages a hierarchical action space mimicking human reach, grasp, and manipulation strategies to reduce data needs.
• The paper demonstrates marked improvement in real-world trials, boosting success rates from 50% to 95% in under an hour per object.

Overview

A paper exploring the development of an Open-World Mobile Manipulation System reports a comprehensive system aimed at manipulating real-world articulated objects, such as doors and cabinets, in unstructured environments. Traditional research in robotics has largely focused on controlled lab environments with a specific set of constraints, but this system aims to bridge the gap towards a more general application in open-world scenarios.

Adaptive Learning Framework

The robotic system described uses an adaptive learning framework that begins its learning process from a small dataset through behavior cloning. This preliminary phase allows for the policy to capture a reasonable starting point for future refinement. Subsequently, the system engages in online reinforcement learning (RL), collecting and learning from interactions with novel objects outside its initial training data domain. This is a critical strategy to generalize across the diverse spectrum of objects encountered in daily environments.

The hierarchical action space utilized in the controller imitates human strategies for manipulating articulated objects: reach, grasp, and execute manipulation while adapting the low-level specifics tailored to each object's physical attributes. This structured action space reduces the amount of data required for proficient learning.

Hardware Platform

The autonomous adaptation of the robot in unstructured settings necessitates not just software and learning algorithms but also a hardware platform capable of supporting such tasks. The document defines specifications for a cost-effective mobile manipulator designed for versatility, agility, and open-world adaptability. Assembling from commercially available components, the robot consists of a high payload capability and can navigate challenging environments. This balance of performance and cost, at approximately USD 20,000, renders it accessible for extensive research purposes.

Experiments and Evaluation

In real-world testing within university buildings, the manipulation system showed a significant improvement in proficiency—achieving a leap from a 50% to a 95% success rate in operations involving articulated objects—after less than an hour of online adaptation per object. These results underscore the efficacy of the adaptive learning process in real-world scenarios.

The paper also touches upon the feasibility of substituting human-given reward feedback with assessments from vision-LLMs (VLMs), pointing to the possibility of fully autonomous learning. The success in autonomous reward generation and adaptation demonstrates the advancing frontier of robotics where minimal human intervention is required during the learning phase.

Conclusion

The described work represents a breakthrough in the domain of robotic mobility and manipulation. By leveraging an adaptive learning approach and creating a versatile and economical robotic platform, the researchers demonstrate that it is possible for robots to efficiently transition from laboratory environments to handling tasks within the complexity of the real world. This advancement holds potential for future proliferation and evolution of robotics, where systems can progressively learn and adapt to operate within the broad spectrum of conditions and objects they would encounter in everyday life.