Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration (1603.06348v3)

Abstract: Dexterous multi-fingered hands can accomplish fine manipulation behaviors that are infeasible with simple robotic grippers. However, sophisticated multi-fingered hands are often expensive and fragile. Low-cost soft hands offer an appealing alternative to more conventional devices, but present considerable challenges in sensing and actuation, making them difficult to apply to more complex manipulation tasks. In this paper, we describe an approach to learning from demonstration that can be used to train soft robotic hands to perform dexterous manipulation tasks. Our method uses object-centric demonstrations, where a human demonstrates the desired motion of manipulated objects with their own hands, and the robot autonomously learns to imitate these demonstrations using reinforcement learning. We propose a novel algorithm that allows us to blend and select a subset of the most feasible demonstrations to learn to imitate on the hardware, which we use with an extension of the guided policy search framework to use multiple demonstrations to learn generalizable neural network policies. We demonstrate our approach on the RBO Hand 2, with learned motor skills for turning a valve, manipulating an abacus, and grasping.

Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrations

The paper at hand presents an exploration into the autonomous training of dexterous manipulation skills for a soft robotic hand, the RBO Hand 2, driven by demonstrations sourced from human manipulations. Aimed at a research-oriented audience, the paper makes significant strides in bridging the gap between high-cost, sophisticated multi-fingered robotic hands, and low-cost, compliant alternatives that are easier to produce but traditionally challenging to control due to their complex dynamics and limited sensing.

Key Contributions

1. Demonstration and Reinforcement Learning Integration: The authors propose an innovative algorithm that integrates object-centric learning from demonstration (LfD) with reinforcement learning (RL). This algorithm assigns correspondence between human demonstrations and multiple controllers trained to follow a feasible demonstration trajectory. A critical component here is the selection phase, which automatically determines which demonstrations are replicable by the soft hand given its morphological constraints compared to humans.
2. Controller Optimization: Reinforcement learning is leveraged to optimize separate controllers, each initialized from different task states. The aim is for these controllers to adopt the closest attainable demonstrations. This optimization considers object-specific segments of the state which are pertinent to demonstrated tasks, realized without requiring precise kinematic models or controlled environments.
3. Neural Network Policy Learning: Through guided policy search (GPS), the authors extend their methodology to generalize across various initial states using a neural network. This network combines insights from individual controllers, thus creating a robust policy capable of managing the versatile initial configurations encountered during tasks.

Experimental Validation

The efficacy of the proposed framework is validated through three distinct dexterous manipulation tasks using the RBO Hand 2: turning a valve, manipulating an abacus, and a grasping task. These tasks were specifically chosen to test the hand's ability to perform complex motions without extensive tactile feedback or precise control mechanisms, which are common limitations of soft hands.

• Valve Turning: Reinforcing the algorithm's validity, the soft hand successfully learns to rotate a valve through an evaluation process involving multiple hand positions relative to the valve. The learned policy outperformed hand-designed baselines and rivaled an oracle model precisely assigning demonstrations, showcasing the method's adaptability to variable task constraints.
• Abacus Manipulation: Here, the task required selective motion of beads, demanding precision and fine motor control. The approach demonstrated its potential by outperforming single-demonstration and hand-designed strategy baselines, pushing only the targeted beads.
• Grasping: Tasked with grasping and maintaining hold of a deodorant bottle, the learned policy equaled the performance of a hand-designed controller. The success underscores the capability of the approach to handle tasks with delayed and varied feedback inherent in grasping routines.

Implications and Future Directions

This research contributes not only practical applications to the field of robotics by offering a cost-effective means of developing dexterous manipulation without intricate mechanical designs but also introduces a scalable method of learning from human demonstrations. The system shows promise for applications requiring adaptable and cost-effective robotic solutions in dynamic environments, such as assistive robots in domestic or medical fields.

Potential future advancements could focus on deploying enhanced computer vision for feature tracking, refining policies to depend more heavily on the onboard soft sensors, or extending this model into tasks requiring more complex tactile feedback. The scalability and flexibility demonstrated could lead to broader applications in robotic tasks that interface directly with human environments.

In summary, this paper offers substantial contributions to the field of soft robotics, utilizing reinforcement learning principles tailored for soft hands. It provides a path forward for cost-effective dexterous manipulation skills, leveraging human demonstrations as a pivotal component in robotic training.