GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning (2409.06427v1)

Abstract: Humans can autonomously learn the relationship between sensation and motion in their own bodies, estimate and control their own body states, and move while continuously adapting to the current environment. On the other hand, current robots control their bodies by learning the network structure described by humans from their experiences, making certain assumptions on the relationship between sensors and actuators. In addition, the network model does not adapt to changes in the robot's body, the tools that are grasped, or the environment, and there is no unified theory, not only for control but also for state estimation, anomaly detection, simulation, and so on. In this study, we propose a Generalized Multisensory Correlational Model (GeMuCo), in which the robot itself acquires a body schema describing the correlation between sensors and actuators from its own experience, including model structures such as network input/output. The robot adapts to the current environment by updating this body schema model online, estimates and controls its body state, and even performs anomaly detection and simulation. We demonstrate the effectiveness of this method by applying it to tool-use considering changes in grasping state for an axis-driven robot, to joint-muscle mapping learning for a musculoskeletal robot, and to full-body tool manipulation for a low-rigidity plastic-made humanoid.

• The paper introduces an autonomous framework that learns a robot’s body schema through multisensory correlation analysis.
• It employs a novel network with mask variables and parametric bias to adapt to sensorimotor changes and environmental variations.
• Experimental results on PR2, Musashi, and KXR robots demonstrate effective control, state estimation, and anomaly detection.

Generalized Multisensory Correlational Model for Body Schema Learning

The paper "GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning" proposes an approach for autonomous learning of a robot's body schema. The model aims to enable robots to estimate and control their body states, simulate their interactions with the environment, and detect anomalies through continuous online adaptation.

Overview and Contributions

GeMuCo, by Kento Kawaharazuka, Kei Okada, and Masayuki Inaba introduces a novel framework that captures the correlation between a robot's sensors and actuators, thus describing the robot's body schema. The research emphasizes four main characteristics:

1. Multisensory Correlation: A capability to express correlations between various sensors and actuators.
2. General Versatility: The model's utility in tasks like control, state estimation, anomaly detection, and simulation.
3. Autonomous Acquisition: An ability for the robot to autonomously learn the models, including their network structures.
4. Change Adaptability: The model's capability to handle gradual changes in the body schema through continuous online updates.

By utilizing masks and parametric bias within the network, GeMuCo can adapt to various changes in its environment, body state, and tools, thereby exhibiting a highly flexible and responsive behavior.

Technical Details

Network Structure and Mask Variable

The network structure of GeMuCo is designed to encode correlations within the sensory and control input space. The mechanism of masking input variables enables the selective activation of different sensory modalities and their correlations. The parametric bias further enriches the network with adaptability by embedding contextual information about the robot's body, tools, and environment.

Training and Structure Determination

To set up GeMuCo, the initial training involves using data collected from random actions and human intervention. The network structure is automatically determined through a two-phase process:

1. Determining the network output by assessing which sensor values can be approximated from other sensors.
2. Defining feasible mask sets and network inputs based on inference errors.

Online Update and Optimization

GeMuCo's online update capability ensures that the model remains effective even as the robot's environment or body changes. The model can be updated incrementally using synthetic or real-world data. Optimization tasks such as control, state estimation, and simulation are addressed using an iterative process of forward and backward propagations.

Experimental Validation

The researchers validated GeMuCo across three distinct experimental setups:

1. Adaptive Tool-Tip Control Learning: This experiment, conducted on the PR2 robot, demonstrated how GeMuCo adapts to different grasping states and varying tool flexibilities. The performance indicated that the robot could effectively estimate and control the tool-tip position despite changes in grasping conditions.
2. Complex Tendon-Driven Body Control Learning: Using the musculoskeletal humanoid Musashi, the experiment validated how GeMuCo could handle complex correlations of muscle lengths, tensions, and joint angles. The model enabled accurate state estimation, improved control strategies, and more realistic simulation of the musculoskeletal system.
3. Full-Body Tool Manipulation for Low-Rigidity Humanoids: For the KXR humanoid, the paper highlighted GeMuCo's ability to manage whole-body manipulation tasks while adjusting to the low-rigidity properties. By considering the body's deflection and tool parameters, the model enabled the humanoid to maintain balance and control tool interactions effectively.

Discussion and Implications

GeMuCo's ability to adapt and function under varying real-world conditions underscores its potential for practical robotics applications. The model's architecture efficiently handles complex sensorimotor correlations and adapts to ongoing changes, which is crucial for dynamic environments. Future research directions could focus on integrating reinforcement learning techniques to improve data collection and enhancing model robustness against potential catastrophic forgetting.

Conclusion

The GeMuCo framework represents a significant contribution to autonomous robotics, providing a robust platform for continuous adaptation and versatility in state estimation, control, and simulation. The experiments conducted validate the effectiveness of the model, showcasing its potential across different robotic configurations and tasks. Future developments could extend its capabilities, making it a cornerstone in the advancement of intelligent robotics.