Advancements in AI Enable Robots to Learn Sensorimotor Skills from Videos and Simulated Interactions (ai.facebook.com)

Summary:

• Two major advancements toward general-purpose embodied AI agents capable of performing challenging sensorimotor skills have been announced.
• An artificial visual cortex (VC-1) supports a diverse range of tasks, environments, and embodiments, and a new approach called adaptive (sensorimotor) skill coordination (ASC) achieves near-perfect performance on the task of robotic mobile manipulation.

Key terms:

• Moravec’s paradox: The thesis that the hardest problems in AI involve sensorimotor skills, not abstract thought or reasoning.
• Artificial visual cortex (VC-1): A single perception model that supports a diverse range of sensorimotor skills, environments, and embodiments.
• Ego4D dataset: A groundbreaking dataset created by Meta AI and academic partners, containing videos of people performing everyday tasks.
• Adaptive (sensorimotor) skill coordination (ASC): A new approach that achieves near-perfect performance on the challenging task of robotic mobile manipulation in physical environments.
• CortexBench: A curated set of 17 different sensorimotor tasks in simulation, spanning locomotion, navigation, and dexterous and mobile manipulation.

Tags:

Research Open Source Alignment embodied AI Robots CortexBench Ego4D dataset Robotic Manipulation embodied AI agents Sensorimotor Skills