AI Meets Physical World -- Exploring Robot Cooking

Abstract: This paper describes our recent research effort to bring the computer intelligence into the physical world so that robots could perform physically interactive manipulation tasks. Our proposed approach first gives robots the ability to learn manipulation skills by "watching" online instructional videos. After "watching" over 200 instructional videos, a functional object-oriented network (FOON) is constructed to represent the observed manipulation skills. Using the network, robots can take a high-level task command such as "I want BBQ Ribs for dinner," decipher the task goal, seek the correct objects to operate on, and then generate and execute a sequence of manipulation motions that can deal with physical interactions in a new condition. To best facilitate manipulation motions in the physical world, we also developed new grasping strategies for robots to hold objects with a firm grasp to withstand the disturbance during physical interactions.