iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks (2108.03272v4)

Abstract: Recent research in embodied AI has been boosted by the use of simulation environments to develop and train robot learning approaches. However, the use of simulation has skewed the attention to tasks that only require what robotics simulators can simulate: motion and physical contact. We present iGibson 2.0, an open-source simulation environment that supports the simulation of a more diverse set of household tasks through three key innovations. First, iGibson 2.0 supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks. Second, iGibson 2.0 implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked. Additionally, given a logic state, iGibson 2.0 can sample valid physical states that satisfy it. This functionality can generate potentially infinite instances of tasks with minimal effort from the users. The sampling mechanism allows our scenes to be more densely populated with small objects in semantically meaningful locations. Third, iGibson 2.0 includes a virtual reality (VR) interface to immerse humans in its scenes to collect demonstrations. As a result, we can collect demonstrations from humans on these new types of tasks, and use them for imitation learning. We evaluate the new capabilities of iGibson 2.0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new research in embodied AI. iGibson 2.0 and its new dataset are publicly available at http://svl.stanford.edu/igibson/.

• The paper introduces extended object state simulation that includes factors like temperature, wetness, and cleanliness to mimic realistic household tasks.
• The paper leverages logical predicate functions to bridge low-level physics with high-level task representations, enabling diverse and scalable scenario generation.
• The paper integrates a VR interface for collecting human demonstrations, supporting both reinforcement learning and imitation learning for complex motor tasks.

iGibson 2.0: Object-Centric Simulation for Robot Learning

The paper presents iGibson 2.0, an advanced simulation environment designed to facilitate research in embodied AI by offering an innovative framework for robot learning, particularly in the context of everyday household tasks. The primary innovations introduced in iGibson 2.0 are centered around object-centric state simulation, logical state predicates, enhanced task generation capabilities, and integration with virtual reality for demonstration collection.

Key Innovations of iGibson 2.0

1. Extended Object States: iGibson 2.0 extends traditional kinodynamic simulations by incorporating additional physical states, such as temperature, wetness, and cleanliness levels, along with toggled and sliced states. This extension is crucial for simulating diverse household tasks like cooking or cleaning that require understanding beyond mere motion and physical interaction.
2. Logical Predicate Functions: The simulator introduces predicate logic functions that map these detailed physical states to semantically meaningful logical states, such as Cooked or Soaked. This ability bridges the gap between low-level simulation (continuous physical states) and high-level task definitions (symbolic states), enhancing both task representation versatility and initial state sampling for diverse task generation.
3. Sampling and Scene Population: iGibson 2.0 allows for the generation of realistic task scenarios by sampling task-relevant logical state predicates, enabling the automatic creation of densely populated and semantically meaningful environments. This functionality supports scalable task and scene specification, facilitating varied and complex learning instances.
4. Virtual Reality Interface: The inclusion of a VR interface enables immersive human interaction and demonstration collection within simulated environments. Demonstrations collected through VR can be used for imitation learning, offering intuitive control over tasks requiring fine motor coordination or complex bimanual actions, which are often challenging to simulate or teach through traditional interfaces.

Evaluation and Implications

The evaluation of iGibson 2.0 focused on novel tasks that demonstrate the environment's capabilities. Tasks requiring nuanced manipulation of extended states, such as Cooking or Bimanual Pick and Place, were explored using both reinforcement learning (RL) and behavioral cloning. While RL showed success in tasks when employing simplifications like fixed grasp constraints, the inclusion of realistic physics presented a noticeable challenge, highlighting areas for methodological advancement.

The paper suggests that iGibson 2.0 can significantly contribute to practical research in embodied AI by providing a rich platform for training robots in tasks traditionally limited by simulator constraints. The addition of realistic physics-based interactions and semantic task definitions in scalable simulation environments opens new avenues for embodied agents, facilitating applications in household robotics and beyond.

Future Directions

Potential future developments highlighted by the paper include improving object and material representation to support a broader range of physical interactions, refining the VR interface for enhanced usability, and potentially extending simulated scenarios to incorporate more complex human-robot interaction tasks. Moreover, transferring policies trained in iGibson 2.0 to real-world robot deployment remains an ongoing research avenue that may yield significant advancements in the capability of autonomous systems to perform everyday activities.

In summary, iGibson 2.0 is positioned as a comprehensive tool for advancing research into robot learning for domestic environments. Its contributions lay foundational groundwork for expanding the horizons of what simulation environments can achieve, fostering both theoretical advancements and practical applications in the field of embodied artificial intelligence.