Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AI (2310.01824v2)
Abstract: We present Mini-BEHAVIOR, a novel benchmark for embodied AI that challenges agents to use reasoning and decision-making skills to solve complex activities that resemble everyday human challenges. The Mini-BEHAVIOR environment is a fast, realistic Gridworld environment that offers the benefits of rapid prototyping and ease of use while preserving a symbolic level of physical realism and complexity found in complex embodied AI benchmarks. We introduce key features such as procedural generation, to enable the creation of countless task variations and support open-ended learning. Mini-BEHAVIOR provides implementations of various household tasks from the original BEHAVIOR benchmark, along with starter code for data collection and reinforcement learning agent training. In essence, Mini-BEHAVIOR offers a fast, open-ended benchmark for evaluating decision-making and planning solutions in embodied AI. It serves as a user-friendly entry point for research and facilitates the evaluation and development of solutions, simplifying their assessment and development while advancing the field of embodied AI. Code is publicly available at https://github.com/StanfordVL/mini_behavior.
- Griddly: A platform for AI research in games. CoRR, abs/2011.06363, 2020. URL https://arxiv.org/abs/2011.06363.
- Rearrangement: A challenge for embodied AI. CoRR, abs/2011.01975, 2020. URL https://arxiv.org/abs/2011.01975.
- On the utility of learning about humans for human-ai coordination. CoRR, abs/1910.05789, 2019. URL http://arxiv.org/abs/1910.05789.
- Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. CoRR, abs/2306.13831, 2023.
- Procthor: Large-scale embodied ai using procedural generation, 2022.
- The threedworld transport challenge: A visually guided task-and-motion planning benchmark for physically realistic embodied AI. CoRR, abs/2103.14025, 2021. URL https://arxiv.org/abs/2103.14025.
- Causal policy gradient for whole-body mobile manipulation. In arXiv preprint arXiv:2305.04866, 2023.
- Modeling dynamic environments with scene graph memory. In International Conference on Machine Learning, pages 17976–17993. PMLR, 2023.
- Hrl4in: Hierarchical reinforcement learning for interactive navigation with mobile manipulators. In CoRL, pages 603–616. PMLR, 2020.
- igibson 2.0: Object-centric simulation for robot learning of everyday household tasks. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 455–465. PMLR, 08–11 Nov 2022a. URL https://proceedings.mlr.press/v164/li22b.html.
- BEHAVIOR-1k: A benchmark for embodied AI with 1,000 everyday activities and realistic simulation. In 6th Annual Conference on Robot Learning, 2022b. URL https://openreview.net/forum?id=_8DoIe8G3t.
- Sim-to-real reinforcement learning for deformable object manipulation. In Conference on Robot Learning, pages 734–743. PMLR, 2018.
- Virtualhome: Simulating household activities via programs. CoRR, abs/1806.07011, 2018. URL http://arxiv.org/abs/1806.07011.
- Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087, 2017.
- Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
- ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. CoRR, abs/1912.01734, 2019. URL http://arxiv.org/abs/1912.01734.
- BEHAVIOR: benchmark for everyday household activities in virtual, interactive, and ecological environments. CoRR, abs/2108.03332, 2021. URL https://arxiv.org/abs/2108.03332.
- Mazebase: A sandbox for learning from games. CoRR, abs/1511.07401, 2015. URL http://arxiv.org/abs/1511.07401.
- Elden: Exploration via local dependencies. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
- Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. CoRR, abs/1910.10897, 2019. URL http://arxiv.org/abs/1910.10897.
- Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Science and Technology, 26(5):674–691, 2021.
- Target-driven visual navigation in indoor scenes using deep reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA), pages 3357–3364. IEEE, 2017.