We Choose to Go to Space: Agent-driven Human and Multi-Robot Collaboration in Microgravity (2402.14299v1)
Abstract: We present SpaceAgents-1, a system for learning human and multi-robot collaboration (HMRC) strategies under microgravity conditions. Future space exploration requires humans to work together with robots. However, acquiring proficient robot skills and adept collaboration under microgravity conditions poses significant challenges within ground laboratories. To address this issue, we develop a microgravity simulation environment and present three typical configurations of intra-cabin robots. We propose a hierarchical heterogeneous multi-agent collaboration architecture: guided by foundation models, a Decision-Making Agent serves as a task planner for human-robot collaboration, while individual Skill-Expert Agents manage the embodied control of robots. This mechanism empowers the SpaceAgents-1 system to execute a range of intricate long-horizon HMRC tasks.
- Robotics: Meet the lunar gateway’s robot caretakers: With people seldom on board, the space station will rely on autonomy. IEEE Spectrum, 59:7–13, 2022.
- Astrobee: A new tool for iss operations. In 2018 SpaceOps Conference, page 2517, 2018.
- Agent ai: Surveying the horizons of multimodal interaction. arXiv preprint arXiv:2401.03568, 2024.
- Interactive hand pose estimation using a stretch-sensing soft glove. ACM Transactions on Graphics (TOG), 38:1 – 15, 2019.
- Maniskill2: A unified benchmark for generalizable manipulation skills. In International Conference on Learning Representations (ICLR), 2023.
- China’s space robotics for on-orbit servicing: the state of the art. National Science Review, 10, 2022.
- A task allocation framework for human multi-robot collaborative settings. 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7614–7620, 2022.
- Visual instruction tuning. In NeurIPS, 2023.
- Generative skill chaining: Long-horizon skill planning with diffusion models. In 7th Annual Conference on Robot Learning (CoRL), 2023.
- Tree of uncertain thoughts reasoning for large language models. ArXiv, abs/2309.07694, 2023.
- OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
- Proximal policy optimization algorithms. ArXiv, abs/1707.06347, 2017.
- The rise and potential of large language model based agents: A survey. ArXiv, abs/2309.07864, 2023.
- Sapien: A simulated part-based interactive environment. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11094–11104, 2020.
- Monocular real-time hand shape and motion capture using multi-modal data. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5345–5354, 2020.