Reinforcement-learning robotic sailboats: simulator and preliminary results (2402.03337v1)
Abstract: This work focuses on the main challenges and problems in developing a virtual oceanic environment reproducing real experiments using Unmanned Surface Vehicles (USV) digital twins. We introduce the key features for building virtual worlds, considering using Reinforcement Learning (RL) agents for autonomous navigation and control. With this in mind, the main problems concern the definition of the simulation equations (physics and mathematics), their effective implementation, and how to include strategies for simulated control and perception (sensors) to be used with RL. We present the modeling, implementation steps, and challenges required to create a functional digital twin based on a real robotic sailing vessel. The application is immediate for developing navigation algorithms based on RL to be applied on real boats.
- Anderson, B. D. (2003). The Physics of Sailing Explained. Sheridan House.
- Cambio: realistic three dimensional simulation of humanoids based on computer vision and robotics. In Proceedings. XV Brazilian Symposium on Computer Graphics and Image Processing, pages 388–395.
- Toward maritime robotic simulation in gazebo. In Proceedings of MTS/IEEE OCEANS Conference, Seattle, WA.
- Fossen, T. I. (2011). Handbook of marine craft hydrodynamics and motion control. Wiley.
- Research on key technology of unmanned surface vehicle motion simulation based on unity3d. In OCEANS 2022, Hampton Roads, pages 1–5.
- Survey of popular robotics simulators, frameworks, and toolkits. In 2011 Proceedings of IEEE Southeastcon, pages 243–249.
- A survey on unmanned surface vehicles for disaster robotics: Main challenges and directions. Sensors, 19(3):702.
- Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627.
- N-boat: An autonomous robotic sailboat. In 2013 Latin American Robotics Symposium and Competition, pages 24–29. IEEE.
- Design and use paradigms for gazebo, an open-source multi-robot simulator. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), volume 3, pages 2149–2154.
- Unmanned surface vehicles: An overview of developments and challenges. Annual Reviews in Control, 41:71–93.
- Unmanned surface vehicle simulator with realistic environmental disturbances. Sensors, 19(5).
- Ros: an open-source robot operating system. In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan.
- Experimentable digital twins—streamlining simulation-based systems engineering for industry 4.0. IEEE Transactions on Industrial Informatics, 14(4):1722–1731.
- Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
- Towards a real-time embedded system for water monitoring installed in a robotic sailboat. Sensors, 16(8):1226.