Deep Reinforcement Learning for Local Path Following of an Autonomous Formula SAE Vehicle (2401.02903v1)
Abstract: With the continued introduction of driverless events to Formula:Society of Automotive Engineers (F:SAE) competitions around the world, teams are investigating all aspects of the autonomous vehicle stack. This paper presents the use of Deep Reinforcement Learning (DRL) and Inverse Reinforcement Learning (IRL) to map locally-observed cone positions to a desired steering angle for race track following. Two state-of-the-art algorithms not previously tested in this context: soft actor critic (SAC) and adversarial inverse reinforcement learning (AIRL), are used to train models in a representative simulation. Three novel reward functions for use by RL algorithms in an autonomous racing context are also discussed. Tests performed in simulation and the real world suggest that both algorithms can successfully train models for local path following. Suggestions for future work are presented to allow these models to scale to a full F:SAE vehicle.
- Szilárd Aradi. Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Transactions on Intelligent Transportation Systems, 23(2):740–759, 2022.
- Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6):26–38, November 2017.
- Formula Student Germany. FSG: Rules & Important Documents, 2023.
- Learning Robust Rewards with Adversarial Inverse Reinforcement Learning. 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, oct 2017.
- A review of motion planning techniques for automated vehicles. IEEE Transactions on Intelligent Transportation Systems, 17(4):1135–1145, April 2016.
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. 35th International Conference on Machine Learning, ICML 2018, 5:2976–2989, jan 2018.
- Navigating occluded intersections with autonomous vehicles using deep reinforcement learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 2034–2039, 2018.
- End-to-end race driving with deep reinforcement learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 2070–2075, 2018.
- AMZ Driverless: The Full Autonomous Racing System. Journal of Field Robotics, 37(7):1267–1294, may 2019.
- Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4:237–285, may 1996.
- Deep Reinforcement Learning for Autonomous Driving: A Survey. IEEE Transactions on Intelligent Transportation Systems, 2021.
- End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research, 17(1):1334–1373, 2016.
- Playing Atari with Deep Reinforcement Learning. dec 2013.
- Andrew Y Ng and S Russell. Algorithms for inverse reinforcement learning. page 2, 2000.
- The magic formula tyre model. Vehicle System Dynamics, 21(sup001):1–18, Jan 1992.
- Bridging the Gap Between Imitation Learning and Inverse Reinforcement Learning. IEEE Transactions on Neural Networks and Learning Systems, 28(8):1814–1826, 2017.
- Racing towards reinforcement learning based control of an autonomous formula sae car. In Australasian Conference on Robotics and Automation, 2022.
- Autonomous overtaking in gran turismo sport using curriculum reinforcement learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9403–9409, 2021.
- Deep reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, pages 1–15, 2022.
- Wayve. Wayve - Autonomous Mobility Driven by AI, 2021.
- A control strategy of autonomous vehicles based on deep reinforcement learning. In 2016 9th International Symposium on Computational Intelligence and Design (ISCID), volume 2, pages 198–201, 2016.
- Explorations and Lessons Learned in Building an Autonomous Formula SAE Car from Simulations. CoRR, abs/1905.0, 2019.
- Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Science and Technology, 26(5):674–691, oct 2021.