In-context Learning for Automated Driving Scenarios (2405.04135v1)
Abstract: One of the key challenges in current Reinforcement Learning (RL)-based Automated Driving (AD) agents is achieving flexible, precise, and human-like behavior cost-effectively. This paper introduces an innovative approach utilizing LLMs to intuitively and effectively optimize RL reward functions in a human-centric way. We developed a framework where instructions and dynamic environment descriptions are input into the LLM. The LLM then utilizes this information to assist in generating rewards, thereby steering the behavior of RL agents towards patterns that more closely resemble human driving. The experimental results demonstrate that this approach not only makes RL agents more anthropomorphic but also reaches better performance. Additionally, various strategies for reward-proxy and reward-shaping are investigated, revealing the significant impact of prompt design on shaping an AD vehicle's behavior. These findings offer a promising direction for the development of more advanced and human-like automated driving systems. Our experimental data and source code can be found here.
- Autonomous Driving Architectures: Insights of Machine Learning and Deep Learning Algorithms, volume 6. Machine Learning with Applications, 12 2021.
- Machine learning-based automatic control of tunneling posture of shield machine. Journal of Rock Mechanics and Geotechnical Engineering, 14(4):1153–1164, August 2022.
- Alaa Khamis. Optimization Algorithms: AI techniques for design, planning, and control problems. Manning Publications, ISBN 9781633438835, New York, United States, 2023.
- Reinforcement learning: An introduction, volume 17. 1999.
- Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(6):4909–4926, 2022.
- Autonomous driving planning and decision making based on game theory and reinforcement learning. Expert Systems, 40(5):e13191, 2023.
- Dense reinforcement learning for safety validation of autonomous vehicles. Nature, 615(7953):620–627, 2023.
- A deep reinforcement learning approach for efficient, safe and comfortable driving. Applied Sciences, 13(9):5272, 2023.
- A survey on multimodal large language models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 958–979, 2024.
- The learnability of in-context learning. Advances in Neural Information Processing Systems, 36, 2024.
- A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022.
- Reward design with language models. arXiv preprint arXiv:2303.00001, 2023.
- A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 2023.
- Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022.
- Training language models to follow instructions with human feedback, 2022.
- Is reinforcement learning (not) for natural language processing: Benchmarks, baselines, and building blocks for natural language policy optimization, 2023.
- Skill reinforcement learning and planning for open-world long-horizon tasks, 2023.
- Inner monologue: Embodied reasoning through planning with language models, 2022.
- Do as i can, not as i say: Grounding language in robotic affordances, 2022.
- The rl/llm taxonomy tree: Reviewing synergies between reinforcement learning and large language models, 2024.
- Text2reward: Automated dense reward function generation for reinforcement learning, 2023.
- Eureka: Human-level reward design via coding large language models, 2023.
- Self-refined large language model as automated reward function designer for deep reinforcement learning in robotics, 2023.
- Alex Place. Adaptive reinforcement learning with llm-augmented reward functions. Authorea Preprints, 2023.
- Dilu: A knowledge-driven approach to autonomous driving with large language models, 2024.
- Drive like a human: Rethinking autonomous driving with large language models, 2023.
- Llm4drive: A survey of large language models for autonomous driving, 2023.
- Driving with llms: Fusing object-level vector modality for explainable autonomous driving, 2023.
- Drive as you speak: Enabling human-like interaction with large language models in autonomous vehicles. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 902–909, 2024.
- Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving, 2023.
- Human feedback as action assignment in interactive reinforcement learning. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 14(4):1–24, 2020.
- Eleurent. Github repository for highwayenv. https://github.com/Farama-Foundation/HighwayEnv, 2021. Accessed: April 30, 2024.
- shi tianyu. Leveraging large language models for intelligent driving scenario. https://medium.com/ai4sm/leveraging-large-language-models-for-intelligent-driving-scenario-c326c4b6ea.
- Github repository for llm-rl. https://github.com/JingYue2000/In-context_Learning_for_Automated_Driving, 2024. Accessed: April 29, 2024.
- On a formal model of safe and scalable self-driving cars. arXiv preprint arXiv:1708.06374, 2017.
- Ziqi Zhou (46 papers)
- Jingyue Zhang (5 papers)
- Jingyuan Zhang (50 papers)
- Boyue Wang (15 papers)
- Tianyu Shi (49 papers)
- Alaa Khamis (1 paper)