Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LC-LLM: Explainable Lane-Change Intention and Trajectory Predictions with Large Language Models (2403.18344v2)

Published 27 Mar 2024 in cs.AI

Abstract: To ensure safe driving in dynamic environments, autonomous vehicles should possess the capability to accurately predict lane change intentions of surrounding vehicles in advance and forecast their future trajectories. Existing motion prediction approaches have ample room for improvement, particularly in terms of long-term prediction accuracy and interpretability. In this paper, we address these challenges by proposing LC-LLM, an explainable lane change prediction model that leverages the strong reasoning capabilities and self-explanation abilities of LLMs. Essentially, we reformulate the lane change prediction task as a LLMing problem, processing heterogeneous driving scenario information as natural language prompts for LLMs and employing supervised fine-tuning to tailor LLMs specifically for lane change prediction task. Additionally, we finetune the Chain-of-Thought (CoT) reasoning to improve prediction transparency and reliability, and include explanatory requirements in the prompts during inference stage. Therefore, our LC-LLM model not only predicts lane change intentions and trajectories but also provides CoT reasoning and explanations for its predictions, enhancing its interpretability. Extensive experiments based on the large-scale highD dataset demonstrate the superior performance and interpretability of our LC-LLM in lane change prediction task. To the best of our knowledge, this is the first attempt to utilize LLMs for predicting lane change behavior. Our study shows that LLMs can effectively encode comprehensive interaction information for driving behavior understanding.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. G. He, X. Li, Y. Lv, B. Gao, and H. Chen, “Probabilistic intention prediction and trajectory generation based on dynamic bayesian networks,” in 2019 Chinese Automation Congress (CAC).   IEEE, 2019, pp. 2646–2651.
  2. S. Mozaffari, E. Arnold, M. Dianati, and S. Fallah, “Early lane change prediction for automated driving systems using multi-task attention-based convolutional neural networks,” IEEE Transactions on Intelligent Vehicles, vol. 7, no. 3, pp. 758–770, 2022.
  3. K. Gao, X. Li, B. Chen, L. Hu, J. Liu, R. Du, and Y. Li, “Dual transformer based prediction for lane change intentions and trajectories in mixed traffic environment,” IEEE Transactions on Intelligent Transportation Systems, 2023.
  4. S. Shi, L. Jiang, D. Dai, and B. Schiele, “Motion transformer with global intention localization and local movement refinement,” Advances in Neural Information Processing Systems, vol. 35, pp. 6531–6543, 2022.
  5. A. Seff, B. Cera, D. Chen, M. Ng, A. Zhou, N. Nayakanti, K. S. Refaat, R. Al-Rfou, and B. Sapp, “Motionlm: Multi-agent motion forecasting as language modeling,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8579–8590.
  6. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
  7. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27 730–27 744, 2022.
  8. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023.
  9. J. Mao, Y. Qian, H. Zhao, and Y. Wang, “Gpt-driver: Learning to drive with gpt,” arXiv preprint arXiv:2310.01415, 2023.
  10. L. Chen, O. Sinavski, J. Hünermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton, “Driving with llms: Fusing object-level vector modality for explainable autonomous driving,” arXiv preprint arXiv:2310.01957, 2023.
  11. D. Wu, W. Han, T. Wang, Y. Liu, X. Zhang, and J. Shen, “Language prompt for autonomous driving,” arXiv preprint arXiv:2309.04379, 2023.
  12. N. F. Rajani, B. McCann, C. Xiong, and R. Socher, “Explain yourself! leveraging language models for commonsense reasoning,” arXiv preprint arXiv:1906.02361, 2019.
  13. S. Huang, S. Mamidanna, S. Jangam, Y. Zhou, and L. H. Gilpin, “Can large language models explain themselves? a study of llm-generated self-explanations,” arXiv preprint arXiv:2310.11207, 2023.
  14. Y. Huang, J. Du, Z. Yang, Z. Zhou, L. Zhang, and H. Chen, “A survey on trajectory-prediction methods for autonomous driving,” IEEE Transactions on Intelligent Vehicles, vol. 7, no. 3, pp. 652–674, 2022.
  15. S. Mozaffari, O. Y. Al-Jarrah, M. Dianati, P. Jennings, and A. Mouzakitis, “Deep learning-based vehicle behavior prediction for autonomous driving applications: A review,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 1, pp. 33–47, 2020.
  16. Z. Yang, X. Jia, H. Li, and J. Yan, “A survey of large language models for autonomous driving,” arXiv preprint arXiv:2311.01043, 2023.
  17. H. M. Mandalia and M. D. D. Salvucci, “Using support vector machines for lane-change detection,” in Proceedings of the human factors and ergonomics society annual meeting, vol. 49, no. 22.   SAGE Publications Sage CA: Los Angeles, CA, 2005, pp. 1965–1969.
  18. N. Lyu, J. Wen, Z. Duan, and C. Wu, “Vehicle trajectory prediction and cut-in collision warning model in a connected vehicle environment,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 2, pp. 966–981, 2020.
  19. J. Hong, B. Sapp, and J. Philbin, “Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8454–8462.
  20. A. Zyner, S. Worrall, J. Ward, and E. Nebot, “Long short term memory for driver intent prediction,” in 2017 IEEE Intelligent Vehicles Symposium (IV).   IEEE, 2017, pp. 1484–1489.
  21. A. Zyner, S. Worrall, and E. Nebot, “A recurrent neural network solution for predicting driver intention at unsignalized intersections,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1759–1764, 2018.
  22. L. Xin, P. Wang, C.-Y. Chan, J. Chen, S. E. Li, and B. Cheng, “Intention-aware long horizon trajectory prediction of surrounding vehicles using dual lstm networks,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC).   IEEE, 2018, pp. 1441–1446.
  23. R. Izquierdo, A. Quintanar, I. Parra, D. Fernández-Llorca, and M. Sotelo, “Experimental validation of lane-change intention prediction methodologies based on cnn and lstm,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC).   IEEE, 2019, pp. 3657–3662.
  24. Y. Chai, B. Sapp, M. Bansal, and D. Anguelov, “Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,” arXiv preprint arXiv:1910.05449, 2019.
  25. H. Cui, V. Radosavljevic, F.-C. Chou, T.-H. Lin, T. Nguyen, T.-K. Huang, J. Schneider, and N. Djuric, “Multimodal trajectory predictions for autonomous driving using deep convolutional networks,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 2090–2096.
  26. N. Deo and M. M. Trivedi, “Multi-modal trajectory prediction of surrounding vehicles with maneuver based lstms,” in 2018 IEEE intelligent vehicles symposium (IV).   IEEE, 2018, pp. 1179–1184.
  27. F. Altché and A. de La Fortelle, “An lstm network for highway trajectory prediction,” in 2017 IEEE 20th international conference on intelligent transportation systems (ITSC).   IEEE, 2017, pp. 353–359.
  28. M. Liang, B. Yang, R. Hu, Y. Chen, R. Liao, S. Feng, and R. Urtasun, “Learning lane graph representations for motion forecasting,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16.   Springer, 2020, pp. 541–556.
  29. J. Gao, C. Sun, H. Zhao, Y. Shen, D. Anguelov, C. Li, and C. Schmid, “Vectornet: Encoding hd maps and agent dynamics from vectorized representation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 525–11 533.
  30. M. Zhu, S. S. Du, X. Wang, Z. Pu, Y. Wang et al., “Transfollower: Long-sequence car-following trajectory prediction through transformer,” arXiv preprint arXiv:2202.03183, 2022.
  31. Z. Xu, Y. Zhang, E. Xie, Z. Zhao, Y. Guo, K. K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end autonomous driving via large language model,” arXiv preprint arXiv:2310.01412, 2023.
  32. X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, “Pre-trained models for natural language processing: A survey,” Science China Technological Sciences, vol. 63, no. 10, pp. 1872–1897, 2020.
  33. C. Cui, Y. Ma, X. Cao, W. Ye, and Z. Wang, “Receive, reason, and react: Drive as you say with large language models in autonomous vehicles,” arXiv preprint arXiv:2310.08034, 2023.
  34. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” Advances in neural information processing systems, vol. 35, pp. 22 199–22 213, 2022.
  35. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems, vol. 35, pp. 24 824–24 837, 2022.
  36. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, O. Bousquet, Q. Le, and E. Chi, “Least-to-most prompting enables complex reasoning in large language models,” arXiv preprint arXiv:2205.10625, 2022.
  37. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021.
  38. R. Krajewski, J. Bock, L. Kloeker, and L. Eckstein, “The highd dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems,” in 2018 21st international conference on intelligent transportation systems (ITSC).   IEEE, 2018, pp. 2118–2125.
  39. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019.
  40. J. Rasley, S. Rajbhandari, O. Ruwase, and Y. He, “Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 3505–3506.
  41. J. Kaplan, S. McCandlish, T. J. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” ArXiv, vol. abs/2001.08361, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:210861095
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Mingxing Peng (7 papers)
  2. Xusen Guo (7 papers)
  3. Xianda Chen (14 papers)
  4. Meixin Zhu (39 papers)
  5. Kehua Chen (17 papers)
Citations (7)

Summary

We haven't generated a summary for this paper yet.