Papers
Topics
Authors
Recent
Search
2000 character limit reached

ESP: Extro-Spective Prediction for Long-term Behavior Reasoning in Emergency Scenarios

Published 7 May 2024 in cs.CV and cs.LG | (2405.04100v1)

Abstract: Emergent-scene safety is the key milestone for fully autonomous driving, and reliable on-time prediction is essential to maintain safety in emergency scenarios. However, these emergency scenarios are long-tailed and hard to collect, which restricts the system from getting reliable predictions. In this paper, we build a new dataset, which aims at the long-term prediction with the inconspicuous state variation in history for the emergency event, named the Extro-Spective Prediction (ESP) problem. Based on the proposed dataset, a flexible feature encoder for ESP is introduced to various prediction methods as a seamless plug-in, and its consistent performance improvement underscores its efficacy. Furthermore, a new metric named clamped temporal error (CTE) is proposed to give a more comprehensive evaluation of prediction performance, especially in time-sensitive emergency events of subseconds. Interestingly, as our ESP features can be described in human-readable language naturally, the application of integrating into ChatGPT also shows huge potential. The ESP-dataset and all benchmarks are released at https://dingrui-wang.github.io/ESP-Dataset/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Florent Altché and Arnaud de La Fortelle. An lstm network for highway trajectory prediction. In 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), pages 353–359. IEEE, 2017.
  2. Intention-aware online pomdp planning for autonomous driving in a crowd. In 2015 ieee international conference on robotics and automation (icra), pages 454–460. IEEE, 2015.
  3. Stable multi-target tracking in real-time surveillance video. In CVPR 2011, pages 3457–3464. IEEE, 2011.
  4. Segmentation and recognition using structure from motion point clouds. In Computer Vision–ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, October 12-18, 2008, Proceedings, Part I 10, pages 44–57. Springer, 2008.
  5. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020.
  6. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8748–8757, 2019.
  7. Harrison Chase. Langchain. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, HP d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, 2022.
  8. Kaist multi-spectral day/night data set for autonomous and assisted driving. IEEE Transactions on Intelligent Transportation Systems, 19(3):934–948, 2018.
  9. A review of motion planning for highway autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 21(5):1826–1848, 2019.
  10. A critical evaluation of the next generation simulation (ngsim) vehicle trajectory dataset. Transportation Research Part B: Methodological, 105:362–377, 2017.
  11. Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9710–9719, 2021.
  12. Drive like a human: Rethinking autonomous driving with large language models. arXiv preprint arXiv:2307.07162, 2023.
  13. Vectornet: Encoding hd maps and agent dynamics from vectorized representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11525–11533, 2020.
  14. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pages 3354–3361. IEEE, 2012.
  15. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  16. Interactive trajectory prediction of surrounding road users for autonomous driving using structural-lstm network. IEEE Transactions on Intelligent Transportation Systems, 21(11):4615–4625, 2019.
  17. The apolloscape open dataset for autonomous driving and its application. IEEE transactions on pattern analysis and machine intelligence, 42(10):2702–2719, 2019.
  18. The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2375–2384, 2019.
  19. The prevention dataset: a novel benchmark for prediction of vehicles intentions. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pages 3114–3121. IEEE, 2019.
  20. Scenario understanding and motion prediction for autonomous vehicles—review and comparison. IEEE Transactions on Intelligent Transportation Systems, 23(10):16962–16982, 2022.
  21. A learning-based stochastic mpc design for cooperative adaptive cruise control to handle interfering vehicles. IEEE Transactions on Intelligent Vehicles, 3(3):266–275, 2018.
  22. Lyft level 5 perception dataset 2020, 2019.
  23. The highd dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems. In 2018 21st international conference on intelligent transportation systems (ITSC), pages 2118–2125. IEEE, 2018.
  24. Crowds by example. In Computer graphics forum, volume 26, pages 655–664. Wiley Online Library, 2007.
  25. Aads: Augmented autonomous driving simulation using data-driven algorithms. Science robotics, 4(28):eaaw0863, 2019.
  26. Grip++: Enhanced graph-based interaction-aware trajectory prediction for autonomous driving. arXiv preprint arXiv:1907.07792, 2019.
  27. Trafficpredict: Trajectory prediction for heterogeneous traffic-agents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6120–6127, 2019.
  28. The exid dataset: A real-world trajectory dataset of highly interactive highway scenarios in germany. In 2022 IEEE Intelligent Vehicles Symposium (IV), pages 958–964. IEEE, 2022.
  29. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  30. You’ll never walk alone: Modeling social behavior for multi-target tracking. In 2009 IEEE 12th international conference on computer vision, pages 261–268. IEEE, 2009.
  31. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023.
  32. Geoscenario: An open dsl for autonomous driving scenario representation. In 2019 IEEE Intelligent Vehicles Symposium (IV), pages 287–294. IEEE, 2019.
  33. Learning social etiquette: Human trajectory understanding in crowded scenes. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, pages 549–565. Springer, 2016.
  34. Graph-based spatial-temporal convolutional network for vehicle trajectory prediction in autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 23(10):17654–17665, 2022.
  35. Motion transformer with global intention localization and local movement refinement. Advances in Neural Information Processing Systems, 35:6531–6543, 2022.
  36. Automatum data: Drone-based highway dataset for the development and validation of automated driving software for research and commercial applications. In 2021 IEEE Intelligent Vehicles Symposium (IV), pages 1372–1377. IEEE, 2021.
  37. An intelligent self-driving truck system for highway transportation. Frontiers in neurorobotics, 16:843026, 2022.
  38. Ltp: Lane-based trajectory prediction for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17134–17142, 2022.
  39. Ganet: Goal area network for motion forecasting. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1609–1615. IEEE, 2023.
  40. A point-based mdp for robust single-lane autonomous driving behavior under uncertainties. In 2011 IEEE international conference on robotics and automation, pages 2586–2592. IEEE, 2011.
  41. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022.
  42. Interaction dataset: An international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. arXiv preprint arXiv:1910.03088, 2019.
  43. Tnt: Target-driven trajectory prediction. In Conference on Robot Learning, pages 895–904. PMLR, 2021.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free