Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction (2403.15098v3)

Published 22 Mar 2024 in cs.CV

Abstract: Vehicle trajectory prediction has increasingly relied on data-driven solutions, but their ability to scale to different data domains and the impact of larger dataset sizes on their generalization remain under-explored. While these questions can be studied by employing multiple datasets, it is challenging due to several discrepancies, e.g., in data formats, map resolution, and semantic annotation types. To address these challenges, we introduce UniTraj, a comprehensive framework that unifies various datasets, models, and evaluation criteria, presenting new opportunities for the vehicle trajectory prediction field. In particular, using UniTraj, we conduct extensive experiments and find that model performance significantly drops when transferred to other datasets. However, enlarging data size and diversity can substantially improve performance, leading to a new state-of-the-art result for the nuScenes dataset. We provide insights into dataset characteristics to explain these findings. The code can be found here: https://github.com/vita-epfl/UniTraj

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Opentraj: Assessing prediction complexity in human trajectories datasets. In Proceedings of the asian conference on computer vision, 2020.
  2. Injecting knowledge in data-driven vehicle trajectory predictors. Transportation research part C: emerging technologies, 128:103010, 2021.
  3. Vehicle trajectory prediction works, but not everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17123–17133, 2022.
  4. Raising context awareness in motion forecasting. In CVPRW, pages 4408–4417. IEEE, 2022.
  5. Ssl-lanes: Self-supervised learning for motion forecasting in autonomous driving. In Conference on Robot Learning, pages 1793–1805. PMLR, 2023.
  6. The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections. In 2020 IEEE Intelligent Vehicles Symposium (IV), pages 1929–1934. IEEE, 2020.
  7. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020.
  8. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810, 2021.
  9. Advdo: Realistic adversarial attacks for trajectory prediction. In European Conference on Computer Vision, pages 36–52. Springer, 2022.
  10. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  11. Human trajectory prediction via counterfactual analysis. In ICCV, pages 9804–9813. IEEE, 2021.
  12. Q-eanet: Implicit social modeling for trajectory prediction via experience-anchored queries. IET Intelligent Transport Systems, 2023.
  13. Long-term path prediction in urban scenarios using circular distributions. Journal on Image and Vision Computing, 69:81–91, 2018.
  14. Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9710–9719, 2021.
  15. William Falcon and The PyTorch Lightning team. PyTorch Lightning, Mar. 2019.
  16. Uncertainty estimation for cross-dataset performance in trajectory prediction. IEEE Int. Conf. on Robotics and Automation Workshop on Fresh Perspectives on the Future of Autonomous Driving, 2022.
  17. Latent variable sequential set transformers for joint multi-agent motion prediction. In International Conference on Learning Representations, 2022.
  18. Deepcore: A comprehensive library for coreset selection in deep learning. In International Conference on Database and Expert Systems Applications, pages 181–195. Springer, 2022.
  19. One thousand and one hours: Self-driving motion prediction dataset. In Conference on Robot Learning, pages 409–418. PMLR, 2021.
  20. Interpretable trajectory prediction for autonomous vehicles viacounterfactual responsibility. In IEEE/RSJ Int. Conf. on Intelligent Robots & Systems, 2023.
  21. trajdata: A unified interface to multiple human trajectory datasets. In Proceedings of the Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, New Orleans, USA, Dec. 2023.
  22. Motiondiffuser: Controllable multi-agent motion prediction using diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9644–9653, 2023.
  23. Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. 1960.
  24. Learning semantic segmentation from multiple datasets with label shifts. In European Conference on Computer Vision, pages 20–36. Springer, 2022.
  25. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
  26. Human trajectory forecasting in crowds: A deep learning perspective. IEEE Transactions on Intelligent Transportation Systems, 2021.
  27. Motion style transfer: Modular low-rank adaptation for deep motion forecasting. In CoRL, volume 205 of Proceedings of Machine Learning Research, pages 774–784. PMLR, 2022.
  28. Scenarionet: Open-source platform for large-scale traffic scenario simulation and modeling. Advances in Neural Information Processing Systems, 2023.
  29. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 45(3):3461–3475, 2022.
  30. Multi-dataset training of transformers for robust action recognition. Advances in Neural Information Processing Systems, 35:14475–14488, 2022.
  31. Learning lane graph representations for motion forecasting. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 541–556. Springer, 2020.
  32. Towards robust and adaptive motion forecasting: A causal representation perspective. In CVPR, pages 17060–17071. IEEE, 2022.
  33. On exposing the challenging long tail in future prediction of traffic actors. In ICCV, pages 13127–13137. IEEE, 2021.
  34. Shifts: A dataset of real distributional shift across multiple large-scale tasks. arXiv preprint arXiv:2107.07455, 2021.
  35. Wayformer: Motion forecasting via simple & efficient attention networks. In IEEE International Conference on Robotics and Automation (ICRA), pages 2980–2987. IEEE, 2023.
  36. Trajeglish: Learning the language of driving scenarios. arXiv preprint arXiv:2312.04535, 2023.
  37. Learn tarot with mentor: A meta-learned self-supervised approach for trajectory prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8384–8393, 2023.
  38. Learning social etiquette: Human trajectory prediction in crowded scenes. In European Conference on Computer Vision (ECCV), volume 2, page 5, 2016.
  39. The atlas benchmark: an automated evaluation framework for human motion prediction. In 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 636–643. IEEE, 2022.
  40. Trajnet: Towards a benchmark for human trajectory prediction. arXiv preprint, 2018.
  41. Adv3d: Generating safety-critical 3d objects through closed-loop simulation. CoRR, abs/2311.01446, 2023.
  42. Caspnet++: Joint multi-agent motion prediction, 2023.
  43. Motionlm: Multi-agent motion forecasting as language modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8579–8590, 2023.
  44. How does traffic environment quantitatively affect the autonomous driving prediction? IEEE Transactions on Intelligent Transportation Systems, 2023.
  45. Multi-dataset pretraining: A unified model for semantic segmentation. arXiv preprint arXiv:2106.04121, 2021.
  46. Motion transformer with global intention localization and local movement refinement. Advances in Neural Information Processing Systems, 35:6531–6543, 2022.
  47. Multipath++: Efficient information fusion and trajectory aggregation for behavior prediction. In 2022 International Conference on Robotics and Automation (ICRA), pages 7814–7821. IEEE, 2022.
  48. FEND: A future enhanced distribution-aware contrastive learning framework for long-tail trajectory prediction. In CVPR, pages 1400–1409. IEEE, 2023.
  49. Whose track is it anyway? improving robustness to tracking errors with affinity-based trajectory prediction. In CVPR, pages 6563–6572. IEEE, 2022.
  50. Argoverse 2: Next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:2301.00493, 2023.
  51. Towards motion forecasting with real-world perception inputs: Are end-to-end approaches competitive? In ICRA, 2024.
  52. Goal-lbp: Goal-based local behavior guided trajectory prediction for autonomous driving. IEEE Transactions on Intelligent Transportation Systems, pages 1–10, 2023.
  53. Improving the generalizability of trajectory prediction models with frenet-based domain normalization. IEEE International Conference on Robotics and Automation (ICRA), 2023.
  54. Dataset distillation: A comprehensive review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  55. Interaction dataset: An international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. arXiv preprint arXiv:1910.03088, 2019.
  56. Trajectory forecasting from detection with uncertainty-aware motion encoding. CoRR, abs/2202.01478, 2022.
  57. Simple multi-dataset detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7571–7580, 2022.
  58. Query-centric trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17863–17873, 2023.
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now