Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Novel Benchmarking Paradigm and a Scale- and Motion-Aware Model for Egocentric Pedestrian Trajectory Prediction (2310.10424v1)

Published 16 Oct 2023 in cs.CV and cs.RO

Abstract: Predicting pedestrian behavior is one of the main challenges for intelligent driving systems. In this paper, we present a new paradigm for evaluating egocentric pedestrian trajectory prediction algorithms. Based on various contextual information, we extract driving scenarios for a meaningful and systematic approach to identifying challenges for prediction models. In this regard, we also propose a new metric for more effective ranking within the scenario-based evaluation. We conduct extensive empirical studies of existing models on these scenarios to expose shortcomings and strengths of different approaches. The scenario-based analysis highlights the importance of using multimodal sources of information and challenges caused by inadequate modeling of ego-motion and scale of pedestrians. To this end, we propose a novel egocentric trajectory prediction model that benefits from multimodal sources of data fused in an effective and efficient step-wise hierarchical fashion and two auxiliary tasks designed to learn more robust representation of scene dynamics. We show that our approach achieves significant improvement by up to 40% in challenging scenarios compared to the past arts via empirical evaluation on common benchmark datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. A. Rasouli, I. Kotseruba, and J. K. Tsotsos, “Agreeing to cross: How drivers and pedestrians communicate,” in Intelligent Vehicles Symposium (IV), 2017.
  2. A. Rasouli and J. K. Tsotsos, “Autonomous vehicles that interact with pedestrians: A survey of theory and practice,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 3, pp. 900–918, 2019.
  3. M.-F. Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan, and J. Hays, “Argoverse: 3d tracking and forecasting with rich maps,” in CVPR, 2019.
  4. A. Rasouli, I. Kotseruba, T. Kunic, and J. K. Tsotsos, “PIE: A large-scale dataset and models for pedestrian intention estimation and trajectory prediction,” in ICCV, 2019.
  5. Y. Yuan, X. Weng, Y. Ou, and K. M. Kitani, “Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting,” in CVPR, 2021.
  6. P. Dendorfer, S. Elflein, and L. Leal-Taixe, “Mg-gan: A multi-generator model preventing out-of-distribution samples in pedestrian trajectory prediction,” in ICCV, 2021.
  7. N. Shafiee, T. Padir, and E. Elhamifar, “Introvert: Human trajectory prediction via conditional 3d attention,” in CVPR, 2021.
  8. Y. Hu, S. Chen, Y. Zhang, and X. Gu, “Collaborative motion prediction via neural motion message passing,” in CVPR, 2020.
  9. A. Mohamed, K. Qian, M. Elhoseiny, and C. Claudel, “Social-STGCNN: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in CVPR, 2020.
  10. J. Sun, Q. Jiang, and C. Lu, “Recursive social behavior graph for trajectory prediction,” in CVPR, 2020.
  11. H. Sun, Z. Zhao, and Z. He, “Reciprocal learning networks for human trajectory prediction,” in CVPR, 2020.
  12. K. Mangalam, H. Girase, S. Agarwal, K.-H. Lee, E. Adeli, J. Malik, and A. Gaidon, “It is not the journey but the destination: Endpoint conditioned trajectory prediction,” in ECCV, 2020.
  13. C. Choi and B. Dariush, “Looking to relations for future trajectory forecast,” in ICCV, 2019.
  14. P. Zhang, W. Ouyang, P. Zhang, J. Xue, and N. Zheng, “SR-LSTM: State refinement for LSTM towards pedestrian trajectory prediction,” in CVPR, 2019.
  15. A. Sadeghian, V. Kosaraju, A. Sadeghian, N. Hirose, H. Rezatofighi, and S. Savarese, “SoPhie: An attentive GAN for predicting paths compliant to social and physical constraints,” in CVPR, 2019.
  16. A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social GAN: Socially acceptable trajectories with generative adversarial networks,” in CVPR, 2018.
  17. H. Damirchi, M. Greenspan, and A. Etemad, “Context-aware pedestrian trajectory prediction with multimodal transformer,” arXiv:2307.03786, 2023.
  18. J. Li, X. Shi, F. Chen, J. Stroud, Z. Zhang, T. Lan, J. Mao, J. Kang, K. S. Refaat, W. Yang, et al., “Pedestrian crossing action recognition and trajectory prediction with 3d human keypoints,” in ICRA, 2023.
  19. M. Halawa, O. Hellwich, and P. Bideau, “Action-based contrastive learning for trajectory prediction,” in ECCV, 2022.
  20. C. Wang, Y. Wang, M. Xu, and D. J. Crandall, “Stepwise goal-driven networks for trajectory prediction,” RAL, vol. 7, no. 2, pp. 2716–2723, 2022.
  21. Y. Yao, E. Atkins, M. Johnson-Roberson, R. Vasudevan, and X. Du, “Bitrap: Bi-directional pedestrian trajectory prediction with multi-modal goal estimation,” RAL, vol. 6, no. 2, pp. 1463–1470, 2021.
  22. L. Neumann and A. Vedaldi, “Pedestrian and ego-vehicle trajectory prediction from monocular camera,” in CVPR, 2021.
  23. O. Makansi, O. Cicek, K. Buchicchio, and T. Brox, “Multimodal future localization and emergence prediction for objects in egocentric view with a reachability prior,” in CVPR, 2020.
  24. S. Malla, B. Dariush, and C. Choi, “TITAN: Future forecast using action priors,” in CVPR, 2020.
  25. T. Yagi, K. Mangalam, R. Yonetani, and Y. Sato, “Future person localization in first-person videos,” in CVPR, 2018.
  26. Y. Yao, M. Xu, C. Choi, D. J. Crandall, E. M. Atkins, and B. Dariush, “Egocentric vision-based future vehicle localization for intelligent driving assistance systems,” in ICRA, 2019.
  27. Y. Yao, M. Xu, Y. Wang, D. J. Crandall, and E. M. Atkins, “Unsupervised traffic accident detection in first-person videos,” in IROS, 2019.
  28. A. Bhattacharyya, M. Fritz, and B. Schiele, “Long-term on-board prediction of people in traffic scenes under uncertainty,” in CVPR, 2018.
  29. R. Chandra, U. Bhattacharya, A. Bera, and D. Manocha, “TraPHic: Trajectory prediction in dense and heterogeneous traffic using weighted interactions,” in CVPR, 2019.
  30. A. Rasouli, M. Rohani, and J. Luo, “Bifold and semantic reasoning for pedestrian behavior prediction,” in ICCV, 2021.
  31. A. Rasouli and I. Kotseruba, “Pedformer: Pedestrian behavior prediction via cross-modal attention modulation and gated multitask learning,” in ICRA, 2023.
  32. N. Nayakanti, R. Al-Rfou, A. Zhou, K. Goel, K. S. Refaat, and B. Sapp, “Wayformer: Motion forecasting via simple & efficient attention networks,” in ICRA, 2023.
  33. A. Rasouli, I. Kotseruba, and J. K. Tsotsos, “Are they going to cross? A benchmark dataset and baseline for pedestrian crosswalk behavior,” in ICCVW, 2017.
  34. H. Girase, H. Gang, S. Malla, J. Li, A. Kanehara, K. Mangalam, and C. Choi, “Loki: Long term and key intentions for trajectory prediction,” in ICCV, 2021.
  35. B. Liu, E. Adeli, Z. Cao, K.-H. Lee, A. Shenoi, A. Gaidon, and J. C. Niebles, “Spatiotemporal relationship reasoning for pedestrian intent prediction,” RAL, vol. 5, no. 2, pp. 3485–3492, 2020.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now