Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Attention-aware Social Graph Transformer Networks for Stochastic Trajectory Prediction (2312.15881v2)

Published 26 Dec 2023 in cs.CV

Abstract: Trajectory prediction is fundamental to various intelligent technologies, such as autonomous driving and robotics. The motion prediction of pedestrians and vehicles helps emergency braking, reduces collisions, and improves traffic safety. Current trajectory prediction research faces problems of complex social interactions, high dynamics and multi-modality. Especially, it still has limitations in long-time prediction. We propose Attention-aware Social Graph Transformer Networks for multi-modal trajectory prediction. We combine Graph Convolutional Networks and Transformer Networks by generating stable resolution pseudo-images from Spatio-temporal graphs through a designed stacking and interception method. Furthermore, we design the attention-aware module to handle social interaction information in scenarios involving mixed pedestrian-vehicle traffic. Thus, we maintain the advantages of the Graph and Transformer, i.e., the ability to aggregate information over an arbitrary number of neighbors and the ability to perform complex time-dependent data processing. We conduct experiments on datasets involving pedestrian, vehicle, and mixed trajectories, respectively. Our results demonstrate that our model minimizes displacement errors across various metrics and significantly reduces the likelihood of collisions. It is worth noting that our model effectively reduces the final displacement error, illustrating the ability of our model to predict for a long time.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. A. Baggag, S. Abbar, A. Sharma, T. Zanouda, A. Al-Homaid, A. Mohan, and J. Srivastava, “Learning spatiotemporal latent factors of traffic via regularized tensor factorization: Imputing missing values and forecasting,” IEEE Trans. Knowl. Data Eng., vol. 33, no. 6, pp. 2573–2587, 2021.
  2. Z. Pan, W. Zhang, Y. Liang, W. Zhang, Y. Yu, J. Zhang, and Y. Zheng, “Spatio-temporal meta learning for urban traffic prediction,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 3, pp. 1462–1476, 2022.
  3. Z. Yang, H. Sun, J. Huang, Z. Sun, H. Xiong, S. Qiao, Z. Guan, and X. Jia, “An efficient destination prediction approach based on future trajectory prediction and transition matrix optimization,” IEEE Trans. Knowl. Data Eng., vol. 32, no. 2, pp. 203–217, 2020.
  4. A. V. Khezerlou, X. Zhou, L. Tong, Y. Li, and J. Luo, “Forecasting gathering events through trajectory destination prediction: A dynamic hybrid model,” IEEE Trans. Knowl. Data Eng., vol. 33, no. 3, pp. 991–1004, 2021.
  5. A. Rudenko, L. Palmieri, M. Herman, K. M. Kitani, D. M. Gavrila, and K. O. Arras, “Human motion trajectory prediction: a survey,” Int. J. Robotics Res., vol. 39, no. 8, 2020.
  6. Y. Ma, X. Zhu, S. Zhang, R. Yang, W. Wang, and D. Manocha, “Trafficpredict: Trajectory prediction for heterogeneous traffic-agents,” in AAAI 2019, pp. 6120–6127.
  7. C. Chen, Y. Liu, S. Kreiss, and A. Alahi, “Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning,” in ICRA 2019, pp. 6015–6022.
  8. N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. S. Torr, and M. Chandraker, “DESIRE: distant future prediction in dynamic scenes with interacting agents,” in CVPR 2017, pp. 2165–2174.
  9. Y. Zhang and Y. Zhou, “Yolov5 based pedestrian safety detection in underground coal mines,” in ROBIO 2021, pp. 1700–1705.
  10. H. Li, A. V. Savkin, and B. Vucetic, “Autonomous area exploration and mapping in underground mine environments by unmanned aerial vehicles,” Robotica, vol. 38, no. 3, pp. 442–456, 2020.
  11. A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “Social LSTM: human trajectory prediction in crowded spaces,” in CVPR 2016,, pp. 961–971.
  12. T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,” in ECCV 2020, pp. 683–700.
  13. X. Li, X. Ying, and M. C. Chuah, “GRIP: graph-based interaction-aware trajectory prediction,” in ITSC 2019, pp. 3960–3966.
  14. N. Deo and M. M. Trivedi, “Convolutional social pooling for vehicle trajectory prediction,” in CVPR Workshops 2018, pp. 1468–1476.
  15. W. Chen, F. Wang, and H. Sun, “S2tnet: Spatio-temporal transformer networks for trajectory prediction in autonomous driving,” in ACML 2021, pp. 454–469.
  16. B. Ivanovic and M. Pavone, “The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs,” in ICCV 2019, pp. 2375–2384.
  17. A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social GAN: socially acceptable trajectories with generative adversarial networks,” in CVPR 2018, pp. 2255–2264.
  18. D. Helbing and P. Molnár, “Social force model for pedestrian dynamics,” Phys. Rev. E, vol. 51, pp. 4282–4286, May 1995.
  19. J. M. Wang, D. J. Fleet, and A. Hertzmann, “Gaussian process dynamical models for human motion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 2, pp. 283–298.
  20. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.
  21. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS 2017, pp. 5998–6008.
  22. J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT 2019, pp. 4171–4186.
  23. D. A. Tedjopurnomo, Z. Bao, B. Zheng, F. M. Choudhury, and A. K. Qin, “A survey on modern deep neural network for traffic prediction: Trends, methods and challenges,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 4, pp. 1544–1561, 2022.
  24. Y. Liu, L. Yao, B. Li, X. Wang, and C. Sammut, “Social graph transformer networks for pedestrian trajectory prediction in complex social scenarios,” in CIKM, 2022, pp. 1339–1349.
  25. C. Yu, X. Ma, J. Ren, H. Zhao, and S. Yi, “Spatio-temporal graph transformer networks for pedestrian trajectory prediction,” in ECCV 2020, pp. 507–523.
  26. P. Zhang, W. Ouyang, P. Zhang, J. Xue, and N. Zheng, “SR-LSTM: state refinement for LSTM towards pedestrian trajectory prediction,” in CVPR 2019, pp. 12 085–12 094.
  27. A. Sadeghian, V. Kosaraju, A. Sadeghian, N. Hirose, H. Rezatofighi, and S. Savarese, “Sophie: An attentive GAN for predicting paths compliant to social and physical constraints,” in CVPR 2019, pp. 1349–1358.
  28. F. Giuliari, I. Hasan, M. Cristani, and F. Galasso, “Transformer networks for trajectory forecasting,” in ICPR 2020, pp. 10 335–10 342.
  29. L. L. Li, B. Yang, M. Liang, W. Zeng, M. Ren, S. Segal, and R. Urtasun, “End-to-end contextual perception and prediction with interaction transformer,” in IROS 2020, pp. 5784–5791.
  30. Y. Liu, J. Zhang, L. Fang, Q. Jiang, and B. Zhou, “Multimodal motion prediction with stacked transformers,” in CVPR 2021, pp. 7577–7586.
  31. V. Kosaraju, A. Sadeghian, R. Martín-Martín, I. D. Reid, H. Rezatofighi, and S. Savarese, “Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks,” in NeurIPS 2019, pp. 137–146.
  32. M. Ye, T. Cao, and Q. Chen, “TPCN: temporal point cloud networks for motion forecasting,” in CVPR 2021, pp. 11 318–11 327.
  33. A. A. Mohamed, K. Qian, M. Elhoseiny, and C. G. Claudel, “Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in CVPR 2020, pp. 14 412–14 420.
  34. S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” in AAAI 2018, pp. 7444–7452.
  35. R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in CVPR 2014, pp. 580–587.
  36. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR 2017.
  37. S. Guo, Y. Lin, H. Wan, X. Li, and G. Cong, “Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 11, pp. 5415–5428, 2022.
  38. Y. Liu, Q. Yan, and A. Alahi, “Social NCE: contrastive learning of socially-aware motion representations,” in ICCV 2021, pp. 15 098–15 109.
  39. S. Pellegrini, A. Ess, K. Schindler, and L. V. Gool, “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in ICCV 2009, pp. 261–268.
  40. L. Leal-Taixé, M. Fenzi, A. Kuznetsova, B. Rosenhahn, and S. Savarese, “Learning an image-based motion context for multiple people tracking,” in CVPR 2014, pp. 3542–3549.
  41. A. Vemula, K. Muelling, and J. Oh, “Social attention: Modeling attention in human crowds,” in ICRA 2018, pp. 1–7.
  42. I. Bae and H. Jeon, “Disentangled multi-relational graph convolutional network for pedestrian trajectory prediction,” in AAAI 2021, pp. 911–919.
  43. J. Sekhon and C. H. Fleming, “SCAN: A spatial context attentive network for joint multi-agent intent prediction,” in AAAI 2021, pp. 6119–6127.
  44. J. Colyar and J. Halkias, “Us highway i-80 dataset: Tech. rep. fhwa-hrt-06-137,” Tech. Rep., 2006.
  45. ——, “Us highway 101 dataset,” Federal Highway Administration, Tech. Rep. FHWA-HRT-07-030, pp. 27–69, 2007.
  46. N. Deo, A. Rangesh, and M. M. Trivedi, “How would surround vehicles move? A unified framework for maneuver classification and motion prediction,” IEEE Trans. Intell. Veh., vol. 3, no. 2, pp. 129–140, 2018.
  47. A. Kuefler, J. Morton, T. A. Wheeler, and M. J. Kochenderfer, “Imitating driver behavior with generative adversarial networks,” in IV 2017.   IEEE, 2017, pp. 204–211.
  48. Y. Zhu, D. Qian, D. Ren, and H. Xia, “Starnet: Pedestrian trajectory prediction using deep neural network in star topology,” in IROS 2019, pp. 8075–8080.
  49. L. Fang, Q. Jiang, J. Shi, and B. Zhou, “Tpnet: Trajectory proposal network for motion prediction,” in CVPR 2020, pp. 6796–6805.
  50. X. Li, X. Ying, and M. C. Chuah, “Grip++: Enhanced graph-based interaction-aware trajectory prediction for autonomous driving,” arXiv preprint arXiv:1907.07792, 2019.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now