TrajPRed: Trajectory Prediction with Region-based Relation Learning (2404.06971v1)
Abstract: Forecasting human trajectories in traffic scenes is critical for safety within mixed or fully autonomous systems. Human future trajectories are driven by two major stimuli, social interactions, and stochastic goals. Thus, reliable forecasting needs to capture these two stimuli. Edge-based relation modeling represents social interactions using pairwise correlations from precise individual states. Nevertheless, edge-based relations can be vulnerable under perturbations. To alleviate these issues, we propose a region-based relation learning paradigm that models social interactions via region-wise dynamics of joint states, i.e., the changes in the density of crowds. In particular, region-wise agent joint information is encoded within convolutional feature grids. Social relations are modeled by relating the temporal changes of local joint information from a global perspective. We show that region-based relations are less susceptible to perturbations. In order to account for the stochastic individual goals, we exploit a conditional variational autoencoder to realize multi-goal estimation and diverse future prediction. Specifically, we perform variational inference via the latent distribution, which is conditioned on the correlation between input states and associated target goals. Sampling from the latent distribution enables the framework to reliably capture the stochastic behavior in test data. We integrate multi-goal estimation and region-based relation learning to model the two stimuli, social interactions, and stochastic goals, in a prediction framework. We evaluate our framework on the ETH-UCY dataset and Stanford Drone Dataset (SDD). We show that the diverse prediction better fits the ground truth when incorporating the relation module. Our framework outperforms the state-of-the-art models on SDD by $27.61\%$/$18.20\%$ of ADE/FDE metrics.
- “Explanatory paradigms in neural networks: Towards relevant and contextual explanations,” IEEE Signal Processing Magazine, vol. 39, no. 4, pp. 59–72, 2022.
- “Open-set recognition with gradient-based representations,” in 2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021, pp. 469–473.
- “A gating model for bias calibration in generalized zero-shot learning,” IEEE Transactions on Image Processing, 2022.
- “On the structures of representation for the robustness of semantic segmentation to input corruption,” in 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020, pp. 3239–3243.
- “Example forgetting: A novel approach to explain and interpret deep neural networks in seismic interpretation,” IEEE Transactions on Geoscience and Remote Sensing, 2022.
- “Joint learning for spatial context-based seismic inversion of multiple data sets for improved generalizability and robustness,” Geophysics, vol. 86, no. 4, pp. O37–O48, 2021.
- “Volumetric supervised contrastive learning for seismic semantic segmentation,” in Second International Meeting for Applied Geoscience & Energy. Society of Exploration Geophysicists and American Association of Petroleum …, 2022, pp. 1699–1703.
- “Patient aware active learning for fine-grained oct classification,” arXiv preprint arXiv:2206.11485, 2022.
- “Multi-modal learning using physicians diagnostics for optical coherence tomography classification,” in 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE, 2022, pp. 1–5.
- “Peeking into the future: Predicting future person activities and locations in videos,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5725–5734.
- “Autotrajectory: Label-free trajectory extraction and prediction from videos using dynamic points,” in European Conference on Computer Vision. Springer, 2020, pp. 646–662.
- “What will happen next? forecasting player moves in sports videos,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 3342–3351.
- “Imitative non-autoregressive modeling for trajectory forecasting and imputation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12736–12745.
- “Dynamic channel: A planning framework for crowd navigation,” in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 5551–5557.
- “A survey on human-aware robot navigation,” Robotics and Autonomous Systems, vol. 145, pp. 103837, 2021.
- Lucy A Suchman, Plans and situated actions: The problem of human-machine communication, Cambridge university press, 1987.
- “Social lstm: Human trajectory prediction in crowded spaces,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 961–971.
- “Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,” in European Conference on Computer Vision. Springer, 2020, pp. 683–700.
- “Social gan: Socially acceptable trajectories with generative adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2255–2264.
- “It is not the journey but the destination: Endpoint conditioned trajectory prediction,” in European Conference on Computer Vision. Springer, 2020, pp. 759–776.
- “Human motion trajectory prediction: A survey,” The International Journal of Robotics Research, vol. 39, no. 8, pp. 895–935, 2020.
- “Will the pedestrian cross? a study on pedestrian path prediction,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 2, pp. 494–506, 2013.
- “Pedestrian’s trajectory forecast in public traffic with artificial neural networks,” in 2014 22nd international conference on pattern recognition. IEEE, 2014, pp. 4110–4115.
- “Social force model for pedestrian dynamics,” Physical review E, vol. 51, no. 5, pp. 4282, 1995.
- “Abnormal crowd behavior detection using social force model,” in 2009 IEEE conference on computer vision and pattern recognition. IEEE, 2009, pp. 935–942.
- “Who are you with and where are you going?,” in CVPR 2011. IEEE, 2011, pp. 1345–1352.
- “Socially-aware large-scale crowd forecasting,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2203–2210.
- “Context-aware trajectory prediction,” in 2018 24th International Conference on Pattern Recognition (ICPR). IEEE, 2018, pp. 1941–1946.
- “Scene-lstm: A model for human trajectory prediction,” arXiv preprint arXiv:1808.04018, 2018.
- “Social attention: Modeling attention in human crowds,” in 2018 IEEE international Conference on Robotics and Automation (ICRA). IEEE, 2018, pp. 4601–4607.
- “The garden of forking paths: Towards multi-future trajectory prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10508–10518.
- “Spatio-temporal graph dual-attention network for multi-agent prediction and tracking,” IEEE Transactions on Intelligent Transportation Systems, 2021.
- “Conditional generative neural system for probabilistic trajectory prediction,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2019, pp. 6150–6156.
- “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
- “Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014, vol. 27.
- “Sophie: An attentive gan for predicting paths compliant to social and physical constraints,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1349–1358.
- “The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 2375–2384.
- “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in 2009 IEEE 12th international conference on computer vision. IEEE, 2009, pp. 261–268.
- “Crowds by example,” in Computer graphics forum. Wiley Online Library, 2007, vol. 26, pp. 655–664.
- “Learning social etiquette: Human trajectory understanding in crowded scenes,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14. Springer, 2016, pp. 549–565.
- “Trajectory forecasting based on prior-aware directed graph convolutional neural network,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 9, pp. 16773–16785, 2022.
- “Cscnet: Contextual semantic consistency network for trajectory prediction in crowded spaces,” Pattern Recognition, vol. 126, pp. 108552, 2022.
- “Analyzing the variety loss in the context of probabilistic trajectory prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9954–9963.
- “An exponential learning rate schedule for deep learning,” arXiv preprint arXiv:1910.07454, 2019.