Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

OpenGait: A Comprehensive Benchmark Study for Gait Recognition towards Better Practicality (2405.09138v1)

Published 15 May 2024 in cs.CV

Abstract: Gait recognition, a rapidly advancing vision technology for person identification from a distance, has made significant strides in indoor settings. However, evidence suggests that existing methods often yield unsatisfactory results when applied to newly released real-world gait datasets. Furthermore, conclusions drawn from indoor gait datasets may not easily generalize to outdoor ones. Therefore, the primary goal of this work is to present a comprehensive benchmark study aimed at improving practicality rather than solely focusing on enhancing performance. To this end, we first develop OpenGait, a flexible and efficient gait recognition platform. Using OpenGait as a foundation, we conduct in-depth ablation experiments to revisit recent developments in gait recognition. Surprisingly, we detect some imperfect parts of certain prior methods thereby resulting in several critical yet undiscovered insights. Inspired by these findings, we develop three structurally simple yet empirically powerful and practically robust baseline models, i.e., DeepGaitV2, SkeletonGait, and SkeletonGait++, respectively representing the appearance-based, model-based, and multi-modal methodology for gait pattern description. Beyond achieving SoTA performances, more importantly, our careful exploration sheds new light on the modeling experience of deep gait models, the representational capacity of typical gait modalities, and so on. We hope this work can inspire further research and application of gait recognition towards better practicality. The code is available at https://github.com/ShiqiYu/OpenGait.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. L. Wang, T. Tan, H. Ning, and W. Hu, “Silhouette analysis-based gait recognition for human identification,” IEEE transactions on pattern analysis and machine intelligence, vol. 25, no. 12, pp. 1505–1518, 2003.
  2. Z. Wu, Y. Huang, L. Wang, X. Wang, and T. Tan, “A comprehensive study on cross-view gait based human identification with deep cnns,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 2, pp. 209–226, 2016.
  3. S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems, vol. 28, 2015.
  4. J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
  5. L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool, “Temporal segment networks: Towards good practices for deep action recognition,” in European conference on computer vision.   Springer, 2016, pp. 20–36.
  6. Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7291–7299.
  7. H. Chao, Y. He, J. Zhang, and J. Feng, “Gaitset: Regarding gait as a set for cross-view gait recognition,” in Proceedings of the AAAI conference on artificial intelligence, 2019, pp. 8126–8133.
  8. Z. Zhu, X. Guo, T. Yang, J. Huang, J. Deng, G. Huang, D. Du, J. Lu, and J. Zhou, “Gait recognition in the wild: A benchmark,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14 789–14 799.
  9. J. Zheng, X. Liu, W. Liu, L. He, C. Yan, and T. Mei, “Gait recognition in the wild with dense 3d representations and a benchmark,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  10. S. Yu, D. Tan, and T. Tan, “A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition,” in 18th International Conference on Pattern Recognition (ICPR’06), vol. 4.   IEEE, 2006, pp. 441–444.
  11. N. Takemura, Y. Makihara, D. Muramatsu, T. Echigo, and Y. Yagi, “Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition,” IPSJ Transactions on Computer Vision and Applications, vol. 10, no. 1, pp. 1–14, 2018.
  12. S. Meng, Y. Fu, S. Hou, C. Cao, X. Liu, and Y. Huang, “Fastposegait: A toolbox and benchmark for efficient pose-based gait recognition,” arXiv preprint arXiv:2309.00794, 2023.
  13. S. Yu, Y. Huang, L. Wang, Y. Makihara, S. Wang, M. A. R. Ahad, and M. Nixon, “Hid 2022: The 3rd international competition on human identification at a distance,” in 2022 IEEE International Joint Conference on Biometrics (IJCB).   IEEE, 2022, pp. 1–9.
  14. S. Yu, W. Chenye, L. Wang, Z. Yuwei, W. Li, W. Ming, L. Qing, L. Wenlong, W. Runsheng, H. Yongzhen, W. Liang, M. Yasushi, and R. A. Md Atiqur, “Human identification at a distance: Challenges, methods and results on hid 2023,” in 2023 IEEE International Joint Conference on Biometrics (IJCB), 2023, pp. 1–7.
  15. C. Fan, Y. Peng, C. Cao, X. Liu, S. Hou, J. Chi, Y. Huang, Q. Li, and Z. He, “Gaitpart: Temporal part-based model for gait recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 14 225–14 233.
  16. B. Lin, S. Zhang, and X. Yu, “Gait recognition via effective global-local feature representation and local temporal aggregation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14 648–14 656.
  17. C. Fan, J. Liang, C. Shen, S. Hou, Y. Huang, and S. Yu, “Opengait: Revisiting gait recognition towards better practicality,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 9707–9716.
  18. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  19. C. Fan, J. Ma, D. Jin, C. Shen, and S. Yu, “Skeletongait: Gait recognition using skeleton maps,” 2023.
  20. W. Li, S. Hou, C. Zhang, C. Cao, X. Liu, Y. Huang, and Y. Zhao, “An in-depth exploration of person re-identification and gait recognition in cloth-changing conditions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 13 824–13 833.
  21. C. Shen, C. Fan, W. Wu, R. Wang, G. Q. Huang, and S. Yu, “Lidargait: Benchmarking 3d gait recognition with point clouds,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 1054–1063.
  22. C. Shen, S. Yu, J. Wang, G. Q. Huang, and L. Wang, “A comprehensive survey on deep gait recognition: Algorithms, datasets and challenges,” arXiv preprint arXiv:2206.13732, 2022.
  23. R. Liao, S. Yu, W. An, and Y. Huang, “A model-based gait recognition method with body pose and human prior knowledge,” Pattern Recognition, vol. 98, p. 107069, 2020.
  24. C. Song, Y. Huang, Y. Huang, N. Jia, and L. Wang, “Gaitnet: An end-to-end network for gait based human identification,” Pattern recognition, vol. 96, p. 106988, 2019.
  25. J. Zheng, X. Liu, S. Wang, L. Wang, C. Yan, and W. Liu, “Parsing is all you need for accurate gait recognition in the wild,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 116–124.
  26. J. Liang, C. Fan, S. Hou, C. Shen, Y. Huang, and S. Yu, “Gaitedge: Beyond plain end-to-end gait recognition for better practicality,” arXiv preprint arXiv:2203.03972, 2022.
  27. X. Li, Y. Makihara, C. Xu, Y. Yagi, S. Yu, and M. Ren, “End-to-end model-based gait recognition,” in Proceedings of the Asian conference on computer vision, 2020.
  28. Y. Wang, X. Zhang, Y. Shen, B. Du, G. Zhao, L. Cui, and H. Wen, “Event-stream representation for human gaits identification using deep neural networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3436–3449, 2022.
  29. Y. Fu, S. Meng, S. Hou, X. Hu, and Y. Huang, “Gpgait: Generalized pose-based gait recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 19 595–19 604.
  30. T. Teepe, A. Khan, J. Gilg, F. Herzog, S. Hörmann, and G. Rigoll, “Gaitgraph: graph convolutional network for skeleton-based gait recognition,” in 2021 IEEE International Conference on Image Processing (ICIP).   IEEE, 2021, pp. 2314–2318.
  31. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “Smpl: A skinned multi-person linear model,” ACM transactions on graphics (TOG), vol. 34, no. 6, pp. 1–16, 2015.
  32. C. Xu, Y. Makihara, X. Li, and Y. Yagi, “Occlusion-aware human mesh model-based gait recognition,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1309–1321, 2023.
  33. Z. Huang, D. Xue, X. Shen, X. Tian, H. Li, J. Huang, and X.-S. Hua, “3d local convolutional neural networks for gait recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14 920–14 929.
  34. M. Wang, X. Guo, B. Lin, T. Yang, Z. Zhu, L. Li, S. Zhang, and X. Yu, “Dygait: Exploiting dynamic representations for high-performance gait recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 424–13 433.
  35. Y. Peng, S. Hou, K. Ma, Y. Zhang, Y. Huang, and Z. He, “Learning rich features for gait recognition by integrating skeletons and silhouettes,” arXiv preprint arXiv:2110.13408, 2021.
  36. H. Dou, P. Zhang, W. Su, Y. Yu, Y. Lin, and X. Li, “Gaitgci: Generative counterfactual intervention for gait recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 5578–5588.
  37. C. Fan, S. Hou, J. Wang, Y. Huang, and S. Yu, “Learning gait representation from massive unlabelled walking videos: A benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 14 920–14 937, 2023.
  38. K. Ma, Y. Fu, D. Zheng, Y. Peng, C. Cao, and Y. Huang, “Fine-grained unsupervised domain adaptation for gait recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 11 313–11 322.
  39. C. Filipi Gonçalves dos Santos, D. d. S. Oliveira, L. A. Passos, R. Gonçalves Pires, D. Felipe Silva Santos, L. Pascotti Valem, T. P. Moreira, M. Cleison S. Santana, M. Roder, J. Paulo Papa et al., “Gait recognition based on deep learning: A survey,” ACM Computing Surveys (CSUR), vol. 55, no. 2, pp. 1–34, 2022.
  40. A. Sepas-Moghaddam and A. Etemad, “Deep gait recognition: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  41. M. Ferrari Dacrema, P. Cremonesi, and D. Jannach, “Are we really making much progress? a worrying analysis of recent neural recommendation approaches,” in Proceedings of the 13th ACM conference on recommender systems, 2019, pp. 101–109.
  42. K. Musgrave, S. Belongie, and S.-N. Lim, “A metric learning reality check,” in European Conference on Computer Vision.   Springer, 2020, pp. 681–699.
  43. ——, “Unsupervised domain adaptation: A reality check,” arXiv preprint arXiv:2111.15672, 2021.
  44. B. Amos, B. Ludwiczuk, M. Satyanarayanan et al., “Openface: A general-purpose face recognition library with mobile applications,” CMU School of Computer Science, vol. 6, no. 2, p. 20, 2016.
  45. K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open mmlab detection toolbox and benchmark,” arXiv preprint arXiv:1906.07155, 2019.
  46. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019.
  47. Y. Fu, Y. Wei, Y. Zhou, H. Shi, G. Huang, X. Wang, Z. Yao, and T. Huang, “Horizontal pyramid matching for person re-identification,” in Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, ser. AAAI’19/IAAI’19/EAAI’19.   AAAI Press, 2019. [Online]. Available: https://doi.org/10.1609/aaai.v33i01.33018295
  48. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022.
  49. P. Zhang, H. Dou, Y. Yu, and X. Li, “Adaptive cross-domain learning for generalizable person re-identification,” in European Conference on Computer Vision.   Springer, 2022, pp. 215–232.
  50. W. Yu, H. Yu, Y. Huang, and L. Wang, “Generalized inter-class loss for gait recognition,” in Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 141–150.
  51. W. Xiang, H. Yang, D. Huang, and Y. Wang, “Multi-view gait video synthesis,” in Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 6783–6791.
  52. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
  53. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  54. X. Huang, D. Zhu, H. Wang, X. Wang, B. Yang, B. He, W. Liu, and B. Feng, “Context-sensitive temporal feature learning for gait recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 12 909–12 918.
  55. H. Luo, Y. Gu, X. Liao, S. Lai, and W. Jiang, “Bag of tricks and a strong baseline for deep person re-identification,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2019, pp. 0–0.
  56. K. Hara, H. Kataoka, and Y. Satoh, “Learning spatio-temporal features with 3d residual networks for action recognition,” in Proceedings of the IEEE international conference on computer vision workshops, 2017, pp. 3154–3160.
  57. Z. Qiu, T. Yao, and T. Mei, “Learning spatio-temporal representation with pseudo-3d residual networks,” in proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 5533–5541.
  58. H. Duan, Y. Zhao, K. Chen, D. Lin, and B. Dai, “Revisiting skeleton-based action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 2969–2978.
  59. M. Liu and J. Yuan, “Recognizing human actions as the evolution of pose estimation maps,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1159–1168.
  60. R. Liao, Z. Li, S. S. Bhattacharyya, and G. York, “Posemapgait: A model-based gait recognition method with pose estimation maps and graph convolutional networks,” Neurocomputing, vol. 501, pp. 514–528, 2022.
  61. K. Ma, Y. Fu, D. Zheng, C. Cao, X. Hu, and Y. Huang, “Dynamic aggregated network for gait recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 076–22 085.
  62. C. Fan, S. Hou, J. Wang, Y. Huang, and S. Yu, “Learning gait representation from massive unlabelled walking videos: A benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–18, 2023.
  63. Z. Wang, S. Hou, M. Zhang, X. Liu, C. Cao, Y. Huang, P. Li, and S. Xu, “Qagait: Revisit gait recognition from a quality perspective,” arXiv preprint arXiv:2401.13531, 2024.
  64. C. Zhang, X.-P. Chen, G.-Q. Han, and X.-J. Liu, “Spatial transformer network on skeleton-based gait recognition,” Expert Systems, vol. 40, no. 6, p. e13244, 2023.
  65. H. Zhu, W. Zheng, Z. Zheng, and R. Nevatia, “Gaitref: Gait recognition with refined sequential skeletons,” in 2023 IEEE International Joint Conference on Biometrics (IJCB).   IEEE, 2023, pp. 1–10.
  66. Y. Dong, C. Yu, R. Ha, Y. Shi, Y. Ma, L. Xu, Y. Fu, and J. Wang, “Hybridgait: A benchmark for spatial-temporal cloth-changing gait recognition with hybrid explorations,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 2, 2024, pp. 1600–1608.
  67. S. Hou, C. Cao, X. Liu, and Y. Huang, “Gait lateral network: Learning discriminative and compact representations for gait recognition,” in European Conference on Computer Vision.   Springer, 2020, pp. 382–398.
  68. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2921–2929.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Chao Fan (48 papers)
  2. Saihui Hou (19 papers)
  3. Junhao Liang (10 papers)
  4. Chuanfu Shen (10 papers)
  5. Jingzhe Ma (34 papers)
  6. Dongyang Jin (6 papers)
  7. Yongzhen Huang (23 papers)
  8. Shiqi Yu (32 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets