Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GaitPT: Skeletons Are All You Need For Gait Recognition (2308.10623v2)

Published 21 Aug 2023 in cs.CV and cs.LG

Abstract: The analysis of patterns of walking is an important area of research that has numerous applications in security, healthcare, sports and human-computer interaction. Lately, walking patterns have been regarded as a unique fingerprinting method for automatic person identification at a distance. In this work, we propose a novel gait recognition architecture called Gait Pyramid Transformer (GaitPT) that leverages pose estimation skeletons to capture unique walking patterns, without relying on appearance information. GaitPT adopts a hierarchical transformer architecture that effectively extracts both spatial and temporal features of movement in an anatomically consistent manner, guided by the structure of the human skeleton. Our results show that GaitPT achieves state-of-the-art performance compared to other skeleton-based gait recognition works, in both controlled and in-the-wild scenarios. GaitPT obtains 82.6% average accuracy on CASIA-B, surpassing other works by a margin of 6%. Moreover, it obtains 52.16% Rank-1 accuracy on GREW, outperforming both skeleton-based and appearance-based approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. 2d human pose estimation: New benchmark and state of the art analysis. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014.
  2. S. Arora and M. S. Bhatia. A computer vision system for iris recognition based on deep learning. In 2018 IEEE 8th International Advance Computing Conference (IACC), pages 157–161. IEEE, 2018.
  3. D. Bryliuk and V. Starovoitov. Access control by face recognition using neural networks. Institute of Engineering Cybernetics, Laboratory of Image Processing and Recognition, 4, 2002.
  4. Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
  5. End-to-end object detection with transformers. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 213–229. Springer, 2020.
  6. From face to gait: Weakly-supervised learning of gender information from walking patterns. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), pages 1–5. IEEE, 2021.
  7. Gaitset: Regarding gait as a set for cross-view gait recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):8126–8133, Jul. 2019.
  8. Hybrid task cascade for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4974–4983, 2019.
  9. Exploring self-supervised vision transformers for gait recognition in the wild. Sensors, 23(5), 2023.
  10. A. Cosma and E. Radoi. Learning gait representations with noisy multi-task learning. Sensors, 22(18):6803, 2022.
  11. A. Cosma and I. E. Radoi. Wildgait: Learning gait representations from raw surveillance streams. Sensors, 21(24):8387, 2021.
  12. Simple and efficient pose-based gait recognition method for challenging environments. Pattern Analysis and Applications, 24(2):497–507, 2021.
  13. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  14. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  15. Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  16. Horizontal pyramid matching for person re-identification. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8295–8302, 2019.
  17. A survey of human gait-based artificial intelligence applications. Frontiers in Robotics and AI, 8, 2022.
  18. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  19. Gait lateral network: Learning discriminative and compact representations for gait recognition. In A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, editors, Computer Vision – ECCV 2020, pages 382–398, Cham, 2020. Springer International Publishing.
  20. A. K. Jain and A. Kumar. Biometric recognition: an overview. Second generation biometrics: The ethical, legal and social context, pages 49–79, 2012.
  21. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661–18673, 2020.
  22. Jointsgait: A model-based gait recognition method based on gait graph convolutional networks and joints relationship pyramid mapping. arXiv preprint arXiv:2005.08625, 2020.
  23. Face recognition in low quality images: A survey. arXiv preprint arXiv:1805.11519, 2018.
  24. A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognition, 98:107069, 2020.
  25. Simple and efficient pose-based gait recognition method for challenging environments. Pattern Analysis and Applications, 24:497–507, 2021.
  26. Gaitgl: Learning discriminative global-local feature representations for gait recognition. arXiv preprint arXiv:2208.01380, 2022.
  27. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  28. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  29. Disentangling and unifying graph convolutions for skeleton-based action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 143–152, 2020.
  30. I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  31. J. Lu and Y.-P. Tan. Gait-based human age estimation. IEEE Transactions on Information Forensics and Security, 5(4):761–770, 2010.
  32. Fingernet: Pushing the limits of fingerprint recognition using convolutional neural network. arXiv preprint arXiv:1907.12956, 2019.
  33. Deep face recognition. 2015.
  34. Spatial temporal transformer network for skeleton-based action recognition. In Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10–15, 2021, Proceedings, Part III, pages 694–701. Springer, 2021.
  35. Biometric recognition: Security and privacy concerns. IEEE security & privacy, 1(2):33–42, 2003.
  36. Learning perceived emotion using affective and deep features for mental health applications. In 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 395–399, 2019.
  37. J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.
  38. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
  39. A. Sepas-Moghaddam and A. Etemad. Deep gait recognition: A survey. IEEE transactions on pattern analysis and machine intelligence, 45(1):264–284, 2022.
  40. Geinet: View-invariant gait recognition using a convolutional neural network. In 2016 international conference on biometrics (ICB), pages 1–8. IEEE, 2016.
  41. Vision-based gait recognition: A survey. Ieee Access, 6:70497–70527, 2018.
  42. L. N. Smith. Cyclical learning rates for training neural networks. In 2017 IEEE winter conference on applications of computer vision (WACV), pages 464–472. IEEE, 2017.
  43. Stronger, faster and more explainable: A graph convolutional baseline for skeleton-based action recognition. In proceedings of the 28th ACM international conference on multimedia, pages 1625–1633, 2020.
  44. Deep high-resolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5693–5703, 2019.
  45. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Transactions on Computer Vision and Applications, 10(1):1–14, 2018.
  46. Towards a deeper understanding of skeleton-based gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1569–1577, 2022.
  47. Gaitgraph: Graph convolutional network for skeleton-based gait recognition. In 2021 IEEE International Conference on Image Processing (ICIP), pages 2314–2318. IEEE, 2021.
  48. Going deeper with image transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 32–42, 2021.
  49. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  50. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696, 2022.
  51. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision, pages 568–578, 2021.
  52. Multi-view gait recognition using 3d convolutional neural networks. In 2016 IEEE International Conference on Image Processing (ICIP), pages 4165–4169, 2016.
  53. A comprehensive study on cross-view gait based human identification with deep cnns. IEEE transactions on pattern analysis and machine intelligence, 39(2):209–226, 2016.
  54. A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(2):209–226, 2017.
  55. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems, 34:12077–12090, 2021.
  56. Real-time gait-based age estimation and gender classification from a single image. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3460–3470, 2021.
  57. Vitpose: Simple vision transformer baselines for human pose estimation. arXiv preprint arXiv:2204.12484, 2022.
  58. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  59. Deep learning for person re-identification: A survey and outlook. IEEE transactions on pattern analysis and machine intelligence, 44(6):2872–2893, 2021.
  60. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In 18th International Conference on Pattern Recognition (ICPR’06), volume 4, pages 441–444, 2006.
  61. Realgait: Gait recognition for person re-identification. arXiv preprint arXiv:2201.04806, 2022.
  62. Gait recognition in the wild with dense 3d representations and a benchmark. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  63. Gait recognition in the wild: A benchmark. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14789–14799, 2021.
  64. Z. Zivkovic. Improved adaptive gaussian mixture model for background subtraction. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., volume 2, pages 28–31. IEEE, 2004.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets