SkeletonGait: Gait Recognition Using Skeleton Maps (2311.13444v2)

Abstract: The choice of the representations is essential for deep gait recognition methods. The binary silhouettes and skeletal coordinates are two dominant representations in recent literature, achieving remarkable advances in many scenarios. However, inherent challenges remain, in which silhouettes are not always guaranteed in unconstrained scenes, and structural cues have not been fully utilized from skeletons. In this paper, we introduce a novel skeletal gait representation named skeleton map, together with SkeletonGait, a skeleton-based method to exploit structural information from human skeleton maps. Specifically, the skeleton map represents the coordinates of human joints as a heatmap with Gaussian approximation, exhibiting a silhouette-like image devoid of exact body structure. Beyond achieving state-of-the-art performances over five popular gait datasets, more importantly, SkeletonGait uncovers novel insights about how important structural features are in describing gait and when they play a role. Furthermore, we propose a multi-branch architecture, named SkeletonGait++, to make use of complementary features from both skeletons and silhouettes. Experiments indicate that SkeletonGait++ outperforms existing state-of-the-art methods by a significant margin in various scenarios. For instance, it achieves an impressive rank-1 accuracy of over 85% on the challenging GREW dataset. All the source code is available at https://github.com/ShiqiYu/OpenGait.

• The paper introduces a novel skeleton map that captures structural gait features through Gaussian heatmap transformation.
• It employs a deep learning framework, SkeletonGait, which outperforms traditional silhouette-based methods in challenging scenarios.
• The study integrates silhouette and skeletal data in SkeletonGait++, significantly improving rank-1 accuracy across multiple benchmark datasets.

SkeletonGait: Gait Recognition Using Skeleton Maps

The paper "SkeletonGait: Gait Recognition Using Skeleton Maps" presents a novel approach to gait recognition, a process of identifying individuals by their walking patterns, by leveraging a unique skeletal representation called the skeleton map. The researchers propose that this representation is effective in capturing the structural features of human gait, thus filling gaps left by traditional methods that rely on binary silhouettes or raw skeletal coordinates.

Key Contributions and Methodology

1. Skeleton Map Representation: The skeleton map is an innovative representation that transforms the coordinates of human joints into a heatmap using Gaussian approximation. This approach creates a silhouette-like image that emphasizes body structure without detailing the exact body shape. The map is a grid-based representation, which allows it to exploit spatial and temporal dimensions effectively when integrated with existing deep learning models.
2. SkeletonGait: The introduction of SkeletonGait leverages the skeleton map within a deep learning framework inspired by DeepGaitV2. By replacing the input silhouettes with skeleton maps, the researchers align the format and architecture, facilitating a comparison between the structural features and the combined attributes of body shape and structural features.
3. Performance Evaluation: The paper reports robust performance across five popular large-scale gait datasets: OU-MVLP, GREW, Gait3D, SUSTech1K, and CCPG. SkeletonGait demonstrated its effectiveness by achieving superior rank-1 accuracy over existing model-based methods, particularly excelling in scenarios where body silhouette data is less reliable, such as occlusion or poor lighting.
4. SkeletonGait++: To integrate the benefits of both skeletal and silhouette data, the researchers proposed a multi-branch architecture named SkeletonGait++. This architecture fuses features from both input types, significantly outperforming current state-of-the-art methods in various challenging environments.

Detailed Analysis

The methodology's efficacy is predominantly highlighted by its capability to achieve competitive results using structural features alone. Importantly, SkeletonGait's rank-1 accuracy improvement across datasets such as GREW and Gait3D provides insight into the potential of purely skeletal models in practical scenarios. Such advancements underscore the importance of body structural attributes in gait recognition tasks.

The authors also address critical insights into popular datasets. For instance, they suggest that the GREW dataset may lack sufficient viewpoint diversity, as indicated by SkeletonGait’s unexpectedly high performance despite its known challenges. Such observations prompt further exploration into dataset composition and its impact on model evaluation.

Future Implications

The findings highlight the potential for future advancements in model-based gait recognition that are less dependent on visual appearance and more on intrinsic structural dynamics. This shift could lead to robust recognition systems that perform consistently across varied environmental conditions and capture scenarios.

Furthermore, the integration observed in SkeletonGait++ suggests that a hybrid approach leveraging both silhouettes and skeletal data could redefine the benchmarks for gait recognition systems. As these models mature, they may find broader applications not only in security and surveillance but also in healthcare and human-computer interaction, where gait analysis offers valuable insights.

Conclusion

The research presents a compelling case for the skeleton map as a viable alternative or complement to traditional representation techniques in gait recognition. By focusing on the inherent structural properties of the human gait, SkeletonGait and its enhanced version, SkeletonGait++, pave the way for more resilient and adaptable recognition systems that can maintain performance across a diverse set of challenging conditions. The implications extend to improving the robustness and scalability of future biometric systems.