PsyMo Benchmark: Gait & Psy Analysis

• PsyMo Benchmark is a multi-modal, privacy-preserving dataset that integrates gait recordings with extensive psychometric annotations.
• It employs high-resolution silhouettes, 2D skeletons, and 3D SMPL meshes captured under various walking conditions and viewpoints.
• The framework supports run-level and subject-level evaluations, facilitating robust benchmarking for gait recognition and psychological trait inference.

The PsyMo benchmark is a multi-modal, privacy-preserving dataset and evaluation framework for the estimation of self-reported psychological traits from gait, designed to advance research at the intersection of computer vision, biometrics, and psychological assessment (Cosma et al., 2023). It integrates high-resolution motion representations with comprehensive psychometric annotations, enabling both psychological trait inference and traditional gait recognition benchmarking across a diverse set of walking variations and camera viewpoints.

1. Dataset Composition

PsyMo comprises gait recordings from 312 healthy subjects under controlled laboratory conditions, employing three synchronized consumer surveillance cameras. Each subject performed seven distinct walking variations:

• Normal walking
• Clothing change
• Carrying a bag
• Slower speed
• Faster speed
• Two dual-task conditions: walking while texting and walking while talking on the phone

Data for each walk was captured from six distinct viewpoints (three physical, three virtual), resulting in 14,976 gait sequences. Each sequence is represented via three modalities:

• Silhouettes: Segmented using Hybrid Task Cascade, resized to $128 \times 128$ pixels; Gait Energy Images (GEI) computed for each walk
• 2D Skeletons: Extracted by AlphaPose, featuring 18 COCO-format joints, each with $(x, y)$ coordinates and confidence scores
• 3D Human Meshes/SMPL: Generated from CLIFF, providing 24 joint coordinates $(x, y, z)$ and 10-dimensional SMPL model parameters

Subjects also completed six standardized psychological questionnaires, collectively measuring 17 attributes. These include:

• Big Five Inventory (OCEAN model) for personality
• Rosenberg Self-Esteem Scale
• Buss-Perry Aggression Questionnaire
• Occupational Fatigue Exhaustion/Recovery Scale
• Depression, Anxiety, and Stress Scale (DASS-21)
• General Health Questionnaire (GHQ)

Demographic data encompass age, gender, height, weight, and computed BMI.

Modality	Algorithm	Output Dimension
Silhouette	Hybrid Task Cascade	128 × 128 + GEI
2D Skeleton	AlphaPose (COCO-18)	$18 \times (x,y)$ + conf
3D Mesh/SMPL	CLIFF	24 × 3 + SMPL: 10-D

2. Methodological Principles

The primary objective is the automated estimation of psychological attributes based on movement features derived from gait representations. Modern gait recognition architectures have been adapted for this multi-modal, multi-task setting:

• GaitFormer, GaitGraph for skeleton-based inputs
• GaitSet, GaitGL for silhouette-based inputs
• SMPLGait for hybrid silhouette and SMPL mesh-based representations

Models are trained to predict ordinal or multi-class trait labels derived from the psychological questionnaires using weighted cross-entropy loss:

$L = -\sum_k y_k \log(p_k)$

where $y_k$ is the true class label and $p_k$ is the predicted probability for class $k$. Some models are specialized for individual questionnaire subscales. The methodology leverages the multi-modal nature of the data and treats outcome prediction as a multi-task ordinal classification problem. No novel model architectures are introduced in PsyMo itself, but the framework's comprehensive design permits benchmarking and extension via state-of-the-art learning techniques.

3. Evaluation Protocols

PsyMo formalizes two evaluation regimes:

• Run-Level: Each gait sequence is independently evaluated for trait estimation; metrics such as precision, recall, and weighted F1 score are computed over sequences, walking variations, and viewpoints. This protocol parallels conventional gait recognition, mapping each run to a psychological trait.
• Subject-Level: Trait prediction is aggregated over all runs for a subject, typically via majority voting, simulating scenarios where multiple gait samples per subject are available. This approach yields increased robustness to observation noise and session variability.

For benchmarking gait recognition itself, a gallery–probe protocol inspired by CASIA-B is implemented. The gallery contains "second normal walking" sequences only, while the probe set includes all remaining variations and viewpoints (excluding gallery viewpoints). This permits fine-grained analysis of recognition method stability against covariates such as varying speed and dual-tasking.

4. Applications and Benchmark Utility

PsyMo addresses dual research motivations:

1. Psychological Trait Estimation: Enables automated inference of mental distress, self-esteem, occupational fatigue, and other psychological indicators from unconstrained gait. The prospect of non-intrusive screening—occupational, clinical, or marketing—rests on the degree to which movement expresses these traits.
2. Gait Recognition Benchmark: Offers a controlled and diverse alternative to canonical benchmarks (e.g., CASIA-B, FVG), including unique dual-task and clothing-change scenarios. Its detailed metadata facilitates evaluation of method robustness against real-world walking variability, providing a testbed for both closed-set and open-set recognition.

Researchers may employ PsyMo for studies of embodied cognition, behavioral profiling, and biometric recognition, leveraging the dataset’s breadth and domain annotations for comparative method development.

5. Anonymization, Privacy, and Ethical Considerations

PsyMo maintains subject anonymity through stringent data processing:

• No raw RGB videos are released
• Processed modalities only (silhouettes, 2D/3D skeletons, SMPL meshes) are distributed, eliminating identifying visual traits
• Explicit, informed consent and Ethics Review Board approval were obtained
• All cues directly linking data to real-world identities are eliminated

These measures ensure compliance with best practices in privacy-conscious research, targeting a balance between dataset utility and ethical responsibility.

6. Benchmarking and Comparative Analysis

PsyMo’s construction—encompassing 14,976 multi-modal sequences, seven walking variations, six viewpoints, and extensive psychometric annotation—renders it suitable for advanced benchmarking:

• Fine-grained analysis: Effects of walking variation, observation angle, and demographic covariates on both biometric and psychological trait prediction can be isolated
• Comparative methodology: Directly supports run-level and subject-level protocol comparison with prior datasets, enabling standardization across studies
• Modal diversity: Silhouettes, 2D skeleton, and 3D SMPL mesh input options permit comprehensive evaluation of recognition and inference algorithms in a unified framework

A plausible implication is that future work leveraging richer modality fusion or temporal context could further resolve trait inference challenges, especially in unconstrained, real-world settings.

7. Impact and Future Directions

PsyMo establishes a new paradigm for movement-informed psychological inference and biometric evaluation. Its multi-modal architecture and psychometric annotation scheme permit synthesis across computational psychology, behavioral biometrics, and human-centered AI. Applications are expected in workforce health screening, consumer modeling, and adaptive surveillance.

Subsequent research—including methods such as hierarchical mixtures of movement experts (Cǎtrunǎ et al., 6 Oct 2025) and open-set enroLLMent via SetTransformers (Basoc et al., 5 May 2025)—utilizes PsyMo to test generalization, contextual aggregation, and multi-task learning paradigms. The benchmark’s extensibility invites methodology advancements in cross-domain generalization, fusion of movement modalities, and interpretable movement-psychology links.

PsyMo thus constitutes a foundational benchmark for interdisciplinary research into the embodied signatures of psychological traits and robust gait-based identification in privacy-sensitive contexts.