VersatileGait: Synthetic Gait Database

• VersatileGait is a synthetic gait dataset featuring around one million silhouette sequences from up to 11,000 subjects across 33 viewpoints.
• It employs a Unity3D pipeline to generate high-contrast, textureless silhouettes enriched with fine-grained attributes such as gender and walking style.
• Evaluations demonstrate that pretraining on VersatileGait notably improves cross-view gait recognition performance, reducing reliance on large real-world datasets.

Searching arXiv for VersatileGait and related gait datasets to ground the article in the cited papers. Searching for the synthetic VersatileGait paper. VersatileGait is a synthetic gait dataset for silhouette-based recognition, generated with a game engine to supply scale, fine-grained attributes, and scenario complexity that are difficult to obtain in real collections. In the principal 2021 description, it comprises around one million silhouette sequences from 11,000 subjects, totaling 72 million frames at $280 \times 200$ resolution, with automatic annotations and 33 viewpoints formed by 11 horizontal views and 3 vertical views (Dou et al., 2021). A later arXiv abstract presents the project as a dataset “towards in-the-wild simulation” and reports more than one million silhouette sequences of 10,000 subjects, indicating a version-dependent discrepancy in the reported subject count (Zhang et al., 2021).

1. Motivation and research setting

VersatileGait was introduced to bridge the gap between controlled, small-scale laboratory gait datasets and the demands of practical gait recognition in visual surveillance, security checks, and video retrieval (Dou et al., 2021). The motivation is rooted in two persistent constraints of real data collection: limited variation in intrinsic and extrinsic factors, and the expense and privacy burden of large-scale capture. The same source contrasts prior benchmarks such as CASIA-B, which contains 124 subjects and 13,632 sequences, with OU-MVLP, which has 10,307 subjects but only two sequences per view without clothing or bag changes. It also notes that real silhouettes often inherit artifacts from background subtraction and segmentation, including irregular cavities and missing body parts.

The dataset’s central design choice is to synthesize silhouettes rather than RGB imagery. The published rationale is that colorless, textureless binary masks reduce the synthetic-to-real domain gap relative to synthetic RGB data, especially for gait recognition tasks whose canonical inputs are already silhouette-based (Dou et al., 2021). This does not eliminate the domain gap, but it reframes it: the primary challenge becomes whether simulated contour dynamics, view geometry, and nuisance factors are sufficiently diverse to transfer to real deployments.

2. Dataset composition and annotation structure

The dataset is reported to contain around one million silhouette sequences from 11,000 subjects, with 72 million frames rendered at $280 \times 200$ resolution and grouped into about one million sequences; on average, sequences contain roughly 72 frames (Dou et al., 2021). The camera design covers 33 viewpoints by combining 11 horizontal views with 3 vertical views. The horizontal views span $0^\circ$ to $180^\circ$ at $18^\circ$ increments: $$0^\circ, 18^\circ, 36^\circ, 54^\circ, 72^\circ, 90^\circ, 108^\circ, 126^\circ, 144^\circ, 162^\circ, 180^\circ$$. The vertical component is realized at three pitch levels, and experiments explicitly analyze $0^\circ, 30^\circ,$ and $60^\circ$. Scenes may contain up to 3 models simultaneously, enabling multi-person gait and occlusion scenarios.

At the attribute level, the modeling stage uses parameterized human models with attributes such as gender, age, and height, while walking styles are diversified through animation selection and modification of stride length and arm swing. Bag and clothing conditions and accessories preferences are also simulated because they affect the silhouette contour. In the experimental pipeline, the fine-grained attributes emphasized most clearly are gender and walking style. Annotation is generated automatically through the Unity3D pipeline and includes identity ID and camera viewpoint metadata, specifically the horizontal yaw among the 11 views and the pitch angle among the 3 vertical views. The data are silhouettes, not textured images, and therefore avoid a separate segmentation stage (Dou et al., 2021).

Aspect	Reported specification
Scale	Around one million sequences; 72 million frames
Subjects	11,000 in (Dou et al., 2021); 10,000 in (Zhang et al., 2021)
View geometry	33 viewpoints = 11 horizontal views $\times$ 3 vertical views
Frame format	Binary silhouettes at $280 \times 200$
Scene complexity	Up to 3 models simultaneously
Key labels	Identity, horizontal view, pitch angle; fine-grained attributes used include gender and walking style

The combination of identity labels, view metadata, and attribute labels makes the dataset suitable not only for canonical cross-view recognition but also for attribute-conditioned retrieval and robustness studies under yaw-pitch mismatch and occlusion. A plausible implication is that VersatileGait was designed as much for controlled ablation as for raw scale.

3. Generation pipeline and rendering design

The generation pipeline consists of four explicit stages: 3D model generation, walking animation collection, animation retargeting, and scene simulation with silhouette capture (Dou et al., 2021). MakeHuman is used to create 150 realistic, parameterized 3D human models with balanced attribute distribution, including genders, ages, and heights. The paper states that artificial constraints are imposed so that accessory preferences differ by gender, thereby making contour differences more salient at the silhouette level.

Walking motions are drawn from 100 Mixamo walking animations, including examples such as standard walking and brutal walking. To increase diversity, these animations are further adjusted by step stride and arm angle. Unity3D’s Mecanim system is then used to bind the animations to humanoid bones of the human models, and manual correction is applied to ensure fluent, natural motion, yielding 11,000 high-quality walking individuals. Scene simulation is performed in Unity3D with dark skyboxes and six orthogonal parallel lights. These choices support consistent, high-contrast silhouettes without texture or color and minimize the need for post-processing. The capture stage uses 33 cameras corresponding to the yaw-pitch layout, and binary silhouettes are rendered directly while annotations are generated automatically (Dou et al., 2021).

This pipeline is notable less for novel rendering primitives than for its explicit control over the factors most disruptive to gait recognition: viewpoint, pitch distortion, contour-changing accessories, inter-person occlusion, and walking-style variability. Because the output modality is the silhouette itself, the pipeline also avoids the propagation of segmentation error from RGB video into the training set.

4. Evaluation protocols and transfer to real datasets

VersatileGait is evaluated primarily as a pretraining and transfer resource rather than through a fully specified internal train/validation/test benchmark (Dou et al., 2021). The CASIA-B evaluation follows standard practice: the gallery uses NM#1–4, the probes use NM#5–6, BG#1–2, and CL#1–2 across 11 views from $280 \times 200$0 to $280 \times 200$1 at $280 \times 200$2 steps, and the principal metric is Rank-1 accuracy averaged across non-identical-view cases.

Two-stage transfer strategies are reported. The first pretrains on VersatileGait and then finetunes on CASIA-B. The second mixes the entire real training set with VersatileGait for pretraining and then finetunes. Both improve over training directly on CASIA-B alone. For GaitSet, the mean Rank-1 accuracies change from 95.0 to 96.2 to 96.4 on NM, from 87.2 to 88.0 to 88.5 on BG, and from 70.4 to 70.6 to 70.6 on CL for origin, pre+finetune, and mix+finetune respectively. For GaitPart, the corresponding means are 96.2 to 96.8 to 97.1 on NM, 91.5 to 91.6 to 91.8 on BG, and 78.7 to 78.9 to 78.9 on CL (Dou et al., 2021).

Model	CASIA-B mean Rank-1 (origin / pre+finetune / mix+finetune)
GaitSet	NM 95.0 / 96.2 / 96.4; BG 87.2 / 88.0 / 88.5; CL 70.4 / 70.6 / 70.6
GaitPart	NM 96.2 / 96.8 / 97.1; BG 91.5 / 91.6 / 91.8; CL 78.7 / 78.9 / 78.9

The paper also reports view-wise gains. For GaitSet under NM, the Rank-1 accuracy at $280 \times 200$3 rises from 90.8 to 92.8 to 93.7; at $280 \times 200$4, from 91.7 to 93.1 to 92.9; and at $280 \times 200$5, from 85.8 to 89.5 to 90.0. An additional observation is that pretraining on VersatileGait increases the state-of-the-art method GaitPart by 1.1% Rank-1 accuracy on CASIA-B. The mix-ratio experiments further indicate that using 50% real data plus VersatileGait can achieve performance comparable to training on 100% real data. The same study reports diminishing returns when pretraining size is scaled from 100 to 10,000 subsets, while inference time rises as the gallery grows (Dou et al., 2021).

These results position VersatileGait as a data-centric mechanism for reducing real-data requirements. They do not, however, establish that synthetic pretraining uniformly closes the gap for all nuisance conditions, since the gains are most pronounced in some settings and modest in others, especially the CL condition.

5. Fine-grained attributes, retrieval acceleration, and multi-pitch analysis

A distinctive feature of VersatileGait is the use of fine-grained attributes as auxiliary supervision. The training objective is reported as

$280 \times 200$6

where $280 \times 200$7 is the metric learning component, $280 \times 200$8 is the cross-entropy loss for attribute $280 \times 200$9, $0^\circ$0 are weighting coefficients, and $0^\circ$1 is the set of attributes (Dou et al., 2021).

Using gender and walking style as attribute labels, the paper reports attribute prediction accuracies of 88.8% for gender and 99.63% for walking style. On CASIA-B, a simplified GaitSet baseline improves substantially when augmented with attribute-guided multi-task learning: NM mean rises from 54.1 to 84.9, BG mean from 51.4 to 83.1, and CL mean from 37.7 to 66.6. Representative NM view-wise improvements include $0^\circ$2 at $0^\circ$3, $0^\circ$4 at $0^\circ$5, and $0^\circ$6 at $0^\circ$7 (Dou et al., 2021).

The same attribute signals are used for retrieval acceleration. Attribute filtering based on reliably predicted gender or walking style reduces the gallery search space and speeds inference by over $0^\circ$8, with modest accuracy drops bounded by $0^\circ$9 when filtering by walking styles and $180^\circ$0 when filtering by genders. In this sense, the attributes are not merely descriptive metadata; they are operational side information for indexing and candidate reduction.

VersatileGait also foregrounds pitch as a first-class nuisance variable. The paper argues that when models are trained without multi-pitch angle coverage, cross-pitch recognition drops significantly. Excluding $180^\circ$1 pitch from training causes dramatic degradation at $180^\circ$2 probes, whereas excluding $180^\circ$3 maintains more stable performance, implying that higher pitch angle data is more critical. Even with training data, recognition accuracy decreases notably for probes at $180^\circ$4, and cross-horizontal view problems become more severe when cross-pitch factors are present, especially around $180^\circ$5 and $180^\circ$6 (Dou et al., 2021). This suggests that yaw invariance and pitch invariance are not separable nuisances in practice; they compound.

6. Limits, ambiguities, and nomenclature

Several limitations are explicit in the published descriptions. The synthetic-to-real gap remains a consideration, even if the use of silhouettes is argued to reduce it, and no quantitative domain-gap metrics such as FID are reported (Dou et al., 2021). Attribute coverage in experiments focuses mainly on gender and walking style, although the generative process uses broader parameters such as age and height. Scenes use dark backgrounds and fixed lighting, and multi-person scenarios are capped at up to three individuals. The paper does not provide explicit train/validation/test splits for VersatileGait itself; instead, the dataset is used chiefly as a pretraining source and as a platform for cross-pitch and cross-view analyses.

There is also a record-level ambiguity in the dataset’s reported size. One arXiv paper describes around one million sequences from 11,000 subjects, whereas a later abstract reports more than one million sequences from 10,000 subjects (Dou et al., 2021, Zhang et al., 2021). This suggests an evolving project specification or a version difference, and it warrants caution when citing the dataset’s exact scale.

A separate nomenclature issue is that “VersatileGait” should not be conflated with the distinct dataset titled “A multi-sensor human gait dataset captured through an optical system and inertial measurement units” (Santos et al., 2021). That resource contains synchronized optical motion capture and IMU accelerations from 25 healthy adults across two sessions and, according to its manuscript and figshare entry, does not use the name “VersatileGait.” The two datasets therefore occupy different methodological niches: one is a large-scale synthetic silhouette corpus for gait recognition and transfer learning, and the other is a real multi-sensor laboratory dataset for optical-inertial gait analysis. A plausible implication is that the term “VersatileGait Database” is most accurately reserved for the synthetic Unity3D-generated silhouette dataset introduced by Dou et al., not for the later optical/IMU collection.