Surpassing Real-World Source Training Data: Random 3D Characters for Generalizable Person Re-Identification (2006.12774v2)

Abstract: Person re-identification has seen significant advancement in recent years. However, the ability of learned models to generalize to unknown target domains still remains limited. One possible reason for this is the lack of large-scale and diverse source training data, since manually labeling such a dataset is very expensive and privacy sensitive. To address this, we propose to automatically synthesize a large-scale person re-identification dataset following a set-up similar to real surveillance but with virtual environments, and then use the synthesized person images to train a generalizable person re-identification model. Specifically, we design a method to generate a large number of random UV texture maps and use them to create different 3D clothing models. Then, an automatic code is developed to randomly generate various different 3D characters with diverse clothes, races and attributes. Next, we simulate a number of different virtual environments using Unity3D, with customized camera networks similar to real surveillance systems, and import multiple 3D characters at the same time, with various movements and interactions along different paths through the camera networks. As a result, we obtain a virtual dataset, called RandPerson, with 1,801,816 person images of 8,000 identities. By training person re-identification models on these synthesized person images, we demonstrate, for the first time, that models trained on virtual data can generalize well to unseen target images, surpassing the models trained on various real-world datasets, including CUHK03, Market-1501, DukeMTMC-reID, and almost MSMT17. The RandPerson dataset is available at https://github.com/VideoObjectSearch/RandPerson.

• The paper demonstrates that fusing synthetic data with real-world datasets boosts person re-identification performance, with RandPerson elevating Rank-1 accuracy by up to 28.8%.
• The authors employ a comprehensive methodology by combining multiple synthetic datasets, including SOMAset, SyRI, PersonX, and RandPerson, with benchmarks like CUHK03-NP, Market-1501, DukeMTMC-reID, and MSMT17.
• The findings indicate that leveraging synthetic data like RandPerson can significantly enhance surveillance and security systems by improving identification precision.

Detailed Analysis of Dataset Fusion in Person Re-Identification

The paper presents an extensive analysis on the fusion of multiple datasets for training models in the domain of person re-identification (ReID). The main datasets considered in this paper include SOMAset, SyRI, PersonX, and RandPerson, which are fused with widely recognized real-world databases such as CUHK03-NP, Market-1501, DukeMTMC-reID, and MSMT17. The primary objective of this research is to understand the impact of different synthetic datasets on ReID performance, measured through Rank-1 accuracy and mean average precision (mAP).

A comprehensive comparative assessment is presented through tables detailing results across the fusion of different datasets. Notably, the paper highlights that RandPerson demonstrates superior performance improvements in various test settings compared to other synthetic datasets like SOMAset, SyRI, and PersonX. Improvements with RandPerson range from 0.2% to 28.8% in Rank-1 accuracy, and from 1.1% to 19.3% in mAP. These figures indicate RandPerson's significant advantage in synthesizing data that closely mimic real-world conditions conducive for improving ReID systems.

Illustrated by the provided data:

1. Testing on CUHK03-NP: Models trained with RandPerson fused with CUHK03-NP yield a Rank-1 accuracy of 50.4% and mAP of 44.4%. This performance is notably higher than others, with PersonX + CUHK03 achieving the next best at 43.5% Rank-1 and 40.7% mAP.
2. Testing on Market-1501: When tested on Market-1501, RandPerson again leads with a Rank-1 accuracy of 87.2% and mAP of 70.9%, pointing towards its effectiveness across multiple test scenarios.
3. Testing on DukeMTMC-reID: The Rank-1 score for RandPerson here is 79.4%, with mAP reaching 60.6%, surpassing others like SyRI and PersonX.
4. Testing on MSMT17: RandPerson achieves a Rank-1 of 65.0% and mAP of 36.8%, further reinforcing its robustness in handling diverse and comprehensive datasets.

The implications of these findings are multifaceted. Practically, the improvement due to RandPerson suggests its potential applicability in enhancing ReID systems that are crucial for surveillance, security, and other similar applications where enhanced identification accuracy is necessary. Theoretically, the results underscore the importance of versatile synthetic data generation models that can expand the generalizability of ReID systems.

Future AI implementations could benefit from integrating methods that incorporate advantages of RandPerson into existing frameworks. This might involve refining the synthetic data generation process or exploring ways to leverage RandPerson-like datasets in conjunction with real-world data to develop more resilient and accurate ReID systems. Moreover, this analysis may drive further investigation into how synthetic datasets can adaptively evolve with real-time data collection, thus continually improving the performance of AI models in ReID and beyond.