Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification

Abstract: Re-identification is generally carried out by encoding the appearance of a subject in terms of outfit, suggesting scenarios where people do not change their attire. In this paper we overcome this restriction, by proposing a framework based on a deep convolutional neural network, SOMAnet, that additionally models other discriminative aspects, namely, structural attributes of the human figure (e.g. height, obesity, gender). Our method is unique in many respects. First, SOMAnet is based on the Inception architecture, departing from the usual siamese framework. This spares expensive data preparation (pairing images across cameras) and allows the understanding of what the network learned. Second, and most notably, the training data consists of a synthetic 100K instance dataset, SOMAset, created by photorealistic human body generation software. Synthetic data represents a good compromise between realistic imagery, usually not required in re-identification since surveillance cameras capture low-resolution silhouettes, and complete control of the samples, which is useful in order to customize the data w.r.t. the surveillance scenario at-hand, e.g. ethnicity. SOMAnet, trained on SOMAset and fine-tuned on recent re-identification benchmarks, outperforms all competitors, matching subjects even with different apparel. The combination of synthetic data with Inception architectures opens up new research avenues in re-identification.

Exploring Synthetic Data in Re-identification with SOMAnet and SOMAset

The paper, "Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification," explores re-identification using deep learning, specifically addressing the limitations posed by varying apparel. Traditional approaches primarily depend on recognizing individuals purely based on their clothing, which introduces challenges when subjects change their apparel. This study proposes a deep convolutional neural network framework named SOMAnet combined with a synthetic dataset called SOMAset to solve these limitations inherent in conventional systems.

The authors introduce several innovations in the field of re-identification. The concept of SOMAnet moves away from the conventional siamese network usage by adopting the Inception architecture. This adaptation enables the network to extract features without necessitating the costly data preparation and pairing of images from different cameras inherent in siamese frameworks. As a result, SOMAnet provides more understanding about what the network learns, focusing on the structural attributes of human figures such as somatotype, height, obesity, and gender, alongside clothing information.

Most notably, SOMAnet is trained using a synthetic dataset, SOMAset, which consists of 100,000 images generated through photorealistic human body generation software. SOMAset encompasses a variety of human prototypes, mixed somatotypes, ethnicities, and camera perspectives, thus capturing significant diversity for training purposes. This synthetic dataset was shown to boost performance, demonstrating that SOMAnet trained on SOMAset outperformed other competitors on benchmarks, even when subjects wore different clothing.

Quantitatively, the integration of synthetic data showcased strong numerical results, with SOMAnet trained on SOMAset achieving state-of-the-art results across multiple re-identification benchmarks like CUHK03, Market-1501, RAiD, and RGBD-ID. The adaptability of SOMAnet was proven as it efficiently handled scenarios where attire changes between camera perspectives—a substantial improvement over previous methods restricted to consistent apparel assumptions.

The implications of this research are multifaceted. Practically, it offers a cost-effective solution to eliminate the need for extensive labeled real-world datasets and circumvents privacy concerns linked with real data usage. Theoretically, it positions synthetic data as a pivotal tool in enhancing deep network training and illustrates the vital role of structural body attributes for recognition, which traditional methods often overlook.

Moving forward, this work opens up new avenues for AI applications that benefit from synthetic data for training, particularly in fields like surveillance and security. As such, leveraging synthetic datasets to improve generalization and accuracy in diverse real-world scenarios could become increasingly significant. The research also prompts a reconsideration of the re-identification domain, suggesting a transition towards broader non-collaborative person recognition techniques.

The insights provided by the paper are pivotal in showcasing how synthetic datasets, paired with innovative network designs like SOMAnet, can reshape the boundaries of what is achievable in human recognition tasks, elevating the efficacy and breadth of re-identification technology. As AI continues to evolve, integrating synthetic data will likely form a cornerstone of developing robust and privacy-conscious recognition systems.