SeqFace: Make full use of sequence information for face recognition (1803.06524v2)

Abstract: Deep convolutional neural networks (CNNs) have greatly improved the Face Recognition (FR) performance in recent years. Almost all CNNs in FR are trained on the carefully labeled datasets containing plenty of identities. However, such high-quality datasets are very expensive to collect, which restricts many researchers to achieve state-of-the-art performance. In this paper, we propose a framework, called SeqFace, for learning discriminative face features. Besides a traditional identity training dataset, the designed SeqFace can train CNNs by using an additional dataset which includes a large number of face sequences collected from videos. Moreover, the label smoothing regularization (LSR) and a new proposed discriminative sequence agent (DSA) loss are employed to enhance discrimination power of deep face features via making full use of the sequence data. Our method achieves excellent performance on Labeled Faces in the Wild (LFW), YouTube Faces (YTF), only with a single ResNet. The code and models are publicly available on-line (https://github.com/huangyangyu/SeqFace).

• The paper introduces a novel dual-dataset training approach that fuses labeled identity images with unlabeled video sequence data.
• The paper adapts Label Smoothing Regularization and proposes a new Discriminative Sequence Agent Loss to boost feature compactness and inter-class separation.
• The framework achieves outstanding results on LFW (99.83%) and YTF (98.12%), demonstrating robust and effective face recognition performance.

An Overview of SeqFace: Leveraging Sequence Information for Enhanced Face Recognition

The rapid advances in deep convolutional neural networks (CNNs) over recent years have significantly propelled face recognition (FR) capabilities. However, the efficacy of these models is often contingent upon the availability of large, high-quality identity datasets which are both costly and labor-intensive to assemble. In response to these resource constraints, the paper presented introduces a novel framework termed SeqFace, which capitalizes on sequence information gathered from video data to enhance face recognition performance.

Core Contributions and Methodology

SeqFace distinguishes itself by blending conventional identity data with sequence data, the latter being derived from video sources. The framework is innovatively designed to train CNNs by amalgamating these two types of data, thereby transcending the limitations posed by scarce identity data.

1. Dual Training Dataset: The SeqFace framework uniquely leverages both identity datasets and sequence datasets, where the latter consists of unlabelled face sequences automatically extracted from videos. Each sequence represents multiple faces of an identity in varying conditions, thereby holding the potential to mitigate intra-identity variations that single images may not address.
2. Enhancements in Loss Functions:
   • Label Smoothing Regularization (LSR) is adapted to facilitate the chief classification loss's ability to incorporate sequence data, thus extending the utility of classification-based training to unlabeled sequences.
   • The Discriminative Sequence Agent (DSA) Loss is introduced as an auxiliary loss to improve both intra-class compactness and inter-class dispersion of face features, outperforming traditional center loss functions. This loss modulates feature learning by leveraging inter-sequence variability that inherently exists in sequence data.

Empirical Evaluation

Empirical results affirm the utility of SeqFace across benchmark datasets such as Labeled Faces in the Wild (LFW) and YouTube Faces (YTF). The proposed framework achieved a verification accuracy of 99.83% on LFW and 98.12% on YTF using a single ResNet model. These results underscore the robustness of SeqFace in employing both labeled identity data and unlabeled sequence data to yield impressively discriminative face representations.

Implications and Future Directions

The SeqFace framework signifies a pivotal development in face recognition methodologies by enabling the effective exploitation of video data. By utilizing sequences, it allows for the circumvention of some limitations associated with single-image datasets, such as inadequate intra-class variations. This can be particularly beneficial in surveillance applications where obtaining labeled data is often impractical.

Theoretically, SeqFace opens avenues for leveraging unlabeled data in improving classifier performance and paves the way for future investigations into similar applications in related domains, such as person re-identification.

Looking forward, exploration into optimizing the SeqFace framework with newer loss functions and improved CNN architectures could further enhance recognition performance. Additionally, further refinement of sequence data processing and integration techniques could yield more efficient models capable of operating under varied conditions and datasets.

In summary, SeqFace represents a significant step towards more resource-efficient and effective face recognition, offering a compelling alternative for researchers and practitioners facing constraints with traditional identity dataset acquisition.