DeepID3: Face Recognition with Very Deep Neural Networks (1502.00873v1)

Abstract: The state-of-the-art of face recognition has been significantly advanced by the emergence of deep learning. Very deep neural networks recently achieved great success on general object recognition because of their superb learning capacity. This motivates us to investigate their effectiveness on face recognition. This paper proposes two very deep neural network architectures, referred to as DeepID3, for face recognition. These two architectures are rebuilt from stacked convolution and inception layers proposed in VGG net and GoogLeNet to make them suitable to face recognition. Joint face identification-verification supervisory signals are added to both intermediate and final feature extraction layers during training. An ensemble of the proposed two architectures achieves 99.53% LFW face verification accuracy and 96.0% LFW rank-1 face identification accuracy, respectively. A further discussion of LFW face verification result is given in the end.

• The paper introduces two very deep network architectures that improve face recognition performance with a 99.53% verification accuracy.
• It employs continuous convolutional and inception layers with intermediate supervisory signals to capture multi-scale features and reduce intra-personal variations.
• Experimental results on LFW reveal marginal gains over previous models, highlighting challenges with dataset quality and labeling.

Overview of "DeepID3: Face Recognition with Very Deep Neural Networks"

The paper "DeepID3: Face Recognition with Very Deep Neural Networks" by Yi Sun et al. presents advancements in face recognition through the introduction of two significantly deeper neural network architectures, referred to as DeepID3. Building on previous works like DeepID2+, this research explores the potential of applying deeper network structures, inspired by architectures like VGG and GoogLeNet, to the domain of face recognition.

Background and Motivation

The context for this research is rooted in the significant progress that deep learning has brought to the field of face recognition. Initial efforts in this domain focused on using face verification as a supervisory signal to reduce intra-personal variations. This evolved into techniques involving large-scale face identity classification, as demonstrated by DeepID and DeepFace, which brought deep learning's performance on par with human levels on certain benchmarks. However, earlier architectures like DeepID2 and DeepID2+ lacked the depth of more recent models used in object recognition, such as VGG and GoogLeNet, prompting the investigation into deeper networks for face recognition.

Architectures of DeepID3

DeepID3 net1

DeepID3 net1 incorporates continuous convolutional layers, similar to VGG net, arranged in pairs before each pooling layer. Additional supervisory signals are integrated into several intermediate fully connected layers, aiding in learning mid-level features and easing the optimization process of deeper networks. The top convolutional layers are replaced with locally connected layers with unshared parameters, which enhance the expressiveness of the features while maintaining a compact feature dimension.

DeepID3 net2

DeepID3 net2 combines convolutional layers and inception layers from GoogLeNet. It starts with convolutional layers akin to DeepID3 net1, but later stages use multiple continuous inception layers before pooling layers. This design allows capturing multi-scale features more efficiently. Similar to DeepID3 net1, supervisory signals are added to fully connected layers following each pooling layer.

Experimental Results

The DeepID3 networks are evaluated on the LFW dataset for both face verification and face identification tasks. The results showcase:

• Face Verification: Achieving an accuracy of 99.53%. This is a marginal improvement over the previous state-of-the-art DeepID2+ which had an accuracy of 99.47%.
• Face Identification: Achieving a rank-1 accuracy of 96.0% for closed-set identification and a rank-1 DIR at 1% FAR of 81.4% for open-set identification.

Discussion on Results

The marginal improvement in face verification accuracy suggests that very deep architectures like VGG and GoogLeNet might not offer substantial benefits over shallower but well-designed networks like DeepID2+ when trained on the relatively smaller datasets typical in face recognition research. The authors also identify that certain mislabeled pairs in the LFW dataset affect the perceived gains of DeepID3 over its predecessors. Additionally, a notable portion of misclassifications across DeepID series algorithms persists due to challenging image pairs, indicating potential areas for further refinement.

Practical and Theoretical Implications

From a practical perspective, the research reinforces the idea that deeper networks can be effective for face recognition, but the advantages might be contingent on the scale and quality of the training data. The marginal improvement in accuracy also highlights the importance of dataset quality and proper labeling.

Theoretically, the paper opens up discussions on the utility of very deep networks in specialized domains like face recognition compared to more general object recognition tasks. Future work might involve exploring these architectures on significantly larger datasets to assess their true potential and conducting more in-depth analysis on the types of errors that persist in very deep networks.

Conclusion

The introduction of DeepID3 networks marks an important contribution to the ongoing development of face recognition technologies. While the gains over DeepID2+ are subtle, this work provides crucial insights into the scalability and optimization of very deep neural networks in face recognition. Future research could further elucidate the potential benefits of deeper architectures as datasets continue to expand in size and complexity.