Deeply learned face representations are sparse, selective, and robust (1412.1265v1)

Abstract: This paper designs a high-performance deep convolutional network (DeepID2+) for face recognition. It is learned with the identification-verification supervisory signal. By increasing the dimension of hidden representations and adding supervision to early convolutional layers, DeepID2+ achieves new state-of-the-art on LFW and YouTube Faces benchmarks. Through empirical studies, we have discovered three properties of its deep neural activations critical for the high performance: sparsity, selectiveness and robustness. (1) It is observed that neural activations are moderately sparse. Moderate sparsity maximizes the discriminative power of the deep net as well as the distance between images. It is surprising that DeepID2+ still can achieve high recognition accuracy even after the neural responses are binarized. (2) Its neurons in higher layers are highly selective to identities and identity-related attributes. We can identify different subsets of neurons which are either constantly excited or inhibited when different identities or attributes are present. Although DeepID2+ is not taught to distinguish attributes during training, it has implicitly learned such high-level concepts. (3) It is much more robust to occlusions, although occlusion patterns are not included in the training set.

• The paper demonstrates that DeepID2+ learns face representations with moderate sparsity, achieving 99.47% verification accuracy on LFW.
• It reveals that neurons in deeper layers selectively respond to specific identities and attributes, even without explicit supervision.
• Empirical results show DeepID2+ maintains high accuracy under occlusions, outperforming traditional methods in robustness.

Deeply Learned Face Representations are Sparse, Selective, and Robust

The paper "Deeply learned face representations are sparse, selective, and robust," authored by Yi Sun, Xiaogang Wang, and Xiaoou Tang, proposes a deep convolutional network (DeepID2+) aiming to push the boundaries of face recognition technology. By refining prior models and systematically increasing the architecture's complexity and training volume, the authors succeed in establishing new state-of-the-art performance benchmarks.

Core Contributions

The primary contributions are threefold:

1. Sparse Neural Activations: The paper demonstrates that the neural activations in DeepID2+ are moderately sparse, balancing between activation and inhibition. This balance enriches the discriminative power of the network and translates raw image data into highly distinguishing features.
2. Selectiveness of Neurons: Neurons in higher layers of DeepID2+ exhibit selective responses to specific identities and attributes, even though they were not explicitly trained to detect these attributes. This implicit learning of high-level concepts underscores the network’s efficacy.
3. Robustness to Occlusions: DeepID2+ features significantly higher resilience to occluded images than traditional handcrafted features like high-dimensional LBP. This robustness is intuitively appealing given the high-level global feature representations in higher layers.

Numerical Performance

Empirical evaluations highlight the efficacy of DeepID2+ across multiple benchmarks:

• LFW (Labeled Faces in the Wild): Achieving a verification accuracy of 99.47%, significantly surpassing previous state-of-the-art performances.
• YouTube Faces Dataset: Reaching an accuracy of 93.2%, underscoring its robustness in more dynamic and variable video data.
• Closed and Open-set Identifications on LFW: Performance ranks at 95.0% and 80.7%, respectively, further showcasing superior identification capabilities.

Technical Insights

Sparse Neural Activations: Histograms of neural activations indicate that only around half of the neurons are activated on any given image, and each neuron is activated on roughly half of the images. This moderate sparsity effectively maximizes the network's ability to differentiate between individual identities. Intriguingly, binarizing the neural responses—thus converting the activations into binary codes—retained high recognition accuracy (e.g., 99.12% combined verification accuracy on LFW).

Neuron Selectiveness: The paper demonstrated that select neurons consistently activate or inhibit upon recognizing specific individuals or attributes. This selectiveness was validated through classification tasks, where neurons achieved high accuracy in identifying particular identities or attributes, indicating neurons have learned complex, high-level distinctions intrinsically.

Robustness to Occlusions: Evaluated under conditions of both partial and random block occlusions, DeepID2+ features--especially in deeper layers--displayed a robust performance. Even with significant occlusions, the accuracy remained higher than traditional LBP features. This can be attributed to deeper layers capturing more abstract and global features, which are less susceptible to local variations.

Practical and Theoretical Implications

Practically, the implications are profound. DeepID2+ sets new standards for face recognition systems, paving the way for applications requiring high accuracy and robustness, like security and surveillance. The binarization technique proposed also introduces a novel approach to optimizing both storage and computational efficiency.

Theoretically, these findings provide deeper insights into the nature of deep learning networks. The moderate sparsity, selectiveness, and robustness properties may inspire further research into the intrinsic characteristics that enable high performance in neural networks. Additionally, understanding such properties accelerates the development of methodologies for handling occlusions and learning discriminative features for other computer vision tasks.

Future Directions

Moving forward, several directions appear promising:

1. Enhanced Training Protocols: Fine-tuning the supervisory signals and diversifying training sets further could push model performance longevity.
2. Cross-Dataset Validation: Applying the principles learned from DeepID2+ to other datasets and tasks could validate its generality and robustness.
3. Exploring Lightweight Models: Investigations into lightweight neural networks using binarized activations may offer similar accuracies with lower resource requirements, catering to real-time applications.

Conclusion

In conclusion, the research by Sun et al. showcases the capabilities of DeepID2+ in achieving high performance in face recognition through sparse, selective, and robust learned face representations. The empirical results substantiate these claims and invite further exploration into the effective use of deep learning architectures for complex visual recognition tasks.