Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition (1702.04710v2)

Abstract: This paper explores multi-task learning (MTL) for face recognition. We answer the questions of how and why MTL can improve the face recognition performance. First, we propose a multi-task Convolutional Neural Network (CNN) for face recognition where identity classification is the main task and pose, illumination, and expression estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss weight to each side task, which is a crucial problem in MTL. Third, we propose a pose-directed multi-task CNN by grouping different poses to learn pose-specific identity features, simultaneously across all poses. Last but not least, we propose an energy-based weight analysis method to explore how CNN-based MTL works. We observe that the side tasks serve as regularizations to disentangle the variations from the learnt identity features. Extensive experiments on the entire Multi-PIE dataset demonstrate the effectiveness of the proposed approach. To the best of our knowledge, this is the first work using all data in Multi-PIE for face recognition. Our approach is also applicable to in-the-wild datasets for pose-invariant face recognition and achieves comparable or better performance than state of the art on LFW, CFP, and IJB-A datasets.

• The paper introduces a multi-task CNN that integrates auxiliary tasks for pose, illumination, and expression estimation to robustly improve identity recognition.
• The proposed dynamic-weighting scheme optimally allocates loss weights for each auxiliary task, effectively regularizing the CNN to separate identity features from variations.
• Empirical results on the Multi-PIE and in-the-wild datasets demonstrate competitive performance, highlighting advancements over state-of-the-art approaches.

Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition

In this comprehensive paper, the authors investigate the application of multi-task learning (MTL) to the domain of pose-invariant face recognition using convolutional neural networks (CNNs). The paper articulates a clear motivation and systematic method to integrate MTL into CNN frameworks, addressing the posed challenges of varying conditions in pose, illumination, and expression through auxiliary tasks complementing the primary task of identity recognition.

The authors introduce a Multi-Task Convolutional Neural Network (m-CNN), where identity classification is augmented with pose, illumination, and expression estimations as side tasks. A novel dynamic-weighting scheme is proposed to automatically assign loss weights for these auxiliary tasks during training, aiming to leverage their potential for enhancing the main task performance. Significantly, this approach identifies the side tasks as a form of regularization, which assists in disentangling variations from the learned identity features without explicitly separating task-specific weights through exhaustive manual tuning.

The deployment of the proposed strategies on the extensive Multi-PIE dataset, which includes the largest set of pose, illumination, and expression variations to date, demonstrates the efficacy of this approach. The validation results indicate not only competent handling of various face variations but also competitive or superior recognition performance compared to state-of-the-art methodologies when applied to in-the-wild datasets like LFW, CFP, and IJB-A.

Key empirical insights from this research include:

• Dynamic-Weight Assignment: This method efficiently allocated side task weights to optimize the contribution of each auxiliary task to the main task, highlighting tasks most beneficial to identity recognition under various conditions.
• Pose-Directed Multi-Task CNN (p-CNN): A further advancement, p-CNN captures pose-specific features alongside generic identity features, and employs a stochastic routing scheme during testing to fuse these features robustly against pose estimation errors.
• Energy-Based Weight Analysis: This analytical approach clarifies the role of side tasks as regularizations in MTL frameworks, showing refined task-induced disentanglement of identity from variation-influencing dimensions.

The m-CNN and p-CNN designs promote significant implications for theoretically advancing the robustness of CNNs under multi-task settings, illustrating the potential of dynamic auxiliary task integration as a foundational component in the design of next-gen face recognition models.

From a practical standpoint, the paper paves the way for improved machine perception systems deployed in more generalized, uncontrolled environments, enhancing the recognition accuracy across diverse and variable datasets. This work also suggests potential iterative advancements that could incorporate more nuanced or additional auxiliary tasks, refined loss weight dynamics, and further integration of domain-related metadata to future neural architectures.

Research possibilities extend to exploring the potential of MTL to address other biometric and identification tasks that undergo frequent real-life variations, thus contributing to a holistic enhancement in the field of artificial intelligence.