Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network (1908.06290v1)

Abstract: Deep Convolutional Neural Networks (CNNs) have been pushing the frontier of the face recognition research in the past years. However, existing general CNN face models generalize poorly to the scenario of occlusions on variable facial areas. Inspired by the fact that a human visual system explicitly ignores occlusions and only focuses on non-occluded facial areas, we propose a mask learning strategy to find and discard the corrupted feature elements for face recognition. A mask dictionary is firstly established by exploiting the differences between the top convoluted features of occluded and occlusion-free face pairs using an innovatively designed Pairwise Differential Siamese Network (PDSN). Each item of this dictionary captures the correspondence between occluded facial areas and corrupted feature elements, which is named Feature Discarding Mask (FDM). When dealing with a face image with random partial occlusions, we generate its FDM by combining relevant dictionary items and then multiply it with the original features to eliminate those corrupted feature elements. Comprehensive experiments on both synthesized and realistic occluded face datasets show that the proposed approach significantly outperforms the state-of-the-arts.

• The paper introduces an innovative Pairwise Differential Siamese Network that learns to generate a Feature Discarding Mask for eliminating corrupted features due to occlusion.
• It employs a mask dictionary to effectively handle diverse partial occlusions, significantly improving accuracy on AR, MegaFace, and LFW datasets.
• Experimental results demonstrate enhanced rank-1 identification rates and robustness in both real-world and synthetic large-scale face recognition scenarios.

Occlusion Robust Face Recognition Using Mask Learning and Pairwise Differential Siamese Network

This paper addresses the challenge of occlusion in face recognition using a strategy based on mask learning with Pairwise Differential Siamese Network (PDSN). The focus is on creating an approach that aligns with the human visual system's capability to ignore occlusions, enhancing the ability of Deep Convolutional Neural Networks (CNNs) to accurately recognize faces even when parts of the face are occluded.

Key Contributions and Methodology

The authors introduce a novel approach that extends the capabilities of deep CNNs for face recognition by employing a feature discarding mechanism. This strategy enables the model to identify and discard corrupted feature elements caused by occlusions. The core of the proposed method revolves around the innovative Pairwise Differential Siamese Network (PDSN). This network is designed to functionally associate occluded facial areas with their corresponding corrupted feature elements, thereby establishing a Feature Discarding Mask (FDM) from a learned mask dictionary.

1. Pairwise Differential Siamese Network (PDSN): The PDSN leverages differential learning by comparing occluded and occlusion-free facial image pairs. The network utilizes differential signals from these pairs to guide the mask generator in identifying features affected by occlusions. This helps in constructing a mask that can eliminate the corrupted elements without needing occlusion labels during live inference.
2. Feature Discarding Mask (FDM): FDMs are generated using the PDSN and represent binary masks that deactivate compromised features in face recognition tasks. These masks are derived from strategically binarizing the learned masks from the PDSN's outputs, enabling the system to handle unanticipated occlusions during real-world application.
3. Mask Dictionary: A mask dictionary is populated by composing FDMs for predefined facial blocks. This dictionary enables the model to tackle random partial occlusions effectively by composing the required mask for any given occlusion scenario, applying a logical combination of relevant pre-learned masks.

Experimental Results

The methodology was evaluated comprehensively using both synthetic and realistic occluded face datasets. The experimental outcomes indicate that the proposed method significantly surpasses state-of-the-art systems in occlusion-robust face recognition, providing evidence for its efficacy:

• AR Dataset: The methodology demonstrated excellent accuracy in handling real-world occlusions such as sunglasses and scarves, achieving higher rank-1 identification accuracy in comparison to baseline models.
• MegaFace Challenge: The proposed approach also maintained robust performance on the MegaFace challenge, proving effective in large-scale recognition scenarios even with synthetic occlusions.
• LFW Benchmark: The process maintained competitive accuracy on the Labeled Faces in the Wild (LFW) dataset, indicating that the proposed approach does not degrade the recognition performance on fully-visible face images.

Implications and Future Directions

This research presents an important advancement in occlusion-robust face recognition by validating the potential of mask learning integrated with PDSN structures. The method not only addresses the challenges posed by partial occlusions but also sets the stage for future developments in real-world applications. Practically, this could enhance security systems and consumer devices in scenarios where occlusion is unavoidable.

Looking forward, further exploration could involve refining the granularity of feature masks and expanding the mask dictionary to account for more complex and variable real-world scenarios. Additionally, integrating traditional occlusion handling techniques with this approach might yield even greater resilience and accuracy in heterogeneous settings.

In conclusion, the paper's proposed method offers a meaningful contribution to the domain of face recognition technology, particularly in environments where occlusions are prevalent. This approach exemplifies how intelligent handling of corrupted data can be achieved, setting a benchmark for subsequent research in deep learning-based face recognition frameworks.