Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition (2109.07270v6)

Abstract: We present a novel facial expression recognition network, called Distract your Attention Network (DAN). Our method is based on two key observations. Firstly, multiple classes share inherently similar underlying facial appearance, and their differences could be subtle. Secondly, facial expressions exhibit themselves through multiple facial regions simultaneously, and the recognition requires a holistic approach by encoding high-order interactions among local features. To address these issues, we propose our DAN with three key components: Feature Clustering Network (FCN), Multi-head cross Attention Network (MAN), and Attention Fusion Network (AFN). The FCN extracts robust features by adopting a large-margin learning objective to maximize class separability. In addition, the MAN instantiates a number of attention heads to simultaneously attend to multiple facial areas and build attention maps on these regions. Further, the AFN distracts these attentions to multiple locations before fusing the attention maps to a comprehensive one. Extensive experiments on three public datasets (including AffectNet, RAF-DB, and SFEW 2.0) verified that the proposed method consistently achieves state-of-the-art facial expression recognition performance. Code will be made available at https://github.com/yaoing/DAN.

Multi-Head Cross Attention Network for Facial Expression Recognition

The research paper titled "Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition" introduces a novel approach to facial expression recognition (FER) with a focus on enhancing network performance through the strategic use of attentional architectures. Addressing the challenge of recognizing subtle variations in facial expressions, the authors propose a unique network architecture called the Distract Your Attention Network (DAN). Their methodology is underpinned by insights drawn from biological visual perception: the inherently subtle differences between classes within facial expression datasets and the need for holistic approaches to capture interactions across multiple facial regions.

Key Components of DAN

The DAN framework is composed of three main components: the Feature Clustering Network (FCN), the Multi-head Attention Network (MAN), and the Attention Fusion Network (AFN). Each component contributes distinctively to the robust extraction and integration of features necessary for accurate facial expression classification.

1. Feature Clustering Network (FCN): FCN employs a large-margin learning objective to enhance class separability. By leveraging an affinity loss, the network maximizes inter-class margins while minimizing intra-class variations. This loss function modifies the conventional center loss by emphasizing the expansion of distance between class centers, which reportedly results in clearer feature clustering. As evidenced by t-SNE visualizations, this approach successfully achieves precise feature clustering leading to improved classification accuracy.
2. Multi-head Attention Network (MAN): Inspired by biological visual perception, MAN introduces multiple attention heads to concurrently focus on distinct facial regions. By adopting both spatial and channel attention strategies, each head captures high-order interactions locally across these different regions. This architecture resolves the limitation of single-head attention mechanisms which might overlook nuances by failing to attend to multiple critical areas simultaneously.
3. Attention Fusion Network (AFN): Finally, AFN employs a partition loss to guide the attention heads towards non-overlapping facial regions, enhancing the model's ability to extract diverse informative features. Attention feature vectors are scaled via log-softmax and subsequently fused for classification, maintaining coherence and reducing redundancy.

Experimental Validation and Implications

Extensive experiments performed on several benchmark datasets, including AffectNet, RAF-DB, and SFEW 2.0, illustrate that DAN surpasses existing methods in achieving state-of-the-art recognition performance. For instance, notable accuracy improvements were observed on both AffectNet configurations (with accuracies of 62.09% on AffectNet-8 and 65.69% on AffectNet-7), and a leading accuracy of 89.70% on the RAF-DB dataset. The method underscores a consistent ability to manage complexities inherent in diverse datasets, while also indicating room for further enhancements, especially with respect to smaller datasets like SFEW 2.0.

Theoretical and Practical Implications

The theoretical implication of this work illustrates a promising direction for FER, advocating for multi-head attention architectures that mirror complex human visual processing traits. Practically, the method holds potential for applications in areas requiring nuanced emotion detection, such as human-computer interaction, sentiment analysis, and emotional diagnostics. The introduction of simplified affinity and partition loss formulations also facilitates more efficient training without exacerbated computational load.

Future Directions

Moving forward, this research could inspire additional exploration into refining attention mechanisms, especially across more challenging, subtle emotion classes as identified in confusion matrices. Future developments might also focus on optimizing computational efficiency further and adapting similar methodologies to related fields in vision processing, extending beyond facial expression recognition to encompass broader aspects of context-aware recognition systems.

In conclusion, the paper presents a substantive advancement in leveraging attention mechanisms within neural networks to achieve enhanced facial expression recognition, paving the way for further innovation in the field of computer vision and emotional intelligence systems.