Analysing Affective Behavior in the First ABAW 2020 Competition (2001.11409v2)

Abstract: The Affective Behavior Analysis in-the-wild (ABAW) 2020 Competition is the first Competition aiming at automatic analysis of the three main behavior tasks of valence-arousal estimation, basic expression recognition and action unit detection. It is split into three Challenges, each one addressing a respective behavior task. For the Challenges, we provide a common benchmark database, Aff-Wild2, which is a large scale in-the-wild database and the first one annotated for all these three tasks. In this paper, we describe this Competition, to be held in conjunction with the IEEE Conference on Face and Gesture Recognition, May 2020, in Buenos Aires, Argentina. We present the three Challenges, with the utilized Competition corpora. We outline the evaluation metrics, present both the baseline system and the top-3 performing teams' methodologies per Challenge and finally present their obtained results. More information regarding the Competition, the leaderboard of each Challenge and details for accessing the utilized database, are provided in the Competition site: http://ibug.doc.ic.ac.uk/resources/fg-2020-competition-affective-behavior-analysis.

• The paper establishes a new benchmark by integrating valence-arousal estimation, basic expression recognition, and AU detection using the Aff-Wild2 dataset.
• The paper details baseline methods and advanced multi-task architectures, including PatchGAN and MobileNetV2, to enhance affective analysis performance.
• The paper demonstrates that ensemble and multi-modal learning techniques can significantly improve the precision of human emotion and behavior recognition.

Analyzing Affective Behavior in the ABAW 2020 Competition

The paper "Analysing Affective Behavior in the First ABAW 2020 Competition" explores the structure, methodology, and outcomes of the Affective Behavior Analysis in-the-wild (ABAW) 2020 Competition. This competition is distinct for its aim to address the automatic analysis of affective behavior through three main tasks: valence-arousal estimation, basic expression recognition, and Action Unit (AU) detection. The competition employs the Aff-Wild2 database, a broad and comprehensive benchmark database annotated for all three tasks, serving as the foundation for challenging real-world scenarios that underpin human-computer interactions.

Context and Motivation

The ABAW 2020 Competition is situated within the broader context of developing human-centered AI technologies that are capable of recognizing and responding to human emotions and behaviors in the wild. This pursuit aligns with the objectives of the iterative societal revolutions targeting the convergence of physical and cyber spaces to enhance human well-being through technology.

Database and Challenges

Aff-Wild2, the backbone of the competition, is an extensive database comprising millions of frames from internet-sourced videos, annotated with valence-arousal scales, basic expressions, and AU occurrences. Notably, Aff-Wild2 is the first dataset of its scale to concurrently support these diverse annotations, reinforcing its critical role in training and evaluating models on affective analysis through three discrete challenges:

1. Valence-Arousal Estimation Challenge: Evaluates the continuous estimation of affect dimensions using the Concordance Correlation Coefficient (CCC) as the evaluatory metric.
2. Basic Expression Classification Challenge: Tasks models with recognizing categorical emotional expressions, merging the $F_1$ score and total accuracy for assessment.
3. AU Detection Challenge: Focuses on identifying the occurrence of specific facial action units, employing metrics that account for both accuracy and precision.

Baseline Methods and Participating Teams

The paper describes baseline models for each challenge, showcasing PatchGAN for valence-arousal estimation and MobileNetV2 for expression classification and AU detection. Performance evaluations reveal that the top-performing teams surpassed these baselines by integrating advanced multi-task and multi-modal learning approaches.

• NISL2020 and TNT teams emerged as top contenders, both excelling in different challenges, using strategies that leverage ensemble learning, multi-task training, and integrating audio-visual data.
• Methodologies of note include the use of convolutional and recurrent architectures, attention mechanisms, and innovative pre-processing techniques, contributing to enhanced performance in expression recognition and AU detection.

Results and Implications

The competition results underscore the efficacy and potential of model architectures that effectively combine spatial, temporal, and auditory information. Achievements by participating teams indicate strides in multi-task learning, suggesting a significant step towards machines capable of nuanced human emotion and behavior recognition across varied stimuli and environments.

Future Directions

The theoretical and practical implications derived from the ABAW 2020 Competition highlight future research opportunities in enhancing affective analysis systems. Building upon the comprehensive data set and benchmarking results laid out, there is scope for refining and expanding learning models to accommodate a wider array of affective states, potentially incorporating contextual and cultural variations in affective behavior interpretation.

The ABAW 2020 Competition thus establishes a foundational pillar for progress in the interdisciplinary field of affective computing, positioning it strategically within the advancements of AI towards empathetic technology development.