- The paper systematically reviews deep learning techniques for elderly facial expression recognition, detailing CNNs, RNNs, and emerging Transformer architectures.
- The paper identifies challenges including inadequate age-annotated datasets, data imbalance, and limited real-world deployment despite promising advances.
- The paper recommends integrating temporal and multimodal data with explainable AI to enhance trustworthiness and privacy in elderly care applications.
This paper, "Deep Learning-Based Facial Expression Recognition for the Elderly: A Systematic Review" (2502.02618), provides a systematic review of deep learning applications for facial expression recognition (FER) specifically tailored for the elderly population. The review highlights the growing need for such technologies due to the rapid global aging trend and its implications for healthcare and well-being.
The authors conducted a systematic literature review following established guidelines, analyzing 31 studies published between 2015 and September 2024 from five databases (SCOPUS, Web of Science, ACM Digital Library, IEEE Xplore, PubMed). The review addresses two primary research questions: (1) What deep learning techniques are applied for FER in elderly populations, and how are they used? and (2) How can these techniques be effectively deployed in real-world environments?
Key Findings on Deep Learning Techniques (RQ1):
- Architectures: Convolutional Neural Networks (CNNs) like VGG, Inception, and ResNet are the most prevalent architectures, often used for feature extraction. Recurrent Neural Networks (RNNs), such as LSTM and GRU, are employed for video-based FER to handle temporal data. Transformer architectures are an emerging trend, used for both visual and multimodal analysis. Lightweight models like MobileNet, EfficientNet, and mini-Xception are favored for deployment on resource-constrained devices. Commercial tools like FaceReader and OpenFace are also sometimes integrated into workflows.
- Datasets: While many studies use general FER datasets like FER-2013, CK+, and AffectNet, these often lack sufficient representation and age annotations for the elderly. Datasets like FACES and LifeSpan include age diversity, and specialized datasets targeting the elderly, such as ElderReact, exist but are less common. Data imbalance across expression classes is a significant issue in many datasets (e.g., AffectNet, LifeSpan), often addressed inadequately through undersampling or removing minority classes, which can reduce data diversity.
- Data Characteristics: Static images are used more frequently than videos, despite the potential benefits of temporal information for FER. Studies using videos typically employ RNNs or Transformers to process sequential features. Multimodal approaches, combining visual data with audio or text, have shown improved performance, although they are less common.
- Facial Landmarks and Action Units (AUs): These features are utilized as inputs for models or for other purposes like system integration. Tools like OpenFace and FaceReader are used to extract them. While landmark/AU-based features can be used for FER, deep learning features often achieve better performance. They can also facilitate privacy preservation by allowing the anonymization of raw visual data.
- Other Tasks: FER is frequently combined with other tasks, including face detection (common preprocessing step using methods like Viola-Jones), age estimation (often used for evaluation or to adapt models), cognitive impairment/apathy detection (using expression patterns), facial recognition, fall detection, and dialogue generation (in social robots). Integrating these tasks can create more comprehensive and useful systems for elderly care.
Key Findings on Real-World Deployment (RQ2):
- Deployment Status: While the potential applications of FER for the elderly are numerous (e.g., assisted living, healthcare), actual deployment of the proposed solutions in real-world environments is limited in the reviewed literature. A few studies demonstrate integration into social robots or adaptive care environments.
- Aging Biases: Age significantly impacts FER performance. Studies have shown that models trained on general datasets often perform poorly on elderly faces. Some approaches attempt to mitigate this by incorporating age information during training, using multi-task learning for age and expression, or specializing models for age groups. However, the scarcity of diverse, age-annotated datasets remains a fundamental challenge.
- Privacy: Privacy is a critical concern when dealing with facial data, especially for vulnerable populations like the elderly in healthcare settings. Few studies explicitly address privacy. Strategies identified include using depth images instead of RGB, employing privacy-preserving transformations (like GANs), using landmarks/AUs to anonymize data, and processing data locally on devices.
- Economic Cost: Economic considerations are mentioned in several studies, often addressed by employing lightweight deep learning models and utilizing affordable hardware platforms (e.g., Raspberry Pi). Client-server architectures can also offload computational burden from edge devices.
- Explainable AI (XAI): The adoption of XAI techniques in FER for the elderly is notably low, despite its importance for building trust and transparency in healthcare applications. A few studies use model-agnostic methods (LIME, RISE, SHAP) or model-specific methods (Grad-CAM) to visualize important image regions for predictions. Analyzing feature clustering also provides some insight into model behavior but is not a full XAI solution.
Discussion and Recommendations:
The review underscores that while significant advancements have been made in DL-based FER, translating this into effective and trustworthy systems for the elderly requires addressing several practical challenges. The lack of suitable datasets remains a primary barrier, necessitating the creation of large, diverse, age-inclusive datasets with accurate annotations. Addressing class imbalance effectively, perhaps using oversampling or weighted loss functions, is also crucial. Incorporating temporal and multimodal data can improve robustness.
For real-world deployment, more research is needed on system integration, user acceptance, and ethical considerations. Privacy must be a first-class concern, with robust techniques implemented. Economic feasibility needs continued attention through lightweight models and efficient architectures. Crucially, the integration of XAI techniques is essential to make FER systems for the elderly understandable, trustworthy, and accountable, particularly in high-stakes healthcare applications. Future research should prioritize interdisciplinary collaboration to develop systems that are not only technically proficient but also user-centered, ethical, and truly beneficial for the aging population.