Semi-Supervised Sparse Representation Based Classification for Face Recognition with Insufficient Labeled Samples (1609.03279v2)

Abstract: This paper addresses the problem of face recognition when there is only few, or even only a single, labeled examples of the face that we wish to recognize. Moreover, these examples are typically corrupted by nuisance variables, both linear (i.e., additive nuisance variables such as bad lighting, wearing of glasses) and non-linear (i.e., non-additive pixel-wise nuisance variables such as expression changes). The small number of labeled examples means that it is hard to remove these nuisance variables between the training and testing faces to obtain good recognition performance. To address the problem we propose a method called Semi-Supervised Sparse Representation based Classification (S$^3$RC). This is based on recent work on sparsity where faces are represented in terms of two dictionaries: a gallery dictionary consisting of one or more examples of each person, and a variation dictionary representing linear nuisance variables (e.g., different lighting conditions, different glasses). The main idea is that (i) we use the variation dictionary to characterize the linear nuisance variables via the sparsity framework, then (ii) prototype face images are estimated as a gallery dictionary via a Gaussian Mixture Model (GMM), with mixed labeled and unlabeled samples in a semi-supervised manner, to deal with the non-linear nuisance variations between labeled and unlabeled samples. We have done experiments with insufficient labeled samples, even when there is only a single labeled sample per person. Our results on the AR, Multi-PIE, CAS-PEAL, and LFW databases demonstrate that the proposed method is able to deliver significantly improved performance over existing methods.

Overview of Semi-Supervised Sparse Representation Based Classification for Face Recognition

The paper presents a novel methodology for face recognition titled Semi-Supervised Sparse Representation based Classification (S$^3$RC), aiming to address the challenge of recognizing faces with a limited number of labeled samples. Utilizing sparse representation methods, the authors propose a framework that integrates semi-supervised learning concepts to enhance recognition performance even when the labeled data is sparsely populated and potentially corrupted by nuisance variables.

Key Concepts and Methodology

In tackling the problem of face recognition with insufficient labeled samples, S$^3$RC builds upon the Sparse Representation based Classification (SRC) framework. SRC has been established as a robust method for face recognition, particularly effective in conditions with occlusions or corruptions. However, its performance is traditionally dependent on a sufficient number of labeled samples.

1. Sparse Representation Implementation: S$^3$RC extends the SRC method by incorporating two dictionaries:
   • Gallery Dictionary: Formulated from one or more labeled examples per individual.
   • Variation Dictionary: Represents linear variations (i.e., lighting, glasses) that can be shared across different individuals.
2. Semi-Supervised Learning:
   • A Gaussian Mixture Model (GMM) is employed to tackle non-linear nuisance variations between labeled and unlabeled samples, facilitating a semi-supervised learning approach that blends labeled and unlabeled data for more accurate gallery dictionary estimation.
   • The variation dictionary aligns linear nuisance variables, while the GMM reduces the impact of non-linear variations through iterative adjustments.
3. Optimization Procedures: The paper details an iterative Expectation-Maximization (EM) algorithm to refine model parameters and enhance face recognition accuracy. This optimization considers the latent variables associated with unlabeled samples, adjusting for variations not captured in limited labeled samples.

Evaluations and Results

The methodology was tested across multiple databases, showcasing a compelling improvement over existing approaches, particularly under conditions of sparse labeled data. In large-scale databases, such as the AR, Multi-PIE, CAS-PEAL, and LFW, S$^3$RC reliably outperformed traditional recognition techniques, including SRC, ESRC, and other dictionary learning methods.

• Numerical results highlighted S$^3$RC's efficacy with remarkable improvements in recognition rates across varied experimental setups—both transductive and inductive—demonstrating robustness to both controlled and uncontrolled images as gallery samples.
• Tests with deep convolution neural network (DCNN) features further verify that the integration of S$^3$RC can enhance performance with state-of-the-art facial recognition models.

Implications and Future Directions

The development of S$^3$RC has significant implications for applications requiring high accuracy in low-data environments, such as security and surveillance scenarios. By leveraging both labeled and unlabeled data within a refined sparse representation framework, this method presents a promising avenue to enhance the reliability and adaptability of face recognition systems under challenging acquisition circumstances.

Future research could explore the application of alternative semi-supervised methods or integrate more complex variation modeling techniques to further enhance the framework's robustness. Additionally, expanding these concepts to other biometric modalities and evaluating their effectiveness across broader conditions could provide valuable insights and adaptations suitable for a wider range of applications.

In conclusion, the proposed S$^3$RC approach offers a substantial advancement in the face recognition domain, especially in contexts where labeled data is minimal and diverse. Its combination of sparse representation, semi-supervised learning, and robust optimization paves the way for improved performance and expanded applicability.