PETALface: Parameter Efficient Transfer Learning for Low-resolution Face Recognition (2412.07771v1)

Abstract: Pre-training on large-scale datasets and utilizing margin-based loss functions have been highly successful in training models for high-resolution face recognition. However, these models struggle with low-resolution face datasets, in which the faces lack the facial attributes necessary for distinguishing different faces. Full fine-tuning on low-resolution datasets, a naive method for adapting the model, yields inferior performance due to catastrophic forgetting of pre-trained knowledge. Additionally the domain difference between high-resolution (HR) gallery images and low-resolution (LR) probe images in low resolution datasets leads to poor convergence for a single model to adapt to both gallery and probe after fine-tuning. To this end, we propose PETALface, a Parameter-Efficient Transfer Learning approach for low-resolution face recognition. Through PETALface, we attempt to solve both the aforementioned problems. (1) We solve catastrophic forgetting by leveraging the power of parameter efficient fine-tuning(PEFT). (2) We introduce two low-rank adaptation modules to the backbone, with weights adjusted based on the input image quality to account for the difference in quality for the gallery and probe images. To the best of our knowledge, PETALface is the first work leveraging the powers of PEFT for low resolution face recognition. Extensive experiments demonstrate that the proposed method outperforms full fine-tuning on low-resolution datasets while preserving performance on high-resolution and mixed-quality datasets, all while using only 0.48% of the parameters. Code: https://kartik-3004.github.io/PETALface/

• The paper introduces PETALface, a parameter-efficient transfer learning model using LoRA modules to improve low-resolution face recognition and mitigate catastrophic forgetting.
• PETALface leverages two distinct LoRA modules based on image quality to effectively adapt feature extraction for both high and low-resolution inputs.
• Training only 0.48% of parameters, PETALface outperforms full fine-tuning on low-resolution datasets while successfully preserving high-resolution performance, demonstrating efficiency and robustness.

Analysis of PETALface: A Comprehensive Approach to Low-Resolution Face Recognition

The paper introduces PETALface, a parameter-efficient transfer learning model tailored for low-resolution face recognition. The authors have made significant strides in addressing two key issues associated with low-resolution datasets: catastrophic forgetting and domain adaptation between high-resolution gallery and low-resolution probe images. Traditional face recognition methodologies, primarily trained on high-resolution datasets, often falter when dealing with the less distinct features of low-resolution images, a gap PETALface aims to bridge.

At the core of PETALface lies the innovation in fine-tuning pre-trained models with minimal additional parameters. This is achieved through Parameter Efficient Transfer Learning (PEFT), specifically utilizing Low-Rank Adaptation (LoRA) modules. These modules are added to the backbone of the model to address catastrophic forgetting by introducing two distinct low-rank adaptation modules. These modules adjust weights based on image quality, enabling the model to maintain high performance across varying data conditions.

Technical Contributions and Impact

The paper outlines several contributions that are noteworthy:

1. Leveraging LoRA Modules: PETALface employs two LoRA modules in the attention layers of the neural network. These modules serve as proxy encoders for high-resolution and low-resolution images, focusing on extracting contextually meaningful features tailored to the input quality. The adaptation hinges on an image-quality-based approach, enhancing model efficacy in resolving domain discrepancies.
2. Preservation of High-Resolution Performance: While most models face a drop in performance on high-resolution datasets after fine-tuning for low-resolution tasks due to catastrophic forgetting, PETALface effectively circumvents this issue. By training a mere 0.48% of the parameters, the model seamlessly adapts without compromising the performance on high-resolution or mixed-quality datasets, such as LFW and IJB-C.
3. Demonstrated Superiority: Through comprehensive experimentation on various low-resolution datasets like TinyFace and BRIAR, PETALface consistently outperformed full fine-tuning approaches. It achieved superior results in rank-retrieval metrics and verification accuracy, establishing its robustness in real-world surveillance scenarios, which often involve images of varying quality.

Implications and Future Directions

The implications of this research are manifold. PETALface promises a paradigm shift in how face recognition systems might be deployed in resource-constrained environments, where computational efficiency is as critical as accuracy. The low overhead in parameter addition makes it viable for real-time applications where low-resolution images are prevalent, such as surveillance and mobile devices.

Theoretical advancements such as these pave the way for further exploration into multifaceted models that dynamically adjust based on input characteristics. Future developments might involve enhancement of the image quality assessment frameworks to capture more nuanced degradation in images or to incorporate other forms of input data such as temporal sequences in video surveillance.

In conclusion, PETALface is a significant step forward in the domain of face recognition, especially in scenarios characterized by low-resolution inputs. The proposed architecture not only promises high adaptability with minimal computational burden but also maintains the integrity of knowledge learned from high-resolution datasets. As the field progresses, such models that balance efficiency with performance will be central to deploying AI systems in broader and more challenging contexts.