DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection (2108.09017v1)

Abstract: Few-shot object detection, which aims at detecting novel objects rapidly from extremely few annotated examples of previously unseen classes, has attracted significant research interest in the community. Most existing approaches employ the Faster R-CNN as basic detection framework, yet, due to the lack of tailored considerations for data-scarce scenario, their performance is often not satisfactory. In this paper, we look closely into the conventional Faster R-CNN and analyze its contradictions from two orthogonal perspectives, namely multi-stage (RPN vs. RCNN) and multi-task (classification vs. localization). To resolve these issues, we propose a simple yet effective architecture, named Decoupled Faster R-CNN (DeFRCN). To be concrete, we extend Faster R-CNN by introducing Gradient Decoupled Layer for multi-stage decoupling and Prototypical Calibration Block for multi-task decoupling. The former is a novel deep layer with redefining the feature-forward operation and gradient-backward operation for decoupling its subsequent layer and preceding layer, and the latter is an offline prototype-based classification model with taking the proposals from detector as input and boosting the original classification scores with additional pairwise scores for calibration. Extensive experiments on multiple benchmarks show our framework is remarkably superior to other existing approaches and establishes a new state-of-the-art in few-shot literature.

• The paper introduces a decoupled architecture that leverages a Gradient Decoupled Layer and a Prototypical Calibration Block for improved few-shot object detection.
• The Gradient Decoupled Layer mitigates overfitting by isolating feature learning stages, enhancing transferability from base to novel domains.
• The Prototypical Calibration Block refines classification accuracy through prototype-based score fusion, resulting in strong performance across VOC and COCO datasets.

Decoupled Faster R-CNN for Few-Shot Object Detection

The paper "DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection" introduces an architectural enhancement for few-shot object detection (FSOD), examining challenges associated with utilizing traditional detection models in data-scarce environments. Despite the established performance of deep neural networks in visual tasks such as object detection, these models struggle when constrained by limited annotated data. The paper identifies key limitations associated with using Faster R-CNN as the base architecture for FSOD and proposes modifications to enhance performance under few-shot settings.

Core Contributions

The work introduces the Decoupled Faster R-CNN (DeFRCN), highlighting two critical components: the Gradient Decoupled Layer (GDL) and the Prototypical Calibration Block (PCB). Each addresses specific issues inherent in the multi-stage, multi-task structure of Faster R-CNN.

1. Gradient Decoupled Layer (GDL):
   • Purpose: Enhance decoupling between backbone, RPN, and RCNN stages to mitigate overfitting during transfer from base to novel domains.
   • Methodology: Introduces learnable transformations for feature maps and scales gradients using a decoupling coefficient, allowing for nuanced control over information exchange between detection stages.
2. Prototypical Calibration Block (PCB):
   • Purpose: Improve classification accuracy by addressing task conflicts between localization and classification branches.
   • Methodology: Utilizes prototype-based score fusion during inference, leveraging pre-trained models to refine softmax outputs and maintain classification robustness without additional training.

Experimental Results and Observations

Extensive experiments validate DeFRCN’s efficacy across multiple benchmarks including PASCAL VOC and COCO datasets. Notable observations include:

• State-of-the-Art Performance: The DeFRCN consistently achieves higher average precision (AP) across diverse dataset splits and shot numbers. It significantly outperforms other methods particularly in novel object detection scenarios, underscoring the benefits of both the GDL and PCB components.
• Cross-Domain Adaptation: The architecture demonstrates strong generalization by maintaining performance across domains, exemplified through experiments from COCO base to VOC novel classes.
• Conventional Detection Improvements: Besides FSOD, GDL shows potential in enhancing conventional object detection tasks by addressing inherent architectural conflicts in Faster R-CNN.

Implications and Future Directions

The paper’s contributions hold significant implications for adaptive learning approaches within computer vision. By decoupling task-specific conflicts and leveraging prototype-based inference adjustments, DeFRCN represents an advancement in efficient learning with few data points. These innovations pave the way for further exploration into fine-tuning strategies within transfer learning frameworks. Future research might focus on refining decoupling coefficients and examining PCB’s applicability to other challenging data-scarce domains.

Given the robust state-of-the-art results achieved by DeFRCN, subsequent studies should consider its applicability in broader contexts, potentially extending methodologies to related fields such as semantic segmentation and other vision applications where data scarcity prevails. Additionally, exploration into dynamic adaptation of decoupling constants during training phases could yield insights into optimizing cross-domain performance further.