Localize to Classify and Classify to Localize: Mutual Guidance in Object Detection (2009.14085v1)

Abstract: Most deep learning object detectors are based on the anchor mechanism and resort to the Intersection over Union (IoU) between predefined anchor boxes and ground truth boxes to evaluate the matching quality between anchors and objects. In this paper, we question this use of IoU and propose a new anchor matching criterion guided, during the training phase, by the optimization of both the localization and the classification tasks: the predictions related to one task are used to dynamically assign sample anchors and improve the model on the other task, and vice versa. Despite the simplicity of the proposed method, our experiments with different state-of-the-art deep learning architectures on PASCAL VOC and MS COCO datasets demonstrate the effectiveness and generality of our Mutual Guidance strategy.

Localize to Classify and Classify to Localize: Mutual Guidance in Object Detection

In the field of computer vision, object detection has emerged as a pivotal task, with numerous methodologies leveraging deep learning techniques for improved accuracy and efficiency. The paper "Localize to Classify and Classify to Localize: Mutual Guidance in Object Detection" addresses the shortcomings of conventional anchor-based object detection frameworks, particularly questioning the efficacy of using Intersection over Union (IoU) as an anchor matching criterion. This research introduces a novel strategy, termed Mutual Guidance, which emphasizes dynamic, task-aware anchor assignment during the training process.

Overview

Traditional state-of-the-art object detection models typically utilize a static IoU-based approach to assign anchors during model training. Anchors that exhibit a higher IoU with target bounding boxes are assigned as positive samples for further training in both localization and classification tasks. However, this approach assumes the IoU metric to be inherently optimal and fails to consider the semantic content and context associated with anchors.

The paper challenges this assumption and proposes Mutual Guidance, a strategy where the dynamism in anchor assignment is orchestrated by the dual tasks of localization and classification. Predominantly, the crux of Mutual Guidance lies in:

• Localize to Classify: Leveraging the precision of bounding box regression to refine classification labels. Anchors exhibiting higher localization accuracy are dynamically selected as positive samples for classification.
• Classify to Localize: Utilizing classification scores to influence the choice of anchors for regression tasks. Here, the classification confidence modulates the IoU, termed as $IoU_{amplified}$, to introduce a content/context-sensitive thresholding for anchor selection in the localization task.

Experimental Findings

The efficacy of the Mutual Guidance approach was rigorously tested under different architectures, including FSSD, RetinaNet, and RFBNet with various backbone networks such as ResNet-18 and VGG-16. Tested across benchmark datasets like PASCAL VOC and MS COCO, the Mutual Guidance method noticeably enhanced performance metrics. Notably, this strategy consistently elevated the Average Precision (AP), with significant gains in strict evaluation settings such as AP75. The effectiveness was particularly pronounced for larger objects in the MS COCO dataset, indicating the method's robustness for diverse object scales.

Implications

This research carries implications beyond marginal improvements in object detection statistics. By addressing the training label assignment from a task-interactive perspective, the Mutual Guidance method encourages an alignment between the localization and classification processes. Alleviating task-misalignment, this strategy ensures that classification confidences and localization accuracy synergize in optimizing anchor points—ultimately leading to more precise object detection models.

Future Directions

Given the substantial improvements demonstrated by the Mutual Guidance strategy, future work might explore its application across other paradigms of object detection, including anchor-free methods. Additionally, there is potential to further refine the dynamic thresholds or explore adaptive learning schemes that can tailor this mutual guidance strategy across varying object densities, scales, and complexities inherent in real-world datasets.

In conclusion, the paper underscores a paradigm shift in object detection training by advocating for a mutualistic approach to anchor assignment. By bridging the gap between localization and classification tasks, it delivers a comprehensive strategy poised to enhance the next generation of object detection models.