Is Heuristic Sampling Necessary in Training Deep Object Detectors? (1909.04868v8)

Abstract: To train accurate deep object detectors under the extreme foreground-background imbalance, heuristic sampling methods are always necessary, which either re-sample a subset of all training samples (hard sampling methods, \eg biased sampling, OHEM), or use all training samples but re-weight them discriminatively (soft sampling methods, \eg Focal Loss, GHM). In this paper, we challenge the necessity of such hard/soft sampling methods for training accurate deep object detectors. While previous studies have shown that training detectors without heuristic sampling methods would significantly degrade accuracy, we reveal that this degradation comes from an unreasonable classification gradient magnitude caused by the imbalance, rather than a lack of re-sampling/re-weighting. Motivated by our discovery, we propose a simple yet effective \emph{Sampling-Free} mechanism to achieve a reasonable classification gradient magnitude by initialization and loss scaling. Unlike heuristic sampling methods with multiple hyperparameters, our Sampling-Free mechanism is fully data diagnostic, without laborious hyperparameters searching. We verify the effectiveness of our method in training anchor-based and anchor-free object detectors, where our method always achieves higher detection accuracy than heuristic sampling methods on COCO and PASCAL VOC datasets. Our Sampling-Free mechanism provides a new perspective to address the foreground-background imbalance. Our code is released at \url{https://github.com/ChenJoya/sampling-free}.

Analysis of the Necessity of Heuristic Sampling in Deep Object Detector Training

The paper "Is Heuristic Sampling Necessary in Training Deep Object Detectors?" authored by Joya Chen et al., presents a comprehensive inquiry into the necessity of heuristic sampling methods when training deep object detectors. The focal point of this research is the foreground-background (fg-bg) imbalance problem prevalent in object detection. This phenomenon is a significant challenge due to the sparse number of foreground samples relative to the vast majority of background samples. Traditional approaches rely heavily on heuristic sampling strategies such as biased sampling and Focal Loss to mitigate this imbalance.

The authors delve into the root cause of performance degradation when heuristic methods are omitted. Previous studies emphasized the importance of these methods, suggesting up to 20% accuracy drops when they are not utilized. However, this paper identifies that the primary issue resides not in the absence of heuristic sampling but rather in the disproportionate gradient magnitudes in classification tasks stemming from fg-bg imbalance.

The authors propose a novel Sampling-Free mechanism, which emphasizes controlling classification gradient magnitudes through strategic weight initialization and loss scaling. This approach circumvents the necessity for heuristic sampling methods, presenting a hyperparameter-free solution to the fg-bg imbalance. The paper details two primary techniques within this mechanism:

1. Optimal Bias Initialization: At the inception of training, the initialization is automated to achieve minimal classification loss, thus balancing gradient magnitudes without the usual heuristic settings (such as pre-determined biases used by Focal Loss).
2. Guided Loss Scaling: This approach aligns classification and localization loss scales dynamically through training. The paper unfolds a loss-adjustment strategy where the localization task guides classification loss scaling, creating a harmonious training balance.

Extensive empirical evidence demonstrates the effectiveness of the Sampling-Free mechanism across multiple detector architectures, including anchor-based models like Faster R-CNN and anchor-free models such as FCOS. The experiments showcase improvements in detection accuracy over state-of-the-art heuristic sampling methods, notably without introducing additional hyperparameters. For instance, Sampling-Free improves AP scores in Faster R-CNN by 1.6 points over biased sampling and achieves comparable accuracy to state-of-the-art label assignment strategies.

On a theoretical front, the paper's implications suggest a reevaluation of fg-bg imbalance handling in object detection, challenging conventional reliance on heuristic sampling. This presents an opportunity for further exploration into alternative strategies for optimizing the training of object detectors, potentially impacting how models address similar imbalance issues in other domains.

Future developments in this area might explore integrating metric-aware loss functions, foreseeing more nuanced object detection models less dependent on heuristic sampling. By emphasizing classification under imbalanced conditions, the paper sets the groundwork for revising training protocols for deep learning networks within the contexts of object detection and beyond.

This paper is a robust contribution to the field of object detection, providing insights and methodologies relevant to researchers and practitioners aiming to refine model training paradigms. The Sampling-Free mechanism marks an important step toward more efficient and less parameter-intensive training methodologies.