Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Training (2004.06002v2)

Abstract: Although two-stage object detectors have continuously advanced the state-of-the-art performance in recent years, the training process itself is far from crystal. In this work, we first point out the inconsistency problem between the fixed network settings and the dynamic training procedure, which greatly affects the performance. For example, the fixed label assignment strategy and regression loss function cannot fit the distribution change of proposals and thus are harmful to training high quality detectors. Consequently, we propose Dynamic R-CNN to adjust the label assignment criteria (IoU threshold) and the shape of regression loss function (parameters of SmoothL1 Loss) automatically based on the statistics of proposals during training. This dynamic design makes better use of the training samples and pushes the detector to fit more high quality samples. Specifically, our method improves upon ResNet-50-FPN baseline with 1.9% AP and 5.5% AP$_{90}$ on the MS COCO dataset with no extra overhead. Codes and models are available at https://github.com/hkzhang95/DynamicRCNN.

• The paper introduces Dynamic R-CNN by dynamically adjusting label assignments and regression loss to align training with evolving proposal qualities.
• It achieves a 1.9% increase in Average Precision and a 5.5% boost at AP90 on the challenging MS COCO dataset using a ResNet-50-FPN baseline.
• The method enhances detection robustness across various architectures without adding computational overhead during inference.

Overview of Dynamic R-CNN for High Quality Object Detection

The paper "Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Training" by Hongkai Zhang et al. presents a notable advancement in object detection, particularly within two-stage frameworks such as Faster R-CNN. The authors identify and address inherent limitations in traditional training methodologies that do not account for the dynamic nature of proposal distributions during model training. These limitations include the fixed label assignment strategies and static regression loss functions, which fail to adapt to the evolving quality of proposals throughout the training process.

The proposed solution, Dynamic R-CNN, introduces two key mechanisms: Dynamic Label Assignment (DLA) and Dynamic SmoothL1 Loss (DSL). DLA adjusts the IoU threshold used for label assignment based on the distribution of proposals, thus tailoring the training samples to better correspond to high IoU thresholds as the training progresses. Conversely, DSL modifies the parameters of the regression loss function to automatically match the proposal distribution changes, enhancing the focus on high-quality samples without computational overhead during training.

Numerical Results and Empirical Validation

The effectiveness of the Dynamic R-CNN framework is empirically validated on the challenging MS COCO dataset. The authors report a substantial improvement in performance metrics, particularly in terms of Average Precision (AP). Specifically, the proposed method exhibits a 1.9% increase in AP and an impressive 5.5% enhancement in AP at a high IoU threshold (AP$_{90}$), when applied to a ResNet-50-FPN baseline. Importantly, these gains are achieved without introducing additional computational burdens during inference, a critical consideration for practical deployment in resource-constrained environments.

Comprehensive experiments demonstrate that these methods are robust across various architectures and compatible with existing enhancements such as multi-scale training and testing, and the use of deformable convolutions. This robustness is further evidenced by the consistent performance improvement across different backbones, including ResNet-101 and variants incorporating deformable convolutional networks (DCN), as well as with Mask R-CNN for instance segmentation.

Implications and Future Prospects

Dynamic R-CNN's contribution to the field of object detection is significant for several reasons. By effectively utilizing the inherent dynamic quality characteristics of modern training processes, this approach facilitates the development of more precise object detectors. The adaptability of Dynamic R-CNN implies potentially superior performance in real-world scenarios, where object characteristics and scene compositions are varied and unpredictable. Additionally, the avoidance of computationally intensive cascaded models suggests a broader applicability to edge devices where resource efficiency is paramount.

Future research directions could explore the extension of dynamic training principles to entirely new types of network architectures and other domains within AI. Furthermore, integrating the dynamic adjustment methodologies into the training of one-stage detectors appears promising, as initial experiments with RetinaNet suggest potential benefits. Another potential area of investigation is the application of these principles to other complex tasks beyond detection, such as segmentation or tracking, where object proposal quality similarly impacts overall performance.

In summary, Dynamic R-CNN represents a substantial step towards more adaptive and effective object detection models, aligning training practices with the evolving landscape of neural network capabilities and application demands.