Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation (2005.03572v4)

Abstract: Deep learning-based object detection and instance segmentation have achieved unprecedented progress. In this paper, we propose Complete-IoU (CIoU) loss and Cluster-NMS for enhancing geometric factors in both bounding box regression and Non-Maximum Suppression (NMS), leading to notable gains of average precision (AP) and average recall (AR), without the sacrifice of inference efficiency. In particular, we consider three geometric factors, i.e., overlap area, normalized central point distance and aspect ratio, which are crucial for measuring bounding box regression in object detection and instance segmentation. The three geometric factors are then incorporated into CIoU loss for better distinguishing difficult regression cases. The training of deep models using CIoU loss results in consistent AP and AR improvements in comparison to widely adopted $\ell_n$-norm loss and IoU-based loss. Furthermore, we propose Cluster-NMS, where NMS during inference is done by implicitly clustering detected boxes and usually requires less iterations. Cluster-NMS is very efficient due to its pure GPU implementation, and geometric factors can be incorporated to improve both AP and AR. In the experiments, CIoU loss and Cluster-NMS have been applied to state-of-the-art instance segmentation (e.g., YOLACT and BlendMask-RT), and object detection (e.g., YOLO v3, SSD and Faster R-CNN) models. Taking YOLACT on MS COCO as an example, our method achieves performance gains as +1.7 AP and +6.2 AR${100}$ for object detection, and +0.9 AP and +3.5 AR${100}$ for instance segmentation, with 27.1 FPS on one NVIDIA GTX 1080Ti GPU. All the source code and trained models are available at https://github.com/Zzh-tju/CIoU

Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation

The paper proposes innovations in the field of object detection and instance segmentation by enhancing the role of geometric factors within model learning and inference processes. Specifically, the authors introduce the Complete-IoU (CIoU) loss and Cluster-NMS techniques to improve bounding box regression and Non-Maximum Suppression (NMS), respectively. These methods collectively lead to notable gains in average precision (AP) and average recall (AR) while maintaining inference efficiency.

Complete-IoU Loss

CIoU loss is designed to address limitations found in earlier loss functions like the $\ell_n$-norm and IoU-based losses. While $\ell_n$-norm functions are not consistent with the IoU evaluation metric, IoU-based losses only maximize the overlap area between boxes, which is insufficient for effective bounding box regression especially in challenging cases.

CIoU enhances performance by considering three geometric factors: overlap area, normalized central point distance, and aspect ratio. This holistic approach ensures that CIoU considers all critical aspects of box matching, leading to faster convergence and improved precision and recall in object detection and instance segmentation tasks.

The simulation experiments and empirical results indicate CIoU outperforms IoU and Generalized IoU (GIoU) losses, especially under conditions of varying scales, distances, and aspect ratios. For instance, CIoU demonstrated significant improvements on YOLO v3, improving AP by 5.67% over GIoU.

Cluster-NMS

Cluster-NMS proposes an efficient alternative to traditional NMS, achieving higher precision and recall by clustering detected boxes. Unlike original and Fast NMS, Cluster-NMS can be efficiently implemented on GPUs, thus providing real-time inference rates.

Geometric factors are incorporated into Cluster-NMS through several variants:

• Score Penalty Mechanism (Cluster-NMS$_S$): Utilizes overlap-based score penalties to enhance recall and precision.
• Normalized Central Point Distance (Cluster-NMS$_D$): Addresses occlusion cases by incorporating distance factors into the suppression process.
• Weighted Coordinates (Cluster-NMS$_W$): Computes weighted averages of box coordinates based on scores and overlap areas, akin to the Weighted-NMS approach but with higher efficiency.

The experimental results validate that leveraging these geometric factors yields consistently improved precision and recall metrics without sacrificing speed. Tests on models like YOLACT and YOLO v3 reflect the robustness and adaptability of the Cluster-NMS framework.

Implications and Future Directions

The introduction of CIoU loss and Cluster-NMS presents promising advancements in improving the efficiency and accuracy of object detection and instance segmentation models. In practice, these methods could be widely adopted across various AI applications involving visual data optimization, including autonomous driving, surveillance, and augmented reality.

Looking ahead, future research could explore further enhancements of geometric factor modeling to refine bounding box regression, potentially integrating these methods into larger and more complex models. Researchers might also investigate more nuanced weighting mechanisms within Cluster-NMS to optimise processing across diverse datasets and environments.