Soft-NMS -- Improving Object Detection With One Line of Code (1704.04503v2)

Abstract: Non-maximum suppression is an integral part of the object detection pipeline. First, it sorts all detection boxes on the basis of their scores. The detection box M with the maximum score is selected and all other detection boxes with a significant overlap (using a pre-defined threshold) with M are suppressed. This process is recursively applied on the remaining boxes. As per the design of the algorithm, if an object lies within the predefined overlap threshold, it leads to a miss. To this end, we propose Soft-NMS, an algorithm which decays the detection scores of all other objects as a continuous function of their overlap with M. Hence, no object is eliminated in this process. Soft-NMS obtains consistent improvements for the coco-style mAP metric on standard datasets like PASCAL VOC 2007 (1.7% for both R-FCN and Faster-RCNN) and MS-COCO (1.3% for R-FCN and 1.1% for Faster-RCNN) by just changing the NMS algorithm without any additional hyper-parameters. Using Deformable-RFCN, Soft-NMS improves state-of-the-art in object detection from 39.8% to 40.9% with a single model. Further, the computational complexity of Soft-NMS is the same as traditional NMS and hence it can be efficiently implemented. Since Soft-NMS does not require any extra training and is simple to implement, it can be easily integrated into any object detection pipeline. Code for Soft-NMS is publicly available on GitHub (http://bit.ly/2nJLNMu).

• The paper proposes Soft-NMS, an algorithm that refines object detection by decaying overlapping bounding box scores instead of outright suppression.
• It demonstrates improved precision and recall, achieving up to a 1.3% increase in mAP on datasets like MS-COCO and PASCAL VOC.
• Soft-NMS is easily integrated into existing systems, providing significant performance gains with negligible computational cost.

Improving Object Detection With One Line of Code

In the domain of object detection, non-maximum suppression (NMS) remains a critical component of the detection pipeline. This paper introduces Soft-NMS, an innovative algorithm intended to enhance the precision of object detection models without necessitating any substantial modifications to existing systems. Authored by Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S. Davis from the University of Maryland, the paper delineates how a subtle yet strategic adjustment to the conventional NMS algorithm can yield significant improvements in detection metrics.

Problem Statement

The primary objective of the traditional NMS algorithm is to discard redundant bounding boxes by applying a threshold to their overlap scores. While effective, this technique has a notable drawback: by setting the scores of overlapping boxes to zero, potential true positives can be eliminated, leading to a decreased recall, especially at higher overlap thresholds.

Proposed Solution: Soft-NMS

Soft-NMS addresses this limitation by modifying the way overlapping boxes are handled. Instead of an abrupt suppression of scores for boxes that surpass the predefined overlap threshold, scores are decayed gradually as a continuous function of their overlap with the highest-scoring detection ($\mathcal{M}$). This decay can be achieved by either a linear or Gaussian function, ensuring that no object is entirely discarded from the detection process.

The linear decay function can be expressed as:

$$s_i = \begin{cases} s_i,& \text{if iou}(\mathcal{M}, b_i) < N_t \ s_i (1 - \text{iou}(\mathcal{M}, b_i)),& \text{if iou}(\mathcal{M}, b_i) \ge N_t \end{cases}$$

Alternatively, a Gaussian decay function is applied such that:

$$s_i = s_i e^{-\frac{\text{iou}(\mathcal{M}, b_i)^2}{\sigma}}, \forall b_i \notin \mathcal{D}$$

Experimental Evaluation and Results

The authors present empirical evaluations on standard datasets such as PASCAL VOC 2007 and MS-COCO. Notable improvements are achieved with minimal computational overhead. For example, with the R-FCN object detector on the MS-COCO dataset, Soft-NMS demonstrated a 1.3% improvement in coco-style mAP (mean Average Precision), translating to an increase from 31.1% to 32.4%. Similarly, Faster-RCNN saw a 1.1% boost with Soft-NMS. These improvements are significant given that no additional training or hyper-parameter tuning was involved.

Sensitivity Analysis

The robustness of Soft-NMS is underscored through a sensitivity analysis of the algorithm's parameters. For traditional NMS, increasing the overlap threshold results in a slight performance gain at higher overlap thresholds but performs poorly at lower thresholds. Conversely, Soft-NMS, with its parameter $\sigma$, maintains superior performance across a broad range of thresholds, particularly excelling at higher overlap thresholds where traditional NMS falls short.

Practical Implications

Soft-NMS is not only easy to integrate but also demonstrates practical efficacy by improving both recall and precision without incurring additional computational costs. These attributes make Soft-NMS an attractive enhancement for existing object detection frameworks. The simplicity and effectiveness of Soft-NMS ensure its wide adoptability, circumventing the performance-affecting bottlenecks associated with overly complex modifications.

Future Directions

While Soft-NMS proves to be competent, the paper leaves room for the exploration of more advanced weighting functions that could further tailor the decay of detection scores based on additional features beyond overlap. Moreover, the impact of Soft-NMS can be extended to other emerging object detection paradigms, potentially enhancing new state-of-the-art models.

Conclusion

By introducing Soft-NMS, this paper presents a compelling case for a modest but impactful modification to traditional NMS. This enhancement aligns well with modern object detection requirements, offering noticeable gains in detection performance with negligible computational implications. As object detection frameworks evolve, methodologies such as Soft-NMS underscore the importance of continuous innovation in refining fundamental components of the detection pipeline.