Dropout Sampling for Robust Object Detection in Open-Set Conditions (1710.06677v2)

Abstract: Dropout Variational Inference, or Dropout Sampling, has been recently proposed as an approximation technique for Bayesian Deep Learning and evaluated for image classification and regression tasks. This paper investigates the utility of Dropout Sampling for object detection for the first time. We demonstrate how label uncertainty can be extracted from a state-of-the-art object detection system via Dropout Sampling. We evaluate this approach on a large synthetic dataset of 30,000 images, and a real-world dataset captured by a mobile robot in a versatile campus environment. We show that this uncertainty can be utilized to increase object detection performance under the open-set conditions that are typically encountered in robotic vision. A Dropout Sampling network is shown to achieve a 12.3% increase in recall (for the same precision score as a standard network) and a 15.1% increase in precision (for the same recall score as the standard network).

• The paper introduces using Dropout Sampling on SSD for uncertainty estimation in object detection under open-set conditions to improve detection performance.
• It demonstrates that the Bayesian SSD increases recall by 12.3% and precision by 15.1% compared to standard SSD configurations.
• The method proves computationally efficient and robust, offering practical improvements for real-world robotic vision tasks.

Dropout Sampling for Robust Object Detection in Open-Set Conditions

The paper "Dropout Sampling for Robust Object Detection in Open-Set Conditions" by Miller et al. explores the application of Dropout Variational Inference, or Dropout Sampling, as an approximation to Bayesian Deep Learning specifically applied to object detection. The research focuses on the unique challenges presented by open-set conditions, which are common in robotics applications where systems frequently encounter objects not included during training.

Context and Objective

Typical deep learning-based object detection systems are designed for closed-set conditions, where the testing data consists of objects seen during training. However, in real-world environments, particularly those navigated by mobile robots, encountering new, untrained object classes is unavoidable. This discrepancy often results in significant performance drops for traditional object detectors due to misclassification of unknown objects as known ones. Thus, operating under open-set conditions remains a formidable challenge.

The primary objective of this research is to extend Dropout Sampling to object detection, providing a better estimate of uncertainty for predictions made by detection networks. This allows the system to reject predictions with low confidence, potentially improving both precision and recall under open-set conditions.

Research Methodology

The authors implement Dropout Sampling on a Single Shot Multibox Detector (SSD), a state-of-the-art object detection model. Dropout Sampling involves keeping dropout layers active during inference, thus enabling the execution of multiple forward passes through the network to estimate a distribution of possible outcomes. This approximation to a Bayesian inference framework allows the network to account for model uncertainty in a computationally feasible manner.

The paper evaluates the effectiveness of this approach using two datasets: the synthetic SceneNet RGB-D dataset and the real-world QUT Campus dataset. Among the three network configurations tested, the Bayesian SSD employing Dropout Sampling exhibited superior performance in terms of precision and recall metrics compared to a standard SSD and SSD with entropy thresholding.

Key Findings

• Performance Improvement: Bayesian SSD achieves a 12.3% increase in recall at the same precision level as a standard SSD and a 15.1% increase in precision for the same recall, underlining the effectiveness of using uncertainty measures to refine detection outcomes.
• Open-Set Identification: The method successfully reduces open-set error, demonstrating improved robustness against misclassification of untrained objects. As evident from the SceneNet RGB-D tests, Bayesian SSD significantly minimized instances where unknown objects were wrongly classified as known.
• Parameter Sensitivity: The benefits of Dropout Sampling were observed even with a modest number of forward passes, suggesting computational feasibility for real-time applications. However, an increase in forward passes enhanced the robustness and accuracy further.

Implications and Future Directions

The implications of this work are significant for robotics applications. A robot operating with a more accurate object detection mechanism in open-set conditions can make more reliable decisions, enhancing safety and functionality. Moreover, this approach can potentially be integrated into other perception tasks where model uncertainty poses a challenge.

Future research could explore the utilization of spatial uncertainty captured via the covariance matrix from bounding box detections for applications such as object-based SLAM. Additionally, optimizing the computational overhead and testing on a broader array of real-world scenarios may further validate and improve this approach.

In conclusion, this paper presents a compelling case for employing Bayesian techniques in object detection to address the challenges posed by open-set conditions, thereby advancing the field of robotic vision.