Do Object Detection Localization Errors Affect Human Performance and Trust?
Abstract: Bounding boxes are often used to communicate automatic object detection results to humans, aiding humans in a multitude of tasks. We investigate the relationship between bounding box localization errors and human task performance. We use observer performance studies on a visual multi-object counting task to measure both human trust and performance with different levels of bounding box accuracy. The results show that localization errors have no significant impact on human accuracy or trust in the system. Recall and precision errors impact both human performance and trust, suggesting that optimizing algorithms based on the F1 score is more beneficial in human-computer tasks. Lastly, the paper offers an improvement on bounding boxes in multi-object counting tasks with center dots, showing improved performance and better resilience to localization inaccuracy.
- Amazon (2023). Mechanical turk. https://www.mturk.com/. Accessed: 2023-02-20.
- Toward Transformer-Based Object Detection.
- Trust measurement in human–automation interaction: A systematic review. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63:1595–1599.
- Cascade r-cnn: Delving into high quality object detection. In CVPR. IEEE/CVF.
- End-to-end object detection with transformers. Lecture Notes in Computer Science, page 213–229.
- How many crowdsourced workers should a requester hire? Annals of Mathematics and Artificial Intelligence, 78:45–72.
- Diffusiondet: Diffusion model for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19830–19843.
- Up-detr: Unsupervised pre-training for object detection with transformers. In CVPR, pages 1601–1610. IEEE/CVF.
- Centernet: Keypoint triplets for object detection. In ICCV. IEEE/CVF.
- Aquarium combined roboflow. https://public.roboflow.com/object-detection/aquarium.
- Roboflow (version 1.0) [software]. https://roboflow.com.
- The pascal visual object classes (voc) challenge. Int. J. Comput. Vision, 88(2):303–338.
- Girshick, R. (2015). Fast r-cnn. In ICCV. IEEE.
- Region-based convolutional networks for accurate object detection and segmentation. PAMI.
- Design, development and evaluation of a human-computer trust scale. Behaviour & Information Technology, 38:1–12.
- Measurement of Trust in Automation: A Narrative Review and Reference Guide. Front Psychol, 12:604977.
- Krippendorff, K. E. (2010). (Vols. 1-0).
- Cornernet: Detecting objects as paired keypoints. In ECCV. Springer.
- Exploring plain vision transformer backbones for object detection. In European Conference on Computer Vision, pages 280–296. Springer.
- Focal loss for dense object detection. In ICCV. IEEE/CVF.
- Microsoft COCO: common objects in context. CoRR, abs/1405.0312.
- Performance evaluating the evaluator. In 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pages 129–136. IEEE.
- Ssd: Single shot multibox detector. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pages 21–37. Springer.
- Trust in a specific technology: An investigation of its components and measures. ACM Transactions on Management Information Systems, 2:12–32.
- Can we trust bounding box annotations for object detection? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4813–4822.
- You only look once: Unified, real-time object detection. In CVPR. IEEE/CVF.
- Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
- Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems.
- Humans disagree with the iou for measuring object detector localization error. In 2022 IEEE International Conference on Image Processing (ICIP), pages 1261–1265. IEEE.
- Computer-aided detection of brain metastasis on 3d mr imaging: Observer performance study. PLOS ONE, 12(6):1–18.
- The value of observer performance studies in dose optimization: a focus on free-response receiver operating characteristic methods. J Nucl Med Technol, 41(2):57–64.
- Bottom-up object detection by grouping extreme and center points. In CVPR. IEEE/CVF.
- Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv preprint arXiv:2010.04159.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.