Papers
Topics
Authors
Recent
Search
2000 character limit reached

Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection

Published 16 May 2024 in cs.CV | (2405.09782v2)

Abstract: This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image. We observe that current metrics are size-sensitive, where larger objects are focused, and smaller ones tend to be ignored. We argue that the evaluation should be size-invariant because bias based on size is unjustified without additional semantic information. In pursuit of this, we propose a generic approach that evaluates each salient object separately and then combines the results, effectively alleviating the imbalance. We further develop an optimization framework tailored to this goal, achieving considerable improvements in detecting objects of different sizes. Theoretically, we provide evidence supporting the validity of our new metrics and present the generalization analysis of SOD. Extensive experiments demonstrate the effectiveness of our method. The code is available at https://github.com/Ferry-Li/SI-SOD.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Frequency-tuned salient region detection. In CVPR, Jun 2009.
  2. Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE TIP, 22(1):55–69, 2013a.
  3. Analysis of scores, datasets, and models in visual saliency prediction. In ICCV, pp.  921–928, 2013b.
  4. Salient object detection: A survey. Computational Visual Media, pp.  117–150, 2019.
  5. What do different evaluation metrics tell us about saliency models? IEEE TPAMI, 41(3):740–757, 2019.
  6. Cnn-based rgb-d salient object detection: Learn, select, and fuse. IJCV, 129(7):2076–2096, 2021.
  7. A general framework for consistent structured prediction with implicit loss embeddings. JMLR, 21(1):3852–3918, 2020.
  8. Consistency analysis for binary classification revisited. In ICML, volume 70, pp.  961–969. PMLR, 06–11 Aug 2017.
  9. Addressing multiple salient object detection via dual-space long-range dependencies. CVIU, 235:103776, 2023.
  10. Imagenet: A large-scale hierarchical image database. In CVPR, pp.  248–255, 2009.
  11. Object detection in aerial images: A large-scale benchmark and challenges. IEEE TPAMI, 44:7778–7796, 2022.
  12. Enhanced-alignment measure for binary foreground map evaluation. In IJCAI, pp.  698–704, 7 2018.
  13. ℓ∞subscriptℓ\ell_{\infty}roman_ℓ start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT vector contraction for rademacher complexity, 2019.
  14. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pp.  580–587, 2014.
  15. Deep residual learning for image recognition. In CVPR, pp.  770–778, 2016.
  16. Accurate rgb-d salient object detection via collaborative learning. In ECCV, pp.  52–69, 2020.
  17. Dmra: Depth-induced multi-scale recurrent attention network for rgb-d saliency detection. IEEE TIP, 31:2321–2336, 2022.
  18. Multispectral video semantic segmentation: A benchmark dataset and baseline. In CVPR, pp.  1094–1104, 2023.
  19. Richer and deeper supervision network for salient object detection. ArXiv, 2019.
  20. Hierarchical set-to-set representation for 3-d cross-modal retrieval. IEEE TNNLS, pp.  1–13, 2023.
  21. Deep contrast learning for salient object detection. In CVPR, pp.  478–487, 2016.
  22. Visual saliency based on multiscale deep features. In CVPR, pp.  5455–5463, June 2015.
  23. Joint semantic mining for weakly supervised rgb-d salient object detection. NeurIPS, 34:11945–11959, 2021.
  24. Dvsod: Rgb-d video salient object detection. In NeurIPS, pp.  8774–8787, 2023a.
  25. Delving into calibrated depth for accurate rgb-d salient object detection. IJCV, 131(4):855–876, 2023b.
  26. Deep collaborative embedding for social image understanding. IEEE TPAMI, 41(9):2070–2083, 2019.
  27. A simple pooling-based design for real-time salient object detection. In CVPR, 2019.
  28. Picanet: Learning pixel-wise contextual attention for saliency detection. In CVPR, 2018a.
  29. Picanet: Pixel-wise contextual attention learning for accurate saliency detection. IEEE TIP, Dec 2018b.
  30. Non-local deep features for salient object detection. In CVPR, 2017.
  31. Pyramidal feature shrinking for salient object detection. In AAAI, volume 35, pp.  2311–2318, 2021.
  32. Saliency-based discriminant tracking. In CVPR, 2009.
  33. How to evaluate foreground maps. In CVPR, Jun 2014.
  34. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, Nov 2002.
  35. Areas beneath the relative operating characteristics (roc) and relative operating levels (rol) curves: Statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society, 128(584):2145–2166, 2002.
  36. Maurer, A. A vector-contraction inequality for rademacher complexities. In Algorithmic Learning Theory, pp.  3–17, 2016.
  37. Michel Ledoux, M. T. Probability in Banach Spaces. Springer-Verlag, New York, 1991.
  38. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 3DV, pp.  565–571, 2016.
  39. Design and perceptual validation of performance measures for salient object segmentation. In CVPR workshop, pp.  49–56, 2010.
  40. Saliency filters: Contrast based filtering for salient region detection. In CVPR, Jun 2012.
  41. Depth-induced multi-scale recurrent attention network for saliency detection. In ICCV, pp.  7254–7263, 2019.
  42. You only look once: Unified, real-time object detection. In CVPR, pp.  779–788, 2016.
  43. Salient object detection by fusing local and global contexts. IEEE TMM, 23:1442–1453, 2021.
  44. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS, 28, 2015.
  45. Region-based saliency detection and its application in object recognition. IEEE TCSVT, pp.  769–779, 2014.
  46. U-net: Convolutional networks for biomedical image segmentation. MICCAI, Jan 2015.
  47. Learnable depth-sensitive attention for deep rgb-d saliency detection with multi-modal fusion architecture search. IJCV, 130(11):2822–2841, 2022.
  48. Tri-clustered tensor completion for social-aware image tag refinement. IEEE TPAMI, 39(8):1662–1674, 2017.
  49. Learning to detect salient objects with image-level supervision. In CVPR, 2017.
  50. Salient object detection in the deep learning era: An in-depth survey. IEEE TPAMI, pp.  3239–3259, 2022.
  51. Pixels, regions, and objects: Multiple enhancement for salient object detection. In CVPR, pp.  10031–10040, 2023.
  52. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600–612, 2004.
  53. Label decoupling framework for salient object detection. In CVPR, June 2020.
  54. Regularized densely-connected pyramid network for salient instance segmentation. IEEE TIP, 30:3897–3907, 2021.
  55. Edn: Salient object detection via extremely-downsampled network. IEEE TIP, pp.  3125–3136, 2022.
  56. What is and what is not a salient object? learning salient object detector by ensembling linear exemplar regressors. In CVPR, pp.  4399–4407, 2017.
  57. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, pp.  3485–3492, 2010.
  58. Few-shot object detection and viewpoint estimation for objects in the wild. In ECCV, volume PP of 3, pp.  192–210, 2020.
  59. Hierarchical saliency detection. In CVPR, 2013.
  60. Saliency detection via graph-based manifold ranking. In CVPR, pp.  3166–3173. IEEE, 2013.
  61. Optimizing two-way partial auc with an end-to-end framework. IEEE TPAMI, 45(8):10228–10246, 2023.
  62. Unitbox: An advanced object detection network. In ACM MM, pp.  516–520, 2016.
  63. Causal intervention for weakly-supervised semantic segmentation. In NeurIPS, volume 33, pp.  655–666, 2020a.
  64. Learning generative vision transformer with energy-based latent space for saliency prediction. NeurIPS, 34:15448–15463, 2021a.
  65. A bi-directional message passing model for salient object detection. In CVPR, Jun 2018.
  66. Memory-oriented decoder for light field salient object detection. NeurIPS, pp.  896–906, 2019.
  67. Lfnet: Light field fusion network for salient object detection. IEEE TIP, 29:6276–6287, 2020b.
  68. Dynamic context-sensitive filtering network for video salient object detection. In ICCV, pp.  1553–1563, 2021b.
  69. C2dfnet: Criss-cross dynamic filter network for rgb-d salient object detection. IEEE TMM, 25:5142–5154, 2023.
  70. Amulet: Aggregating multi-level convolutional features for salient object detection. arXiv, 2017a.
  71. Learning uncertain convolutional features for accurate saliency detection. In ICCV, pp.  212–221, 2017b.
  72. Suppress and balance: A simple gated network for salient object detection. In ECCV, 2020.
  73. Salient object detection via integrity learning. IEEE TPAMI, 2022.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free