Salient Object Detection in the Deep Learning Era: An In-Depth Survey (1904.09146v5)

Abstract: As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research attention over the years. Recent advances in SOD are predominantly led by deep learning-based solutions (named deep SOD). To enable in-depth understanding of deep SOD, in this paper, we provide a comprehensive survey covering various aspects, ranging from algorithm taxonomy to unsolved issues. In particular, we first review deep SOD algorithms from different perspectives, including network architecture, level of supervision, learning paradigm, and object-/instance-level detection. Following that, we summarize and analyze existing SOD datasets and evaluation metrics. Then, we benchmark a large group of representative SOD models, and provide detailed analyses of the comparison results. Moreover, we study the performance of SOD algorithms under different attribute settings, which has not been thoroughly explored previously, by constructing a novel SOD dataset with rich attribute annotations covering various salient object types, challenging factors, and scene categories. We further analyze, for the first time in the field, the robustness of SOD models to random input perturbations and adversarial attacks. We also look into the generalization and difficulty of existing SOD datasets. Finally, we discuss several open issues of SOD and outline future research directions.

• The paper presents a comprehensive taxonomy of deep learning-based SOD methods, categorizing them by network architecture, supervision levels, and learning paradigms.
• The paper benchmarks 44 models across renowned datasets, demonstrating that deep models significantly outperform traditional methods despite robustness challenges.
• The paper highlights empirical insights, noting limitations in small object detection and perturbation sensitivity, and suggests future research directions for enhanced model adaptability.

Salient Object Detection in the Deep Learning Era: An Expert Overview

The paper "Salient Object Detection in the Deep Learning Era: An In-depth Survey" by Wang et al. presents a comprehensive survey of salient object detection (SOD) leveraging deep learning technologies. It provides an extensive review and categorization of SOD methods, datasets, and evaluation metrics, along with insightful empirical analyses. This essay will summarize the key contributions and implications, focusing on the impact of the deep learning paradigm on SOD.

Overview of Deep Learning-Based SOD Models

The authors categorize existing deep SOD models based on network architecture, supervision levels, learning paradigms, and object-level versus instance-level detection. The survey reveals that while earlier approaches relied on multi-layer perceptron (MLP)-based techniques that leveraged local features, more recent methods use fully convolutional networks (FCNs), which allow for end-to-end training and capture richer spatial information. FCN architectures are predominantly single-stream or hierarchical networks, distinguishing themselves through their enhanced information flow and learning capacity.

Evaluation and Benchmarking

The survey benchmarks 44 deep SOD models on several renowned datasets and contrasts their performance against top-performing heuristic methods. The results show a significant performance advantage of data-driven models, underscoring the efficacy of deep learning in capturing complex visual patterns. Models such as PoolNet and EGNet demonstrate state-of-the-art results but highlight challenges such as performance saturation on specific datasets.

Empirical Insights and Challenges

The paper conducts an attribute-based evaluation, partitioning images based on salient object categories, contextual challenges, and scene complexities. This analysis identifies specific strengths and limitations of current models, such as difficulties with small objects and complex scenes, suggesting pathways for future research. It also finds that deep models significantly outperform traditional methods in capturing semantically rich objects but still struggle with determining relative object importance.

Robustness and Generalization

The paper examines the robustness of SOD models against both random input perturbations and adversarial attacks. The results reveal surprising sensitivities in deep models to common perturbations, hinting at a need for more robust design strategies. Moreover, a cross-dataset generalization paper highlights the generalization limits of current datasets, identifying DUTS as a well-generalizing training set.

Theoretical and Practical Implications

The survey’s findings demonstrate that while deep SOD models have made substantial progress, there remain several open challenges. Future research could benefit from exploring adaptive computation strategies, more robust learning paradigms, and better integration of classical psychological theories to enhance model robustness and interpretability. The paper suggests improvements in dataset collection, advocating for more consistent annotations and domain-specific datasets to advance the field further.

Conclusion and Future Directions

This comprehensive survey by Wang et al. offers an insightful examination of the state of salient object detection, emphasizing deep learning's pivotal role in advancing the field. Researchers are encouraged to consider the identified challenges and implications, such as adversarial robustness, dataset generalization, and model adaptability, to drive further innovations in SOD applications. The paper effectively sets the stage for continued exploration into these promising directions, ensuring that SOD models continue to evolve and adapt to new technological and theoretical landscapes.