Deep Contrast Learning for Salient Object Detection (1603.01976v1)

Abstract: Salient object detection has recently witnessed substantial progress due to powerful features extracted using deep convolutional neural networks (CNNs). However, existing CNN-based methods operate at the patch level instead of the pixel level. Resulting saliency maps are typically blurry, especially near the boundary of salient objects. Furthermore, image patches are treated as independent samples even when they are overlapping, giving rise to significant redundancy in computation and storage. In this CVPR 2016 paper, we propose an end-to-end deep contrast network to overcome the aforementioned limitations. Our deep network consists of two complementary components, a pixel-level fully convolutional stream and a segment-wise spatial pooling stream. The first stream directly produces a saliency map with pixel-level accuracy from an input image. The second stream extracts segment-wise features very efficiently, and better models saliency discontinuities along object boundaries. Finally, a fully connected CRF model can be optionally incorporated to improve spatial coherence and contour localization in the fused result from these two streams. Experimental results demonstrate that our deep model significantly improves the state of the art.

Deep Contrast Learning for Salient Object Detection

The paper "Deep Contrast Learning for Salient Object Detection" by Guanbin Li and Yizhou Yu presents a novel end-to-end deep contrast network tailored for the task of salient object detection. The primary contribution of this work is the design and implementation of a computationally efficient deep learning model that improves the accuracy of saliency maps, particularly focusing on boundaries and spatial coherence.

The authors identify several limitations in existing CNN-based saliency detection methods, which include operational inefficiencies and the production of blurry saliency maps, especially near boundaries. Traditional models process at the patch level, leading to cumbersome computations and storage requirements due to redundancy among overlapping patches.

To address these issues, the authors propose an architecture that comprises two main streams:

1. Pixel-level Fully Convolutional Stream:
   • Utilizes a multi-scale fully convolutional network (MS-FCN) which processes the input image to produce a saliency map at pixel-level accuracy.
   • The MS-FCN is designed to generate semantic features across different scales while maintaining computational efficiency.
   • The convolutional network skips subsampling in the last two pooling layers and uses the "hole algorithm" (a trous algorithm) for larger receptive fields without increased computational cost.
2. Segment-wise Spatial Pooling Stream:
   • Operates on superpixels (segments) of the image.
   • Efficiently extracts features by masking segment areas on feature maps and performing spatial pooling over these segments.
   • This stream complements the fully convolutional stream by modeling visual contrast between regions and along boundaries.

A key enhancement of this approach is its ability to merge outputs from both streams to produce a refined saliency map. Additionally, the authors incorporated a fully connected Conditional Random Field (CRF) to further enhance spatial coherence and contour localization of the resulting saliency maps.

Numerical Results and Evaluations

The paper thoroughly evaluates the proposed method against recent state-of-the-art techniques using several benchmark datasets including MSRA-B, HKU-IS, DUT-OMRON, PASCAL-S, and SOD. The evaluation metrics include precision-recall (PR) curves, maximum F-measure (maxF), and mean absolute error (MAE).

The results indicate substantial improvements:

• On the MSRA-B dataset, the method achieved a maxF of 0.916 and an MAE of 0.047.
• Across various datasets, the proposed model demonstrated higher precision and recall, resulting in consistently better PR curves compared to other methods.
• Specifically, in more challenging datasets like DUT-OMRON, the method outperformed others by notable margins in both maxF and MAE metrics, emphasizing its robustness.

One important insight is the effect of incorporating CRFs, which led to an overall improvement in spatial coherence as observed in the enhanced saliency maps. The fusion of multi-scale convolutional network outputs and segment-wise pooling demonstrated complementary benefits, particularly in the precise detection of boundaries of salient objects.

Implications and Future Work

Practically, the implications of this research are significant for tasks such as object detection, image editing, and video summarization where accurate identification of salient regions is crucial. The proposed deep contrast network promises enhanced efficiency by reducing redundant computations and storage.

Theoretically, this work paves the way for further research into integrating multi-scale fully convolutional networks with region-based methods. Future developments could explore the scalability of this approach to other pixel-labeling tasks beyond saliency detection, including semantic segmentation and optical flow estimation.

Expanding on this research, future work may delve into improving the efficiency of training processes and exploring the utility of larger, more diverse datasets for fine-tuning. Additionally, leveraging advancements in hardware acceleration can further reduce inference times, making it more applicable for real-time applications.

In conclusion, the paper by Li and Yu represents a significant advancement in the domain of salient object detection by proposing a synergized approach combining pixel-level and segment-level insights efficiently, leading to better precision, recall, and spatial coherence in saliency maps. This paper is instrumental for researchers looking to build on contemporary methods and push the boundaries of visual saliency detection.