Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion (1710.05174v1)

Abstract: Stereoscopic perception is an important part of human visual system that allows the brain to perceive depth. However, depth information has not been well explored in existing saliency detection models. In this letter, a novel saliency detection method for stereoscopic images is proposed. Firstly, we propose a measure to evaluate the reliability of depth map, and use it to reduce the influence of poor depth map on saliency detection. Then, the input image is represented as a graph, and the depth information is introduced into graph construction. After that, a new definition of compactness using color and depth cues is put forward to compute the compactness saliency map. In order to compensate the detection errors of compactness saliency when the salient regions have similar appearances with background, foreground saliency map is calculated based on depth-refined foreground seeds selection mechanism and multiple cues contrast. Finally, these two saliency maps are integrated into a final saliency map through weighted-sum method according to their importance. Experiments on two publicly available stereo datasets demonstrate that the proposed method performs better than other 10 state-of-the-art approaches.

Saliency Detection for Stereoscopic Images: An Examination of Depth Confidence and Cue Integration

The paper "Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion" presents a novel approach geared toward enhancing the precision of saliency detection in stereoscopic images. The focus of this work lies in utilizing the depth cues inherently present in 3D stereo images, which remain underexplored in conventional methods primarily centered on 2D RGB data.

The authors introduce a saliency detection model that significantly relies on two main contributions: the introduction of a depth confidence measure and the integration of multiple visual cues. Depth maps, essential for representing stereoscopic data, vary widely in quality. The developed depth confidence measure serves to assess and mitigate the influence of unreliable depth maps. This proposed metric is a calculated synthesis involving mean depth value, coefficient of variation, and an entropy-based assessment of depth distribution. This quantitative measure of depth reliability ensures more robust saliency predictions by discounting erroneous depth data.

On the methodological front, the proposed model initializes through a graph-based representation of the input image, partitioning it into superpixels using the SLIC method. The calculated affinity between these superpixels incorporates both color and depth differences, effectively utilizing the stereoscopic nature of the data. This graph representation then feeds into two innovative saliency calculations—compactness saliency and foreground saliency.

Compactness saliency is calculated using both color and depth cues, acknowledging that salient objects typically appear more spatially compact compared to their diffuse background counterparts. This method benefits from manifold ranking to propagate similarities across the graph and is further refined by incorporating an objectness measure, predicting the likelihood of superpixels belonging to distinct objects.

Foreground saliency is derived from a novel mechanism coined as Depth-Refined Foreground Seeds Selection (DRSS). This procedure integrates compactness and depth analyses to identify reliable foreground seeds, thus improving the distinguishing of salient features even when foreground and background share color similarities. By incorporating multiple cues such as color, depth, texture, and spatial positioning, this approach uses a manifold ranking scheme to propagate the computed saliency, ensuring coherent object segmentation.

Performance evaluation conducted on the NJU-400 and NJU-1985 datasets shows the proposed method outperforming ten state-of-the-art saliency detection techniques, both 2D and stereoscopic. This is demonstrated through precision-recall curves, F-measure, and MAE metrics. The model achieves notable enhancements in ensuring saliency maps align well with ground truth.

The implications of this research are twofold. Practically, the approach advances the state of the field in applications like object detection, image retrieval, and adaptive image compression, where precise object localization is paramount. Theoretically, it extends the understanding of integrating depth information into image processing tasks, which could influence future developments in AI-driven image recognition and manipulation.

The paper also opens up new avenues for future exploration, particularly in addressing scenarios where depth data quality significantly affects output results. Improved models incorporating sophisticated machine learning algorithms to predict and adaptively compensate for unreliable depth input are a potential area of development.

In summary, the integration of a depth confidence metric and multifaceted cue approach in stereoscopic saliency detection exemplifies a considerable advancement in leveraging 3D information for computational vision tasks. This work not only bolsters performance metrics over prior methodologies but also contributes vital insights into the handling and utilization of depth data for machine vision-based applications.