F3Net: Fusion, Feedback and Focus for Salient Object Detection (1911.11445v1)

Abstract: Most of existing salient object detection models have achieved great progress by aggregating multi-level features extracted from convolutional neural networks. However, because of the different receptive fields of different convolutional layers, there exists big differences between features generated by these layers. Common feature fusion strategies (addition or concatenation) ignore these differences and may cause suboptimal solutions. In this paper, we propose the F3Net to solve above problem, which mainly consists of cross feature module (CFM) and cascaded feedback decoder (CFD) trained by minimizing a new pixel position aware loss (PPA). Specifically, CFM aims to selectively aggregate multi-level features. Different from addition and concatenation, CFM adaptively selects complementary components from input features before fusion, which can effectively avoid introducing too much redundant information that may destroy the original features. Besides, CFD adopts a multi-stage feedback mechanism, where features closed to supervision will be introduced to the output of previous layers to supplement them and eliminate the differences between features. These refined features will go through multiple similar iterations before generating the final saliency maps. Furthermore, different from binary cross entropy, the proposed PPA loss doesn't treat pixels equally, which can synthesize the local structure information of a pixel to guide the network to focus more on local details. Hard pixels from boundaries or error-prone parts will be given more attention to emphasize their importance. F3Net is able to segment salient object regions accurately and provide clear local details. Comprehensive experiments on five benchmark datasets demonstrate that F3Net outperforms state-of-the-art approaches on six evaluation metrics.

• The paper introduces F3Net that enhances SOD by integrating novel modules like the CFM, CFD, and a pixel-aware loss to refine feature fusion and segmentation.
• The Cross Feature Module non-linearly fuses multi-level features to reduce redundancy and boost the complementarity of semantic and detailed cues.
• The cascaded feedback decoder and pixel position aware loss iteratively refine saliency maps, achieving superior performance on five benchmark datasets.

F$^3$Net: Fusion, Feedback and Focus for Salient Object Detection

The paper "F$^3$Net: Fusion, Feedback and Focus for Salient Object Detection" presents an advanced approach to improve Salient Object Detection (SOD) by addressing inherent challenges in feature fusion and detail enhancement. This method introduces novel techniques such as Cross Feature Module (CFM), Cascaded Feedback Decoder (CFD), and a Pixel Position Aware (PPA) loss, resulting in improved segmentation of salient regions with refined local details.

Key Contributions

1. Cross Feature Module (CFM): The CFM selectively aggregates multi-level features by employing a non-linear fusion strategy based on element-wise multiplication and addition. This mechanism reduces the interference of redundant information and enhances the complementarity between high-level semantic features and low-level detailed features. Unlike conventional fusion strategies like addition or concatenation, the CFM effectively mitigates discrepancies in features stemming from varied receptive fields.
2. Cascaded Feedback Decoder (CFD): This component supports iterative refinement of features through a feedback loop mechanism. By incorporating multi-stage feedback, the CFD enables a bottom-up and top-down feature propagation across multiple sub-decoders. Consequently, it enhances feature consistency and saliency map precision through iterative refinement, effectively correcting and enriching feature representations.
3. Pixel Position Aware Loss (PPA): To enhance boundary accuracy and tackle prediction inconsistencies on complex pixels, the PPA loss adapts conventional loss formulations by considering local structure information. This design assigns different weights to pixels based on their spatial context, focusing more on challenging regions like boundaries or areas prone to errors.

Experimental Results

The experimental validation of F$^3$Net was conducted on five benchmark datasets (ECSSD, PASCAL-S, DUTS-TE, HKU-IS, and DUT-OMRON), where it outperforms existing state-of-the-art models on six evaluation metrics, including mean absolute error (MAE), mean F-measure ($mF$), structural similarity measure ($S_\alpha$), and E-measure ($E_\xi$). Notably, F$^3$Net offers superior precision-recall performance, highlighting its robust capability in discerning salient object regions amidst complex backgrounds.

Implications and Future Directions

The integration of advanced feature fusion and feedback mechanisms in F$^3$Net provides a significant enhancement in SOD, particularly in environments with complex object boundaries and ambiguous contexts. The methodological innovations may serve as a foundation for addressing similar challenges across other computer vision tasks that require precise boundary delineation and semantic segmentation.

Looking forward, further exploration into the adaptive selection mechanisms for feature fusion could yield models that better generalize across diverse datasets. Additionally, the principles underlying PPA loss may inspire novel loss functions that effectively balance local and global information for improved model training. Extending the framework to accommodate dynamic input scales and aspect ratios without performance degradation remains a pertinent avenue for future research.

In summary, F$^3$Net introduces significant architectural and theoretical advancements that pave the way for more accurate and efficient salient object detection models, embodying meaningful progress in understanding and resolving key challenges in the field.