Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (2110.06530v1)

Abstract: Weakly supervised semantic segmentation produces pixel-level localization from class labels; however, a classifier trained on such labels is likely to focus on a small discriminative region of the target object. We interpret this phenomenon using the information bottleneck principle: the final layer of a deep neural network, activated by the sigmoid or softmax activation functions, causes an information bottleneck, and as a result, only a subset of the task-relevant information is passed on to the output. We first support this argument through a simulated toy experiment and then propose a method to reduce the information bottleneck by removing the last activation function. In addition, we introduce a new pooling method that further encourages the transmission of information from non-discriminative regions to the classification. Our experimental evaluations demonstrate that this simple modification significantly improves the quality of localization maps on both the PASCAL VOC 2012 and MS COCO 2014 datasets, exhibiting a new state-of-the-art performance for weakly supervised semantic segmentation. The code is available at: https://github.com/jbeomlee93/RIB.

• The paper addresses the information bottleneck in neural networks to improve weakly supervised semantic segmentation using image-level labels.
• It proposes removing the final layer activation function and introducing Global Non-Discriminative Region Pooling (GNDRP) to capture broader object regions.
• Experiments on PASCAL VOC 2012 and MS COCO 2014 demonstrate state-of-the-art performance, making the method valuable for tasks with limited annotations.

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation

Introduction

The paper "Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation" presents a nuanced approach to improving semantic segmentation under weak supervision, specifically focusing on the use of image-level class labels. It is well-recognized in the domain that weakly supervised methods, while offering ease of data annotation compared to fully supervised approaches, face significant challenges in achieving precise pixel-level segmentation. A critical issue identified in this paper is the problem of classifiers focusing disproportionately on small discriminative regions of target objects due to the information bottleneck at the final layers of neural networks.

Proposed Method

The authors explore the information bottleneck theory to analyze how information is compressed across the layers of a deep neural network (DNN). They observe that the final layer of a network activates using saturating functions such as sigmoid or softmax, leading to significant information bottleneck effects. To alleviate this, the authors propose removing the final activation function during training. This seemingly simple modification ensures a broader range of information, including non-discriminative but relevant regions of the target object, is preserved and utilized in the production of class activation mappings (CAMs).

Furthermore, the paper introduces a novel pooling method referred to as Global Non-Discriminative Region Pooling (GNDRP). This pooling mechanism selectively enhances features from less discriminative regions, ensuring a more comprehensive object region identification in the final segmentation maps.

Experimental Results

Extensive experiments are conducted on the PASCAL VOC 2012 and MS COCO 2014 datasets. The results show significant improvements in the quality of the generated localization maps, with the approach reaching new state-of-the-art performances. On the validation and test datasets of PASCAL VOC 2012, the proposed method achieves mean Intersection over Union (mIoU) gains, illustrating the effectiveness of their strategy in overcoming limitations associated with traditional CAM-based methods.

Theoretical and Practical Implications

Theoretically, this paper underscores the importance of considering neural network information flow characteristics - particularly the adverse effects of information bottleneck in the final layers. By revisiting activation functions traditionally applied in classification networks' final layers, this paper foregrounds a path to enriching information propagation and representation within weakly supervised learning frameworks.

Practically, this work offers an easily implementable tweak to existing models, making weak supervision more viable without significant additional computational overheads. As semantic segmentation tasks extend to various applications such as medical image analysis, autonomous vehicles, and robotics, the approach proposed in this paper provides a valuable tool to enhance the quality and applicability of models trained with limited annotations.

Conclusions and Future Directions

This research contributes a critical perspective on addressing the laser-focused attention of classifiers by mitigating the information bottleneck effect without necessitating exhaustive pixel-level annotations. Future works may explore the alignment of such weakly supervised techniques with emerging paradigms like self-supervised learning and investigate their applicability across other domains requiring semantic understanding from sparse annotations. The interplay between model explainability and performance, especially under different forms of weak supervision, could further extend the insights garnered from this paper.

In conclusion, the paper presents a pragmatic contribution to the field of computer vision, advancing the potential and reliability of weakly supervised semantic segmentation through an insightful amalgamation of theoretical analysis and practical innovation.