Simple Does It: Weakly Supervised Instance and Semantic Segmentation

Abstract: Semantic labelling and instance segmentation are two tasks that require particularly costly annotations. Starting from weak supervision in the form of bounding box detection annotations, we propose a new approach that does not require modification of the segmentation training procedure. We show that when carefully designing the input labels from given bounding boxes, even a single round of training is enough to improve over previously reported weakly supervised results. Overall, our weak supervision approach reaches ~95% of the quality of the fully supervised model, both for semantic labelling and instance segmentation.

• The paper introduces a recursive training approach that iteratively refines bounding box annotations, achieving nearly 95% of fully supervised performance.
• It combines classic methods like GrabCut with CNN predictions to enhance initial labels and generate high-quality segmentation masks.
• The study demonstrates that leveraging inexpensive bounding box annotations can effectively bridge the gap to costly pixel-wise labeling in segmentation tasks.

Simple Does It: Weakly Supervised Instance and Semantic Segmentation

The paper "Simple Does It: Weakly Supervised Instance and Semantic Segmentation" by Anna Khoreva et al. addresses the significant challenge of reducing the annotation cost for tasks like semantic labeling and instance segmentation. The authors propose a methodology that utilizes readily available bounding box annotations to achieve high-quality results comparable to fully supervised models, thereby reducing reliance on costly pixel-wise annotations.

Key Contributions

The paper elucidates the following primary contributions:

1. Recursive Training Approach: A method for weakly supervised semantic labeling using bounding box annotations, which iteratively refines the annotations through recursive training rounds.
2. Box-driven Segmentation Techniques: Enhancement of initial bounding box labels using algorithms such as GrabCut and MCG, to provide more precise input segments for the training process.
3. High-Quality Results with Weak Supervision: The proposed methodology achieves approximately 95% of the accuracy of fully supervised models using only bounding box annotations.
4. Firsts in Weakly Supervised Instance Segmentation: Introducing and achieving competitive results in the domain of weakly supervised instance segmentation using proposed techniques.

Methodology Overview

Recursive Training with Bounding Boxes

The authors first explore a naive baseline where the entirety of each bounding box is labeled as the object. While trivial, this initialization is enhanced through a recursive training regimen where the object labels are iteratively refined using the predictions of a convolutional neural network (CNN) from prior rounds. This recursive process, combined with techniques to reinforce minimal quality controls like area constraints and leveraging spatial continuity priors, reaches substantial quality improvement over naive methods.

Figure 1 and Table 1 detail the notable improvement as a result of this iterative denoising approach. Using this method alone, significant performance metrics are achieved, showcasing the inherent robustness of recursive training in refining noisy label inputs.

Box-driven Segmentation

The methodology further explores leveraging classic computer vision techniques to improve upon the baseline recursive training. By employing the GrabCut algorithm and its variants, including a modification termed GrabCut+, which uses a boundary detector, the initial segmentation quality sees marked improvement. Furthermore, combining GrabCut+ with MCG segments (denoted M ∩ G+), where regions of concordance between methods provide higher confidence labels, allows achieving state-of-the-art results in weak supervision.

Compared to existing methodologies such as BoxSup and WSSL, which modify the training procedure or classification architecture, the proposed method keeps the same training pipeline as a fully-supervised scenario and instead enhances the input labels before training. This approach brings forth simplicity and avoids procedural complexities while capitalizing on well-designed input label generation.

Results

The results are compelling. As depicted in Tables 2 and 3, this methodology achieves about 95% of the performance of fully supervised models, when tested on the Pascal VOC 2012 dataset, and similarly competitive results using additional COCO data. Furthermore, the approach also shows promise in the semi-supervised paradigm, where partial pixel-wise annotations are available alongside numerous bounding boxes.

The approach extends beyond semantic labeling, presenting the first reported instance segmentation results under weak supervision. Here, the results again show that models like DeepMask, trained using the refined bounding box-driven segments, attain performance levels close to fully supervised training, thereby setting a notable precedent in the field.

Implications and Future Directions

This research posits significant practical implications by demonstrating that bounding box annotations, which are significantly cheaper and more ubiquitous, can be effectively repurposed to train high-performance semantic labeling and instance segmentation models. It highlights a pragmatic route to mitigate the extensive resource investments typically required for comprehensive pixel-wise annotations.

Future steps could include exploring co-segmentation strategies, where multiple images are jointly analyzed to propagate annotation refinements across datasets. Additionally, exploring even weaker forms of supervision, such as image-level labels or sparse annotations, could unlock further efficiencies.

Overall, this research illustrates that methodically leveraging and denoising simpler annotations can bridge the performance gap with more complex and expensive fully supervised approaches, driving impactful advancements in the usability and accessibility of computer vision technologies.