InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting (1908.07801v1)

Abstract: Instance segmentation requires a large number of training samples to achieve satisfactory performance and benefits from proper data augmentation. To enlarge the training set and increase the diversity, previous methods have investigated using data annotation from other domain (e.g. bbox, point) in a weakly supervised mechanism. In this paper, we present a simple, efficient and effective method to augment the training set using the existing instance mask annotations. Exploiting the pixel redundancy of the background, we are able to improve the performance of Mask R-CNN for 1.7 mAP on COCO dataset and 3.3 mAP on Pascal VOC dataset by simply introducing random jittering to objects. Furthermore, we propose a location probability map based approach to explore the feasible locations that objects can be placed based on local appearance similarity. With the guidance of such map, we boost the performance of R101-Mask R-CNN on instance segmentation from 35.7 mAP to 37.9 mAP without modifying the backbone or network structure. Our method is simple to implement and does not increase the computational complexity. It can be integrated into the training pipeline of any instance segmentation model without affecting the training and inference efficiency. Our code and models have been released at https://github.com/GothicAi/InstaBoost

• The paper introduces two InstaBoost methods—random and heatmap guided—that boost instance segmentation mAP by up to 2.2 on the COCO dataset.
• It employs probability map guidance to maintain appearance consistency, ensuring realistic augmentation with minimal computational overhead.
• Empirical results on COCO and Pascal VOC demonstrate robust improvements in both instance segmentation and object detection tasks.

Analyzing "InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting"

The paper entitled "InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting" introduces a novel approach to enhance instance segmentation performance by augmenting training data. Instance segmentation demands considerable annotated data, which is a limiting factor due to the labor-intensive nature of obtaining pixel-wise annotations. The authors propose two primary methods—random InstaBoost and appearance consistency heatmap guided InstaBoost—to create visually coherent augmented data for effective training.

Key Contributions

1. Random InstaBoost: This method applies random jittering to images by sampling transformations in proximity to the identity transform. The transformations include translation, scaling, and rotation, which are constrained to small ranges to preserve image coherence. This simple approach results in a 1.7 mAP improvement on Mask R-CNN using the COCO dataset. It harnesses the redundancy in background pixels, making it straightforward to implement within existing training pipelines.
2. Appearance Consistency Heatmap Guided InstaBoost: Building on the random InstaBoost, this method utilizes a heatmap to guide the transformation of object placement. The heatmap is generated based on local appearance similarities, enabling objects to be pasted in semantically consistent locations without introducing additional computational complexity. This method yields a 2.2 mAP improvement on COCO instance segmentation and manifests a more refined augmentation strategy that respects image context.

Methodology

The paper explores the potential of augmentation by separating instances from backgrounds using matting techniques, followed by inpainting for background completion. The similarity of textures near object boundaries is evaluated using an appearance consistency heatmap, ensuring augmented instances integrate seamlessly with their new contexts. By employing these methods, object placement remains consistent and realistic, bypassing the need for additional complex models.

Empirical Results

The authors conduct exhaustive experiments on the COCO and Pascal VOC datasets, leveraging the InstaBoost methods across different neural network architectures, such as Mask R-CNN with ResNet-FPN backbones and Cascade R-CNN. The performance improvement is notable not only in instance segmentation but also in object detection tasks. The results illustrate substantial augmentation benefits without inducing overfitting, as demonstrated by sustained performance gains across extended training epochs.

Implications and Future Directions

InstaBoost represents a practical augmentation strategy that can be readily integrated with existing instance segmentation frameworks. Its low computational demand and ability to work effectively on various models suggest robust adaptability and potential for widespread adoption. Future research could explore dynamic adjustment of transformation ranges or further enhancements to the heatmap to refine instance consistency further.

In conclusion, InstaBoost elevates the efficacy of instance segmentation by creatively exploiting annotated data, mitigating the constraints imposed by limited datasets. This methodology offers significant performance gains without necessitating modifications to the underlying model architecture, thus presenting a valuable tool for researchers in computer vision.