Group R-CNN for Weakly Semi-supervised Object Detection with Points (2205.05920v1)

Abstract: We study the problem of weakly semi-supervised object detection with points (WSSOD-P), where the training data is combined by a small set of fully annotated images with bounding boxes and a large set of weakly-labeled images with only a single point annotated for each instance. The core of this task is to train a point-to-box regressor on well-labeled images that can be used to predict credible bounding boxes for each point annotation. We challenge the prior belief that existing CNN-based detectors are not compatible with this task. Based on the classic R-CNN architecture, we propose an effective point-to-box regressor: Group R-CNN. Group R-CNN first uses instance-level proposal grouping to generate a group of proposals for each point annotation and thus can obtain a high recall rate. To better distinguish different instances and improve precision, we propose instance-level proposal assignment to replace the vanilla assignment strategy adopted in the original R-CNN methods. As naive instance-level assignment brings converging difficulty, we propose instance-aware representation learning which consists of instance-aware feature enhancement and instance-aware parameter generation to overcome this issue. Comprehensive experiments on the MS-COCO benchmark demonstrate the effectiveness of our method. Specifically, Group R-CNN significantly outperforms the prior method Point DETR by 3.9 mAP with 5% well-labeled images, which is the most challenging scenario. The source code can be found at https://github.com/jshilong/GroupRCNN

• The paper presents a novel point-to-box regressor that leverages grouped proposals around single-point annotations.
• It employs instance-level proposal assignment and instance-aware representation learning to enhance accuracy in crowded scenes.
• The method achieves a 3.9 mAP improvement on MS-COCO with just 5% fully labeled data, underscoring its cost-effective potential.

Insightful Overview of "Group R-CNN for Weakly Semi-supervised Object Detection with Points"

The paper, "Group R-CNN for Weakly Semi-supervised Object Detection with Points," addresses the problem of weakly semi-supervised object detection enhanced by point annotations (WSSOD-P). The research focuses on a scenario where the training data comprises a small subset of fully annotated images with bounding boxes and a larger subset of weakly-labeled images annotated only with a single point per instance. The central thrust of this investigation is the development of the Group R-CNN, a CNN-based architecture, counters the assertion that CNN detectors are unsuitable for translating point annotations into bounding boxes efficiently within this context.

Key Contributions

The authors propose a novel point-to-box regressor, Group R-CNN, which builds on the standard R-CNN architecture featuring several core innovations, including:

1. Instance-level Proposal Grouping: By aggregating proposals generated by feature points surrounding a given annotation point, this method enhances the recall rate and robustness of the detector against annotation inaccuracies.
2. Instance-level Proposal Assignment: Unlike the conventional strategy, the proposed assignment method ensures that proposals are assigned exclusively to their relevant instance, thus improving precision in crowded scenes.
3. Instance-aware Representation Learning: This proposal resolves convergence issues by incorporating both instance-aware feature enhancement and parameter generation using relative coordinates and category embeddings.

These key elements contribute to the substantial performance increase demonstrated by Group R-CNN compared to its predecessor, Point DETR. The implementation results in a 3.9 mAP improvement when using only 5% well-labeled data on the MS-COCO dataset, illustrating marked improvements in object localization accuracy particularly in the low data scenarios.

Implications and Future Research Directions

The implications of this paper are far-reaching both practically and theoretically. On a practical level, Group R-CNN provides a more cost-effective solution to object detection by minimizing the dependency on costly bounding box annotations while maintaining a high level of accuracy. Theoretically, the introduction of instance-aware representation learning may inspire further research into dynamic parameter adaptation in deep learning networks, particularly in semi-supervised and weakly-supervised domains.

Given the promising results demonstrated in this paper, future research might explore integrating more advanced semi-supervised methods, thereby possibly enhancing performance further. Additionally, the adaptation of this framework to other weakly-annotated structures beyond points could extend its utility across different computer vision tasks.

In conclusion, while Group R-CNN delivers a commendable stride in the WSSOD-P landscape by efficiently leveraging minimal annotation data, it simultaneously opens new avenues for research aimed at reducing computational costs and improving the practicality of deploying advanced object detection systems in real-world scenarios.