HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection (2406.19394v1)

Abstract: Most WSOD methods rely on traditional object proposals to generate candidate regions and are confronted with unstable training, which easily gets stuck in a poor local optimum. In this paper, we introduce a unified, high-capacity weakly supervised object detection (WSOD) network called HUWSOD, which utilizes a comprehensive self-training framework without needing external modules or additional supervision. HUWSOD innovatively incorporates a self-supervised proposal generator and an autoencoder proposal generator with a multi-rate resampling pyramid to replace traditional object proposals, enabling end-to-end WSOD training and inference. Additionally, we implement a holistic self-training scheme that refines detection scores and coordinates through step-wise entropy minimization and consistency-constraint regularization, ensuring consistent predictions across stochastic augmentations of the same image. Extensive experiments on PASCAL VOC and MS COCO demonstrate that HUWSOD competes with state-of-the-art WSOD methods, eliminating the need for offline proposals and additional data. The peak performance of HUWSOD approaches that of fully-supervised Faster R-CNN. Our findings also indicate that randomly initialized boxes, although significantly different from well-designed offline object proposals, are effective for WSOD training.

• The paper introduces HUWSOD, a unified framework for Weakly Supervised Object Detection using holistic self-training, eliminating the need for external object proposal modules.
• HUWSOD features a unified network with self-supervised and autoencoder object proposal generators (SSOPG, AEOPG) and a Multi-rate Resampling Pyramid (MRRP) for multi-scale context.
• The framework employs a holistic self-training scheme with Step-wise Entropy Minimization (SEM) and Consistency-constraint Regularization (CCR), achieving competitive performance on PASCAL VOC and MS COCO without additional data.

The paper "HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection" introduces an integrated framework for Weakly Supervised Object Detection (WSOD) that leverages a holistic self-training approach to overcome the reliance on external object proposal modules. The proposed HUWSOD framework innovates in two primary dimensions: a unified network structure and a holistic self-training scheme. These advancements address the limitations observed in traditional methods that depend heavily on offline object proposal techniques, which are inefficient and prone to local optimization traps.

Unified Network Structure

1. Object Proposal Generators:
   • Self-supervised Object Proposal Generator (SSOPG): This module employs a self-learning approach where object proposals are generated based on the predictions of the WSOD model itself, without external supervision or modules. It fundamentally implements a small fully convolutional network to map image feature maps to objectness scores and coordinate proposals.
   • Autoencoder Object Proposal Generator (AEOPG): Operating in an unsupervised manner, AEOPG focuses on salient object detection by conducting a low-rank approximation of feature maps. This module uses an encoder-decoder architecture to project significant feature characteristics into a compressed representation, providing crucial insights into object locations.
2. Multi-rate Resampling Pyramid (MRRP):
   • The MRRP aggregates multi-scale contextual information by incorporating various dilation rates within shared backbone parameters. This novel feature hierarchy efficiently exploits diverse spatial contexts for feature representation, significantly enhancing the model's capacity to address scale variations without parameter crowding.

Holistic Self-training Scheme

1. Step-wise Entropy Minimization (SEM):
   • SEM progressively reduces classification entropy, refining detection scores and bounding-box coordinates in a structured pipeline. By optimizing at multiple IoU thresholds with the number of instance refinement branches, SEM balances the trade-off between precision and recall effectively at each step.
2. Consistency-constraint Regularization (CCR):
   • CCR enforces prediction consistency across stochastic augmentations, employing a consistency regularization strategy that harmonizes outputs of different transformation scenarios. The integration of exponential moving average schemes within branches further enhances the transfer and propagation of knowledge throughout the network, thereby fortifying WSOD predictions with internal regularities.

Experimental Results and Observations

The comprehensive experiments across PASCAL VOC and MS COCO datasets highlight HUWSOD's competitive performance against state-of-the-art WSOD methods without needing additional data or external modules. Key findings and results include:

• The framework eliminates the historical dependency on externally generated proposals, achieving superior localization (as measured by CorLoc) and detection accuracy (in terms of mAP).
• Significant improvements were shown in fully-supervised settings, approaching the performance parity with methods like Faster R-CNN.
• It reports consistent AP metrics and further underscores the effectiveness of harnessing learnable proposal generators and self-training schemes for WSOD tasks.

Conclusion and Future Directions

HUWSOD shows substantial promise in enhancing WSOD methodologies through self-contained, unified detection frameworks. Future research directions suggest refining object proposal learning mechanisms by embedding traditional proposal wisdom into end-to-end trainable modules and exploring comprehensive augmentation strategies to optimize consistency regularization aspects further. The approach marks a notable progression in the field of computer vision, reducing the gap towards achieving FSOD-level efficacy and scalability.