Webly Supervised Learning of Convolutional Networks (1505.01554v2)

Abstract: We present an approach to utilize large amounts of web data for learning CNNs. Specifically inspired by curriculum learning, we present a two-step approach for CNN training. First, we use easy images to train an initial visual representation. We then use this initial CNN and adapt it to harder, more realistic images by leveraging the structure of data and categories. We demonstrate that our two-stage CNN outperforms a fine-tuned CNN trained on ImageNet on Pascal VOC 2012. We also demonstrate the strength of webly supervised learning by localizing objects in web images and training a R-CNN style detector. It achieves the best performance on VOC 2007 where no VOC training data is used. Finally, we show our approach is quite robust to noise and performs comparably even when we use image search results from March 2013 (pre-CNN image search era).

• The paper presents a two-stage framework that first trains on easy search engine images before adapting to complex Flickr photos.
• It uses a relationship graph during fine-tuning to mitigate the impact of noisy labels and enhance feature robustness on VOC and MIT datasets.
• Empirical results show that webly supervised CNNs can match or outperform traditional ImageNet-pretrained models, offering a scalable alternative to manual labeling.

Webly Supervised Learning of Convolutional Networks: An Overview

The paper "Webly Supervised Learning of Convolutional Networks" by Xinlei Chen and Abhinav Gupta presents a novel approach for training Convolutional Neural Networks (CNNs) using vast amounts of web data without requiring manual labeling. The methodology is inspired by curriculum learning and comprises a two-stage training strategy. This research aims to leverage the structure of data to improve performance on vision tasks such as object detection and scene classification without heavy reliance on manually labeled datasets like ImageNet.

Key Contributions and Methodology

The research introduces a two-stage, webly supervised learning framework that involves:

1. Initial Network Training: The initial CNN is trained on "easy" images sourced from search engines like Google. These images typically exhibit a high signal-to-noise ratio and are often bias towards iconic representations of objects, thus facilitating an effective start for the CNN learning procedure.
2. Representation Adaptation with Harder Images: The initially trained CNN is then adapted to handle more complex and realistic images obtained from photo-sharing websites like Flickr. This stage incorporates a relationship graph constructed from initial training data to guide and constrain the fine-tuning process. The graph captures inter-category visual similarities, improving robustness to noisy labels in the Flickr dataset.

The paper reports that this two-stage CNN surpasses the performance of a fine-tuned CNN trained solely on ImageNet, particularly on the PASCAL VOC 2012 dataset. Additionally, an object localization pipeline based on the CNN's learned features allows for effective training of object detectors without using any ground-truth bounding box annotations.

Empirical Findings

• No VOC Fine-Tuning: The webly supervised CNN achieves competitive performance to ImageNet-trained models on the VOC 2007 dataset without any PASCAL training data, achieving a mean Average Precision (mAP) of 44.7%.
• With VOC Fine-Tuning: When fine-tuned on the PASCAL data, the webly supervised CNNs perform on par with, or superior to, ImageNet-pretrained models, particularly for datasets such as VOC 2012.
• Scene Classification: The research extends beyond object detection to scene classification on the MIT Indoor-67 dataset, demonstrating the applicability of webly learned features to different vision tasks.

Implications and Future Prospects

This work implicates a shift towards leveraging publicly available web data in place of expensive manual annotation processes traditionally employed in computer vision. The demonstrated ability of CNNs to excel using webly sourced training data suggests potential scalability to billions of images without human supervision.

Future research could explore the dynamic updating of the relationship graph as additional data becomes available, enhancing the adaptability and generalization of the learned representations. The integration of semantic understanding and context-aware object detection can be augmented further by leveraging advanced neural architectures that continuously learn from evolving web data landscapes.

In conclusion, this paper presents a compelling case for webly supervised learning as an alternative paradigm in vision research. The proposed methodology efficiently harnesses the inherently rich and diverse web data, underscoring its capability to match the traditional, heavily human-supervised learning frameworks in computer vision tasks.