Automatically Discovering and Learning New Visual Categories with Ranking Statistics (2002.05714v1)

Abstract: We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. This setting is similar to semi-supervised learning, but significantly harder because there are no labelled examples for the new classes. The challenge, then, is to leverage the information contained in the labelled images in order to learn a general-purpose clustering model and use the latter to identify the new classes in the unlabelled data. In this work we address this problem by combining three ideas: (1) we suggest that the common approach of bootstrapping an image representation using the labeled data only introduces an unwanted bias, and that this can be avoided by using self-supervised learning to train the representation from scratch on the union of labelled and unlabelled data; (2) we use rank statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images; and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data. We evaluate our approach on standard classification benchmarks and outperform current methods for novel category discovery by a significant margin.

• The paper addresses novel category discovery in image datasets by leveraging labelled data without examples for new classes, aiming to create a general clustering model.
• A proposed approach combines self-supervised learning for bias reduction, rank statistics for knowledge transfer via pseudo-labels, and a joint objective function for unified optimization.
• Numerical results demonstrate significant performance improvements on benchmarks like CIFAR-10 and ImageNet, advancing capabilities for discovering novel classes in real-world dynamic datasets.

Automatically Discovering and Learning New Visual Categories with Ranking Statistics

In the presented research, the authors address the critical challenge of discovering novel classes in an image dataset where labelled examples exist only for other classes. This problem sits beyond the boundaries of traditional semi-supervised learning (SSL) due to the complete absence of labelled examples for the new, undiscovered classes. The goal is to leverage the labelled data to create a general-purpose clustering model that can identify and learn the new classes from unlabelled data.

The authors propose an approach combining three key ideas:

1. Self-Supervised Learning for Representation Bias Reduction: The paper suggests that reliance solely on labelled data for bootstrapping a model inevitably introduces bias. Instead, the authors advocate for the model's image representation (usually a Convolutional Neural Network, CNN) to be initially trained with self-supervised learning. This method utilizes both the labelled and unlabelled data, effectively mitigating bias toward the known classes and yielding a more generalized feature representation.
2. Transfer Learning through Rank Statistics: The proposed method employs rank statistics to transfer knowledge from labelled data to unlabelled data efficiently. This is operationalized by comparing pairs of unlabelled images using their representation vectors. Robust rank statistics determine whether two images activate the same subset of top representation components, consequently identifying if they belong to the same class. This mechanism generates pseudo-labels that facilitate learning a similarity function for unlabelled data.
3. Joint Objective Function Optimization: To leverage both the labelled and unlabelled data, the authors propose a joint objective function. This function optimizes both the supervised classification on the labelled subset and the clustering of unlabelled data. Such an approach prevents the model from 'forgetting' previously learned knowledge by integrating new class information dynamically throughout the learning process.

Numerical results on several standard benchmarks, including CIFAR-10, CIFAR-100, and ImageNet, demonstrate significant improvements in performance over existing methods. The paper showcases how the integration of consistency constraints, BCE loss for unlabelled data, and a self-supervised learning baseline contribute to these improvements. The proposed technique outperforms state-of-the-art methods in novel category discovery tasks, as indicated by higher ACC (accuracy) scores across datasets.

Implications and Future Directions

This research implies substantial advancements in machine learning, particularly in tasks with dynamic and unlabeled datasets. In practical terms, this could revolutionize how novel classes are discovered in real-world applications like retail, surveillance, or autonomous driving, where new object categories frequently emerge.

Theoretically, this work paves the way for more sophisticated AI systems that can seamlessly extend their understanding to new categories without explicit re-training on annotated data. Further research could investigate refining self-supervised objectives or explore alternative ranking strategies and their efficacy under various conditions.

Overall, the insights from this research contribute to the broader narrative of creating more autonomous, adaptable machine learning systems capable of overcoming one of the core challenges in AI: learning from unstructured, unlabeled environments.