Invariant Information Clustering for Unsupervised Image Classification and Segmentation

Abstract: We present a novel clustering objective that learns a neural network classifier from scratch, given only unlabelled data samples. The model discovers clusters that accurately match semantic classes, achieving state-of-the-art results in eight unsupervised clustering benchmarks spanning image classification and segmentation. These include STL10, an unsupervised variant of ImageNet, and CIFAR10, where we significantly beat the accuracy of our closest competitors by 6.6 and 9.5 absolute percentage points respectively. The method is not specialised to computer vision and operates on any paired dataset samples; in our experiments we use random transforms to obtain a pair from each image. The trained network directly outputs semantic labels, rather than high dimensional representations that need external processing to be usable for semantic clustering. The objective is simply to maximise mutual information between the class assignments of each pair. It is easy to implement and rigorously grounded in information theory, meaning we effortlessly avoid degenerate solutions that other clustering methods are susceptible to. In addition to the fully unsupervised mode, we also test two semi-supervised settings. The first achieves 88.8% accuracy on STL10 classification, setting a new global state-of-the-art over all existing methods (whether supervised, semi-supervised or unsupervised). The second shows robustness to 90% reductions in label coverage, of relevance to applications that wish to make use of small amounts of labels. github.com/xu-ji/IIC

• The paper introduces IIC, a novel method that maximizes mutual information to achieve state-of-the-art unsupervised image classification and segmentation.
• Its approach avoids clustering degeneracy by using entropy maximization and an auxiliary overclustering head for balanced semantic assignments.
• IIC demonstrates versatility by excelling in both unsupervised and semi-supervised tasks on benchmarks like STL10 and CIFAR10 with significant performance gains.

Invariant Information Clustering for Unsupervised Image Classification and Segmentation

The paper presents a novel approach termed Invariant Information Clustering (IIC) that addresses the challenges of unsupervised image classification and segmentation by leveraging mutual information as the primary objective function. IIC is grounded in statistical learning theory and operates by training a neural network to maximize the mutual information between paired samples, which are generated through random transformations of input images. This method shows significant improvements over existing techniques and establishes new benchmarks in various unsupervised learning tasks.

Key Contributions

The paper introduces IIC as a robust clustering algorithm that excels in multiple benchmarks for unsupervised tasks across datasets such as STL10, CIFAR10, and MNIST. Key aspects of this approach include:

1. Clustering Objective: IIC trains a neural network from scratch to predict cluster identities with high fidelity to semantic classes by maximizing mutual information.
2. Mutual Information Maximization: The clustering objective of IIC does not suffer from the degeneracy issues seen in other methods, thanks to its intrinsic use of entropy maximization, which ensures balanced cluster assignments.
3. General Applicability: Though demonstrated primarily in the context of image data, IIC is a generic algorithm applicable to any paired dataset samples.
4. State-of-the-Art Performance: IIC sets new performance records on multiple unsupervised datasets with significant margins, particularly on STL10 and CIFAR10, where it surpasses the closest competitors by 6.6 and 9.5 percentage points, respectively.
5. Semantic Label Outputs: Unlike many existing methods that require post-processing of high-dimensional representations, IIC directly outputs semantic labels, simplifying the overall clustering pipeline.
6. Auxiliary Overclustering: To handle noisy data and enhance robustness, IIC employs an auxiliary head trained for an overclustering task, which aids in leveraging a larger context even with distractor classes in the data.

Experimental Findings

Image Clustering: The paper demonstrates substantial improvements in unsupervised image clustering, achieving 59.6% accuracy on STL10 and 61.7% on CIFAR10, significantly outperforming state-of-the-art methods such as DeepCluster and ADC. The method's resilience to noise and its capability to maintain balanced and meaningful clusters underscore its robustness.

Image Segmentation: When applied to image segmentation, IIC excels on datasets such as COCO-Stuff and Potsdam. For example, it achieves 72.3% accuracy on COCO-Stuff-3, exhibiting superior performance compared to various baselines. The method's convolutional implementation optimizes segmentation tasks efficiently, which is reflected in both the accuracy and computational speed.

Semi-supervised Learning: IIC demonstrates its versatility by excelling in semi-supervised scenarios as well, achieving the highest reported accuracy of 88.8% on STL10 even in settings with significant reductions in labeled data coverage.

Implications and Future Directions

The approach's rigorous grounding in information theory and its successful application to various imaging tasks suggest several theoretical and practical implications:

1. Theoretical: The use of mutual information as a primary clustering objective offers a principled way to avoid degenerate clustering solutions and could inspire further research into information-theoretic objectives for clustering and other unsupervised learning tasks.
2. Practical: The demonstrated success of IIC in handling both fully unsupervised and semi-supervised learning scenarios makes it a versatile tool for real-world applications where labeled data is sparse or expensive to obtain.
3. Future Developments: The method's generalizability to non-vision datasets presents intriguing opportunities for applying IIC to time-series data, social network analysis, and other domains where paired data structures are prevalent.

In conclusion, the Invariant Information Clustering approach provides a robust and efficient solution to unsupervised image classification and segmentation, notably advancing the state of the art in these tasks. With its theoretical rigor and practical efficacy, IIC represents a significant step forward in the unsupervised learning landscape.