Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning (1706.08653v2)

Published 27 Jun 2017 in cs.CV

Abstract: Prevalent techniques in zero-shot learning do not generalize well to other related problem scenarios. Here, we present a unified approach for conventional zero-shot, generalized zero-shot and few-shot learning problems. Our approach is based on a novel Class Adapting Principal Directions (CAPD) concept that allows multiple embeddings of image features into a semantic space. Given an image, our method produces one principal direction for each seen class. Then, it learns how to combine these directions to obtain the principal direction for each unseen class such that the CAPD of the test image is aligned with the semantic embedding of the true class, and opposite to the other classes. This allows efficient and class-adaptive information transfer from seen to unseen classes. In addition, we propose an automatic process for selection of the most useful seen classes for each unseen class to achieve robustness in zero-shot learning. Our method can update the unseen CAPD taking the advantages of few unseen images to work in a few-shot learning scenario. Furthermore, our method can generalize the seen CAPDs by estimating seen-unseen diversity that significantly improves the performance of generalized zero-shot learning. Our extensive evaluations demonstrate that the proposed approach consistently achieves superior performance in zero-shot, generalized zero-shot and few/one-shot learning problems.

User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Shafin Rahman (38 papers)
  2. Salman H. Khan (17 papers)
  3. Fatih Porikli (141 papers)
Citations (162)
View on Semantic Scholar

Summary

Unified Approach for Zero-shot, Generalized Zero-shot, and Few-shot Learning

The paper introduces a comprehensive approach addressing the challenges of conventional zero-shot learning (ZSL), generalized zero-shot learning (GZSL), and few-shot learning (FSL) by proposing the Class Adapting Principal Directions (CAPD) framework. This framework aims to bridge the gap between seen and unseen classes in visual understanding tasks.

Key Contributions and Methodology

  1. Class Adapting Principal Directions (CAPD): At the core of the paper is the concept of CAPD, which provides a mechanism to transfer information from seen classes to unseen classes efficiently. CAPD involves generating principal directions from class-specific discriminative models for seen classes. These directions are then combined to create principal directions for unseen classes, facilitating a robust embedding in the semantic space.
  2. Semantic Space and Metric Learning: The authors emphasize the importance of learning a metric in the semantic space, particularly when dealing with noisy or unsupervised semantic embeddings such as those derived from word2vec or GloVe. By modeling the relationships between the visual features and semantics of seen classes, the framework can better approximate the semantic embeddings of unseen classes, leveraging a learned distance metric.
  3. Reduced Set Description for Unseen Classes: By proposing a strategy to automatically select the most relevant seen classes for describing unseen classes, the framework introduces sparsity, resulting in significant performance improvements. This selection relies on the semantic relationship among classes, thus reducing the complexity involved.
  4. Generalized Zero-shot Learning Solution: Addressing the inherent bias towards seen classes in GZSL scenarios, the authors propose balancing seen-unseen diversity without relying on direct image feature supervision. This allows for a balanced and unprejudiced prediction mechanism when both seen and unseen classes are present during testing.
  5. Few-shot Learning Adaptation: Extending the framework to FSL settings, the authors provide a mechanism for updating unseen CAPDs using a few labeled examples. This iterative update process involves combining CAPDs derived from seen classes with those generated by a few-shot classifier, thus refining the prediction for unseen classes.

Experimental Validation

The paper validates its approach through extensive experiments on multiple benchmark datasets, demonstrating consistent superior performance across various settings:

  • ZSL and GZSL: The framework not only maintains high accuracy in standard ZSL but also shows enhanced performance in more complex GZSL scenarios by reducing the bias towards seen classes and improving harmonic mean scores.
  • FSL: The framework effectively utilizes limited labeled instances of unseen classes, yielding improved accuracies in FSL settings and minimizing the performance gap between unsupervised and supervised semantic embeddings.

Implications and Future Directions

The paper's unified approach to zero-shot, generalized zero-shot, and few-shot learning offers practical implications for the development of more adaptable machine learning models in real-world applications. By leveraging semantic embeddings and demonstrating adaptability across several contexts, the CAPD framework sets a foundation for future advancements in visual object classification without excessive reliance on labeled data.

Furthermore, the research suggests avenues for extending this framework to transductive learning and domain adaptation tasks, promising tools for handling evolving data and class distributions in dynamic environments.

In conclusion, by presenting a modular solution adaptable to various learning scenarios, this paper provides significant insights and methodologies contributing to the landscape of visual recognition tasks, encouraging further exploration and application of CAPD in more diverse and challenging settings.