Zero-shot Learning: Methods and Challenges

• Zero-shot learning is a machine learning paradigm that recognizes unseen classes by leveraging auxiliary semantic information like attributes, word embeddings, or ontologies.
• It employs methods such as cross-modal embedding, GAN-based synthesis, and probabilistic approaches to bridge the gap between seen and unseen classes.
• ZSL addresses challenges like domain shift and data imbalance while expanding applications in computer vision, language processing, and beyond.

Zero-shot learning (ZSL) is a paradigm in machine learning and computer vision where the objective is to recognize classes (typically categories of objects, concepts, or relations) for which no instance-level training examples are available. Instead, ZSL enables recognition by leveraging auxiliary semantic information—such as human-defined attributes, word embeddings, textual descriptions, or formal ontologies—that serves to relate previously seen (source/“seen”) classes to those that are unseen during training. This transfer of knowledge is intrinsic to ZSL and stands in contrast to traditional supervised learning, which requires labeled data for every class of interest.

1. Theoretical Foundations and Problem Formulation

The canonical formulation of ZSL is a function $f: X \to Y$ that must generalize to a label space $Y = Y^{s} \cup Y^{u}$, where $Y^{s}$ and $Y^{u}$ denote the sets of seen and unseen classes, respectively, and $Y^{s} \cap Y^{u} = \varnothing$. Given a training set $\{(x_{i}, y_{i})\}_{i=1}^{N}, y_{i} \in Y^{s}$ and auxiliary information for all $Y$, ZSL aims to correctly assign a label from $Y^{u}$ to a test sample $x \in X$ sampled from an unseen class.

Auxiliary information $A$ may take the form of attribute vectors, semantic embeddings, or formal class definitions. Standard ZSL is evaluated in the “conventional” (CZSL) setting (test instances from $Y^{u}$ only) and the “generalized” (GZSL) setting (test instances may originate from $Y^{s} \cup Y^{u}$). A critical challenge in ZSL is that there is no data overlap between train and test classes, so successful models must induce transferable representations driven by higher-order semantic relationships.

2. Methodological Taxonomy

2.1. Embedding-Based Methods

Most ZSL methods employ a cross-modal embedding strategy—projecting samples in the visual feature space $X$ and semantic space $A$ into a joint or compatible space, with a distance or similarity function to enable classification. Examples include:

• Linear or nonlinear mapping-based approaches: Learning a regression or compatibility function mapping $X \to A$, $A \to X$, or both (e.g., ESZSL, SAE, SJE).
• Multi-modal fusion: MBFA-ZSL (Ji et al., 2016) formulates a unified embedding by simultaneously mapping visual features and multiple side information types (attributes, word vectors) into a shared latent space via Multi-Battery Factor Analysis. This approach exploits inter-modal covariances with a closed-form solution and supports efficient multi-modal fusion.
• Dictionary learning/coupled representations: Several works use sparsity-driven coupling between visual and semantic representations (see (Rostami et al., 2019, Jiang et al., 2018)), enforcing that both modalities share a sparse or low-rank code—yielding improved transfer.

2.2. Generative Model-Based Methods

Generative models (GANs, VAEs) are utilized to synthesize artificial features or images for unseen classes, effectively casting ZSL as a standard supervised problem after feature augmentation:

• GAN-based synthesis: Generating pseudo visual features conditioned on semantic descriptors (attribute vectors, enriched text, ontology embeddings). Augmentations such as model-specific attribute scoring for instance-level variability (Shohag et al., 18 Jun 2025), or knowledge sharing to enhance semantic input (Ting et al., 2021), improve synthesis fidelity and mitigate domain shift.
• Meta-learning frameworks: Meta-learning with sample synthesis, as in (Verma et al., 2019), constructs inner- and outer-loop training splits for WGANs to simulate the ZSL process and learns initializations that adapt rapidly to novel (unseen) classes.

2.3. Probabilistic and Bayesian Approaches

Probabilistic generative models, including hierarchical Bayesian frameworks (Badirli et al., 2019), estimate class distributions in the image feature space. Unseen class distributions are inferred using meta-classes—clusters of semantically similar seen classes—and posterior predictive distributions accommodate statistical uncertainty and facilitate a tunable trade-off between seen and unseen class accuracies.

2.4. Transductive and Manifold Transfer Strategies

Transductive methods utilize unlabelled test data from unseen classes. Approaches such as “missing data” ZSL (Zhao et al., 2016) transfer manifold structures from the semantic space to synthesize the data distribution (mean, covariance) of unseen classes, refine these parameters with EM algorithms in a GMM framework, and classify by likelihood. Other methods employ clustering or spectral graph analysis to align latent distributions between spaces, preserving topological and geometric relationships (Zhao et al., 2017, Gune et al., 2020).

2.5. Ontology and Structure-Enhanced Representations

Ontology-guided ZSL augments or replaces traditional side information with formal knowledge representations (e.g., OWL ontologies) that encode compositional, taxonomic, and relational semantics (Chen et al., 2020, Geng et al., 2021). Embedding strategies treat each class as a geometric or translation-embedded entity, with logical constraints formulated as loss functions to maintain subsumption and relational structure.

3. Multi-Modality, Semantic Enrichment, and Side Information

ZSL performance is contingent upon the expressivity and transferability of side information:

• Attribute and word vector fusion: Simultaneous integration of multiple modalities yields stronger signals and complements domain-specific weaknesses (e.g., attributes often contain more distinctive information than word vectors, but both provide unique benefits (Ji et al., 2016)).
• Semantic augmentation: Techniques such as Knowledge Sharing (KS) enrich class semantics by incorporating contextual information from related classes or hierarchical neighbors (Ting et al., 2021, Fei et al., 2018), addressing both sparse and inadequate original descriptors.
• Ontology-based embeddings: Ontologies enable formal composition and capture complex inter-class relationships, boosting recognition in both image classification and language-centric tasks (Chen et al., 2020, Geng et al., 2021).

Table: Modalities in ZSL and Utilization Paradigms

Side Information Modality	Utilization Approach	Example Methods
Attributes	Embedding, GAN conditioning	MBFA-ZSL, GAN-ZSL, SAP
Word embeddings	Embedding, joint fusion	MBFA-ZSL, Joint-Graph ZSL
Ontology	Embedding, logical constraints	OntoZSL, Ontology-guided ZSL
Text descriptions	Knowledge sharing, augmentation	KS-GAN-ZSL, Creativ. ZSL
Visual context	Context-aware model components	Context-Aware ZSL

4. Empirical Evaluations and Performance

ZSL methods are benchmarked on standard datasets (AwA, CUB, SUN, aPY, ImageNet, NELL-ZS, Wikidata-ZS) with evaluation under both conventional and generalized settings. Key empirical observations include:

• Unified and joint embedding strategies (e.g., MBFA-ZSL, SRG, JCMSPL) typically outperform methods treating each modality independently.
• Generative approaches with semantic augmentation (OntoZSL, FSIGenZ, KS-GAN) achieve superior balance on GZSL tasks, especially when addressing data imbalance with specialized regularization (DPSR, ReMSE).
• Transductive and manifold alignment models (Zhao et al., 2016, Gune et al., 2020) improve accuracy significantly, particularly on fine-grained and sparse annotation settings.
• Ontology-driven frameworks provide not only accuracy improvements (e.g., on AwA, ImageNet), but also interpretable, compositional representations for tasks such as zero-shot visual question answering.

5. Advanced Topics and Open Challenges

5.1. Domain Shift and Hubness

Domain shift denotes the tendency for mappings learned on seen classes to generalize poorly to the distribution of unseen classes—a major source of ZSL failure. Architectural elements such as latent space reconstruction constraints, regularization, sparse coding with locality, and embedding entropy minimization directly address domain shift and hubness (the latter referring to over-concentration of nearest neighbors in high-dimensional embedding space; see (Rostami et al., 2019, Gune et al., 2020, Zhao et al., 2017)).

5.2. Data Imbalance

Traditional ZSL methods often produce imbalanced semantic predictions, primarily due to intrinsic differences in semantic label values rather than sample counts (Ye et al., 2022). The ReMSE loss addresses this by reweighting errors across semantic dimensions and classes, minimizing both mean and variance of errors in semantic regression.

5.3. Continual and Incremental Learning

Continual ZSL (Gautam et al., 2020) merges the zero-shot transfer paradigm with streaming or task-incremental learning. Strategies such as experience replay, episodic memory, and knowledge distillation are employed to retain performance on both prior and new (seen/unseen) classes, enabling class-incremental expansion in single-head classifier settings.

5.4. Optimization and Computational Considerations

Efficient closed-form solutions (as in MBFA-ZSL), block-coordinate descent (JCMSPL), and convex formulations enable scalability to large datasets (e.g., ImageNet). Methods that replace large-scale per-instance feature synthesis with compact group-level prototypes (e.g., FSIGenZ (Shohag et al., 18 Jun 2025)) reduce both storage and computation, aligning ZSL with few-shot regimes.

6. Prospects, Implications, and Future Research Directions

Zero-shot learning continues to evolve with advances in semantic information modeling, generative modeling, optimization, and theoretical understanding. Current literature highlights several promising directions:

• Unified multi-modal and cross-domain frameworks able to simultaneously leverage attributes, word vectors, ontologies, and contextual signals.
• Generative ZSL with semantic regularization, few-shot-inspired prototype modeling, and transductive self-supervision, which reduce dependence on large amounts of synthetic data and better model instance variability, as in FSIGenZ (Shohag et al., 18 Jun 2025).
• Neural-symbolic integration, with ontology-enhanced embeddings supporting comprehensive, interpretable semantic transfer (Geng et al., 2021, Chen et al., 2020).
• Dynamic reweighting and balancing strategies (DPSR, ReMSE) that equitably distribute error and regularization effects across semantics and classes, leading to greater robustness and fairness (Ye et al., 2022).
• Applications beyond vision, including open-set recognition, language understanding, KG completion, and lifelong/continual learning (Geng et al., 2021, Gautam et al., 2020).

Ongoing challenges remain in fully bridging the semantic-visual gap, managing open-world and out-of-distribution settings, and scaling to arbitrary classes or knowledge bases without exhaustive engineering of side information. Nonetheless, recent research demonstrates that the principled integration of multi-modal semantics, structure-aware embedding, efficient prototypes, and semantic regularization continues to advance the frontier of zero-shot learning.