Prototype-Guided Curriculum Learning

• Prototype-Guided Curriculum Learning is a curriculum design method that quantifies sample difficulty via cosine similarity between visual embeddings and semantic prototypes.
• It schedules training from easy to hard instances by dynamically weighting samples, thereby mitigating instance-level mismatches and improving mapping generalization.
• Integrating a Prototype Update module, this approach demonstrates improved performance on benchmarks like AWA2, SUN, and CUB, enhancing zero-shot transfer.

Prototype-Guided Curriculum Learning (PCL) is a targeted curriculum learning framework for zero-shot learning (ZSL) in embedding-based models, where the goal is to map samples from seen classes onto semantic prototypes and transfer this alignment to unseen classes. PCL constructs a curriculum over seen-class training data by quantifying per-sample alignment difficulty between each sample's visual embedding and its manually defined semantic prototype, prioritizing well-aligned (i.e., “easy”) samples early in training. This strategy aims to mitigate instance-level mismatches and improve generalization by leveraging the structure imposed by prototypes throughout training. PCL is typically used together with a Prototype Update (PUP) module, which dynamically refines class-level prototypes to attenuate class-level imprecision introduced by manual definitions (Wang et al., 11 Aug 2025).

1. Mathematical Formulation and Notation

Let $x$ be a sample image and $f(x) \in \mathbb{R}^d$ its visual embedding, produced by a feature extractor such as a ResNet or ViT. Given a manually defined semantic prototype $a_c \in \mathbb{R}^d$ for class $c$, the cosine similarity is defined as

$$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$

The per-sample “difficulty” for a training pair $(x,c)$ is

$d(x, c) = 1 - \mathrm{sim}(f(x), a_c)$

where $d(x,c) \approx 0$ denotes “easy” samples (strong alignment), and higher values indicate greater misalignment.

2. Curriculum Construction and Scheduling

The curriculum in PCL is dynamically shaped using per-sample difficulties:

1. Ranking by Difficulty: For each mini-batch or epoch, $d_i = d(x_i, y_i)$ is computed for all training samples $(x_i, y_i)$. Samples are sorted in non-decreasing order of $f(x) \in \mathbb{R}^d$0.
2. Scheduling Function $f(x) \in \mathbb{R}^d$1: The inclusion ratio $f(x) \in \mathbb{R}^d$2 (where $f(x) \in \mathbb{R}^d$3 is iteration or epoch, $f(x) \in \mathbb{R}^d$4 total iterations) determines which fraction of the easiest samples are used. Typical schedules include:
   • Linear: $f(x) \in \mathbb{R}^d$5
   • Exponential: $f(x) \in \mathbb{R}^d$6
   • Multi-stage: $f(x) \in \mathbb{R}^d$7 increases in $f(x) \in \mathbb{R}^d$8 piecewise-constant stages
3. Sample Selection/Weighting:
   • Subset selection: Include only lowest-$f(x) \in \mathbb{R}^d$9 samples up to $a_c \in \mathbb{R}^d$0 fraction (hard curriculum).
   • Continuous weighting: Assign weight $a_c \in \mathbb{R}^d$1 where $a_c \in \mathbb{R}^d$2 is strictly decreasing in $a_c \in \mathbb{R}^d$3.

Hyperparameters include number of stages $a_c \in \mathbb{R}^d$4, schedule exponent $a_c \in \mathbb{R}^d$5, epoch boundaries $a_c \in \mathbb{R}^d$6, and thresholds $a_c \in \mathbb{R}^d$7 corresponding to empirical CDF quantiles. In practice, soft weighting (e.g., $a_c \in \mathbb{R}^d$8) or learned weights via a small MLP is also used.

3. Curriculum-Weighted Loss and Optimization

The curriculum is embedded into the training objective as a per-sample weight:

$a_c \in \mathbb{R}^d$9

where $c$0 is a matching loss (e.g., negative cosine similarity or cross-entropy over class scores derived from cosine similarities), and $c$1 the curriculum weight. The implementation (mini-batch version) pseudocode: $(x,c)$2 This cycling from the easiest to hardest examples ensures early focus on well-aligned instances, followed by gradual broadening to samples with greater instance-level variation.

4. Interaction with Prototype Update (PUP)

The Prototype Update (PUP) module complements PCL by addressing prototype imprecision:

• After every $c$2 epochs, for each seen class $c$3, its prototype $c$4 is updated via:

$c$5

where $c$6 is the set of seen-class samples for class $c$7, and $c$8 is a momentum parameter (e.g., $c$9). For unseen classes, prototype updates are propagated using the $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$0 nearest seen neighbors in prototype space. This mechanism reduces class-level imprecision and helps anchor the curriculum to more representative semantics as training progresses.

5. Theoretical Intuition and Computational Complexity

Intuition: By focusing first on “easy” examples (highly prototypical samples), the model learns a robust mapping $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$1 insulated from noise and annotation artifacts. Sequential inclusion of harder samples allows refinement of the mapping while minimizing the risk of catastrophic forgetting.

Justification: The curriculum is explicitly tied to semantic supervision, with difficulty quantified as misalignment between instance embedding and class prototype—directly reflecting the knowledge transfer target. Empirically, this yields embeddings with enhanced intra-class compactness and greater inter-class separability.

Computational Overhead:

• Each mini-batch requires $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$2 for dot products/norms (batch size $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$3), $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$4 for sorting, or $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$5 for threshold approximation via histograms.
• Prototype updates impose $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$6 costs per update (infrequent relative to forward/backward passes).
• Overall, the marginal overhead is negligible compared to the visual encoder computation.

6. Empirical Performance Across Benchmarks

PCL, evaluated both independently and in combination with PUP (“CLZSL”), demonstrates robust improvements on AWA2, SUN, and CUB benchmarks for both Generalized ZSL (harmonic mean $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$7 between seen/unseen) and conventional ZSL (unseen-only):

Dataset	Baseline $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$8	PCL only $$\mathrm{sim}(f(x), a_c) = \frac{f(x) \cdot a_c}{\|f(x)\|_2 \|a_c\|_2}$$9	PUP only $(x,c)$0	CLZSL (PCL+PUP) $(x,c)$1	Unseen-Only Acc.
AWA2	69.0	70.7 (+1.7)	70.6 (+1.6)	71.6 (+2.6)	67.4
SUN	52.2	52.8 (+0.6)	53.3 (+1.1)	54.1 (+1.9)	72.3
CUB	66.3	69.0 (+2.7)	68.9 (+2.6)	70.4 (+4.1)	78.0

These results confirm consistent improvements from PCL across highly diverse domains, with the most substantial benefits observed when combined with prototype refinement (Wang et al., 11 Aug 2025).

7. Context and Significance

Prototype-Guided Curriculum Learning provides a principled strategy for curriculum design in embedding-based ZSL by operationalizing prototype alignment as a difficulty measure. This approach generalizes the curriculum learning paradigm by grounding sample curricula in the supervision structure of semantic prototypes, which are central to zero-shot transfer. The integration with prototype updating addresses limitations of manual prototypes, supporting both stable early training and improved knowledge transfer. These empirical findings suggest PCL is broadly applicable in ZSL settings where prototype definitions are noisy or imperfect, and where resistance to instance-level peculiarity is critical for generalization.