Prototypical Prompting (Propot)

• Prototypical Prompting is a technique that combines prototype-based representation learning and prompt adaptation to create representative centroids for classes, identities, or tasks.
• It leverages learnable prompts to steer pre-trained models and dynamically update prototypes across diverse applications such as vision-language tasks, NLP, and continual learning.
• The framework achieves state-of-the-art performance in few-shot recognition, cross-modal retrieval, and rehearsal-free continual learning while enhancing robustness and interpretability.

Prototypical Prompting (Propot) is a family of methods that integrate prototype-based representation learning with prompt-based parameter adaptation in pre-trained models. These approaches leverage the construction or adaptation of “prototypes”—centroidal or representative embedding vectors for classes, identities, or tasks—and use prompts to both steer the backbone model and mediate prototype construction or retrieval. Propot methods offer a unified framework that bridges instance-level, class-level, and task-level adaptation, and are applied in diverse settings including vision-LLMs (VLMs), transformer architectures, continual learning, few-shot recognition, and cross-modal retrieval (Yan et al., 2024, Zhang et al., 2022, Luo et al., 8 Jan 2026, Wei et al., 2022, Li et al., 2023). Key instantiations include text-to-image person re-identification, prototype-based few-shot prompt learning, rehearsal-free continual learning, and prototypical prompt verbalization in NLP.

1. Core Principles and Prototypical Prompting Paradigm

Propot approaches are characterized by two tightly coupled operations: prototype construction/maintenance and prompt-driven adaptation. Prototypes serve as identity-, class-, or task-level representations—class means or centroids in an embedding space derived from a (frozen or lightly adapted) pre-trained encoder. Prompts—learnable vectors injected at the token or layer level—allow per-domain, per-task, or per-instance conditioning of representations and prototype construction (Zhang et al., 2022, Li et al., 2023, Yan et al., 2024).

The essential operational flow in Propot encompasses:

• Prototype initialization: Compute initial prototypes as averages/centroids of embedding vectors for each category/identity/task using a pre-trained model (e.g., CLIP, ViT, LMs) (Yan et al., 2024, Zhang et al., 2022, Wei et al., 2022).
• Prompt adaptation: Apply learnable prompts to align the backbone model’s representations to the specifics of the downstream domain or task (e.g., via domain-conditional or task-specific prompt tokens) (Yan et al., 2024, Li et al., 2023, Luo et al., 8 Jan 2026).
• Prototype enrichment: Update prototypes with instance- or task-conditional information (e.g., through cross-attention or batchwise statistics) to produce more discriminative and robust representations (Yan et al., 2024, Zhang et al., 2022).
• Prototype aggregation: Fuse candidate prototypes (initial, adapted, enriched) into final identity- or class-enriched vectors with learnable weighting (e.g. via softmax attention over similarity scores) (Yan et al., 2024).
• Contrastive objectives: Employ losses that pull embeddings toward their prototype and push them away from others (e.g., prototype–instance contrast, prototype–prototype repulsion), ensuring tight class or identity clustering and mitigation of interference (Yan et al., 2024, Li et al., 2023, Wei et al., 2022).

This paradigm enables a trade-off between parameter efficiency and adaptation specificity, supports both zero/few-shot and continual learning, and unifies the handling of both instance and identity/task-level discrimination.

2. Methodological Variants and Technical Implementations

Vision–Language and Cross-Modal Instantiations

In (Yan et al., 2024), Prototypical Prompting is applied to Text-to-Image Person Re-identification (TIReID). The framework maintains modality- and identity-level prototypes, adapts CLIP encoders via domain-conditional prototypical prompting (DPP), and further enriches prototypes through instance-conditional prototypical prompting (IPP). Adaptive prototype aggregation yields the final representations, which are optimized with a loss that propagates prototype information back into instance features. Both instance-level (cross-modal matching) and identity-level (prototype–instance contrast) losses are jointly optimized.

Few-Shot Learning with Prototype-based Prompt Routing

The “Prompting through Prototype” (PTP) approach (Zhang et al., 2022) introduces prototype learning for both visual features and prompt representations in few-shot VLM settings. Each query instance is softly associated with learned image prototypes using a similarity-based weighting. The resulting distribution then linearly combines a corresponding set of prompt prototypes, yielding an instance-adaptive soft prompt. Only the prototypes are updated; the underlying VLM remains frozen. Classification is performed via the frozen vision and text branches with the instance-specific soft prompt.

Continual Learning: Task-Specific Prompt-Prototype and Contrastive Steering

Recent methods (Luo et al., 8 Jan 2026, Li et al., 2023) use prompt-prototype coupling to address rehearsal-free continual learning. Each new task receives its own prompt and a set of class prototypes constructed from that prompt-conditioned feature extractor. The model avoids key–value pools and directly binds prototypes to task prompts, preventing interference and scaling issues present in prior key-based methods. During training, contrastive loss terms are computed to cluster new samples around their prototypes and to displace new representations from previous task prototypes. This mechanism counters semantic drift and catastrophic forgetting (Li et al., 2023).

Prototypical Prompt Verbalizer in NLP

The Prototypical Prompt Verbalizer (PPV) (Wei et al., 2022) replaces discrete verbalizers in prompt-tuned masked LLMs with continuous prototypes for each class. Embeddings at the [MASK] token (after a projection) are matched to class prototypes via cosine similarity, and a combination of contrastive losses is used to refine the prototype space. Zero-shot initialization leverages the LLM itself to construct class prototypes from unlabeled sentences containing the label word.

3. Formalization and Mathematical Underpinnings

The Propot framework formalizes prototype construction and prompt-mediated adaptation with a consistent mathematical scaffold:

• Prototype construction: For each category/identity/task $k$, the prototype is typically computed as the average of feature vectors: $p_k = \frac{1}{|S_k|} \sum_{i \in S_k} f(x_i)$, where $S_k$ is the set of support/examples for class $k$ and $f$ is the (possibly prompt-conditioned) encoder (Yan et al., 2024, Zhang et al., 2022, Luo et al., 8 Jan 2026, Li et al., 2023).
• Prompt adaptation: Prompt tokens $P$ are injected at the input or at each transformer block. For example, for task $t$, $P^t = \{P^t_1, ..., P^t_S\}$, and each is concatenated to the representation sequence at the corresponding layer (Li et al., 2023). Prompt parameters may be shared across domains/classes or uniquely assigned.
• Instance–prototype association: Cosine similarity or dot product is used to associate query features with prototypes. Weighting schemes (typically softmax normalization) produce an adaptive prompt or prediction score (Zhang et al., 2022, Wei et al., 2022).
• Contrastive/prototype-to-instance losses: For embedding $z_i$ (instance), prototype $c_{y_i}$ (class), and negatives from other classes or tasks, the loss is of the form:

$$\mathcal{L}_{\text{proto-inst}} = -\frac{1}{N} \sum_{i=1}^N \log \frac{\exp(\mathrm{sim}(z_i, c_{y_i})/\tau)}{\sum_{j} \exp(\mathrm{sim}(z_i, c_{j})/\tau)}$$

(Yan et al., 2024, Li et al., 2023, Wei et al., 2022).

• Complete objectives: Typically, an instance-level loss is combined with the prototype-based loss and optional regularization:

$$\mathcal{L} = \mathcal{L}_{\text{instance}} + \lambda_1 \mathcal{L}_{\text{proto-inst}} + \lambda_2 \mathcal{L}_{\text{aux}}$$

(Yan et al., 2024).

4. Empirical Efficacy and Practical Applications

Empirical evaluations indicate Propot approaches offer improvement across several axes:

• State-of-the-art performance in text-to-image person re-ID, with Propot achieving R@1 74.89% (+2.16% gain) on CUHK-PEDES and competitive results on ICFG-PEDES and RSTPReid (Yan et al., 2024).
• Superior few-shot classification compared to linear probes, BitFit, and other prompt-based methods, with the PTP method outperforming both task/instance-level prompting and more parameter-intensive adaptation schemes in low-data regimes (Zhang et al., 2022).
• Minimized catastrophic forgetting and semantic drift in continual learning; Contrastive Prototypical Prompt (CPP) attains up to 91.1% accuracy on Split CIFAR-100 (no rehearsal), outperforming DualPrompt and L2P, with low forgetting scores (Li et al., 2023). Task-specific Prompt-Prototype (ProP) achieves 85.99% on CIFAR and 68.78% on ImageNet-R, outstripping all key–value baselines (Luo et al., 8 Jan 2026).
• Enhanced interpretability and robustness: Learned prototype vectors in NLP remain well-separated and interpretable, even under limited parameter adaptation (Wei et al., 2022).

The parameter efficiency, decoupling from upstream model weights, and flexibility for incorporating per-task or per-domain information underscore practical scalability to large, pre-trained architectures.

5. Limitations, Open Challenges, and Future Directions

Key limitations include:

• Dependency on class and label semantics: Performance can degrade on datasets with semantically opaque labels, as meaningful prototype construction may be hampered (Zhang et al., 2022).
• Prototype selection and updating: Choice of prototype number, update rules (e.g., running mean, clustering, EMA), and enrichment mechanism can impact stability and adaptation, but optimal strategies remain underexplored (Li et al., 2023).
• Resource and inference overhead: Some Propot instantiations require additional forward passes for each candidate prompt or prototype set, which can become costly in large-task regimes (Li et al., 2023).
• Initialization sensitivity: Prototype initialization in zero-shot settings relies on the coverage and quality of available unlabeled data or canonical label words; noisy initializations can propagate errors (Wei et al., 2022).
• Competition with full fine-tuning: In abundant data scenarios, standard fine-tuning may close or surpass the performance gap, especially when the pre-trained model limitations are encountered (Zhang et al., 2022).

Future research directions highlighted include online prototype updating, adaptive prompt–prototype pool growth, prompt selection networks for more efficient inference, and extension to multi-modal and domain-incremental learning scenarios (Luo et al., 8 Jan 2026, Li et al., 2023). The Propot recipe also admits extensions to open-vocabulary, few-shot sequential, and reinforcement learning settings via meta-learned or language-derived prototypes.

6. Comparative Overview of Representative Approaches

Approach	Domain	Prompt–Prototype Coupling	Prototyping Strategy
Propot (TIReID)	Cross-modal retrieval	DPP/IPP prompts + APA	Multi-stage & aggregated
PTP	Few-shot image classification	Instance-adaptive prompt	Learned visual + prompt centers
ProP (ProP-CIL)	Continual learning (vision)	Task-specific prompt	Per-task class mean
CPP	Rehearsal-free CIL (vision)	Deep task prompts	Key/value prototypes, contrast.
PPV	Few-shot text classification	Prototype verbalizer	LM-elicited then contrastively refined

All cited works consistently demonstrate improved performance over prior prompt-tuning, prototype-only, and parameter-efficient baselines, while offering a modular design that is easily adapted to the target domain and application setting (Yan et al., 2024, Zhang et al., 2022, Luo et al., 8 Jan 2026, Wei et al., 2022, Li et al., 2023).