Prototype Alignment in Representation Learning

• Prototype Alignment (PA) is a design pattern that uses class-level prototypes as semantic anchors to align samples, features, and subdomains.
• It enhances performance in tasks like few-shot segmentation and domain adaptation by leveraging prototype-aware losses and bidirectional alignment techniques.
• Implementations range from masked average pooling and medoid selection to orthogonal constraints, emphasizing tailored prototype construction for specific challenges.

Prototype Alignment (PA) denotes a family of representation-learning and transfer mechanisms in which class-level prototypes serve as alignment anchors for samples, features, subdomains, or even other prototypes. Across the literature, a prototype may be a masked average of support features, a batch-wise class centroid, a server-aggregated public reference, a medoid selected by K-Medoids, an orthogonal class anchor, or a learnable ordinal level vector. Correspondingly, alignment may mean Euclidean attraction, prototype-aware contrastive learning, bidirectional support–query consistency, class-conditional domain matching, structural matching of inter-class relations, or hyperspherical rearrangement of global prototypes (Wang et al., 2019, Wu et al., 21 Jan 2026, Wu et al., 7 May 2026).

1. Prototype as representation primitive

A defining characteristic of PA is that it replaces unconstrained feature matching with class-structured anchors. In few-shot semantic segmentation, PANet computes a foreground prototype by masked average pooling over support pixels,

$p_c = \cfrac{1}{K} \sum_{k} \cfrac{\sum_{x,y} F_{c,k}^{(x,y)} \mathds{1}[M_{c,k}^{(x,y)}=c]}{\sum_{x,y} \mathds{1}[M_{c,k}^{(x,y)}=c]},$

and also learns a background prototype from all background pixels in the support set (Wang et al., 2019). In multimodal intent recognition, MVCL-DAF++ defines a batch-wise class prototype as

$$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$

with prototypes L2-normalized before use in the loss (Huang et al., 22 Sep 2025). In spoken language assessment, LOPA uses learnable level prototypes

$$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$

initialized from the $k$-th CEFR level centroid and updated during training, so that alignment operates simultaneously at the sample-to-prototype and prototype-to-prototype levels (Lin et al., 30 Jun 2026).

Not all PA methods use mean centroids. In cross-plant unknown attack detection for industrial control systems, Medoid Prototype Alignment first standardizes source and target traffic, projects them into PCA spaces, and then extracts medoid prototypes with K-Medoids. The clustering objective is

$$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$

so each prototype is a real observation rather than an average, with the stated aim of reducing sensitivity to outliers and preserving operational structure (Wang, 28 Apr 2026). In domain-adaptive retrieval, PSCA instead learns orthogonal prototypes $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$ in a shared latent subspace under the constraint $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$, explicitly maximizing inter-class separability (Hu et al., 4 Dec 2025).

These formulations imply that PA is better understood as a design pattern than as a single loss. What remains invariant is the use of prototypes as semantic anchors; what changes is how those anchors are constructed, which geometric properties they are required to satisfy, and whether they summarize pixels, samples, clients, classes, or latent subgroups.

2. Dense prediction and bidirectional prototype consistency

In dense prediction, PA is often used to enforce reciprocity between support- and query-derived semantics. PANet’s Prototype Alignment Regularization (PAR) makes few-shot segmentation bidirectional: support-derived prototypes first segment the query, then query-derived prototypes segment the support back. The training objective is

$$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$

with $\lambda=1$. Empirically, PANet reports $48.1\%$ mIoU and $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$0 mIoU on PASCAL-5$$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$1 for 1-shot and 5-shot settings, compared with $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$2 and $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$3 without PAR, and the Euclidean distance between support and query prototypes decreases from $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$4 to $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$5 when PAR is used (Wang et al., 2019).

MSPA extends the same broad principle to semi-supervised medical image segmentation through Mutual- and Self- Prototype Alignment. Its Mutual-Prototype Alignment contains Labeled Prototype Alignment, which uses labeled prototypes to regularize unlabeled predictions, and Unlabeled Prototype Alignment, which uses voting-based pseudo-labels to compute unlabeled prototypes that supervise labeled images in the reverse direction. Self-Prototype Alignment then aligns relatively uncertain and relatively reliable regions within the same unlabeled image:

$$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$6

The full objective is

$$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$7

where $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$8. On Kvasir-SEG, the reported Dice score rises from $$\mathbf{r}_{c}=\frac{1}{|\mathcal{I}_{c}|}\sum_{i\in\mathcal{I}_{c}}\mathbf{h}_{i},$$9 for supervised $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$0 training to $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$1 for MSPA with $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$2 (Zhang et al., 2022).

Generalizable semantic segmentation introduces a different variant: Prototypical Progressive Alignment and Reweighting. PPAR replaces naive source-batch prototypes with CLIP-derived Original Text Prototypes and Visual Text Prototypes, then aligns shallow features to VTP and deep features to OTP in an easy-to-difficult schedule:

$$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$3

The paper’s stated critique is that naive prototypes are source-biased, single-layer alignment is too weak, and naive multi-layer alignment is too coarse (Zhang et al., 16 Jul 2025).

3. Domain adaptation, transfer, and retrieval

In domain adaptation, PA is typically used to replace coarse marginal matching with class-conditional or structure-aware matching. In cross-domain object detection, GPA constructs a graph over region proposals, propagates features and class confidences,

$$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$4

and forms each class prototype as a confidence-weighted mean,

$$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$5

Alignment is then defined through an intra-class pull and an inter-class margin loss. GPA reports $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$6 mAP on Cityscapes $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$7 Foggy Cityscapes and $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$8 AP on SIM 10k $$\mathcal{C} = \{\mathbf{c}_1, \dots, \mathbf{c}_K\},$$9 Cityscapes (Xu et al., 2020).

Universal domain adaptation takes a more cautious route. Subsidiary Prototype Alignment operates in an auxiliary Bag-of-Visual-Words-like mid-level space rather than in the semantic class-label space, precisely to avoid negative transfer between shared and private classes. The paper frames SPA as a closed-set alignment problem over word-prototypes, coupled with entropy minimization on word assignments and a grid-shuffling pretext task that predicts the number of mixed instances from word-histogram structure (Kundu et al., 2022).

Other transfer settings use prototypes to control correspondence granularity. Medoid Prototype Alignment for cross-plant industrial intrusion detection aligns target medoid prototypes to source medoid prototypes through a temperature-scaled soft correspondence

$k$0

with the overall objective

$k$1

The method reports average accuracy $k$2 and average F1-score $k$3 across four unknown-attack transfer tasks (Wang, 28 Apr 2026). Prototype-Based Semantic Consistency Alignment for domain-adaptive retrieval likewise rejects pair-wise sample alignment in favor of orthogonal class prototypes, a reliability-aware soft membership matrix $k$4, and feature reconstruction prior to hashing; reported gains include $k$5 average over the second-best method on MNIST$k$6USPS cross-domain retrieval (Hu et al., 4 Dec 2025).

Prototype guidance also appears at test time. PGFA for zero-shot skeleton-based action recognition first predicts pseudo-labels for unseen skeletons using static unseen text embeddings, then replaces those text embeddings with prototypes computed from high-confidence unseen skeleton features:

$k$7

The reported absolute accuracy improvements over SMIE are $k$8, $k$9, and $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$0 on NTU-60, NTU-120, and PKU-MMD (Zhou et al., 1 Jul 2025). By contrast, the abstract of "Online Prototype Alignment for Few-shot Policy Transfer" describes a framework that learns a mapping from functional similarity rather than visual clues and enables few-shot policy transfer within only several episodes, but the supplied material also states that the provided document is only an ICML 2022 template and does not contain the algorithmic or experimental details of OPA (Yi et al., 2023).

4. Federated learning: from coordinate pull to structural matching

Federated learning has produced several distinct and sometimes conflicting meanings of PA. In RefProtoFL, External-Referenced Prototype Alignment stabilizes representation learning across heterogeneous clients by forcing each client’s feature space to agree with a shared class-level reference. For classes covered by a small public dataset, the server forms an external-reference prototype $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$1; for uncovered classes, it uses a weighted global reference prototype $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$2; the client-side target is

$$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$3

The local objective augments cross-entropy with a squared Euclidean prototype pull. On CIFAR-10 with $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$4, the full method reports $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$5 accuracy, versus $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$6 without ERPA and $$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$7 without APUD (Wu et al., 21 Jan 2026).

FedSAP argues that early prototype alignment is actively harmful because immature local embeddings and immature global prototypes produce a large distance-dependent gradient,

$$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$8

Its remedy is a deterministic alignment curriculum

$$\min_{\{C_k\},\{p_k\}} \sum_{k=1}^{K}\sum_{z_i\in C_k}\|z_i-p_k\|_2,$$9

implemented in the experiments by introducing the prototype loss after round 20 and increasing it linearly from $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$0 to $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$1 until round 100. On CIFAR-100 under the classifier-head inference setting, FedSAP reports $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$2 versus $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$3 for FedProto (Casado-Diez et al., 1 Jun 2026).

FedSAF rejects the standard coordinate-alignment assumption more fundamentally. Its claim is that MSE- or cosine-based prototype alignment in heterogeneous federated learning conflates semantic structure transfer with rigid basis matching. FedSAF therefore aligns Gram matrices or representational dissimilarity matrices rather than raw coordinates, and reports improvements of up to $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$4 over state-of-the-art prototype-based HtFL methods (Wu et al., 7 May 2026).

ProtoNorm represents yet another geometry. Its server-side PA normalizes global prototypes onto the unit sphere and iteratively repels them according to a Thomson-problem-inspired energy,

$\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$5

The paper’s explicit conclusion is that PA alone gives little benefit when $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$6; Prototype Upscaling is needed to turn angular separation into useful Euclidean separation on clients. The full PA+PU pipeline reports $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$7 on CIFAR-10 and $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$8 on CIFAR-100 in the $\mathbf{O}=[\mathbf{o}_1,\dots,\mathbf{o}_c]$9 setting, compared with $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$0 and $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$1 for FedProto (Lee et al., 6 Jul 2025).

5. Prototype alignment as semantic, ordinal, causal, and physical regularization

Outside transfer and federation, PA is used to impose latent geometry suited to the task. MVCL-DAF++ introduces Prototype-aware Contrastive Alignment in multimodal intent recognition. Instead of only performing instance-level cross-modal InfoNCE, it aligns each sample embedding $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$2 to the prototype of its ground-truth class $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$3 and away from other class prototypes through a prototype-aware softmax over classes. On MIntRec and MIntRec2.0, the full model reports ACC/WF1 of $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$4 and $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$5; the loss ablation shows that classifier + prototype is stronger than classifier only on both datasets (Huang et al., 22 Sep 2025).

LOPA uses PA to encode ordinality rather than mere class separation. Its attraction term pulls each latent utterance representation $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$6 to the prototype of its proficiency level, while an ordinal constraint forces pairwise prototype distances to match score gaps:

$\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$7

Coupled with SALR, the full model reports RMSE $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$8 and PCC $\mathbf{O}^{\top}\mathbf{O}=\mathbf{I}_c$9, versus RMSE $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$0 and PCC $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$1 without LOPA; the global ordinality score improves from $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$2 to $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$3 (Lin et al., 30 Jun 2026).

PITE adapts PA to causal inference by learning $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$4 prototypes per treatment arm, assigning each sample to its nearest within-arm prototype, aligning prototype $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$5 in treatment and control through

$$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$6

and preserving diversity with an explicit repulsion term. The method reports the lowest within-sample $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$7 and out-of-sample $$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$8 on IHDP among the listed methods (Cao et al., 13 Nov 2025).

In molecular foundation-model fine-tuning, Prototype-Guided Latent Alignment uses source prototypes of local atomic environments and aligns target nodal energy latents to source energy-prototype statistics through the Nodal Energy Alignment Loss,

$$\mathcal{L} = \mathcal{L}_{\text{seg}} + \lambda \mathcal{L}_{\text{PAR}},$$9

Alignment is active only early in fine-tuning, and energy MAE reductions of up to $\lambda=1$0 over standard fine-tuning baselines are reported (Pawar et al., 28 May 2026).

6. Recurring themes, misconceptions, and limitations

Several recurring themes cut across these otherwise heterogeneous methods. First, PA is not reducible to a single attraction loss. Some works align samples to class centroids; some align target prototypes to source prototypes; some align prototype geometry; some even regard the absolute coordinates as secondary and align only inter-class relations (Wu et al., 7 May 2026). This suggests that the phrase "prototype alignment" names a representational principle rather than a canonical algorithm.

Second, prototype quality is repeatedly treated as the bottleneck. RefProtoFL states that PA there is not a generic prototype regularizer but a reference-guided mechanism built around public-data anchors and server-aggregated fallback prototypes (Wu et al., 21 Jan 2026). FedSAP argues that even correct alignment objectives can be harmful when prototypes are immature (Casado-Diez et al., 1 Jun 2026). PPAR argues that naive prototypes computed by averaging source batch features are source-biased and may be negatively affected by unrelated source data (Zhang et al., 16 Jul 2025). Medoid Prototype Alignment argues that direct sample-level alignment is brittle in bursty, imbalanced, noisy, and heterogeneous ICS traffic, so medoid summaries are preferable (Wang, 28 Apr 2026). PSCA similarly rejects pair-wise sample alignment as computationally inefficient, limited in coverage, and sensitive to noisy pseudo-labels (Hu et al., 4 Dec 2025).

Third, many results indicate that PA often requires a companion mechanism. PANet pairs PAR with query segmentation loss (Wang et al., 2019); MSPA combines mutual and self alignment (Zhang et al., 2022); FedSAP adds a proxy separation loss (Casado-Diez et al., 1 Jun 2026); ProtoNorm requires Prototype Upscaling to convert hyperspherical separation into practical client-side discrimination (Lee et al., 6 Jul 2025); PAA-M couples prototype-guided local class-conditional alignment with contrastive semantic regularization and boundary-aware dual-classifier discrepancy, reporting average improvements of $\lambda=1$1, $\lambda=1$2, $\lambda=1$3, and $\lambda=1$4 under four cross-corpus EEG protocols (Li et al., 18 Mar 2026).

A common misconception is therefore that PA is inherently sufficient once prototypes are defined. The supplied literature does not support that view. Rather, PA is typically embedded in broader systems that address pseudo-label reliability, boundary distortion, feature fusion, communication constraints, or task-specific geometry. The most stable conclusion is narrower: prototypes are effective anchors when their construction, geometry, and training schedule are matched to the failure mode of the target problem.