Prototype Normalization (ProtoNorm)

• Prototype Normalization (ProtoNorm) is a family of techniques that uses prototype-based statistics as anchors to normalize activations, replacing raw feature statistics.
• It addresses failures of traditional normalization (like BatchNorm and LayerNorm) by dynamically calibrating features, especially in imbalanced and multi-distribution settings.
• Empirical studies in person search, time-series pretraining, and federated learning demonstrate that ProtoNorm leads to improved performance metrics such as higher mAP, rank-1 accuracy, and forecasting accuracy.

Searching arXiv for papers on "Prototype Normalization" / "ProtoNorm" and closely related usages. Prototype Normalization (ProtoNorm) is not a single universally standardized method but a family of prototype-conditioned normalization strategies whose meaning depends on research context. In the literature provided here, the term ranges from a prototype-based standardization layer placed before $L_2$ normalization in person search, to a prototype-guided replacement for LayerNorm in time-series Transformer pretraining, to a prototype-geometry optimization procedure in heterogeneous federated learning. A related few-shot learning line does not use the name “ProtoNorm,” but analyzes why normalizing feature embeddings before constructing class prototypes—especially by $L_2$ normalization—can make prototype classifiers competitive with both Prototypical Networks and retrained linear classifiers (Hou et al., 2021). Conversely, “Proxy Normalization” is a distinct batch-independent normalization method and is not Prototype Normalization in the prototype or centroid sense (Labatie et al., 2021).

1. Terminological scope and conceptual commonalities

“ProtoNorm” appears explicitly in at least three different technical senses in the material considered here. In person search, it is a normalization layer that computes statistics from within-batch identity prototypes rather than directly from raw instance features, and then applies $L_2$ normalization (Lee et al., 2022). In time-series foundation model pretraining, it is a prototype-guided dynamic normalization mechanism that replaces LayerNorm with a bank of LayerNorm modules selected by nearest learned distribution prototypes (Gong et al., 15 Apr 2025). In heterogeneous federated learning, it is a prototype-based framework that improves class-prototype geometry by Prototype Alignment and Prototype Upscaling (Lee et al., 6 Jul 2025).

Despite these differences, the shared motif is the use of prototypes as normalization anchors rather than relying only on raw instance statistics or a single static normalization parameterization. This suggests that ProtoNorm is better understood as a design pattern than as a single operator: prototypes summarize structure—identity structure, latent distribution structure, or class geometry—and the normalization mechanism is conditioned on that summary.

A second commonality is that ProtoNorm is typically introduced in response to a failure mode of conventional normalization. In few-shot prototype classification, raw pre-trained features plus a prototype classifier perform substantially worse than ProtoNet and than pre-trained features plus a newly trained linear classifier, and the paper identifies variation in feature norms as a key unmodeled source of error (Hou et al., 2021). In person search, the authors argue that direct $L_2$ normalization is unreliable when proposal features are not standardized and that BatchNorm is biased by long-tail identity frequency (Lee et al., 2022). In time-series pretraining, the criticism is that standard LayerNorm has one shared learned affine parameterization for all samples despite substantial distribution mismatch across datasets (Gong et al., 15 Apr 2025). In prototype-based federated learning, the weakness is insufficient separation of aggregated global prototypes under data and model heterogeneity (Lee et al., 6 Jul 2025).

2. Few-shot prototype classification as a precursor

A direct antecedent to ProtoNorm-like reasoning appears in “A Closer Look at Prototype Classifier for Few-shot Image Classification” (Hou et al., 2021). The paper studies few-shot image classification in the standard $N$-way $K$-shot setting with feature extractor $\phi:\mathcal{X}\to\mathbb{R}^D$, prototype

$$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$

and nearest-prototype classification in squared Euclidean distance: $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$

The core empirical observation is that directly using feature vectors extracted by standard pre-trained models to construct a prototype classifier in meta-testing does not perform as well as Prototypical Networks or training a new linear classifier on the same features. The paper attributes this to a mismatch in training objectives: ProtoNet encourages features to lie close to class means, whereas standard cross-entropy training only requires linear separability after projection by class weights $W_c$, with

$L_2$0

The paper’s theoretical contribution is a generalization bound for a prototypical classifier whose error depends not only on a term related to the ratio of within-class variance to between-class variance, but also on the variance of feature norms: $L_2$1 This directly motivates $L_2$2 normalization,

$L_2$3

because it makes all feature norms equal to $L_2$4, driving the variance of norms to zero.

The paper evaluates six transformations before constructing prototypes: $L_2$5 normalization, centering+$L_2$6, LDA, EST, EST+$L_2$7, and LDA+$L_2$8. Its strongest general conclusion is that the same performance can be obtained by using the $L_2$9 normalization and minimizing the ratio of the within-class variance to the between-class variance without training a new classifier or meta-learning (Hou et al., 2021). It also states explicitly that there is no named method called “Prototype Normalization” or “ProtoNorm” in that work. The closest interpretation is simply normalizing feature embeddings before constructing class prototypes and classifying by nearest prototype.

3. ProtoNorm in person search

The explicit term “ProtoNorm” is introduced in “OIMNet++: Prototypical Normalization and Localization-aware Learning for Person Search” (Lee et al., 2022). There, ProtoNorm is a normalization layer inserted before the final $L_2$0 normalization in the projection module. Its purpose is to calibrate features from pedestrian proposals while considering a long-tail distribution of person IDs, thereby enabling $L_2$1-normalized person representations to be discriminative.

The motivation is specific to person search. Proposal features are extracted from detector outputs rather than clean, tightly cropped re-identification images, so they may be misaligned, include background clutter, contain overlapping people, or only partially cover the target person. The paper states that existing person search methods implicitly assume that features obtained from pedestrian proposals have zero-mean and unit variance and then simply apply $L_2$2 normalization; when that assumption fails, $L_2$3 normalization can degrade discriminative power. A straightforward BatchNorm fix is also argued to be inadequate because mean and variance estimated from the current batch are dominated by majority identities in a long-tail distribution, biasing feature calibration toward frequent IDs and weakening discrimination for minority IDs (Lee et al., 2022).

ProtoNorm therefore computes its mean from within-batch identity prototypes. For a mini-batch

$L_2$4

the prototype for identity $L_2$5 is

$L_2$6

If $L_2$7 is the number of unique IDs in the batch, ProtoNorm computes

$L_2$8

Each feature is then standardized channel-wise and only afterwards projected onto the unit hypersphere by $L_2$9 normalization.

Two details distinguish this method from BatchNorm. First, the mean is computed from prototypes rather than raw batch instances, so each identity contributes more equally regardless of occurrence count. Second, the main ProtoNorm used before $L_2$0 normalization omits a learnable affine transform because the authors found that scaling and offset converge to constant and zero values, respectively, and inferred that the effect is canceled out by $L_2$1 normalization (Lee et al., 2022). During inference, labels are unavailable, so ProtoNorm uses running estimates of mean and variance accumulated during training, analogous to BatchNorm.

The reported ablation on PRW isolates ProtoNorm’s effect. With OIM only, Search mAP and rank-1 are $L_2$2 and $L_2$3, and ReID mAP and rank-1 are $L_2$4 and $L_2$5. With BatchNorm + OIM, these become $L_2$6, $L_2$7, $L_2$8, and $L_2$9. With ProtoNorm + OIM, they become $N$0, $N$1, $N$2, and $N$3 (Lee et al., 2022). The paper also reports that when combined with the localization-aware OIM update, the full OIMNet++ reaches Search mAP $N$4, rank-1 $N$5, ReID mAP $N$6, and rank-1 $N$7.

4. Prototype-guided normalization in time-series models

In time-series research, ProtoNorm denotes a different mechanism. “Bridging Distribution Gaps in Time Series Foundation Model Pretraining with Prototype-Guided Normalization” proposes a domain-aware adaptive normalization strategy within Transformer architectures by replacing LayerNorm with a prototype-guided dynamic normalization mechanism (Gong et al., 15 Apr 2025). The paper’s argument is that multi-dataset time-series pretraining is harmed by substantial distribution mismatch across datasets—differences in value ranges, morphology, temporal dependencies, noise level, sampling rate, channel count, and sequence length—and that standard LayerNorm is too rigid because it has a single shared learned affine parameterization for all samples.

Standard LayerNorm is written as

$N$8

with one shared pair $N$9 per layer. ProtoNorm instead contains $K$0 LayerNorm modules $K$1, a prototype-guided gating network $K$2, and prototypes $K$3. For a sample feature representation $K$4, the routing rule is

$K$5

and the normalization is best summarized as

$K$6

The dynamic part is therefore branch selection: the mean and variance remain LN-style per-sample statistics, while the affine parameters are selected by nearest-prototype hard routing.

The prototypes are updated by exponential moving average,

$K$7

and are regularized by an orthogonality loss

$K$8

where $K$9 is the prototype matrix. In the reported pretraining framework, the total loss is

$\phi:\mathcal{X}\to\mathbb{R}^D$0

with $\phi:\mathcal{X}\to\mathbb{R}^D$1 in the main experiments (Gong et al., 15 Apr 2025).

The empirical findings are broad. In classification averaged over groups, ProtoN-FM reaches $\phi:\mathcal{X}\to\mathbb{R}^D$2 accuracy and $\phi:\mathcal{X}\to\mathbb{R}^D$3 Macro-F1, compared with $\phi:\mathcal{X}\to\mathbb{R}^D$4 and $\phi:\mathcal{X}\to\mathbb{R}^D$5 for vanilla multi-dataset pretraining. In forecasting on Monash, integrating ProtoNorm into Moirai improves normalized MAE from $\phi:\mathcal{X}\to\mathbb{R}^D$6 to $\phi:\mathcal{X}\to\mathbb{R}^D$7, reported as an $\phi:\mathcal{X}\to\mathbb{R}^D$8 improvement. Under a controlled distribution-shift analysis on IMS-N3, ProtoNorm yields accuracy $\phi:\mathcal{X}\to\mathbb{R}^D$9 versus $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$0 for vanilla and Macro-F1 $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$1 versus $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$2 (Gong et al., 15 Apr 2025). The paper further reports that prototype-guided gating matters: on MFD average, removing ProtoGate reduces performance from $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$3 to $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$4 in accuracy/Macro-F1.

A related but differently framed time-series method is APT: Affine Prototype Timestamp (Li et al., 17 Nov 2025). That paper does not use the label “ProtoNorm,” but structurally it is described as a prototype-conditioned affine normalization/de-normalization layer: timestamp embeddings are matched to a learned prototype library, the top-$$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$5 sparse assignment produces a prototype aggregate, and two MLPs generate $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$6 used in a forward affine transform before the forecasting backbone and an inverse affine transform after it. This suggests that the time-series literature contains both explicit ProtoNorm methods (Gong et al., 15 Apr 2025) and ProtoNorm-like prototype-conditioned normalization modules that are named differently (Li et al., 17 Nov 2025).

5. ProtoNorm in heterogeneous federated learning

In “Heterogeneous Federated Learning with Prototype Alignment and Upscaling,” ProtoNorm is a prototype-based heterogeneous federated learning framework designed to improve the discriminative quality of shared class prototypes (Lee et al., 6 Jul 2025). The setting is a central server with $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$7 clients, each with its own private distribution $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$8, possibly different model architecture, and a shared classification task over $$\overline{\phi(S_c)}=\frac{1}{K}\sum_{x\in S_c}\phi(x),$$9 classes. The client model has a feature extractor $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$0 and classifier $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$1, with penultimate-layer feature $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$2.

Prototype-based federated learning communicates only class prototypes, namely class-wise mean feature vectors from the penultimate layer. The local prototype for class $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$3 on client $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$4 is

$$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$5

ProtoNorm’s critique of prior PBFL methods is that global prototypes obtained by aggregation alone are insufficiently separated under non-IID data and model heterogeneity.

Its remedy has two sequential components. Prototype Alignment (PA) first normalizes aggregated global prototypes onto the unit sphere,

$$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$6

and then optimizes them using a Thomson-problem-inspired repulsive energy. The surrogate objective used for $$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$7 is

$$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$8

with force

$$\mathcal{M}(\phi, x, S)_c = \frac{\exp\left(-\|\phi(x)-\overline{\phi(S_c)}\|^2\right)} {\sum_{l=1}^N\exp\left(-\|\phi(x)-\overline{\phi(S_l)}\|^2\right)}.$$9

The server then applies momentum updates

$W_c$0

followed by prototype update and renormalization. Prototype Upscaling (PU) then multiplies each aligned prototype by a scalar $W_c$1 inside the local regularization term,

$W_c$2

so that the full local objective becomes

$W_c$3

A notable design decision is that ProtoNorm replaces weighted global prototype averaging with simple class-wise averaging,

$W_c$4

which the paper states avoids transmitting $W_c$5 and therefore improves privacy relative to weighted aggregation (Lee et al., 6 Jul 2025).

The reported gains are substantial. In the heterogeneous pathological setting, ProtoNorm reaches $W_c$6 on CIFAR-10, $W_c$7 on CIFAR-100, and $W_c$8 on Tiny ImageNet, outperforming FedProto, FedTGP, and other heterogeneous FL baselines. In the practical setting with $W_c$9, it reaches $L_2$00 on CIFAR-10, $L_2$01 on CIFAR-100, $L_2$02 on Flowers-102, and $L_2$03 on Tiny ImageNet (Lee et al., 6 Jul 2025). The ablation is especially diagnostic: PA alone is not enough. On CIFAR-100, $L_2$04 yields $L_2$05, almost the same as FedProto at $L_2$06, whereas $L_2$07 yields $L_2$08. This establishes that the method’s effect depends on the combination of alignment and upscaling rather than on spherical alignment alone.

6. Distinctions, misconceptions, and adjacent methods

A recurrent misconception is that any method with “proxy” in its name is a ProtoNorm method. “Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence” explicitly introduces “Proxy Normalization,” not Prototype Normalization (Labatie et al., 2021). Its proxy is a per-channel Gaussian reference distribution,

$L_2$09

used to normalize post-activations through

$L_2$10

The paper is explicit that the proxy is not learned prototypes, class centroids, memory-bank representatives, or reference activations taken from data. It is therefore adjacent only in the very broad sense of replacing batch statistics with reference statistics (Labatie et al., 2021).

A second misconception is that ProtoNorm always means normalization before $L_2$11 projection. That description is exact for OIMNet++ (Lee et al., 2022) and is a close practical interpretation of the few-shot prototype-classifier analysis (Hou et al., 2021), but it does not characterize the time-series foundation-model method, which replaces LayerNorm branches by prototype-routed LayerNorm modules (Gong et al., 15 Apr 2025), nor the federated-learning method, which optimizes prototype geometry rather than channel-wise activation statistics (Lee et al., 6 Jul 2025).

A third misconception is that prototypes are always persistent class centers used directly for inference. In OIMNet++, the prototypes used by ProtoNorm are ephemeral within-batch constructs used only to estimate normalization statistics, and running mean and variance are used at test time (Lee et al., 2022). In the time-series Transformer method, prototypes are latent distribution anchors that gate normalization branches (Gong et al., 15 Apr 2025). In the federated-learning method, prototypes are the central communicated objects of PBFL and directly shape local representation learning through nearest-prototype geometry (Lee et al., 6 Jul 2025).

The limitations also differ sharply across settings. In few-shot classification, the paper explicitly notes that prototype classifiers work best when each class is approximately unimodal and isotropic; strongly multimodal classes make a single centroid a poor summary (Hou et al., 2021). In person search, ProtoNorm depends on labeled identity supervision during training and on repeated identities within mini-batches to form informative prototypes (Lee et al., 2022). In time-series foundation model pretraining, several operational details are left unspecified, including the exact feature extraction used for routing and whether prototypes are layer-specific or globally shared (Gong et al., 15 Apr 2025). In federated learning, performance depends strongly on the scaling factor $L_2$12, and the paper states that PA alone improves little without PU (Lee et al., 6 Jul 2025).

Taken together, these works establish ProtoNorm as a heterogeneous technical term unified less by a single formula than by a recurring principle: normalization or geometry control is conditioned on prototype structure rather than treated as globally fixed or purely instance-statistical. This suggests a broader research direction in which prototypes function as compact intermediates between raw sample statistics and domain- or class-aware adaptation, but the concrete mechanism remains highly task-specific across few-shot learning, person search, time-series modeling, and federated learning.