ProtoTTA: Prototype-Guided Test-Time Adaptation

• ProtoTTA is a test-time adaptation framework that leverages intermediate prototype activations to improve robustness in prototypical models under distribution shifts.
• It minimizes the entropy of the prototype-similarity distribution and employs geometric filtering to select reliable samples for gradient-based updates.
• By updating only normalization layers and structural add-ons, ProtoTTA enhances both prediction accuracy and interpretability while preserving semantic focus.

ProtoTTA, short for Prototype-Guided Test-Time Adaptation, is a test-time adaptation framework for prototypical models that uses intermediate prototype activations rather than only final output logits to improve robustness under distribution shift. In the reported formulation, ProtoTTA minimizes the entropy of the prototype-similarity distribution, restricts updates to a geometrically filtered reliable set of samples, and regularizes adaptation by prototype-importance weights and model-confidence scores. It is presented as a general method spanning fine-grained vision, histopathology, and NLP, with experiments on four prototypical backbones and four benchmarks showing improved robustness over standard output entropy minimization together with restored semantic focus in prototype activations (Abootorabi et al., 16 Apr 2026).

1. Problem setting and motivation

Prototype-based networks such as ProtoPNet, Deformable ProtoPNet, ProtoViT / ProtoPFormer, and ProtoLens classify by comparing an input representation to a bank of learned prototypes. This yields interpretable evidence chains of the form “this looks like that,” because specific input regions or textual fragments can be matched to class-relevant prototypes (Abootorabi et al., 16 Apr 2026).

Under distribution shifts, however, the prototype-selection mechanism itself can degrade. The reported failure mode is not limited to a drop in class confidence: correct prototypes may be suppressed, while spurious prototypes, including background artifacts, may become activated. This affects both prediction quality and interpretability. Standard test-time adaptation methods such as Tent, SAR, and MEMO are described as treating the model as a black box and minimizing Shannon entropy on output logits,

$$L_{\mathrm{ent}}(x) = - \sum_c q_c(x)\log q_c(x), \qquad q_c(x)=\mathrm{softmax}_c(\mathrm{logits}(x)).$$

ProtoTTA departs from this output-only view by adapting the model through prototype-level activations themselves (Abootorabi et al., 16 Apr 2026).

The central motivation is therefore twofold. First, prototype models expose internal signals that are more structured than output logits. Second, distribution shift in such models is partly a failure of semantic prototype focus, so an adaptation objective placed directly on prototype activations can target the mechanism that becomes corrupted. The paper explicitly characterizes ProtoTTA as the first TTA framework to leverage these prototype-level activations directly (Abootorabi et al., 16 Apr 2026).

2. Formalization of prototype signals

ProtoTTA is defined for models with a learned prototype bank $\{p_k\}_{k=1}^K$. For an input $x$, a feature extractor $f(\cdot)$ produces an embedding $f(x)$, and the model computes a prototype-similarity vector

$s(x) = [s_1(x), \ldots, s_K(x)].$

The exact similarity depends on the backbone (Abootorabi et al., 16 Apr 2026).

For cosine-similarity backbones, the prototype score is

$$s_k(x) = \cos(f(x), p_k) = \frac{f(x)\cdot p_k}{\|f(x)\|\;\|p_k\|}.$$

For Euclidean-distance backbones such as ProtoPNet, raw distances

$d_k(x)=\|f(x)-p_k\|^2$

are converted into similarities through a log-inverse distance kernel,

$s_k(x)=\log \frac{d_k(x)+1.0}{d_k(x)+10^{-4}}.$

In both cases, the similarity vector is the intermediate signal that quantifies how much the input resembles each prototype (Abootorabi et al., 16 Apr 2026).

ProtoTTA first induces a probability distribution over prototypes by applying a softmax:

$$p_k(x)=\frac{\exp(s_k(x))}{\sum_{j=1}^K \exp(s_j(x))}.$$

It then minimizes the Shannon entropy of this prototype-similarity distribution,

$\{p_k\}_{k=1}^K$0

The intended effect is to push the prototype distribution toward a one-hot regime in which one prototype dominates, thereby restoring decisive prototype-level reasoning (Abootorabi et al., 16 Apr 2026).

The implementation described in the paper further maps each raw similarity $\{p_k\}_{k=1}^K$1 to a probability-like activation $\{p_k\}_{k=1}^K$2, either by linear scaling or, for text, with a sigmoid:

$\{p_k\}_{k=1}^K$3

ProtoTTA then applies a binary entropy to each mapped activation,

$\{p_k\}_{k=1}^K$4

so that uncertain mid-range activations near $\{p_k\}_{k=1}^K$5 are penalized, whereas decisive matches or decisive rejections are favored (Abootorabi et al., 16 Apr 2026).

3. Reliable-set construction and regularized adaptation objective

A key design element is geometric filtering, introduced to avoid adapting on samples whose prototype activations are themselves unreliable or corruption-dominated. For a batch of test samples, ProtoTTA defines a reliable set

$\{p_k\}_{k=1}^K$6

where $\{p_k\}_{k=1}^K$7 is a threshold; the paper gives 0.5 as an example choice. Only samples with at least one strong prototype match are allowed to drive test-time updates. The description also notes that one may optionally require low output entropy on pseudo-label logits for additional confidence (Abootorabi et al., 16 Apr 2026).

ProtoTTA further weights adaptation by two quantities. The first is the prototype-importance weight $\{p_k\}_{k=1}^K$8, defined as the weight of prototype $\{p_k\}_{k=1}^K$9 in the final classification head for class $x$0. The second is the model confidence $x$1, for example the maximum softmax logit or $x$2 of the output distribution. Using the pseudo-label $x$3, the paper defines the per-sample loss

$x$4

where $x$5 is the set of prototypes for the pseudo-label class (Abootorabi et al., 16 Apr 2026).

Aggregating over the reliable set yields the full ProtoTTA objective:

$x$6

This construction makes the method more selective than plain entropy minimization on model outputs. Adaptation is driven only by samples with strong prototype evidence, and within those samples it emphasizes prototypes that matter most to the current class decision. A plausible implication is that ProtoTTA is designed to reduce the risk that corrupted or semantically irrelevant prototype activations dominate the update.

4. Optimization procedure and implementation regime

The reported algorithm proceeds batch by batch on the test stream. Starting from a pre-trained prototype model $x$7, ProtoTTA initializes adapted parameters $x$8 and then performs, for each test batch, a forward pass to compute prototype similarities and pseudo-labels, reliable-set selection through geometric filtering, optional consensus aggregation for sub-prototypes, loss computation via $x$9, and one gradient update step (Abootorabi et al., 16 Apr 2026).

The implementation details are deliberately restrictive about what may change at test time. Only normalization layers + structural add-ons are adapted: LayerNorm/BatchNorm, attention biases in ViTs, and $f(\cdot)$0 convolutions in CNNs. All other weights remain frozen. Optimization uses one gradient step per batch with Adam and $f(\cdot)$1 (Abootorabi et al., 16 Apr 2026).

For architectures with sub-prototypes, ProtoTTA uses top-K mean consensus aggregation to pool them into representative $f(\cdot)$2. The paper states that geometric filtering, top-K mean aggregation, and updating only normalization + structural add-ons are all critical to ProtoTTA’s stability and performance. It also reports that long-sequence experiments show no catastrophic forgetting (Abootorabi et al., 16 Apr 2026).

This update regime clarifies an important property of ProtoTTA: it is not a purely cache-based or backpropagation-free method. Its adaptation signal is prototype-centric, but the actual mechanism is gradient-based test-time optimization over a limited subset of parameters. That makes it conceptually closer to entropy-minimization TTA methods than to retrieval-based adaptation, while still differing from them in where the entropy is applied.

5. Empirical performance, interpretability, and efficiency

The empirical study spans four prototypical backbones and four diverse benchmarks covering fine-grained vision, histopathology, and NLP. Across these settings, ProtoTTA is reported to improve robustness over standard output entropy minimization, and the improvements are paired with gains in interpretability-oriented diagnostics rather than accuracy alone (Abootorabi et al., 16 Apr 2026).

Representative task results are summarized below.

Setting	Comparator	Reported result
CUB-200-C, ProtoViT	Unadapted / Tent / EATA / ProtoTTA	51.9 ± 13.0% / 54.0 ± 12.8% / 58.9 ± 10.8% / 60.1 ± 10.6%
Stanford Dogs-C, ProtoPFormer	EATA / ProtoTTA+	57.2 ± 8.4% / 57.7 ± 8.4%
SICAPv2-C, ProtoPNet	EATA / ProtoTTA+	55.6 ± 4.7% / 55.8 ± 4.9%
Amazon-C, ProtoLens	Unadapted / Tent / EATA / ProtoTTA	80.81 ± 7.43% / 80.08 ± 7.80% / 80.84 ± 6.91% / 81.33 ± 7.45%

The paper also introduces three interpretability-aware metrics. Prototype Activation Consistency (PAC) measures cosine agreement between clean and adapted prototype-activation vectors. Weighted Prototype Alignment (PCA-W) measures whether highly activated prototypes belong to the ground-truth class, weighted by activation strength and classifier weight. Prediction Stability measures how often adaptation preserves clean predictions. On ProtoViT on CUB-200-C, the reported values are PAC = 91.9 ± 2.9%, PCA-W = 82.6 ± 7.1%, and Stability = 68.7%; the accompanying efficiency metrics are Selection Rate = 58.0% and Relative Speed = 95.7% (Abootorabi et al., 16 Apr 2026).

A notable aspect of the evaluation is the explicit use of a vision-LLM evaluation framework for adaptation dynamics. The protocol constructs reasoning boards containing a corrupted image, predicted-class prototypes, and any-class prototypes, and submits them to a VLM agent such as Qwen3-VL, which returns integer scores from 1 to 5 for Focus Relevance, Prototype Match, and Overall Adaptation Quality. On the reported setup, ProtoTTA obtains 4.30 ± 0.90 for Focus Relevance, 3.86 ± 0.81 for Prototype Match, and 3.78 ± 0.97 for Overall Quality, compared with EATA at 4.18 ± 0.90, 3.79 ± 0.82, and 3.75 ± 0.94, and the Unadapted model at 4.14 ± 0.98, 3.66 ± 0.87, and 3.53 ± 1.02. The paper further reports that PCA-W correlates with VLM scores with Pearson $f(\cdot)$3 pooled and $f(\cdot)$4 on ProtoTTA alone (Abootorabi et al., 16 Apr 2026).

These results are significant because the evaluation does not treat interpretability as an anecdotal by-product. Instead, interpretability is operationalized with explicit metrics and an external VLM-based assessment, and the reported gains suggest that ProtoTTA improves not only robustness but also the semantic correctness of prototype activations under shift.

6. Relationship to contemporaneous prototype-based TTA methods

The name ProtoTTA should be distinguished from a separate April 2026 work, "Prototype-Based Test-Time Adaptation of Vision-LLMs", which introduces PTA rather than ProtoTTA (Huang et al., 23 Apr 2026). The two methods share a prototype-centered perspective, but they target different model classes and use different update mechanisms.

PTA is formulated for vision-LLMs such as CLIP under unlabeled test streams. It replaces cache-based TTA designs with a fixed set of class-specific knowledge prototypes, one per class, updated by an adaptive exponential moving average based on zero-shot class confidence. The reported design is backpropagation-free, eliminates cache maintenance and retrieval, and has per-sample complexity $f(\cdot)$5, independent of any cache capacity (Huang et al., 23 Apr 2026).

By contrast, ProtoTTA is a general framework for prototypical models and adapts by backpropagating a prototype-level entropy objective over selected parameters. In other words, ProtoTTA changes model parameters at test time, whereas PTA accumulates test-stream information directly in class prototypes without gradient-based optimization (Abootorabi et al., 16 Apr 2026, Huang et al., 23 Apr 2026).

The distinction is also reflected empirically. PTA reports, for ViT-B/16, a cross-domain average accuracy of 69.38%, compared with CLIP (zero-shot) 65.64%, cache-based TDA 67.97%, and cache-based ADAPT 68.74%. On ImageNet-1K on a single NVIDIA RTX 3090, it reports 81.8 FPS (92%), compared with CLIP 89.3 FPS (100%), TDA 44.6 FPS (50%), and ADAPT 12.9 FPS (14%). It also reports 55.13% on ModelNet-C average at Severity=2, compared with ULIP (zero-shot) 47.52% and Point-Cache 53.70% (Huang et al., 23 Apr 2026).

A useful way to read the two papers together is that they delineate two prototype-based TTA regimes. ProtoTTA focuses on prototype-guided gradient adaptation in interpretable prototypical networks, whereas PTA focuses on prototype accumulation for cache-free, backpropagation-free adaptation in VLMs. This suggests that “prototype-based TTA” is not a single method family but a broader design space organized around where prototype information enters the adaptation loop: as an optimization target, as an online memory, or both.

7. Significance and open technical implications

Within the scope of the reported results, ProtoTTA advances test-time adaptation in two linked directions. First, it treats intermediate prototype signals as first-class adaptation targets rather than secondary explanations of output behavior. Second, it evaluates adaptation with metrics and procedures that explicitly address semantic focus, prototype-class alignment, and externally judged reasoning quality (Abootorabi et al., 16 Apr 2026).

This framing also clarifies what ProtoTTA is not. It is not merely output entropy minimization applied to a prototype model, because its core loss is placed on the prototype-similarity distribution and its reliable-set construction depends on prototype geometry. It is also not restricted to visual CNNs: the paper reports results on ProtoViT / ProtoPFormer and ProtoLens, extending the setting to Vision Transformers and textual embeddings (Abootorabi et al., 16 Apr 2026).

Several technical implications follow from the reported ablations. The importance of geometric filtering indicates that prototype-level uncertainty must itself be controlled before adaptation is trusted. The importance of top-K mean aggregation indicates that sub-prototype architectures require a robust pooling step. The effectiveness of updating only normalization layers + structural add-ons suggests that preserving learned prototype representations while adjusting a limited adaptation surface is central to stability. These are direct experimental findings; a plausible broader implication is that interpretable models under shift may benefit from adaptation mechanisms that preserve prototype semantics while recalibrating how those prototypes are selected and weighted.

Taken together, ProtoTTA occupies a specific position in the 2026 TTA landscape: a prototype-guided, gradient-based framework for interpretable prototypical models that seeks to restore both robustness and semantically correct prototype focus under distribution shift, and that evaluates those goals with both standard robustness benchmarks and explicit interpretability criteria (Abootorabi et al., 16 Apr 2026).