Reliable Test-time Adaptation (ReTA)

• Reliable Test-time Adaptation (ReTA) is an approach that emphasizes stable, well-calibrated predictions and resilience to error accumulation during dynamic adaptation.
• It integrates mechanisms such as uncertainty estimation, controlled plasticity, and anti-forgetting regularization to sustain performance under continual domain shifts.
• Empirical studies indicate that ReTA methods improve both calibration and recovery in challenging scenarios like evolving, non-i.i.d. data streams and adversarial environments.

Searching arXiv for the primary ReTA paper and closely related test-time adaptation reliability work. Reliable Test-time Adaptation (ReTA) denotes a line of work in test-time adaptation (TTA) that treats reliability as a first-class objective rather than a by-product of adaptation accuracy. In this framing, the central question is not only whether a deployed model can recover performance under distribution shift, but whether it can do so while remaining stable, well-calibrated, observable, and resistant to error accumulation when both the data stream and the adapted model evolve over time. A concise formulation appears in “Towards Reliable Test-Time Adaptation: Style Invariance as a Correctness Likelihood” (Nam et al., 8 Dec 2025), which argues that the missing ingredient in TTA is not only accuracy under shift, but also well-calibrated, instance-wise uncertainty while the model is continuously changing. Across the broader literature, ReTA encompasses several complementary mechanisms: reliability-aware uncertainty estimation, failure detection and recovery, controlled plasticity, anti-forgetting regularization, sample-quality filtering, drift-aware resetting, domain-specialized model routing, and task-specific reliability criteria in settings such as vision-language modeling and monocular 3D object detection (Nam et al., 8 Dec 2025).

1. Conceptual scope and problem formulation

Reliable Test-time Adaptation arises from a limitation of standard TTA: most methods are designed to adapt a source-trained model to unlabeled target data, but many rely on entropy minimization, pseudo-labeling, or batch-statistics updates that can become unstable when the test stream is long, dynamic, non-i.i.d., or repeatedly changing. Several papers identify this unreliability explicitly. “AETTA: Label-Free Accuracy Estimation for Test-Time Adaptation” (Lee et al., 2024) characterizes the problem as the need to monitor whether adaptation is succeeding or failing under dynamic test streams, since standard methods such as TENT, EATA, SAR, CoTTA, RoTTA, and SoTTA may “blindly” keep adapting and can enter adaptation failure regimes. “AR-TTA: A Simple Method for Real-World Continual Test-Time Adaptation” (Sójka et al., 2023) argues that reliable test-time adaptation requires both stability and controlled plasticity, especially in continual TTA where the stream keeps changing over time and the model is not reset between domains.

A recurring theme is that reliability in TTA is broader than top-1 accuracy. In (Nam et al., 8 Dec 2025), reliability is tied to well-calibrated, instance-wise uncertainty under evolving distributions and adaptive model updates. In (Sójka et al., 2023), reliability refers to preserving source knowledge and preventing drift on long, temporally correlated streams. In “Persistent Test-time Adaptation in Recurring Testing Scenarios” (Hoang et al., 2023), the reliability criterion is persistence: a TTA method should not only adapt quickly, but also remain stable under repeated exposure to changing and recurring environments. “RDumb++: Drift-Aware Continual Test-Time Adaptation” (Mishra, 22 Jan 2026) further narrows the notion to stable performance over very long horizons without prediction collapse.

This suggests that ReTA is best viewed as an umbrella perspective on TTA rather than a single algorithm. The literature uses the term to refer to methods that address at least one of the following failure modes under deployment shift: poorly calibrated uncertainty, error accumulation from noisy pseudo-labels, overconfident collapse, forgetting of source knowledge, instability under temporally correlated streams, brittleness to recurring or rapidly changing domains, and inability to tell when adaptation is harming rather than helping (Nam et al., 8 Dec 2025).

2. Reliability failure modes in test-time adaptation

The literature identifies several concrete failure modes that motivate ReTA. A central issue is that calibration methods are mostly designed for a fixed model and static test distribution, so they break down in TTA where both the data stream and the adapted model evolve over time (Nam et al., 8 Dec 2025). Under changing corruption types or temporal streams, the calibration pattern itself shifts substantially, so a temperature learned for one batch or corruption does not remain valid later. At the same time, most TTA methods rely on entropy minimization or similar self-training losses, which tend to make the model increasingly overconfident, worsening reliability as adaptation proceeds (Nam et al., 8 Dec 2025).

Another major failure mode is catastrophic forgetting. AR-TTA reports that repeated self-training without source anchors leads to error accumulation and catastrophic forgetting, and that methods such as TENT, EATA, CoTTA, and SAR often degrade below the frozen source model on realistic continual streams (Sójka et al., 2023). ResiTTA similarly argues that continual distribution changes and temporally correlated test streams cause adaptation methods to overfit to recent batches, leading to model degradation and forgetting (Zhou et al., 2024). ReservoirTTA formalizes the same phenomenon as inter-domain interference and error accumulation caused by a single parameter trajectory serving multiple distinct domain optima (Vray et al., 20 May 2025).

A third failure mode is unreliable supervision. “Decoupled Prototype Learning for Reliable Test-Time Adaptation” (Wang et al., 2024) attributes unreliability to noisy pseudo-labels and to the vulnerability of cross-entropy loss to label noise. The paper argues that standard sample-centric fitting actively amplifies mistakes by pushing the model toward incorrect pseudo-labels, biasing future pseudo-labels and creating a self-reinforcing failure loop. A related argument appears in the open-set TTA work “Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization” (Lee et al., 2023), which distinguishes noisy signals from incorrect predictions and open-set predictions and shows that long-term stable adaptation is hampered by such noisy signals.

Further failure modes arise in specialized settings. In monocular 3D object detection, Fully TTA becomes unreliable because OOD test data causes a remarkable decline in object detection scores, which in turn leads to severe object omissions, rare positive detections, and many false negatives; under this regime, limited positive detections and noisy predictions cause test-time adaptation to fail (Lin et al., 2024). In adversarially contaminated streams, SAFER argues that perturbed samples can not only be misclassified themselves, but can also corrupt batch statistics, pseudo-labels, and gradients, thereby destabilizing future adaptation (Koziak et al., 21 Jun 2026). For vision-LLMs, ReTA reports that entropy often becomes unreliable under distribution shifts, causing error accumulation in a cache, while final predictions may remain unreliable because of inflexible decision boundaries (Liang et al., 13 Jul 2025).

These analyses collectively define the ReTA agenda: reliable adaptation must address not only distribution shift but also the dynamics of online learning under missing labels.

3. Reliability estimation and correctness likelihood

One major branch of ReTA focuses on estimating whether a prediction is likely correct under current model and stream conditions. The most explicit formulation is SICL in (Nam et al., 8 Dec 2025). SICL estimates instance-wise correctness likelihood by measuring prediction consistency across style-altered variants. Given an input $\mathbf{x}_i$ and its feature representation $\mathbf{z}_i$, it generates $N$ style variants $\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$ using feature-level style perturbation, and defines the core signal as the fraction of variants that preserve the original prediction: $$\gamma_i^{\text{style}} = p_{\text{init}}(y_i)= \frac{1}{N}\sum_{j=1}^{N}\mathbb{I}\!\left(\hat{y}_i=\hat{y}_{i,j}^{s\prime}\right).$$ The intuition in the paper is causal: if prediction is truly driven by content rather than nuisance style, it should remain stable across style-preserving perturbations. SICL then introduces a relaxation term using consistency under content distortion,

$$\gamma_i^{\text{content}}=\frac{1}{N}\sum_{j=1}^{N}\mathbb{I}\!\left(\hat{y}_i=\hat{y}_{i,j}^{c\prime}\right),$$

and calibrates confidence as

$$p_{\text{calibrated}}(y_i)=\omega_{\text{relaxation}}\cdot p_{\text{init}}(y_i), \qquad \omega_{\text{relaxation}} = 1-\gamma_i^{\text{content}}.$$

The method is explicitly presented as a principled, test-time-only way to estimate whether a prediction is likely correct (Nam et al., 8 Dec 2025).

AETTA addresses the same problem from an accuracy-estimation perspective rather than a calibration perspective. It estimates adaptation quality using prediction disagreement with dropout inferences: $${\tt PDD}_{\mathcal{D}^{\mathcal{T}}}(h) \triangleq \mathbb{E}_{\mathcal{D}^{\mathcal{T}}}\left[\frac{1}{N}\sum_{i=1}^{N}\mathbb{1}\left[h(X;\Theta)\neq h(X;\Theta^{\tt dropout}_i)\right]\right].$$ Under the paper’s dropout independence and confidence-prediction calibration assumptions, expected test error is approximately expected PDD (Lee et al., 2024). Because adaptation failures often violate ordinary calibration, AETTA further introduces robust confidence-prediction calibration and the practical estimator

$${\tt Err}_{\mathcal{D}^{\mathcal{T}}}(h)\approx \left(\frac{E^{\tt avg}}{E^{\tt max}}\right)^{-\alpha} {\tt PDD}_{\mathcal{D}^{\mathcal{T}}}(h), \qquad {\tt Acc}=1-{\tt Err}.$$

This makes reliability observable in unlabeled streams and enables recovery decisions such as resetting the model when estimated accuracy shows consecutive degradation or a sudden drop below a threshold (Lee et al., 2024).

DualTTA adopts a third notion of reliability: prediction stability under semantic-preserving perturbations and sensitivity under semantic-altering perturbations (Phuong et al., 19 Apr 2026). It computes discrepancy scores $\mathrm{Diff}_{sp}$ and $\mathbf{z}_i$0, uses the top-1 probability rather than the entire distribution because top-1 is less sensitive to small fluctuations and has lower variance, and partitions samples into likely-correct and likely-incorrect subsets $\mathbf{z}_i$1 and $\mathbf{z}_i$2. Reliable samples are then optimized with entropy minimization, while unreliable ones are optimized with entropy maximization or unlearning: $\mathbf{z}_i$3 This reframes reliable TTA as asymmetric optimization conditioned on a reliability estimate (Phuong et al., 19 Apr 2026).

A plausible implication is that ReTA methods increasingly replace static confidence heuristics with dynamic, sample-dependent trust signals derived from invariance, disagreement, or semantic perturbation behavior.

4. Controlled adaptation, anti-forgetting, and recovery mechanisms

A second major branch of ReTA focuses on controlling the update process so that adaptation remains stable over time. AR-TTA is exemplary in this regard. It modifies self-training TTA with a weight-averaged teacher–student setup, a small class-balanced replay buffer with mixup, and dynamic batch-normalization statistics adaptation based on estimated shift intensity (Sójka et al., 2023). The student is updated by

$\mathbf{z}_i$4

the teacher follows an EMA update

$\mathbf{z}_i$5

and BN statistics are interpolated according to estimated shift severity: $\mathbf{z}_i$6 The paper’s central reliability claim is that source BN statistics should not be discarded wholesale; instead, adaptation strength should increase when the shift is larger and decrease when the stream is closer to source-like data (Sójka et al., 2023).

ResiTTA addresses the same instability-plasticity tradeoff through parameter resilience and data quality (Zhou et al., 2024). Its ResiBN component maintains global target statistics via EMA and softly aligns them back to source BN statistics with the Wasserstein distance

$\mathbf{z}_i$7

Its EntroBank stores $\mathbf{z}_i$8 and replaces memory entries in the order of outdated samples, long-persisted over-confident samples, and uncertainty-based replacement. The stated purpose is to avoid stale samples, over-confident self-reinforcing samples, and high-uncertainty noisy samples (Zhou et al., 2024).

EATA and EATA-C pursue reliability by selective adaptation and anti-forgetting regularization (Tan et al., 2024). EATA chooses only reliable and non-redundant samples via

$\mathbf{z}_i$9

and regularizes important parameters with a Fisher penalty

$N$0

EATA-C then replaces blind entropy minimization with consistency between the full network and a sub-network and a min-max entropy regularizer based on disagreement of predicted labels, so that certain samples have entropy decreased while uncertain ones have entropy increased (Tan et al., 2024).

Other papers solve control and recovery differently. RDumb++ introduces entropy-based drift scoring and KL-divergence drift scoring,

$N$1

and triggers either full reset or soft reset

$N$2

when standardized drift exceeds a threshold (Mishra, 22 Jan 2026). PeTTA instead senses divergence from the source model in feature space via

$N$3

then adapts regularization and EMA strength dynamically: $N$4 Both methods reflect the ReTA view that reliable adaptation must include mechanisms to avoid, detect, and recover from failure (Hoang et al., 2023).

5. Specialized ReTA paradigms across tasks and architectures

Although much ReTA work is centered on image classification, the literature extends the reliability perspective to diverse settings. In vision-LLMs, “Advancing Reliable Test-Time Adaptation of Vision-LLMs under Visual Variations” (Liang et al., 13 Jul 2025) defines ReTA through two complementary modules. Consistency-aware Entropy Reweighting (CER) penalizes inconsistent samples by scaling entropy with

$N$5

where $N$6 is a consistency factor and $N$7 measures committee stability across adjacent text embeddings. Diversity-driven Distribution Calibration (DDC) models class-wise text embeddings as multivariate Gaussian distributions,

$N$8

and combines cache-based and Gaussian-based predictions via

$N$9

The paper’s stated goal is to protect both the memory and the classifier under substantial visual variation (Liang et al., 13 Jul 2025).

In monocular 3D object detection, MonoTTA defines reliability at the level of detected objects rather than class probabilities. It uses a self-adaptive threshold

$\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$0

to select reliable high-score objects and combines this with negative regularization over low-score objects: $\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$1 This is explicitly designed to address object omissions, scarce reliable positives, and many false negatives under OOD shift (Lin et al., 2024).

For adversarially contaminated streams, SAFER provides a reliability-guided ensembling wrapper. It generates stochastic augmentations, computes cross-view feature agreement,

$\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$2

drops the least reliable view with the cc_drop rule, and pools the remaining predictions (Koziak et al., 21 Jun 2026). The method is explicitly training-free and preserves the wrapped TTA objective while replacing brittle single-view predictions with a multi-view, reliability-weighted predictor (Koziak et al., 21 Jun 2026).

ReservoirTTA addresses prolonged TTA by keeping a reservoir of domain-specialized experts and routing incoming batches according to online clustering over style features,

$\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$3

Only the selected expert is updated,

$\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$4

while prediction uses a soft weighted ensemble of experts (Vray et al., 20 May 2025). This makes reliability domain-aware rather than monolithic.

These task-specific variants indicate that ReTA is not tied to one modality or backbone. The literature explicitly discusses compatibility with ViTs in SICL (Nam et al., 8 Dec 2025), GN/LN-based architectures and LayerNorm-based ViTs in DualTTA (Phuong et al., 19 Apr 2026), CLIP-based VLMs in ReTA for visual variations (Liang et al., 13 Jul 2025), and segmentation or driving-style domains in PeTTA, EATA-C, and ReservoirTTA (Hoang et al., 2023).

6. Empirical evidence, benchmarks, and evaluation criteria

Empirical evaluation in ReTA is shaped by the claim that reliability must be assessed under dynamic conditions rather than static held-out test sets. SICL uses three corrupted-image benchmarks—CIFAR10-C, CIFAR100-C, and ImageNet-C—each at corruption severity level 5, and evaluates three architectures: ResNet-50, ResNet-101, and ViT (Nam et al., 8 Dec 2025). It distinguishes a benign scenario with a fixed corruption type and i.i.d. test stream from a dynamic scenario with temporally correlated corruption streams simulated with a Dirichlet process. Its primary metric is cumulative Expected Calibration Error over online test batches,

$\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$5

The paper reports that SICL achieves the best calibration in almost all settings, winning 63 out of 72 configuration combinations, and reduces calibration error by 13.0 percentage points relative to the baselines on average; the improvement is larger in the dynamic scenario, with 15.6\%p on average versus 10.4\%p in the benign setting (Nam et al., 8 Dec 2025).

AETTA uses mean absolute error between estimated and true batch accuracy as its reliability metric and reports an average 19.8 percentage-point improvement over baselines in estimation error (Lee et al., 2024). Its model-recovery case study yields an average 11.7\%p performance improvement and outperforms DistShift by 3.0\%p (Lee et al., 2024). These numbers are not adaptation metrics in the usual sense; they are measurements of observability and actionability under unlabeled deployment.

AR-TTA emphasizes realistic continual streams such as CLAD-C and SHIFT-C rather than only synthetic corruptions. On CLAD-C, source accuracy is 81.3%, and AR-TTA reaches 83.7% with 63.1 AMCA, while several baselines perform worse than the frozen source model (Sójka et al., 2023). On SHIFT-C, source is 89.5% and AR-TTA reaches 90.2% with 90.2 AMCA, while TENT drops to 57.6% and CoTTA to 47.2% (Sójka et al., 2023). The explicit message is that adaptation can hurt under realistic continual shift unless reliability mechanisms are built in.

Other ReTA papers adapt evaluation to their own claims. ResiTTA reports best average classification error on CIFAR10-C, CIFAR100-C, and ImageNet-C under practical TTA at severity 5 (Zhou et al., 2024). PeTTA proposes episodic TTA with 20 revisits to quantify error accumulation in recurring environments and reports average errors of 23.0 on CIFAR-10-C, 35.2 on CIFAR-100-C, and 42.5 on DomainNet, all lower than RoTTA (Hoang et al., 2023). RDumb++ stresses long-horizon CTTA on CCC-medium with 1 million samples per run and reports mean accuracies showing consistent gains over RDumb across speeds 1000, 2000, and 5000 (Mishra, 22 Jan 2026). DualTTA reports that the reliable subset remains the most accurate subset even under severe noise, though on ImageNet-C under extreme noise the reliable subset can still contain up to 29% incorrect predictions (Phuong et al., 19 Apr 2026).

A plausible implication is that ReTA evaluation increasingly shifts away from single-pass, short-horizon average accuracy toward streaming calibration, collapse resistance, recovery efficacy, recurrence robustness, and trustworthiness under adversarial or open-set perturbations.

7. Debates, limitations, and emerging directions

The ReTA literature shares a broad diagnosis but differs on what reliability should mean operationally. One perspective emphasizes calibrated uncertainty and instance-wise correctness likelihood, as in SICL and EATA-C (Nam et al., 8 Dec 2025). Another emphasizes stability and anti-forgetting under continual or recurring streams, as in AR-TTA, PeTTA, RDumb++, and ReservoirTTA (Sójka et al., 2023). A third emphasizes the ability to monitor and intervene in adaptation without labels, as in AETTA (Lee et al., 2024). These perspectives are not mutually exclusive, but the coexistence of different reliability objectives implies that no single metric fully captures ReTA.

Several papers also note limitations. AR-TTA requires a source replay memory, which may be problematic in memory-constrained or privacy-sensitive settings (Sójka et al., 2023). AETTA’s theory relies on dropout independence and a calibration-like condition, and the robust version omits the intractable term $\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$6 in practice (Lee et al., 2024). EATA needs a small ID sample set for Fisher estimation (Tan et al., 2024). SAFER is effective mainly under black-box transfer attacks and its robustness drops sharply under a defense-aware EOT attack (Koziak et al., 21 Jun 2026). TEA directly addresses covariate shift and calibration through an energy-based perspective, but SGLD is time-consuming and sometimes unstable, and mild shifts may not benefit (Yuan et al., 2023). DPL is specialized for TTA and explicitly underperforms cross-entropy in ordinary supervised training (Wang et al., 2024).

A separate debate concerns whether reliability should be handled through better sample selection, better uncertainty estimation, stronger memory and replay, or multiple hypotheses instead of one. The most explicit challenge to single-trajectory TTA appears in “Multi-Hypothesis Test-Time Adaptation to Mitigate Underspecification” (Shamsi et al., 30 Jun 2026), which argues that entropy-based adaptation is fundamentally underconstrained and maintains a population of adaptation particles: $\{\mathbf{z}_{i,j}^{s\prime}\}_{j=1}^N$7 Its argument is that reliable TTA should be framed as multi-hypothesis inference rather than point estimation (Shamsi et al., 30 Jun 2026). This suggests a possible synthesis with earlier ReTA work: sample reliability, uncertainty calibration, drift detection, and anti-forgetting may all address symptoms of the deeper underspecification of entropy-based adaptation.

Another emerging direction is explicit supervision at test time. “Active Test-Time Adaptation: Theoretical Analyses and An Algorithm” (Gui et al., 2024) argues that standard fully test-time adaptation may be too brittle under true domain shifts and introduces ATTA, in which the model actively requests a limited number of labels from streaming test data. The paper proves that there exists a choice of weights in ATTA that yields a strictly tighter test error bound than plain TTA and uses entropy to split current test data into source-like pseudo-labeled samples and informative high-entropy samples for active annotation (Gui et al., 2024). This suggests that one future interpretation of ReTA may include budgeted intervention rather than insisting on purely unsupervised adaptation.

Overall, the literature portrays Reliable Test-time Adaptation as a transition from “adapt as much as possible” to “adapt only when and how it is trustworthy.” The field’s common premise is that test-time adaptation in deployment is not only an optimization problem but also a reliability problem: predictions should remain calibrated, updates should remain stable, failure should be detectable, and long-horizon adaptation should not collapse under the very conditions it seeks to solve (Nam et al., 8 Dec 2025).