Mitigating the ID-OOD Tradeoff in Open-Set Test-Time Adaptation

Abstract: Open-set test-time adaptation (OSTTA) addresses the challenge of adapting models to new environments where out-of-distribution (OOD) samples coexist with in-distribution (ID) samples affected by distribution shifts. In such settings, covariate shift-for example, changes in weather conditions such as snow-can alter ID samples, reducing model reliability. Consequently, models must not only correctly classify covariate-shifted ID (csID) samples but also effectively reject covariate-shifted OOD (csOOD) samples. Entropy minimization is a common strategy in test-time adaptation to maintain ID performance under distribution shifts, while entropy maximization is widely applied to enhance OOD detection. Several studies have sought to combine these objectives to tackle the challenges of OSTTA. However, the intrinsic conflict between entropy minimization and maximization inevitably leads to a trade-off between csID classification and csOOD detection. In this paper, we first analyze the limitations of entropy maximization in OSTTA and then introduce an angular loss to regulate feature norm magnitudes, along with a feature-norm loss to suppress csOOD logits, thereby improving OOD detection. These objectives form ROSETTA, a $\underline{r}$obust $\underline{o}$pen-$\underline{se}$t $\underline{t}$est-$\underline{t}$ime $\underline{a}$daptation. Our method achieves strong OOD detection while maintaining high ID classification performance on CIFAR-10-C, CIFAR-100-C, Tiny-ImageNet-C and ImageNet-C. Furthermore, experiments on the Cityscapes validate the method's effectiveness in real-world semantic segmentation, and results on the HAC dataset demonstrate its applicability across different open-set TTA setups.

• The paper's primary contribution is the ROSETTA framework that decouples entropy objectives, balancing classification accuracy and OOD rejection.
• It introduces angular alignment for ID samples and norm suppression for OOD, avoiding uniform feature norm inflation.
• Empirical results show improved AUROC, balanced accuracy, and mIoU across diverse benchmarks, validating its robustness.

Mitigating the ID–OOD Tradeoff in Open-Set Test-Time Adaptation: ROSETTA

Problem Formulation and Motivation

Open-set test-time adaptation (OSTTA) targets a critical challenge in real-world deployment: adapting neural models to covariate-shifted environments that concurrently contain both in-distribution (ID) samples (covariate-shifted ID, denoted csID) and out-of-distribution (csOOD) samples. Conventional TTA approaches—designed mainly for closed-set scenarios—are insufficient because models must not only maintain csID classification under distribution shift but also effect robust csOOD rejection. Prior methods attempt to merge test-time entropy minimization (for ID) and entropy maximization (for OOD), but their intrinsic objectives are antagonistic, producing a marked trade-off between csID and csOOD performance.

The paper’s primary contribution is an in-depth analysis of this entropy-based conflict and the formulation of ROSETTA, a decoupled objective that regulates csID and csOOD with diverse mechanisms. The method is robustly validated across classification, semantic segmentation, and multimodal video-audio settings.

Analysis of Entropy-Based OSTTA and Its Limitations

Standard entropy minimization, as in TENT, enhances confidence in corrupted but ID samples by tightening the predictive distribution. However, as demonstrated analytically and empirically, this is achieved mainly by inflating the feature vector norms, not by improved alignment with class prototypes. This uniform norm amplification also lifts csOOD sample norms, reducing separability and degrading downstream OOD detection.

Conversely, employing confidence-based (entropy or energy-based) OOD detectors to gate entropy maximization and minimization on presumed OOD/ID partitions is inherently limited: no available OOD detector can reliably split incoming batches, and misclassification (csID as csOOD or vice versa) leads to counterproductive updates on both groups. These imperfections are consistently visualized across benchmarks, as shown in misclassification exemplars.

Figure 1: Examples of misclassification under the optimal entropy threshold, revealing the inability of energy or entropy OOD detectors to reliably split csID and csOOD under significant covariate shift.

The ROSETTA Framework: Objective Decoupling and Loss Formulation

ROSETTA circumvents the ID–OOD objective conflict by decoupling adaptation for classification and detection:

• For ID (csID): Besides entropy minimization, it introduces an angular alignment loss, optimizing the cosine similarity between sample embeddings and running class prototypes via EMA. This avoids indiscriminate norm inflation and strengthens semantic consistency.
• For OOD (csOOD): Rather than maximizing entropy, ROSETTA explicitly suppresses feature norms by minimizing the $l_1$-norm of activations for samples identified as csOOD, thus encouraging lower logit magnitudes and cleaner separation.

  Figure 2: Overview of the ROSETTA pipeline, illustrating the combination of angular alignment and norm suppression for dual ID and OOD objectives.

The total objective is:

$$\mathcal{L}_{\text{ROSETTA}} = \mathcal{L}_{\text{csID}} + \gamma_1 \mathcal{L}_{\text{ang}} + \gamma_2 \mathcal{L}_{\text{norm}}$$

This formulation removes the oscillatory effects of switching entropy policies and aligns test-time adaptation with theoretical underpinnings of discriminative feature geometry.

Empirical Analysis and Validation

Empirical results cover four domains: image recognition (CIFAR-10-C, CIFAR-100-C, Tiny-ImageNet-C, ImageNet-C), real-world semantic segmentation (Cityscapes-to-ACDC), a multimodal action recognition dataset (HAC), as well as robustness analyses on class ratios and unknown-class counts.

In classification tasks, all competing entropy-based TTA methods (e.g., UniEnt, UniEnt+) manifest strong ID/OOD trade-offs—models tuned for OOD detection sacrifice classification accuracy, and vice versa. ROSETTA consistently dominates both axes, establishing state-of-the-art OSCR and AUROC without degrading ID classification.

• On CIFAR-10-C, AUROC improves to 93.75 with balanced Acc (84.34), and analogous trends hold on all other benchmarks.
• Semantic segmentation on adverse-weather datasets shows meaningful mIoU gains, especially on challenging corruptions (Rain: +0.24, Snow: +0.25).

ROSETTA’s performance is robust to high imbalance in csID/csOOD ratios, as well as the absence of csOOD at test time, where entropy-maximization-based TTA methods typically collapse.

Figure 3: OSCR heatmaps of ROSETTA across varying adaptation hyperparameters on CIFAR-10-C and CIFAR-100-C, confirming stability and robustness.

Ablation studies confirm the necessity of both angular and norm-based loss terms: norm suppression alone closes csID/csOOD feature magnitude gaps, but the angular term is critical for semantic alignment and csID accuracy.

Figure 4: Effect of increasing OOD loss weight $\tau$ on csID accuracy and csOOD AUROC, demonstrating no pronounced trade-off in ROSETTA compared to baseline entropy policies.

Further, t-SNE and logit-profiling analyses (see paper) clarify that ROSETTA distributes logit mass more evenly across csOOD while tightening one-hotness for csID, boosting separability.

Implications and Future Directions

ROSETTA presents a principled method to address the dual objectives of open-set TTA. The explicit decoupling of test-time objectives enables stable, reliable deployment in dynamic environments, including vision and multimodal recognition under severe corruption, weak OOD detector reliability, or shifting input priors.

Practically, the framework can be integrated atop most entropy-minimization-based TTA methods with minimal computational overhead. Theoretically, it highlights the limitations of confidence-based OOD detection and the importance of direct feature space control for robust adaptation.

Open questions remain regarding the detection/allocation of OOD assignments and extension to adaptation strategies beyond entropy-driven frameworks, including self-supervised or non-classification-centric open-world deployments.

Conclusion

The paper delivers a thorough diagnosis of entropy-based losses for open-set test-time adaptation and introduces ROSETTA, which leverages angular and norm-based losses for decoupled, simultaneous optimization of csID and csOOD objectives. Empirical evidence across multiple benchmarks demonstrates that ROSETTA reliably mitigates the ID–OOD trade-off and is robust, generalizable, and extensible to complex, real-world and multimodal domains. This positions ROSETTA as a strong candidate mechanism for future research and deployment in adaptive, open-set environments.