TINS: Test-time ID-prototype-separated Negative Semantics Learning for OOD Detection

Abstract: Vision-LLMs enable OOD detection by comparing image alignment with ID labels and negative semantics. Existing negative-label-based methods mainly rely on static negative labels constructed before inference, limiting their ability to cover diverse and evolving OOD concepts. Although test-time expansion provides a natural solution, naively learning negative semantics from potential OOD samples may introduce hard ID contamination. To address this issue, we propose a \textbf{T}est-time \textbf{I}D-prototype-separated \textbf{N}egative \textbf{S}emantics learning method, termed \textbf{TINS}. TINS learns sample-specific negative text embeddings via image-to-text modality inversion and introduces ID-prototype-separated regularization to keep them separated from ID semantics. To further stabilize negative semantics expansion, TINS employs group-wise aggregation scoring and a buffer update strategy. Extensive experiments across Four-OOD, OpenOOD, Temporal-shift, and Various ID settings show consistent improvements over strong baselines. Notably, on the Four-OOD benchmark with ImageNet-1K as ID, TINS reduces the average FPR95 from 14.04\% to 6.72\%. Our code is available at https://github.com/zxk1212/tins.

• The paper presents TINS, a novel approach that generates sample-specific negative semantics while explicitly distancing from ID prototypes to improve OOD detection.
• It employs modality inversion with a regularizer, group-wise aggregation scoring, and a dynamic bank-buffer system to adapt to evolving OOD distributions.
• Experimental results demonstrate significant improvements in AUROC and FPR95 metrics, validating TINS' robustness across various benchmarks and temporal shifts.

Test-Time Adaptive Negative Semantics via ID-Prototype Separation for OOD Detection

Introduction

This paper introduces TINS (Test-time ID-prototype-separated Negative Semantics learning), a method designed to enhance out-of-distribution (OOD) detection with vision-LLMs (VLMs) by dynamically learning semantic negatives during test time while explicitly preventing contamination from in-distribution (ID) prototypes (2605.10756). Classical OOD detection strategies in VLMs primarily rely on static, pre-constructed negative label sets and fail to adaptively capture the spectrum of OOD concepts encountered during deployment. While recent test-time adaptive methods attempt to expand negative semantics using online data, they are prone to the inclusion of ambiguous or hard ID-aligned features, significantly degrading ID/OOD separability. TINS addresses these limitations by (1) inverting image features to generate sample-specific negative semantics while regularizing these away from ID prototypes, and (2) introducing a stable group-wise aggregation mechanism and a dynamic bank-buffer management system for negative features.

Methodology

TINS is grounded in the hypothesis that effective OOD detection in VLMs requires adaptive, dynamically learned negative semantics, which remain robustly separated from the semantic centers of ID categories (ID prototypes).

ID-Prototype-Separated Modality Inversion

For a test image identified as potentially OOD (using an initial alignment-based score function), TINS learns a negative text embedding by optimizing a pseudo-token appended to a CLIP-style prompt. Crucially, the inversion loss not only aligns the learned text embedding with the image feature, but also adds an explicit regularizer enforcing a margin from all ID prototypes:

$$\mathcal{L}_{\mathrm{ours}} = 1 - \cos(\boldsymbol{t}^-, \boldsymbol{v}) + \lambda \cdot \frac{1}{C}\sum_{c=1}^C (1 + \cos(\boldsymbol{t}^-, \boldsymbol{\mu}_c))$$

where $\boldsymbol{v}$ and $\boldsymbol{\mu}_c$ represent the candidate image feature and class prototypes, respectively; $\lambda$ controls the regularization strength. This prevents semantic negatives from drifting toward ambiguous or hard-to-distinguish ID semantics.

Two initialization strategies for the pseudo-token embedding are explored:

• Random initialization (higher flexibility but slower convergence)
• Vocabulary-prior initialization (selects a corpus word minimizing the overall objective, leading to faster and more stable convergence)

Figure 1: t-SNE visualization contrasting static and dynamically learned negative features, highlighting the enhanced separation between ID and OOD samples yielded by TINS.

Group-Wise Aggregation Scoring

Expanding the set of negative semantics at test-time could inflate the denominator in the scoring function, artificially reducing scores for both ID and OOD and degrading calibration. To address this, TINS partitions static and dynamic negatives into balanced groups and computes the average of group-level ID-versus-negative scores. Theoretical analysis shows that more balanced groupings, achieved by random permutation, make the OOD detection score more robust and discriminative—a result substantiated by both ablations and performance metrics.

Dynamic Bank-Buffer and Buffer Update

To ensure adaptation to temporal shifts in OOD distributions, TINS maintains a capped dynamic negative bank and a sliding buffer. When the bank is full and new high-quality negatives are generated, the most separated negatives are retained, and overflow is moved to the buffer. If both are saturated, the buffer is merged using a random selection protocol to avoid semantic collapse and maintain diversity. This mechanism counters overfitting to early OOD distributions and secures long-term robustness under distributional drift.

Figure 2: Overview of the TINS framework, highlighting online adaptive learning of negative semantic embeddings, dynamic bank/buffer management, and group-wise aggregation scoring.

Experimental Results

TINS is evaluated extensively on Four-OOD and OpenOOD benchmarks, as well as under temporal OOD shifts and diverse ID dataset settings.

Key empirical findings:

• On Four-OOD with ImageNet-1K as ID and the CLIP ViT-B/16 backbone, TINS achieves an AUROC of 98.51% and an average FPR95 of 6.72%, representing a significant {\bf FPR95 reduction (7.32% absolute) over the strongest contemporary baseline, InterNeg}.
• On the OpenOOD benchmark, TINS delivers state-of-the-art performance on both near-OOD (FPR95: 57.88%) and far-OOD (FPR95: 12.99%) scenarios, outstripping test-time adaptation contemporaries.

  Figure 3: Comparative OOD detection results on ImageNet-1K across methods and effect of pseudo-token initialization strategy.

• Robust performance transfers across various ID distributions (Food-101, ImageNet-Sketch, ImageNet-R, ImageNet-V2) and CLIP backbones.
• Ablations confirm the necessity of all three major components: (1) ID-prototype separation regularizer, (2) group-wise aggregation with permutation, and (3) the buffer-based update strategy. Omission of any component leads to a marked drop in OOD detection accuracy.

Figure 4: Sensitivity analyses for regularization coefficient $\lambda$, ID-shot number for prototype estimation, and expansion threshold $\beta$; performance is stable across a broad range of hyperparameters.

Temporal-shift experiments (distributional drift) demonstrate the buffer’s efficacy, with substantial FPR95 improvements on all orderings of OOD dataset arrival (reducing average FPR95 to 9.60% compared to 14.11% without buffer).

Figure 5: Ablation results on batch size, bank capacity $M$, and buffer ratio $\rho$, confirming robustness to key implementation choices.

Discussion

The explicit separation from ID prototypes confers robust resistance to label-space contamination—a key limitation in earlier test-time dynamic negative expansion approaches. The group-wise aggregation scoring framework offers a calibrated and theoretically principled antidote to denominator inflation, a subtle statistical artifact often ignored in prior work. The dynamic bank-buffer protocol provides resilience to both short-term and long-term semantic drift in real-world OOD samples, rendering TINS suitable for streaming and continually evolving environments.

From a practical standpoint, TINS expands the reliable operational envelope of OOD detectors in safety-critical and open-world VLM applications, such as medical image analysis and autonomous systems, where hard ID/OOD boundaries are required and OOD concepts rapidly evolve. Theoretically, TINS sets a new standard for fully test-time, training-free, OOD-aware adaptation with minimal hyperparameter sensitivity and strong empirical guarantees across architectures and semantic domains.

Open questions involve efficiently scaling dynamic negative expansion for resource-constrained deployment, integrating richer textual priors, and extending to multimodal OOD tasks. Future advances could explore hybrid score calibration schemes, more principled online sampling for the dynamic bank, and applying the prototype-separation regularizer to hierarchical or few-shot OOD detection regimes.

Conclusion

TINS presents a comprehensive and effective framework for test-time adaptive OOD detection in VLMs, systematically addressing the core problems of negative semantic drift and ID contamination. Experiments substantiate its superiority over both static and prior adaptive baselines, with compelling improvements on community benchmarks. By maintaining a dynamic, prototype-separated semantic negative bank, TINS advances OOD detection toward robust open-world vision-language intelligence.