Activation-Aware Negative Label Mining for OOD

• Activation-aware Negative Label Mining is a technique that dynamically selects negative labels at test time based on activation patterns to distinguish in-distribution from OOD samples.
• It leverages both historical and batch-level activation metrics, using FIFO queues and interpolation to adaptively update negative label sets without retraining models.
• The method achieves significant reductions in false positive rates across benchmarks, offering robust performance for CLIP-style vision-language models in diverse environments.

Activation-aware Negative Label Mining denotes a family of training-free techniques for out-of-distribution (OOD) detection in vision-LLMs, where negative labels are adaptively selected at test time based on observed activation patterns, rather than fixed a priori. Central to this approach is Test-time Activated Negative Labels (TANL), which mines negative labels with strong response to OOD samples using dynamically accumulated activation metrics drawn from historical and mini-batch test data. The resulting methodology enables online, robust, and adaptive distribution alignment for OOD detection across diverse data regimes without requiring re-training or modification of pre-trained CLIP-style models (Zhang et al., 26 Mar 2026).

1. Formal Framework and Notation

Let $\mathcal{Y}_{ID} = \{y_1, \dots, y_C\}$ be the set of in-distribution (ID) labels and $$\mathcal{Y}^{\mathrm{cor}} = \{\hat{y}_1, \dots, \hat{y}_N\}$$ the corpus of candidate labels. For any test example $x$, pre-trained encoders yield a normalized image feature $v = f_{img}(x) \in \mathbb{R}^D$ and a normalized text feature $t(y) = f_{txt}(\rho(y)) \in \mathbb{R}^D$ for any label $y$, where $\rho$ is a prompt template (e.g., “a photo of a $<y>$”).

The zero-shot softmax probability over labels is given by: $$p(y \mid v) = \frac{\exp(v \cdot t(y))}{\sum_{y' \in \mathcal{Y}_{ID} \cup \mathcal{Y}_{neg}} \exp(v \cdot t(y'))}$$ where $$\mathcal{Y}_{neg} = \{\tilde{y}_1, \dots, \tilde{y}_M\} \subseteq \mathcal{Y}^{\mathrm{cor}} \setminus \mathcal{Y}_{ID}$$ are the mined negative labels ($M \ll N$).

This framework supports dynamic, test-time re-evaluation of $\mathcal{Y}_{neg}$ as label activations shift during inference.

2. Label Activation Metrics

The informational content of a candidate negative label $\hat{y}\in\mathcal{Y}^{\mathrm{cor}}$ is quantified by its ability to activate upon OOD versus ID samples. The oracle activation difference is: $$A_d(\hat{y}) = Act(\mathcal{X}_{OOD}, \hat{y}) - Act(\mathcal{X}_{ID}, \hat{y})$$ where

$$Act(\mathcal{X}, \hat{y}) = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} p(\hat{y} \mid f_{img}(x))$$

In the absence of ground-truth OOD labels at inference, TANL maintains two size-$L$ FIFO queues—$\mathcal{X}_{pos}$ (predicted ID, high-confidence) and $\mathcal{X}_{neg}$ (predicted OOD, high-confidence)—to estimate the empirical activation gap: $$\widehat{A}_d(\hat{y}) = Act(\mathcal{X}_{neg}, \hat{y}) - Act(\mathcal{X}_{pos}, \hat{y})$$ The high-confidence sets are updated per batch by thresholding an OOD score $S_{aa}(v)$ (defined below) with gap $g$ around $\gamma$.

3. Batch-Adaptive Activation Mining

In addition to historical activation statistics, TANL leverages batch-level adaptivity by extracting high-confidence positive and negative feature vectors within the current test batch ($$\mathcal{X}_{pos}^b, \mathcal{X}_{neg}^b \subseteq \mathcal{B}$$). The combined batch-adaptive metric is: $$\widehat{A}_b(\hat{y}) = Act_b(\mathcal{X}_{neg}, \hat{y}) - Act_b(\mathcal{X}_{pos}, \hat{y})$$ with

$$Act_b(\mathcal{Q}, \hat{y}) = \begin{cases} \alpha \, Act(\mathcal{Q}, \hat{y}) + (1-\alpha) \, Act(\mathcal{Q}^b, \hat{y}) & |\mathcal{Q}^b| > 0 \ Act(\mathcal{Q}, \hat{y}) & |\mathcal{Q}^b| = 0 \end{cases}$$

where $\alpha \in [0, 1]$ interpolates global and batch-local statistics. This enables TANL to rapidly adapt negative label selection in response to distribution drift or diversity in test-time batches.

4. Activation-aware OOD Scoring

Once the $M$ most activated candidate negatives $\{\tilde{y}_1, \dots, \tilde{y}_M\}$ are selected according to $\widehat{A}_d$ or $\widehat{A}_b$, OOD scoring for any test image feature $v$ is performed via: $$S_{aa}(v) = \frac{1}{M} \sum_{m=1}^{M} \sum_{i=1}^{C} \frac{\exp(v \cdot t(y_i))}{\sum_{j=1}^{C} \exp(v \cdot t(y_j)) + \sum_{j=1}^{m} \exp(v \cdot t(\tilde{y}_j))}$$ This formula endows negatives with higher activation greater implicit weight: $\tilde{y}_1$ appears in all denominators, $\tilde{y}_2$ in all but one, etc. The score is maximized when $v$ is close to ID features and far from top-activated negatives, reflecting robust OOD/ID discrimination.

5. Algorithmic Workflow

The TANL procedure, summarized in the following pseudocode, alternates between online activation mining and OOD scoring:

Inputs: - ID labels Y_ID - Corpus labels Y_cor - Pretrained CLIP encoders f_img, f_txt - Test set X_test - Queue length L, negatives M, threshold γ, gap g, batch-tradeoff α Initialize: - X_pos ← L features from ID prompts - X_neg ← L features from random noise images For each test batch B ⊂ X_test: 1. Compute S_aa(v) for all v∈B with current top-M negatives 2. X_pos^b ← {v∈B | S_aa(v) ≥ γ+(1-γ)g} X_neg^b ← {v∈B | S_aa(v) < γ-γg} 3. Update X_pos, X_neg with respective FIFO queues 4. Compute batch-adaptive activations ĤA_b(ŷ) for all ŷ∈Y_cor 5. Select top-M labels by ĤA_b as Y_neg 6. Recompute S_aa for the next batch Output: - S_aa(v) for each test image

In this process, negative labels are dynamically aligned to historic and in-batch activation, yielding distribution-adaptive OOD scoring without requiring model or corpus retraining (Zhang et al., 26 Mar 2026).

6. Theoretical Properties

Building on multilabel detection theory, the change in false positive rate at a fixed decision threshold $\lambda$ is controlled by: $$\frac{\partial FPR_{\lambda}}{\partial M} \propto \frac{p_1 - p_2}{\sqrt{M\,p_2(1-p_2)}}$$ where $$p_1 = \Pr(sim(x_{ID}, \tilde{y}) \geq \psi \mid ID)$$, $$p_2 = \Pr(sim(x_{OOD}, \tilde{y}) \geq \psi \mid OOD)$$. Effectiveness of additional negatives ($M$) for decreasing $FPR$ hinges on $p_1-p_2<0$; i.e., the selected negatives must activate more on OOD than ID. TANL’s metric $\widehat{A}_d(\tilde{y})$ is explicitly constructed to enforce this relationship, ensuring systematic selection of informative negatives (Zhang et al., 26 Mar 2026).

7. Empirical Performance and Robustness

The performance benefits of activation-aware mining are empirically validated across diverse benchmarks and architectures:

• On ImageNet-1K vs. {iNat, SUN, Places, Textures}, NegLabel (agnostic) achieves FPR95 ≈ 25.4%, while TANL attains FPR95 ≈ 9.8% (15.6 percentage point absolute reduction).
• Compared to recent conjugated-pool methods (FPR95 ≈ 17.5%), TANL achieves a 7.7 percentage point lower FPR95.
• TANL is robust to the number $M$ of negatives: for small $M$ it outperforms NegLabel, and for large $M$ the activation-aware score prevents the performance collapse observed in prior approaches.
• Ablations on OpenOOD ImageNet demonstrate progressive improvement: distribution-adaptive only yields FPR95 ≈ 61.6% (vs. NegLabel 69.5%), with further reductions from batch-adaptive and activation-aware variants.
• The method generalizes to various backbones (ResNet50, ViT-B/32, ViT-L/14) and is effective on CIFAR-10/100 and medical X-ray OOD benchmarks.

TANL integrates a principled activation metric, test-time adaptation, and an activation-aware scoring rule, mining the most informative negatives in an online fashion and effecting significant advances in CLIP-style OOD detection (Zhang et al., 26 Mar 2026).