TV-OOD Detection Method

• The paper introduces a TV-OOD approach that uses a lightweight neural estimator for unbiased OOD scoring with minimal computational overhead.
• TV-OOD employs a linear variational form of total variation distance to overcome bias issues common with KL-divergence-based methods.
• Empirical results demonstrate that TV-OOD consistently matches or outperforms state-of-the-art techniques on standard OOD detection benchmarks.

The Total Variation Out-of-Distribution (TV-OOD) detection method is a principled approach for addressing distributional shift in machine learning deployments, specifically in image classification tasks. TV-OOD leverages the total variation (TV) distance—a statistical measure of distributional difference—to create a robust, unbiased, and information-theoretically justified criterion for distinguishing in-distribution (ID) from out-of-distribution (OOD) samples. The core innovation lies in estimating the contribution of each example to the overall TV objective using a lightweight neural estimator, yielding effective OOD scores with minimal computational overhead. Empirical results demonstrate TV-OOD consistently matches or outperforms prior state-of-the-art techniques across standard OOD detection benchmarks (Ma et al., 22 Jan 2026).

1. Motivation and Information-Theoretic Foundations

Traditional OOD detection practices evaluate OOD-ness via scores such as Maximum Softmax Probability (MSP), energy-based confidence, or surrogate divergences like KL-divergence (often approximated using Donsker–Varadhan’s variational representation). However, KL-divergence–based approaches introduce a log-expectation over the entire background distribution, causing estimation bias in mini-batch training and requiring mitigation strategies such as exponential moving averages.

Total variation distance, in contrast, is a member of the f-divergence family with the following properties:

• Admits a direct “linear” variational form without log-terms, enabling unbiased estimation from mini-batches.
• By Pinsker’s inequality ($D_{TV}^2 \leq D_{KL}/2$) and the Bretagnolle–Huber bound ($D_{TV} \leq \sqrt{1-\exp(-D_{KL})}$), TV can be more sensitive than KL in distinguishing diverging distributions.
• Empirical evidence indicates superior OOD separation versus other f-divergences and KL, as shown in detailed ablations (Section 4.4).

Formally, for probability densities (or mass functions) $p$ and $q$:

$D_{TV}(p\,\|\,q) = \frac{1}{2}\int |p(u)-q(u)|\,du$

Variationally,

$$D_{TV}(p\,\|\,q) = \sup_{T : |T| \leq 1/2} \Big( \mathbb{E}_{u\sim p}[T(u)] - \mathbb{E}_{u\sim q}[T(u)] \Big)$$

with $T$ ranging over all measurable functions bounded in $[-1/2, +1/2]$.

2. Total Variation Network Estimator Architecture

The TV-OOD architecture decomposes into:

• A fixed, pre-trained $K$-way classifier $f$ for the target task, producing penultimate feature vectors $h=f_{feat}(x)\in\mathbb{R}^d$.
• A trainable Total Variation Network Estimator (TVNE), denoted $g_\theta$, which is a single dense layer $g_c(h) = W h + b$ with $K+1$ outputs (one per class plus a synthetic OOD label $K+1$).

The estimator is implemented as:

• For sample $x$, compute $h = f_{feat}(x)$;
• Output logits $t = W h + b \in\mathbb{R}^{K+1}$;
• For each class $j = 1, ..., K+1$, set $T_\theta(x,j) = \text{clip}(t_j, -0.5, +0.5)$.

Only $W$ and $b$ are learned; the backbone $f$ remains frozen. This architecture ensures negligible compute and memory overhead in both training and inference phases.

3. Training Objective, Scoring, and Decision Procedure

Training Objective

Let $m$ represent the fraction of ID samples in the training set, and $m_j$ the class prior for class $j$ among ID samples. Viewing training pairs $(x, y)$ as drawn from $X\sim P_X$ (union support of ID and OOD) and $Y\sim P_Y$ (with $Y\in\{1, ..., K\}$ for ID, $Y=K+1$ otherwise), the objective derived from the TV variational form is:

$$\mathcal{L}(\theta) = \frac{1}{N}\sum_{i=1}^N \Big[ T_\theta(x_i, y_i) - m\sum_{j=1}^K m_j T_\theta(x_i, j) - (1-m) T_\theta(x_i, K+1) \Big]$$

which may be denoted for each example as $Sc(x_i, y_i)$.

TV-OOD Score

For any sample $x$, with classifier prediction $\hat{y} = f(x)$:

$$Sc(x, \hat{y}) = T_\theta(x, \hat{y}) - m\sum_{j=1}^K m_j T_\theta(x, j) - (1-m) T_\theta(x, K+1)$$

Inference and Thresholding

At test time, compute $S(x) = Sc(x, f(x))$. Classification into ID or OOD proceeds via thresholding:

• $S(x) \geq \tau$: declare $x$ in-distribution;
• $S(x) < \tau$: declare $x$ out-of-distribution; where $\tau$ is chosen (typically via a validation set) to meet a specified true-positive rate (commonly 95%).

4. Theoretical Justification

Pinsker’s inequality and the Bretagnolle–Huber bound guarantee that optimizing an estimator of TV distance maintains or increases KL-based discrepancy, thus preserving information-theoretic rigor. The “linear” variational form of TV ensures unbiased mini-batch gradient estimates without recourse to exponential moving averages, in contrast to the KL-divergence variational form which introduces a log-moment over $q$ and is inherently biased in finite samples.

For a misclassified OOD input $x$, the expected score obeys:

$$\mathbb{E}_{j'\sim m_j}[Sc(x, j')] = -\frac{1-m}{m} Sc(x, K+1)$$

As a consequence, training drives $Sc(x, K+1)$ large for genuine OOD samples, which in turn forces the expected $Sc$ on such samples lower, creating a robust separation between ID (high $Sc$) and OOD (low $Sc$) instances.

5. Empirical Evaluation and Results

Datasets and Experimental Setup

Dataset Type	Data Source	Purpose
ID (D_in)	CIFAR-100, ImageNet-1k	Training/Test
Auxiliary OOD (D_out^train)	80M Tiny Images (CIFAR); OpenImage-O, ImageNet-O (ImageNet)	Train aux OOD
Test OOD (D_out^test)	Textures, SVHN, Places365, LSUN-crop/pad, iSUN (CIFAR); DTD, iNaturalist, Places365, SUN (ImageNet)	Evaluation
Feature Extractors	DenseNet121, WideResNet, ViT-B_16	Backbone

Metrics

• FPR95: False positive rate (OOD declared ID) at 95% ID true positive rate (TPR).
• AUROC: Area under the receiver operating characteristic.
• AUPR: Area under the precision-recall curve.

Results Overview

Selected average results for TV-OOD versus comparators:

Model + Data	Scenario	FPR95 (↓)	AUROC (↑)	AUPR (↑)
CIFAR-100 + DN121	w/o aux	46.01	88.34	97.05
(comparators)		34.16–78.55	78.34–88.24	94.45–96.76
+	with aux	24.93	93.87	98.42
(comparators)		47.58–64.93	85.71–91.35	96.68–98.01
WideResNet	w/o aux	41.58	88.12	96.91
(comparators)		51.53–80.60	75.06–86.55	93.74–96.59
+	with aux	27.39	92.49	97.91
(comparators)		39.08–48.99	87.63–91.34	96.93–97.98
ImageNet1k + ViT-B_16	w/o aux	30.01	92.34	98.75
(comparators)		32.31	93.00	98.52
+	with aux	22.28	94.81	98.77
(comparators)		22.95–22.97	93.38–95.00	98.35–98.56

Across backbones, TV-OOD is consistently competitive and frequently outperforms reference techniques (e.g., MSP, Energy, Mahalanobis, OE, BEOE, WOODS), especially when auxiliary OOD or augmented data are incorporated.

6. Implementation Aspects and Practical Considerations

• The hyperparameter $m$ (proportion of ID to OOD in loss) was held constant at 0.5 for all reported experiments.
• The estimator $g_\theta$ is implemented as a shallow, single fully-connected layer of dimensionality $d\rightarrow (K+1)$, with output clipping to $[-0.5, 0.5]$.
• Standard mini-batch optimizers (e.g., Adam) and learning-rate schedules adapted from baseline methods are used.
• For further improvement, the auxiliary OOD dataset $D_{aug}$ is generated by shuffling 64 uniform patches per training image, a simple data augmentation that enhances OOD detection performance.

This implementation design maintains scalability and does not introduce significant computational burden.

7. Limitations and Prospective Directions

TV-OOD, in its present form, requires access to an auxiliary OOD dataset—either real or synthetic—at training time. The estimator network $g_\theta$ is intentionally shallow to preserve efficiency, but there is potential for future exploration of deeper or transformer-based architectures. Empirical ablations confirm TV’s advantage among f-divergences; however, in some regimes, alternative f-divergences may perform comparably or better. Ongoing research directions include:

• Eliminating dependence on auxiliary OOD data.
• Automating the selection of the $m$ hyperparameter.
• Extending theoretical analyses to situations involving non-overlapping support (“shifting supports”).
• Investigating the optimality conditions for TV versus alternative divergences in OOD detection contexts.

These avenues are aimed at further enhancing the generality, robustness, and theoretical grounding of divergence-based OOD detection frameworks (Ma et al., 22 Jan 2026).