Surrogate for OOD-free novelty-sensitive tuning

Determine whether a surrogate exists that can carry the kind of novelty to which a constant selected using only in-distribution data responds, thereby removing the need to use OOD data for tuning.

Background

The paper examines post-hoc out-of-distribution (OOD) detectors whose parameters or scoring directions are fitted using OOD data. It finds that such fitting can exploit dataset identity rather than genuine novelty, producing gains under within-dataset validation that do not survive when the entire OOD dataset is held out.

The authors investigate an alternative in which a constant is selected using in-distribution data alone. They test a construction based on retraining the classifier on subsets of its own classes, but report that it fails to reach the zero-capacity reference on CIFAR-100 and that the selected value varies by an order of magnitude across class splits. The unresolved problem is whether some other surrogate can provide novelty information of the relevant kind without access to OOD data.