Distance-Aware Temperature Scaling

• Distance-Aware Temperature Scaling (DATS) is a calibration method that assigns adaptive, class-dependent temperatures using embedding-space distances to mitigate catastrophic forgetting in continual learning.
• It leverages prototype-based distance estimation to infer task proximity, allowing the calibration process to adjust temperatures based on how close a class is to the current task.
• DATS improves stability and reduces calibration errors across tasks by replacing a single global temperature with a nuanced, distance-aware approach that has shown effectiveness on standard and imbalanced biomedical benchmarks.

Searching arXiv for the DATS paper and closely related calibration/CIL work. Distance-Aware Temperature Scaling (DATS) is a post-hoc calibration method for class-incremental learning (CIL) that adapts temperature scaling to the inferred proximity between previously learned classes and the current task, rather than using a single global temperature across all tasks. It is motivated by the observation that, in continual learning, catastrophic forgetting induces substantial task-dependent variation in predictive accuracy and confidence, so a shared scalar temperature cannot jointly calibrate recent and older tasks. DATS addresses the absence of task identity at deployment by combining prototype-based distance estimation with distance-aware calibration, and is reported to reduce calibration error across tasks on standard benchmarks and imbalanced biomedical datasets (Serra et al., 25 Sep 2025).

1. Problem formulation in calibrated class-incremental learning

Continual Learning (CL) is described as enabling a single model to learn incrementally from a sequence of new classes, while maintaining consistent predictive performance across all classes and mitigating catastrophic forgetting. In this setting, predictive performance alone is insufficient in safety-critical applications; the model must also communicate uncertainty reliably, with confidence scores aligned to the true frequencies of target events (Serra et al., 25 Sep 2025).

In standard single-task settings, a single global temperature $T$ can be learned by dividing all logits by $T$ so that post-softmax confidences better match observed frequencies. In class-incremental learning, however, the model encounters a sequence of tasks $1,\dots,T$, and catastrophic forgetting causes predictive accuracy and average confidence to vary markedly between older and newer tasks. A single scalar temperature therefore cannot simultaneously “sharpen” outputs on recent tasks and “soften” outputs on forgotten tasks. This motivates task-sensitive calibration.

A central complication is that task identity is unavailable at deployment time. DATS is constructed around the idea that calibration should depend on “task proximity,” but this proximity must be inferred indirectly. The method therefore uses distances in the embedding space as a surrogate for the relationship between a class and the current task. This suggests a shift from data-centric recalibration with one shared temperature toward a task-aware post-hoc mechanism that remains applicable when explicit task labels are absent.

2. Prototype-based estimation of task proximity

After training on task $t$, DATS retains a small calibration buffer $\mathcal{B}_t$ of representative samples from all seen classes, with $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$, and uses the current task’s validation set $D_{t,\mathrm{val}}$ (Serra et al., 25 Sep 2025). For every class $c$, the method defines a prototype in the penultimate embedding space $\phi(\cdot)$ as

$$\mu_c = \frac{1}{|D_c|} \sum_{(x,y)\in D_c} \phi(x),$$

where $T$0 is the set of samples of class $T$1, drawn either from $T$2 if $T$3 or from $T$4 otherwise.

The distance between classes can be computed using either Euclidean or cosine distance. For cosine similarity,

$T$5

and the corresponding distance is defined as

$T$6

To assign each buffer class a single proximity score relative to the current task, DATS takes the minimum distance over current-task classes:

$T$7

These distances are then min–max normalized across all $T$8, so that $T$9. All samples in class $1,\dots,T$0 inherit the same score $1,\dots,T$1.

This construction makes class-level distance the operational proxy for task proximity. A plausible implication is that classes whose embeddings lie near the current task’s prototypes are treated as less affected by forgetting, whereas distant classes are treated as candidates for stronger recalibration.

3. Distance-aware temperature assignment

DATS learns a base temperature $1,\dots,T$2 together with a per-class weight $1,\dots,T$3, producing a class-dependent temperature

$1,\dots,T$4

Under this parameterization, classes close to the current task, where $1,\dots,T$5, receive $1,\dots,T$6, while more distant classes receive a different temperature determined by the learned slope term. The parameters $1,\dots,T$7 and $1,\dots,T$8 are initialized, for example, to $1,\dots,T$9 and $t$0, and are then jointly optimized (Serra et al., 25 Sep 2025).

The formulation is explicitly extensible. More generally, one may use a monotonic mapping $t$1, such as $t$2 or $t$3, yielding

$t$4

with trainable $t$5, or with separate $t$6 per class.

The calibrated predictive distribution is obtained by dividing each raw logit $t$7 by the corresponding distance-dependent temperature:

$t$8

This differs from ordinary temperature scaling in that the rescaling can vary across classes as a function of inferred distance. The method is therefore not merely a re-estimation of one scalar calibration parameter; it is a structured calibration rule conditioned on embedding-space geometry.

4. Training, calibration, and test-time inference

The method specifies separate procedures for post-task calibration and deployment-time inference. After finishing task $t$9, the training and calibration workflow is:

1. Update calibration buffer $\mathcal{B}_t$0 with a held-out fraction of $\mathcal{B}_t$1.
2. Compute prototypes $\mathcal{B}_t$2 for all $\mathcal{B}_t$3.
3. Compute pairwise cosine similarities $\mathcal{B}_t$4.
4. Assign class distances $\mathcal{B}_t$5 and normalize.
5. Define $\mathcal{B}_t$6.
6. Learn $\mathcal{B}_t$7 by minimizing a calibration loss on $\mathcal{B}_t$8, for instance the Brier score:

$\mathcal{B}_t$9

using L-BFGS or Adam (Serra et al., 25 Sep 2025).

At calibration time, logits are rescaled only for samples in the buffer $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$0. At test time, the system first estimates a global distance for the test batch and then derives a deployment temperature from it. The inference workflow on $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$1 is:

1. For each test sample $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$2, find its nearest prototype among $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$3.
2. Collect the set $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$4 of assigned classes that cover at least $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$5 of the test set, to filter out spurious matches.
3. Compute an overall test distance

$\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$6

1. Set $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$7, or more generally $\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$8.
2. Produce calibrated probabilities

$\mathcal{C}_{\text{buf}} = \cup_{i=1}^t C_i$9

A notable feature is that class-dependent temperatures are used during calibration learning, whereas test-time deployment uses a global test-batch distance estimate. This suggests an operational compromise between fine-grained class-aware calibration and task-agnostic deployment constraints.

5. Evaluation protocol and empirical behavior

The evaluation uses several calibration metrics. Expected Calibration Error (ECE) divides confidences in $D_{t,\mathrm{val}}$0 into bins $D_{t,\mathrm{val}}$1 and computes

$D_{t,\mathrm{val}}$2

where $D_{t,\mathrm{val}}$3 and $D_{t,\mathrm{val}}$4. Negative Log-Likelihood (NLL) is

$D_{t,\mathrm{val}}$5

The study also reports $D_{t,\mathrm{val}}$6LECE, defined as the change in ECE on the last task before versus after calibration, and max $D_{t,\mathrm{val}}$7ECE, the worst-case increase in ECE across all tasks after calibration (Serra et al., 25 Sep 2025).

The main empirical findings are reported on CIFAR-10, CIFAR-100, TinyImageNet, and two imbalanced medical datasets, BloodCell and SkinLesions. On these benchmarks, DATS achieves lower average ECE and NLL than single-temperature TS, ETS, PTS, and the recent CIL-specific methods RC and T-CIL. In particular, $D_{t,\mathrm{val}}$8LECE for DATS is typically negative, indicating improvement on the most recent task, whereas RC and T-CIL often show positive values, indicating worsened calibration. The worst-case $D_{t,\mathrm{val}}$9ECE across tasks under DATS stays near zero or negative, while RC and T-CIL can exhibit large positive spikes on some tasks, which the paper identifies as undesirable in safety-critical settings (Serra et al., 25 Sep 2025).

The reported computational profile is also part of the method’s characterization: DATS is described as computationally efficient, with a calibration phase that adds only a modest overhead over RC and is far faster than the adversarial-perturbation-based T-CIL. Within the stated evidence, the defining empirical property of DATS is not only lower mean calibration error, but also greater stability across the task sequence.

6. Relation to single-temperature and continual-learning calibrators

DATS is positioned against approaches that handle calibration primarily from a data-centric perspective while relying on a single temperature shared across all tasks. The core criticism is that such solutions overlook task-specific differences and therefore exhibit large fluctuations in calibration error across tasks (Serra et al., 25 Sep 2025).

This point can be summarized by contrasting the calibration assumptions of the methods mentioned in the study.

Method family	Temperature structure	Limitation highlighted
Single-temperature TS, ETS, PTS	One shared temperature across tasks	Overlooks task-specific differences
RC, T-CIL	CIL-specific baselines used for comparison	Can show positive $c$0LECE and large positive max $c$1ECE spikes
DATS	Distance-aware adaptive temperature	Infers task proximity without prior task information

The distinction is methodological as well as operational. Standard temperature scaling assumes that one scalar can adequately correct confidence bias globally. DATS instead assumes that calibration should vary with distance to the current task, but that this dependence must be inferred from the embedding space rather than from explicit task labels. This suggests a broader interpretation of calibration in continual learning: confidence correction may need to track representational drift induced by sequential training.

At the same time, DATS remains a post-hoc calibrator rather than a retraining strategy. Its integration point is the logits-to-probabilities transformation, and its optimization objective is a calibration loss on a held-out buffer. This places it conceptually closer to temperature scaling and related post-hoc methods than to continual-learning algorithms designed primarily to prevent forgetting.

7. Limitations, extensions, and scope

The paper identifies several limitations. First, DATS assumes that, at test time, the dominant classes inferred through nearest-prototype assignment accurately reflect the true task distribution. Under extreme non-stationarity or very noisy embeddings, this assumption may be violated (Serra et al., 25 Sep 2025). Second, the current formulation learns one scalar per class, $c$2, together with $c$3; for very large numbers of classes, this may become burdensome.

Several extensions are proposed. These include exploring other distance mappings $c$4, including nonlinear or kernelized mappings; using Euclidean rather than cosine distance, or combining multiple metrics; extending the method to online continual learning with per-sample adaptive temperatures; integrating DATS with other post-hoc calibrators such as Platt or isotonic in a distance-aware fashion; and applying it to domain-incremental or task-agnostic continual learning scenarios (Serra et al., 25 Sep 2025).

In summary, DATS injects task-awareness into temperature-based calibration by estimating how far a sample or class is from the most recent task in embedding space and scaling logits with a distance-dependent temperature learned post-hoc. The reported effect is more stable and robust calibration across a sequence of tasks than can be obtained with a single global temperature. A plausible implication is that the method’s broader significance lies in treating calibration as a sequential, representation-dependent problem rather than a one-parameter correction applied uniformly after continual training.