Temperature Scaling in Neural Networks

• Temperature scaling is a recalibration method that rescales logits using a positive scalar to adjust confidence while preserving hard predictions.
• It fits a single temperature parameter via minimizing the negative log-likelihood on a validation set, leading to improved NLL and ECE measurements.
• Variants of TS extend its application to adaptive and out-of-domain calibration, though its uniform scaling may struggle under severe distribution shifts.

Temperature scaling (TS) denotes the rescaling of logits by a positive scalar before the softmax. In classification, it is a post-hoc calibration method: the trained model weights are held fixed, logits are divided by a temperature $T>0$, and the resulting probabilities are used to better align confidence with correctness while preserving the hard prediction. In language-model and reinforcement-learning settings, the same transformation is also used to control output sharpness, entropy, and sampling behavior, although its sequence-level effects differ from its classifier-calibration role (Mattei et al., 16 Feb 2026, Mozafari et al., 2019).

1. Mathematical definition and invariant properties

Let $z=(z_1,\dots,z_K)$ be the logits of a $K$-class model. Temperature scaling replaces the standard softmax by

$$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$

Equivalently, with inverse temperature $\beta=1/T$,

$$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$

For $T=1$, one recovers the base model; for $T>1$, the distribution softens; for $T<1$, it sharpens. Because multiplying logits by a positive scalar preserves their ordering, TS does not change the hard prediction $\arg\max_k p(y=k\mid z)$; in this sense it is accuracy-preserving for the classifier’s decision rule (Mattei et al., 16 Feb 2026).

The theory in "The Well-Tempered Classifier" gives three elementary properties. First, increasing temperature increases uncertainty in a very general sense. In particular, if $z=(z_1,\dots,z_K)$0 is the tempered classifier, then the Shannon entropy satisfies

$z=(z_1,\dots,z_K)$1

so entropy decreases in $z=(z_1,\dots,z_K)$2 and therefore increases in $z=(z_1,\dots,z_K)$3. The same work further states a majorization result: if $z=(z_1,\dots,z_K)$4, then $z=(z_1,\dots,z_K)$5 majorizes $z=(z_1,\dots,z_K)$6, implying monotonic behavior for Schur-concave uncertainty measures. Second, TS admits a geometric characterization: among distributions on the simplex with a prescribed entropy, the one minimizing $z=(z_1,\dots,z_K)$7 is exactly a temperature-scaled version of $z=(z_1,\dots,z_K)$8. Third, among linear post-hoc scalers such as matrix scaling and Dirichlet calibration, TS is the only linear family that does not change the hard predictions (Mattei et al., 16 Feb 2026).

These results clarify why TS occupies a special position among post-hoc recalibrators. It is not merely a heuristic softmax flattening; it is a one-parameter submodel with a precise entropy monotonicity, an information-projection interpretation, and a unique accuracy-preserving status within linear scaling families (Mattei et al., 16 Feb 2026).

2. Fitting the temperature and measuring calibration

Standard TS uses a labeled validation set $z=(z_1,\dots,z_K)$9 that is disjoint from weight training. The network weights are frozen, and $K$0 is chosen by minimizing the negative log-likelihood (NLL),

$K$1

or, in equivalent notation,

$K$2

Because this is a single-parameter optimization, the cited works describe it as convex or strictly convex and solvable by simple line search, gradient-based methods, L-BFGS, or simple gradient descent initialized at $K$3 (Mozafari et al., 2019, Chanda et al., 29 Sep 2025).

A common external calibration metric is expected calibration error (ECE). With $K$4 confidence bins $K$5,

$K$6

where $K$7 is the empirical accuracy in bin $K$8 and $K$9 is the mean confidence. The definition operationalizes the ideal calibration condition $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$0: a perfectly calibrated classifier has confidence frequencies matching empirical correctness frequencies at each confidence level (Mozafari et al., 2019).

The workflow is correspondingly simple. One carves out a held-out calibration split, fits $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$1 solely on that split, and evaluates both predictive metrics and calibration metrics on test data. The medical-imaging study recommends reserving approximately $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$2 to $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$3 of the data for temperature tuning, not using test labels for calibration, and using approximately $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$4 to $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$5 bins for ECE to avoid extreme sparsity while retaining stable estimates (Chanda et al., 29 Sep 2025).

3. Empirical behavior in deep classifiers

A detailed case study is provided by "Evaluating Temperature Scaling Calibration Effectiveness for CNNs under Varying Noise Levels in Brain Tumour Detection" (Chanda et al., 29 Sep 2025). The model is a light CNN with 4 convolutional blocks followed by 2 fully connected layers and a dropout of $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$6; inputs are $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$7, training uses Adam with learning rate $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$8, batch size $$\hat p_i(T)=\frac{\exp(z_i/T)}{\sum_{j=1}^K \exp(z_j/T)}.$$9, and up to 60 epochs with early stopping. The dataset split is $\beta=1/T$0 training, $\beta=1/T$1 validation for TS tuning, and $\beta=1/T$2 testing. To simulate realistic MRI artifacts, five synthetic noise types are applied only to the test set: Gaussian, Poisson, Salt & Pepper, Speckle, and Uniform noise (Chanda et al., 29 Sep 2025).

On clean data, the uncalibrated model attains accuracy $\beta=1/T$3, NLL $\beta=1/T$4, and ECE $\beta=1/T$5. After TS, accuracy remains approximately $\beta=1/T$6, NLL decreases from $\beta=1/T$7 to $\beta=1/T$8, and ECE changes from approximately $\beta=1/T$9 to $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$0. Under noise, the reductions are larger. For Gaussian noise with $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$1 and $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$2, the uncalibrated model has accuracy $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$3, NLL $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$4, and ECE $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$5; after TS with $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$6, accuracy is $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$7 ($$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$8), NLL falls to $$\pi_{k,\beta}=\mathrm{Softmax}(\beta z)_k \quad\Longleftrightarrow\quad \pi_{k,\beta}=\frac{\pi_k^\beta}{\sum_{j=1}^K \pi_j^\beta}.$$9, and ECE falls to $T=1$0, corresponding to NLL $T=1$1 and ECE $T=1$2 without meaningful accuracy loss (Chanda et al., 29 Sep 2025).

Across other corruptions, the same pattern recurs. For Poisson noise at scale $T=1$3, NLL changes from $T=1$4 to $T=1$5 and ECE from $T=1$6 to $T=1$7, with $T=1$8. For Salt & Pepper noise with $T=1$9 and $T>1$0, NLL changes from $T>1$1 to $T>1$2, ECE from $T>1$3 to $T>1$4, and $T>1$5. Across Uniform and Speckle noise up to scale $T>1$6, NLL reductions are $T>1$7 and $T>1$8, while ECE reductions are $T>1$9 for both; $T<1$0 ranges from approximately $T<1$1 under light noise to approximately $T<1$2 under heavier noise. Repeating optimal noise cases on ResNet50 shows the same trend, with Gaussian $T<1$3 yielding NLL $T<1$4 and ECE $T<1$5, and Uniform $T<1$6 yielding NLL $T<1$7 and ECE $T<1$8, while accuracy remains approximately $T<1$9 (Chanda et al., 29 Sep 2025).

The reported operational cost is correspondingly small. TS adds only a single scalar optimization step on the validation set, took less than one second on GPU with L-BFGS in those experiments, requires no changes to the model architecture, and does not retrain weights. This makes it attractive whenever the objective is to improve NLL and ECE without altering the learned decision rule (Chanda et al., 29 Sep 2025).

4. Limitations, failure modes, and interaction with conformal prediction

The simplicity of TS is also its principal limitation. Because it rescales confidence uniformly across all samples, it cannot correct class-specific or region-specific miscalibration. The brain-tumour study therefore notes that under severe distribution shift TS may not suffice and suggests adaptive TS variants or full retraining with calibration-aware loss when the shift is strong (Chanda et al., 29 Sep 2025).

A second limitation concerns the calibration set. "Attended Temperature Scaling" shows that the standard TS objective depends directly on labels in the held-out validation set, and identifies two concrete failure modes: small or noisy validation sets, and highly accurate networks. If $\arg\max_k p(y=k\mid z)$0 is small or labels are corrupted, the estimate of $\arg\max_k p(y=k\mid z)$1 can become highly unstable; when a network makes almost no errors on the calibration split, there are too few misclassified examples to pull down overconfidence, so TS can remain sub-optimally calibrated (Mozafari et al., 2018).

A third limitation appears when TS is composed with adaptive conformal prediction. "On Temperature Scaling and Conformal Prediction of Deep Classifiers" reports that TS improves the class-conditional coverage of adaptive CP methods, but negatively affects their prediction-set sizes. For CIFAR-100 with ResNet50 at $\arg\max_k p(y=k\mid z)$2, optimal $\arg\max_k p(y=k\mid z)$3 leaves LAC set size nearly unchanged at $\arg\max_k p(y=k\mid z)$4, but increases APS set size from $\arg\max_k p(y=k\mid z)$5 to $\arg\max_k p(y=k\mid z)$6 and RAPS set size from $\arg\max_k p(y=k\mid z)$7 to $\arg\max_k p(y=k\mid z)$8, while coverage remains approximately $\arg\max_k p(y=k\mid z)$9. The same work proves that APS and RAPS thresholds decrease monotonically with $z=(z_1,\dots,z_K)$00, yet the mean set size typically increases, often non-monotonically but with an overall upward trend. Its practical guideline is explicit: do not apply TS before APS or RAPS when the classifier’s top-1 accuracy is below $z=(z_1,\dots,z_K)$01, and note that LAC is unaffected in set size because TS preserves the softmax ranking (Dabah et al., 2024).

These limits are not contradictions of the core TS theory. They instead delimit the regime in which a single global temperature is well matched to the structure of the miscalibration problem.

5. Major variants and extensions

A substantial literature generalizes standard TS without abandoning the central idea of logit rescaling. The variants differ primarily in what data are used to fit the temperature and whether the temperature remains global, becomes sample-dependent, or is structured by class, domain, or network depth (Mozafari et al., 2019, Mozafari et al., 2018, Joy et al., 2022, Choi et al., 2024, Guo et al., 2023, Khalifa et al., 2022).

Variant	Core mechanism	Reported behavior
UTS	Fit $z=(z_1,\dots,z_K)$02 on unlabeled samples via class-wise self-selection	Matches or outperforms TS in NLL and ECE
ATS	Attend to borderline examples from other classes	More stable with small or noisy validation sets
AdaTS	Predict $z=(z_1,\dots,z_K)$03 per sample	Lowers ECE and AdaECE beyond global TS
CTS	Add style/content consistency losses	Improves OOD calibration using only source domains
Dual-TS	Fuse class-adaptive and equal-size-bin temperatures	Improves calibration in long-tailed recognition
LATES	Learn nonnegative weights across layer probes	Improves calibration and also accuracy/AUC

UTS removes the need for labeled calibration data. It assumes that high-confidence samples for class $z=(z_1,\dots,z_K)$04 are likely to come from the true class-$z=(z_1,\dots,z_K)$05 distribution, constructs sets

$z=(z_1,\dots,z_K)$06

and minimizes an unsupervised NLL over these pseudo-labeled subsets. On ResNet-110/CIFAR-10, NLL changes from $z=(z_1,\dots,z_K)$07 uncalibrated to $z=(z_1,\dots,z_K)$08 with TS and $z=(z_1,\dots,z_K)$09 with UTS, while ECE changes from $z=(z_1,\dots,z_K)$10 to $z=(z_1,\dots,z_K)$11 with TS and $z=(z_1,\dots,z_K)$12 with UTS. On DenseNet-100/CIFAR-100, NLL changes from $z=(z_1,\dots,z_K)$13 to $z=(z_1,\dots,z_K)$14 with TS and $z=(z_1,\dots,z_K)$15 with UTS, while ECE changes from $z=(z_1,\dots,z_K)$16 to $z=(z_1,\dots,z_K)$17 with TS and $z=(z_1,\dots,z_K)$18 with UTS (Mozafari et al., 2019).

ATS enlarges each class’s calibration subset by including true class-$z=(z_1,\dots,z_K)$19 samples together with borderline non-$z=(z_1,\dots,z_K)$20 samples satisfying $z=(z_1,\dots,z_K)$21. The reported effect is robustness when the validation set is small or contains noisy labels. On DenseNet40/CIFAR-10, ECE changes from $z=(z_1,\dots,z_K)$22 uncalibrated to $z=(z_1,\dots,z_K)$23 with TS and $z=(z_1,\dots,z_K)$24 with ATS; on ISIC skin-lesion detection with ResNet200, ECE changes from $z=(z_1,\dots,z_K)$25 uncalibrated to $z=(z_1,\dots,z_K)$26 with TS and $z=(z_1,\dots,z_K)$27 with ATS (Mozafari et al., 2018).

AdaTS replaces the single scalar with a predicted $z=(z_1,\dots,z_K)$28 for each sample, using a small VAE and an MLP on pseudo-likelihood features. On CIFAR-10 and CIFAR-100, it improves both ECE and AdaECE beyond TS; on CIFAR-10-C, ResNet50 ECE drops from $z=(z_1,\dots,z_K)$29 uncalibrated to $z=(z_1,\dots,z_K)$30 with TS and $z=(z_1,\dots,z_K)$31 with AdaTS (Joy et al., 2022).

CTS is designed for out-of-domain calibration when target-domain validation labels are unavailable. It augments the TS NLL with KL consistency terms under style-shifted and content-shifted feature recombinations. On PACS, Office-Home, Digits-DG, and VLCS, the reported average ECE values for CTS are $z=(z_1,\dots,z_K)$32, $z=(z_1,\dots,z_K)$33, $z=(z_1,\dots,z_K)$34, and $z=(z_1,\dots,z_K)$35, compared with $z=(z_1,\dots,z_K)$36, $z=(z_1,\dots,z_K)$37, $z=(z_1,\dots,z_K)$38, and $z=(z_1,\dots,z_K)$39 for source-validation TS and $z=(z_1,\dots,z_K)$40, $z=(z_1,\dots,z_K)$41, $z=(z_1,\dots,z_K)$42, and $z=(z_1,\dots,z_K)$43 for oracle TS using target labels (Choi et al., 2024).

Dual-TS addresses long-tailed recognition by combining class-adaptive temperatures $z=(z_1,\dots,z_K)$44 with equal-size-bin temperatures $z=(z_1,\dots,z_K)$45 through

$z=(z_1,\dots,z_K)$46

On CIFAR-10-LT with imbalance factor $z=(z_1,\dots,z_K)$47, Dual-TS reports accuracy $z=(z_1,\dots,z_K)$48, ECE $z=(z_1,\dots,z_K)$49, and Esbin-ECE $z=(z_1,\dots,z_K)$50, versus $z=(z_1,\dots,z_K)$51 accuracy and ECE $z=(z_1,\dots,z_K)$52 for standard TS. On CIFAR-100-LT with imbalance factor $z=(z_1,\dots,z_K)$53, Dual-TS reports accuracy $z=(z_1,\dots,z_K)$54, ECE $z=(z_1,\dots,z_K)$55, and Esbin-ECE $z=(z_1,\dots,z_K)$56, versus $z=(z_1,\dots,z_K)$57 accuracy and ECE $z=(z_1,\dots,z_K)$58 for TS (Guo et al., 2023).

LATES distributes the scaling operation across all layers by attaching probes after each block and learning a nonnegative weight vector $z=(z_1,\dots,z_K)$59 for their logits. On ImageNet with ResNet50, TS reports ECE $z=(z_1,\dots,z_K)$60, NLL $z=(z_1,\dots,z_K)$61, ACC $z=(z_1,\dots,z_K)$62, and AUC $z=(z_1,\dots,z_K)$63, whereas LATES reports ECE $z=(z_1,\dots,z_K)$64, NLL $z=(z_1,\dots,z_K)$65, ACC $z=(z_1,\dots,z_K)$66, and AUC $z=(z_1,\dots,z_K)$67. Its stated overhead is less than $z=(z_1,\dots,z_K)$68 extra parameters, less than $z=(z_1,\dots,z_K)$69 extra FLOPs at inference, and less than $z=(z_1,\dots,z_K)$70 extra training time (Khalifa et al., 2022).

6. Temperature scaling beyond classifier calibration

Outside standard classifier calibration, TS appears in three closely related but distinct roles. The first is language-model decoding and test-time scaling. The second is entropy restoration in reinforcement learning. The third is token-level recalibration for semantic uncertainty in question answering (Mattei et al., 16 Feb 2026, Wu et al., 2 Oct 2025, Yang et al., 30 May 2026, Lamb et al., 8 Apr 2026).

For autoregressive decoding, token-level temperature rescales next-token logits exactly as in classification. However, "The Well-Tempered Classifier" challenges the common claim that increasing temperature increases diversity for LLMs. A full temperature scaling of the joint autoregressive model is intractable, so practitioners apply a myopic token-level version; in that setting, sequence-level entropy or diversity can be non-monotonic in temperature (Mattei et al., 16 Feb 2026).

At inference time, this does not preclude useful multi-temperature search. "On the Role of Temperature Sampling in Test-Time Scaling" reports that different sampling temperatures solve different subsets of reasoning problems. Averaged over Qwen3 $z=(z_1,\dots,z_K)$71B, $z=(z_1,\dots,z_K)$72B, $z=(z_1,\dots,z_K)$73B, and $z=(z_1,\dots,z_K)$74B on AIME 2024/2025, MATH500, LiveCodeBench, and Hi-ToM, multi-temperature scaling raises Pass.All from $z=(z_1,\dots,z_K)$75 to $z=(z_1,\dots,z_K)$76, an additional $z=(z_1,\dots,z_K)$77 points over single-temperature TTS. For Qwen3 $z=(z_1,\dots,z_K)$78B on AIME 2025, performance rises from $z=(z_1,\dots,z_K)$79 to $z=(z_1,\dots,z_K)$80 (Wu et al., 2 Oct 2025).

In RL from verifiable rewards, TS can be internalized into the model rather than used only at rollout time. TS-OPSD constructs a self-teacher by applying high-temperature scaling to the model’s own logits, then distills that smoother distribution back into the student. On Qwen3-4B, macro-averaged avg@8 and pass@8 change from approximately $z=(z_1,\dots,z_K)$81 and $z=(z_1,\dots,z_K)$82 for baseline DAPO to approximately $z=(z_1,\dots,z_K)$83 and $z=(z_1,\dots,z_K)$84 for policy reheating with forward KL; on Qwen3-8B they change from approximately $z=(z_1,\dots,z_K)$85 and $z=(z_1,\dots,z_K)$86 to approximately $z=(z_1,\dots,z_K)$87 and $z=(z_1,\dots,z_K)$88. The reported analysis further states that linear CKA between the collapsed and reheated checkpoints remains near $z=(z_1,\dots,z_K)$89, while top-$z=(z_1,\dots,z_K)$90 overlap and overlapping probability mass both exceed $z=(z_1,\dots,z_K)$91 (Yang et al., 30 May 2026).

For semantic uncertainty in QA, token-level scalar TS is treated as a post-hoc recalibration method for confidence measures derived from token probabilities. On TriviaQA with Llama-3 and the E-SC measure, ACE changes from $z=(z_1,\dots,z_K)$92 at $z=(z_1,\dots,z_K)$93 to $z=(z_1,\dots,z_K)$94 at $z=(z_1,\dots,z_K)$95, while AUROC changes from $z=(z_1,\dots,z_K)$96 to $z=(z_1,\dots,z_K)$97. On Natural Questions with Qwen-2.5, ACE changes from $z=(z_1,\dots,z_K)$98 to $z=(z_1,\dots,z_K)$99, while AUROC changes from $K$00 to $K$01. The same study argues that a single scalar provides a suitable inductive bias, whereas more expressive token-level recalibration methods can overfit (Lamb et al., 8 Apr 2026).

Taken together, these works show that the phrase “temperature scaling” names a shared logit-rescaling mechanism whose consequences depend strongly on the object being scaled. For classifiers, TS is a one-parameter recalibration that preserves hard predictions and usually improves NLL and ECE. For adaptive prediction sets, domain shift, long-tailed recognition, and sample-specific miscalibration, structured variants modify the fitting procedure or the temperature itself. For LLMs and RL, the same operation governs sharpness and entropy, but its sequence-level and policy-level consequences are not reducible to the standard classifier-calibration picture.