Scaling Law of Miscalibration: Rates and Implications

• Scaling law of miscalibration is a set of asymptotic relationships that quantify how discrepancies between reported and true uncertainty change as scale parameters vary.
• In scored reporting, strategic perturbations induce a quadratic miscalibration loss, with flat scoring rules like the Brier score uniquely achieving second-best screening.
• In auditing and second-order calibration, these laws reveal near-parametric estimation rates and verification challenges, highlighting the roles of sample size, curvature, and tail behavior.

The expression scaling law of miscalibration denotes a family of asymptotic relationships describing how calibration error, calibration-induced welfare loss, or the difficulty of estimating or verifying calibration changes with a governing scale parameter. Recent work uses the phrase in several technically distinct regimes: perturbation strength in scored reporting, sample size in second-order calibration, dataset or model scale in autoregressive language modeling, and labeled evaluation budget in rare-error auditing. In all of these settings, the central object is not accuracy alone but the discrepancy between a reported uncertainty-related quantity and the quantity it is supposed to represent, together with the rate at which that discrepancy shrinks, persists, or becomes undetectable (Lovén et al., 8 May 2026, Ciosek et al., 8 May 2026, Cao et al., 15 Nov 2025, Wang, 14 Apr 2026).

1. Scope of the term across recent literature

Recent arXiv work uses the phrase in at least four precise senses.

Setting	Scaling law	Quantity of interest
Scored reporting with strategic agents	Quadratic perturbation loss in $\gamma$; welfare lower bound $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$	miscalibration and welfare loss
Second-order calibration in binary classification	$\tilde\Theta(n^{-1/2})$ minimax rate	estimation and correction of second-order calibration error
Entropy calibration of LLMs	$\text{EntCE} \propto m^{1/\alpha - 1}$	entropy calibration error
Rare-error calibration auditing	$\Theta\!\left((L\varepsilon/m)^{1/3}\right)$ passively; $\Theta\!\left(\sqrt{\varepsilon/m}\right)$ actively	verification error

These results are not interchangeable definitions of calibration. In scored reporting, miscalibration is endogenous to incentives and mechanism design. In second-order calibration, the object is a two-moment notion attached to epistemic uncertainty. In entropy calibration, the discrepancy is between model entropy over generations and log loss on human text. In rare-error auditing, the focus is the minimax difficulty of estimating calibration error when model errors are sparse. A plausible implication is that the phrase functions best as an umbrella term for asymptotic regularities of calibration failure, rather than as the name of a single invariant quantity.

2. Endogenous miscalibration in scored reporting

In the scored-reporting framework, a reporter with private type $\theta$ submits a report $r$ and is evaluated under the combined objective

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$

where $S$ is a strictly proper scoring rule and $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$0 is a report-based non-accuracy payoff. The truthful report function $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$1 is defined as the unique maximizer of $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$2, but the strategic report under perturbation is $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$3, which need not equal $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$4. The paper’s central claim is an endogeneity of miscalibration: the principal’s own optimal approval rule creates the non-accuracy incentive that then distorts reports (Lovén et al., 8 May 2026).

The formal obstruction is given by the Perturbation Lemma and the Credibility Impossibility. If $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$5 is strictly proper and $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$6 is $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$7, then adding $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$8 shifts the maximizer unless $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$9 is constant on the truthful-report image $\tilde\Theta(n^{-1/2})$0. At a truthful report,

$\tilde\Theta(n^{-1/2})$1

Hence, if $\tilde\Theta(n^{-1/2})$2, truthful reporting is not even a local optimum. The impossibility theorem states that under NT1 (binding conflict) and NT2 (non-affine perturbation), no strategy $\tilde\Theta(n^{-1/2})$3 simultaneously achieves truthfulness, $\tilde\Theta(n^{-1/2})$4 a.e., and rationality, $\tilde\Theta(n^{-1/2})$5 for some $\tilde\Theta(n^{-1/2})$6.

The first-order perturbation is explicit: $\tilde\Theta(n^{-1/2})$7 The associated loss in score is

$\tilde\Theta(n^{-1/2})$8

Accordingly, miscalibration loss is quadratic in $\tilde\Theta(n^{-1/2})$9 to leading order. The role of curvature is direct: the report moves in the direction of the perturbation gradient, and the movement is larger when the scoring rule is flatter.

The paper distinguishes sharply between affine and non-affine perturbations. Affine $\text{EntCE} \propto m^{1/\alpha - 1}$0 can be reinterpreted as a uniform shift in truth, whereas non-affine $\text{EntCE} \propto m^{1/\alpha - 1}$1 changes incentives differently across types and destroys truthfulness. This distinction underwrites the broader claim that smooth scoring-based oversight cannot preserve truthful calibration when the agent also benefits from approval.

3. Threshold escape, curvature heterogeneity, and welfare scaling

The constructive escape from the impossibility is a step-function approval rule. In the AI oversight model, the principal chooses an approval function $\text{EntCE} \propto m^{1/\alpha - 1}$2, and the optimal choice is

$\text{EntCE} \propto m^{1/\alpha - 1}$3

Under this rule, the agent faces a binary choice: report truthfully and receive no approval if below threshold, or inflate to the threshold and receive approval. The strategic condition becomes

$\text{EntCE} \propto m^{1/\alpha - 1}$4

so the induced approval rule is

$\text{EntCE} \propto m^{1/\alpha - 1}$5

which is exactly the first-best screening rule (Lovén et al., 8 May 2026).

This escape is not restricted to the Brier score. Theorem $\text{EntCE} \propto m^{1/\alpha - 1}$6 states that for any strictly proper scoring rule with strictly convex generator $\text{EntCE} \propto m^{1/\alpha - 1}$7, the agent’s binary “inflate or not” choice generates a unique threshold in type space, because the inflation cost is strictly increasing in the report distance. The principal can therefore choose $\text{EntCE} \propto m^{1/\alpha - 1}$8 so that the threshold in agent types coincides with the socially optimal cutoff $\text{EntCE} \propto m^{1/\alpha - 1}$9.

The paper then isolates a scaling law for the welfare loss of smooth approval functions $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$0: $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$1 summarized in the abstract as

$\Theta\!\left((L\varepsilon/m)^{1/3}\right)$2

For binary-outcome scoring rules, $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$3 is a scalar, and the variance term measures heterogeneity in inverse curvature across the relevant binding type region. For non-Brier rules, $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$4 is not constant, so different types face different effective inflation costs under smooth approval, and the principal cannot perfectly offset that heterogeneity with a single smooth incentive schedule.

The special role of the Brier score follows from constant curvature: $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$5 Because inflation costs are type-independent, the paper proves that second-best equals first-best under the Brier score, and that this welfare equivalence is unique to Brier. The continuity result for the power family,

$\Theta\!\left((L\varepsilon/m)^{1/3}\right)$6

shows the welfare gap vanishes as $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$7, i.e., as the rule approaches Brier.

4. Sample-size scaling in second-order calibration

In second-order calibration for binary classification, a higher-order predictor outputs

$\Theta\!\left((L\varepsilon/m)^{1/3}\right)$8

where $\Theta\!\left((L\varepsilon/m)^{1/3}\right)$9 is a mean prediction and $\Theta\!\left(\sqrt{\varepsilon/m}\right)$0 is an epistemic-uncertainty estimate. The data consist of $\Theta\!\left(\sqrt{\varepsilon/m}\right)$1 i.i.d. 2-snapshots

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$2

with

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$3

The law of total variance is written as

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$4

thereby decomposing total predictive uncertainty into aleatoric and epistemic components (Ciosek et al., 8 May 2026).

The calibration functions are

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$5

and with

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$6

the second-order calibration error is

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$7

Since

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$8

a calibrated predictor satisfies

$\Theta\!\left(\sqrt{\varepsilon/m}\right)$9

The paper also proves a quantitative equivalence with a bucketed Wasserstein notion: for each partition cell $\theta$0,

$\theta$1

The main scaling law is minimax. Under the sech perturbation scheme, Proposition 1 states that with perturbation bandwidth $\theta$2,

$\theta$3

the polynomial-regression estimator satisfies

$\theta$4

Proposition 2 gives a matching lower bound: there exists an absolute constant $\theta$5 such that for every $\theta$6, every $\theta$7, and every estimator,

$\theta$8

Hence the minimax rate is

$\theta$9

The technical reason this rate improves over generic two-dimensional smoothing is analyticity. The sech perturbation kernel

$r$0

has nearest poles at

$r$1

so $r$2 is analytic in the strip

$r$3

This yields exponentially accurate polynomial approximation,

$r$4

which in turn supports the near-parametric estimation rate. The same framework produces the finite-sample guarantee for second-order Platt scaling via

$r$5

and its empirical analogue

$r$6

with

$r$7

5. Model-scale and data-tail effects in entropy calibration of LLMs

For autoregressive LLMs, miscalibration is formulated as entropy calibration: whether a model’s entropy over generations matches its log loss on human text. The model $r$8 is entropy calibrated if

$r$9

where

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$0

Per-step versions are

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$1

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$2

The entropy calibration error is

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$3

and in the reported experiments models are generally high-entropy relative to their log loss over long generations, reflecting error accumulation (Cao et al., 15 Nov 2025).

The scaling analysis is built from a simplified theoretical setting. Training data consist of $$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$4 i.i.d. samples from an $$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$5-power-law vocabulary distribution,

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$6

During generation, if the context contains any token seen only once in training, the model enters a “derailed” high-entropy mode. Let $$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$7 be the number of items seen exactly once in $$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$8 samples. The asymptotic relation stated in the paper is

$$V(r;\theta,\gamma) = S(r;\theta) + \gamma \cdot h(r), \quad \gamma > 0,$$9

for $S$0 infinite and $S$1 large. The derailing probability therefore scales like $S$2, and the simplified entropy accumulation model is

$S$3

so that over $S$4 steps

$S$5

This yields the scaling law

$S$6

The dependence on tail heaviness is central. If $S$7, then $S$8, so miscalibration improves extremely slowly with scale. The paper reports

• WikiText: $S$9,
• WritingPrompts: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$00,
• CodeContests: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$01,

with corresponding predicted scaling exponents

• WikiText: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$02,
• WritingPrompts: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$03,
• CodeContests: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$04.

The empirical evaluation covers Qwen2.5 models from 0.5B to 72B, Llama 3 models from 1B to 70B, Llama 2 models from 7B to 70B, and Pythia models from 410M to 12B, on WikiText-103, WritingPrompts, and CodeContests, using 5000 examples and generation length up to 1024 tokens. The fitted exponents for text are reported as close to $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$05, while the code dataset exhibits a more negative exponent. The paper explicitly notes that “a scaling exponent of $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$06 means that to reduce calibration error by a factor of 10, dataset size must increase by a factor of $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$07.”

The same paper connects these observations to truncation methods such as temperature reduction, top-$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$08, top-$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$09, and min-$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$10. Reducing temperature below 1 reduces entropy but increases log loss; instruction tuning often reduces entropy but also increases log loss. The standard quality–diversity tradeoff is therefore presented as contingent rather than necessary. Under the assumption of a black-box regression algorithm that predicts future entropy with small test error $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$11, the adjusted model

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$12

satisfies

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$13

and

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$14

This establishes that reducing entropy calibration error without worsening log loss is theoretically possible under the stated assumption.

6. Verification limits, detectability thresholds, and comparative interpretation

In rare-error auditing, the relevant scaling law concerns not the generation of miscalibration but the minimax difficulty of verifying it. Let a classifier or LLM output a confidence score $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$15 and predicted label $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$16. The conditional accuracy function is

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$17

the calibration gap is

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$18

and the expected calibration error is represented in binned form as

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$19

with the continuous version essentially

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$20

Under the assumption that $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$21 is $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$22-Lipschitz,

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$23

and with model error rate

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$24

the main scaling law is

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$25

up to logarithmic factors (Wang, 14 Apr 2026).

The matched lower and upper bounds are

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$26

and, for histogram binning with

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$27

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$28

This is the paper’s verification tax: as $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$29 decreases, verification becomes harder. The implied precision requirement is

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$30

and the verification floor is

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$31

Any claimed calibration improvement smaller than this floor is statistically indistinguishable from noise.

Three further asymptotic results complete the picture. First, label-free self-evaluation is information-theoretically uninformative in the worst case: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$32 Second, there is a phase transition at

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$33

with detection impossible when $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$34 and possible when $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$35. Third, active querying removes the Lipschitz penalty: $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$36 For a $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$37-component pipeline with homogeneous per-component Lipschitz constants $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$38, the compositional verification cost obeys

$$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$39

The empirical validation spans MMLU, TruthfulQA, ARC-Challenge, HellaSwag, and WinoGrande, with six LLMs from five families and about 27,000 items total. The paper reports 95% bootstrap confidence intervals, 10,000-permutation tests for self-evaluation claims, self-evaluation non-significance in 80% of pairs, and 23% of pairwise comparisons indistinguishable from noise.

Taken together, these literatures support several common corrections to oversimplified intuitions. First, miscalibration does not uniformly vanish with scale: in language modeling, fitted exponents for text are close to $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$40; in strategic oversight, smooth incentives can make miscalibration endogenous; and in auditing, lower error rates can make calibration harder to verify rather than easier to estimate. Second, smoothness is not always beneficial. In scored reporting, smooth approval creates the impossibility, whereas sharp thresholds preserve screening. In verification, passive smooth-function recovery incurs the $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$41-rate tax, whereas active querying collapses the problem toward detection. Third, curvature matters in multiple senses: the curvature of the scoring-rule generator $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$42 controls strategic distortion in scored reporting, and analyticity induced by the sech kernel controls the near-parametric $$\Omega\!\left(\operatorname{Var}(1/G'')(\gamma/\beta)^2\right)$$43 rate in second-order calibration.

A plausible synthesis is that the scaling law of miscalibration is best understood as a set of rate statements about how calibration failure interacts with structure: incentive structure, smoothness structure, tail structure, and observational structure. The direction of the rate depends on which structure dominates. In some settings, additional structure enables fast correction; in others, the same kind of structure makes truthful calibration impossible or verification fundamentally expensive.