Bounded Calibration with Contestability

• Bounded calibration with contestability is defined as attaching explicit miscalibration bounds to probabilistic predictions along with mechanisms to challenge these bounds through tests and remappings.
• Recent approaches utilize methods like h-calibration, smooth calibration, and finite-sample certificates to deliver rigorous error bounds in binary classification and forecasting.
• Incorporated contestability mechanisms include swap maps, bounded scoring rules, and community value frameworks that enable forecasts to be auditable and revisable.

Bounded calibration with contestability denotes, in an interpretive synthesis of recent work, the conjunction of two requirements: probabilistic predictions should come with explicit bounds on miscalibration, and those bounds or predictions should remain open to challenge through auditable assumptions, adversarial tests, remappings, benchmark comparisons, or decision-review procedures. The phrase is not used uniformly across the literature. Some papers study finite-sample certificates for true calibration error, smooth-calibration and calibration-distance bounds, or probabilistic error-bounded canonical calibration without discussing contestability in the legal or governance sense; others formalize contestability through swap maps, bounded downstream scoring rules, post-hoc challenges, or evidence for overturning decisions (Ciosek et al., 15 Dec 2025, Luo et al., 23 Feb 2025, Freiesleben et al., 15 May 2026).

1. Conceptual scope

In the calibration literature, “bounded” refers to several distinct objects. One strand gives high-probability, non-asymptotic, distribution-free upper bounds on the true $L_1$ calibration error of a binary classifier (Ciosek et al., 15 Dec 2025). A second strand studies smooth calibration and the distance from calibration as robust measures of miscalibration, with quantitative control through Earth Mover’s Distance and sample-complexity bounds (Gopalan et al., 16 Mar 2026, Qiao, 19 Mar 2026). A third strand introduces $h$-calibration as a uniform eventwise error bound,

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$

and relates it to bounded canonical calibration (Huang et al., 22 Jun 2025). A fourth strand studies worst-case calibration under bounded perturbation sets, producing Certified Brier Score and Certified Calibration Error style guarantees (Emde et al., 2024).

“Contestability” is likewise heterogeneous. In technical online-learning and forecasting papers, contestability is represented by swap maps $\sigma:[0,1]\to[0,1]$, by comparison against all bounded proper scoring rules, by post-processings of smoothed forecasts, by external forecasts whose informativeness is challenged through calibeating, or by adaptive and oblivious adversaries (Luo et al., 23 Feb 2025, Kleinberg et al., 2023, Chen et al., 23 Mar 2026, Dagan et al., 2024). In socio-technical papers, contestability is the capacity to challenge and potentially overturn a decision through predictive multiplicity, incorrect feature values, neglected overruling evidence, or community-defined value profiles bounded by meta-rules (Freiesleben et al., 15 May 2026, Mayer, 7 Jul 2025). This suggests that bounded calibration with contestability is best understood as a family of related technical and institutional constructions rather than a single settled formalism.

2. Formal notions of bounded calibration

Recent papers use different calibration primitives, and they impose boundedness at different levels: true error, local smoothness, eventwise deviation, transport distance, or bounded downstream loss. The following notions recur.

Notion	Representative formula	Bounded object
$L_1$ expected calibration error	$\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$	True calibration error (Ciosek et al., 15 Dec 2025)
$h$-calibration	$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$	Eventwise conditional deviation (Huang et al., 22 Jun 2025)
Smooth calibration error	$$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$	Lipschitz-tested residual (Gopalan et al., 16 Mar 2026)
Calibration distance	$CalDist_D(f) = \inf_{g\in C(D)} d_D(f,g)$	Distance to a calibrated predictor (Qiao, 19 Mar 2026)
U-calibration	$h$0	Worst-case bounded-score regret (Kleinberg et al., 2023)
KL-calibration	$h$1	Swap-regret under log loss (Luo et al., 23 Feb 2025)

For binary probabilistic classifiers, the paper “Measuring Uncertainty Calibration” models labels by

$h$2

and treats exact calibration as $h$3 (Ciosek et al., 15 Dec 2025). It distinguishes exact calibration, approximate calibration, and estimated calibration, where the last notion means a computable upper bound on the true calibration error derived from finite data. Under a bounded-variation assumption on $h$4, or under bounded derivatives induced by a score perturbation scheme, it upper-bounds the true $h$5 rather than merely an estimator.

The paper “h-calibration” strengthens canonical calibration by requiring a uniform eventwise bound over all $h$6, and proves that $h$7-calibration is sufficient for generalized canonical calibration with bounded error,

$h$8

It then reformulates this condition as $h$9-$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$0 boundedness and derives a differentiable objective intended to learn approximately equivalent error-bounded calibrated probabilities (Huang et al., 22 Jun 2025).

The paper “The Importance of Being Smoothly Calibrated” defines

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$1

for $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$2-Lipschitz $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$3, and shows that smooth calibration is equivalent up to constants to Earth Mover’s Distance to the nearest calibrated prediction-label distribution: $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$4 The same paper proves

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$5

thereby linking smooth calibration to the lower distance to calibration (Gopalan et al., 16 Mar 2026).

The paper “Computational and Statistical Hardness of Calibration Distance” uses

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$6

as the central quantitative notion of miscalibration and characterizes it through partitions of the instance space. This turns calibration into an optimization problem over all partitions, or equivalently over all perfectly calibrated predictors (Qiao, 19 Mar 2026).

3. Finite-sample certificates, robustness certificates, and tractability

A major bounded-calibration line asks not merely whether a predictor is calibrated, but whether one can certify a finite-sample upper bound on miscalibration. “Measuring Uncertainty Calibration” gives two such routes for binary classification (Ciosek et al., 15 Dec 2025). Under bounded variation,

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$7

the paper uses one-dimensional total variation denoising on a training split and empirical Bernstein on a validation split to produce a genuine upper confidence bound on the true CE. Under bounded derivatives,

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$8

it uses a Nadaraya–Watson surrogate and a computable error term $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h,$$9. The paper also proposes modifying any classifier by sampling a perturbed score

$\sigma:[0,1]\to[0,1]$0

from a truncated hyperbolic secant kernel, and proves that the perturbed calibration function satisfies

$\sigma:[0,1]\to[0,1]$1

This yields a smooth enough calibration function for the derivative-based certificate without any assumption on the original classifier (Ciosek et al., 15 Dec 2025).

A closely related, but adversarially robust, notion appears in “Towards Certification of Uncertainty Calibration under Adversarial Attacks” (Emde et al., 2024). That paper assumes per-sample certificates

$\sigma:[0,1]\to[0,1]$2

and

$\sigma:[0,1]\to[0,1]$3

then aggregates them into dataset-level worst-case bounds on calibration metrics. For top-label Brier score, the worst-case bound is exact and analytic: $\sigma:[0,1]\to[0,1]$4 For empirical ECE, it defines

$\sigma:[0,1]\to[0,1]$5

and gives a mixed-integer formulation, solved approximately by ADMM as ACCE (Emde et al., 2024).

The computational picture is more mixed for calibration distance. “Computational and Statistical Hardness of Calibration Distance” proves that $\sigma:[0,1]\to[0,1]$6 can be computed exactly in polynomial time when the distribution has a uniform marginal and noiseless labels, but becomes $\sigma:[0,1]\to[0,1]$7-hard when either assumption is removed (Qiao, 19 Mar 2026). It extends the exact algorithm to a PTAS for the general case and proves that $\sigma:[0,1]\to[0,1]$8 samples are sufficient and necessary for the empirical calibration distance to be upper bounded by the true distance plus $\sigma:[0,1]\to[0,1]$9. The paper explicitly identifies an asymmetry: “small calibration distance” has succinct witnesses, while proving “large calibration distance” is much harder; in decision form, the complementary lower-bound problem is described as co-NP-complete (Qiao, 19 Mar 2026). This matters directly for contestability, because negative certificates are harder than positive ones.

4. Contestability as remapping, bounded evaluation, benchmarks, and adversarial pressure

Several papers formalize contestability as a challenger’s ability to improve, remap, or robustly stress-test a forecaster. The cleanest formulation appears in “Simultaneous Swap Regret Minimization via KL-Calibration” (Luo et al., 23 Feb 2025). For a proper loss $L_1$0 and a swap map $L_1$1, actual swap regret is

$L_1$2

The paper shows that for log loss,

$L_1$3

Thus KL-calibration is exactly the maximum gain obtainable by a post-hoc remapping challenger under log loss. It then proves transfer results to bounded proper losses with twice continuously differentiable or $L_1$4-smooth univariate form, giving uniform control of challenger advantage through $L_1$5 or $L_1$6 (Luo et al., 23 Feb 2025).

“U-Calibration: Forecasting for an Unknown Agent” gives a complementary bounded-loss contestability model (Kleinberg et al., 2023). There, the challenger is an unknown downstream agent with bounded utility $L_1$7, and the central quantity is

$L_1$8

where $L_1$9 is the class of bounded proper scoring rules. The paper proves that sublinear U-calibration error is a necessary and sufficient condition for all bounded agents to achieve sublinear regret guarantees. It also shows that standard calibration is sufficient but not necessary for universal bounded-agent usefulness, and provides an online algorithm achieving $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$0 U-calibration error (Kleinberg et al., 2023). This suggests a bounded contestability notion in which a forecast remains defensible under any bounded proper scoring rule, not just under one chosen metric.

A third route appears in “The Importance of Being Smoothly Calibrated” (Gopalan et al., 16 Mar 2026). That paper smooths the predictor by adding noise $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$1, writing

$\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$2

and proves that for every bounded proper loss $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$3, every benchmark PLD $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$4 with the same label marginal, and every post-processing $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$5,

$\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$6

In particular, for $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$7, smooth calibration plus random smoothing yields bounded excess loss against all post-processings of the smoothed predictor. The paper is explicit that smooth calibration alone does not imply the unsmoothed post-processing guarantee; smoothing is essential (Gopalan et al., 16 Mar 2026).

Contestability against external forecasts is formalized most directly in “Calibeating Made Simple” (Chen et al., 23 Mar 2026). For an external forecast sequence $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$8, the learner is $\mathrm{CE} \triangleq \mathbb{E}_{s}[|s-\eta(s)|]$9-calibeating if

$h$0

where $h$1 is the external forecaster’s refinement score. The paper proves that calibeating is minimax-equivalent to regret minimization, and that multi-calibeating is minimax-equivalent to the combination of calibeating and the classical expert problem. For bounded proper losses it gives optimal rates

$h$2

for calibeating and

$h$3

for multi-calibeating; for mixable losses, including Brier and log, it gives logarithmic rates (Chen et al., 23 Mar 2026). In this setting, contestability is benchmark-relative and explicitly tied to informativeness.

Finally, “Breaking the $h$4 Barrier for Sequential Calibration” places contestability in an adversarial sequential setting (Dagan et al., 2024). It studies binary forecasting under adaptive and oblivious adversaries, defines the standard unnormalized $h$5 calibration error

$h$6

and shows that progress on sequential calibration is equivalent to progress on sign preservation with reuse (SPR). It also proves an oblivious-adversary lower bound

$h$7

which the paper presents as the first $h$8 lower bound for oblivious adversaries (Dagan et al., 2024). This supplies a worst-case challenge model for sequential calibration.

5. Joint constructions and broader contestability frameworks

The strongest direct simultaneous result for calibration and contestability appears in “Calibeating Made Simple” for the Brier loss (Chen et al., 23 Mar 2026). The paper constructs a meta-algorithm that combines a reference calibeating or multi-calibeating algorithm with a Blum–Mansour-style reduction, discretization, and a lopsided two-expert aggregation scheme. For binary Brier loss it gives the first calibrated algorithm that at the same time also achieves the optimal

$h$9

multi-calibeating rate, with calibration

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$0

with high probability (Chen et al., 23 Mar 2026). This is a genuine bounded-loss simultaneous calibration-plus-contestability statement, but only for Brier.

A more cautionary joint picture comes from “The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting” (Lovén et al., 8 May 2026). That paper studies a principal-agent setting where a report is both scored for accuracy and used for approval, allocation, or control. The agent maximizes

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$1

so strict properness of $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$2 no longer guarantees truthfulness once $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$3 is non-affine. The paper proves a general impossibility: under binding conflict and non-affine perturbation, no strategy simultaneously achieves truthfulness and rationality. It also derives the perturbation formula

$$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$4

Its constructive escape is a step-function approval threshold, which recovers first-best screening for every strictly proper scoring rule, though not truthful reporting. Under the Brier score specifically, the paper proves a welfare equivalence between second-best and first-best, and argues that this equivalence is unique to Brier under smooth oversight (Lovén et al., 8 May 2026). This suggests that contestability may be endogenous to mechanism design rather than a property of scoring rules alone.

The socio-technical literature widens the notion of contestability beyond forecasting games. “Explainable AI Isn’t Enough! Rethinking Algorithmic Contestability” defines a decision as normatively contestable if $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$5, and epistemically contestable relative to a feature set $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$6 if $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$7 (Freiesleben et al., 15 May 2026). The paper argues that standard XAI explanations establish at most “somewhere contestability” or “somewhere inaccuracy,” and identifies three evidence types that can justify reversal: predictive multiplicity, incorrect feature values, and neglected overruling evidence. A plausible implication is that bounded calibration, even if technically rigorous, is insufficient for decision-level contestability when evidence, features, or model multiplicity remain disputed.

“Infrastructuring Contestability: A Framework for Community-Defined AI Value Pluralism” moves the problem to normative system design (Mayer, 7 Jul 2025). It introduces Community-Defined AI Value Pluralism (CDAVP), built from community-defined value profiles, user-controlled context-sensitive activation, and systemic conflict moderation under fundamental, non-negotiable meta-rules. The paper states that “Value-conformity thus becomes a measurable and competitive criterion,” but leaves formal languages and open standards for value profiles as future work (Mayer, 7 Jul 2025). This suggests a distinct sense of bounded calibration: not calibration to frequencies, but calibration of system behavior to explicit, contestable value profiles bounded by constitutional constraints.

A more local, model-internal variant appears in “Contestability in Quantitative Argumentation” (Yin et al., 15 Jul 2025). In an interpretive bounded-calibration lens, that paper studies an inverse problem over Edge-Weighted Quantitative Bipolar Argumentation Frameworks: given a topic argument $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$8 and a desired strength $$|p_{\mu}(Y \in A\mid F) - p_c(Y \in A\mid F)| \le h$$9, modify edge weights $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$0 so that $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$1. It defines gradient-based relation attribution explanations

$$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$2

and a projected iterative update

$$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$3

The paper does not use the statistical meaning of calibration, but it does provide a constrained local revision mechanism for contestable outputs (Yin et al., 15 Jul 2025).

6. Limitations, controversies, and open problems

The literature is explicit that many guarantees remain narrow. “Measuring Uncertainty Calibration” treats only binary classification, relies either on an untestable bounded-variation assumption or on score perturbation, and stresses that even the best method remains sample-intensive, with roughly $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$4 samples needed to get CE bounded to about $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$5 (Ciosek et al., 15 Dec 2025). “Breaking the $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$6 Barrier for Sequential Calibration” improves the conceptual understanding of online calibration but does not itself prove a concrete unconditional $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$7 algorithm; the upper-bound improvement remains conditional on progress on SPR (Dagan et al., 2024).

Bounded robustness certificates are also estimator-specific. In “Towards Certification of Uncertainty Calibration under Adversarial Attacks,” the Brier bound is exact and tight given confidence certificates, but ECE certification is only approximate, depends on binning, and remains empirical and dataset-level rather than population-level (Emde et al., 2024). “h-calibration” provides a theoretically motivated bounded canonical objective, but its end-to-end guarantees combine exact reformulations, asymptotic empirical approximations, and high-probability discrepancy bounds rather than a simple deterministic deployment certificate (Huang et al., 22 Jun 2025).

On the contestability side, several technical notions remain incomplete. “Simultaneous Swap Regret Minimization via KL-Calibration” gives constructive pseudo guarantees and existential actual guarantees, but no multiclass extension and no explicit institutional notion of contestability (Luo et al., 23 Feb 2025). “U-Calibration” protects external regret against all bounded proper scoring rules, yet the paper is explicit that stronger notions such as swap regret return standard calibration to a special role (Kleinberg et al., 2023). “Calibeating Made Simple” solves bounded-loss contestability in a unified way, but its simultaneous calibration-plus-contestability theorem is only for the Brier loss; the paper leaves open whether one can generalize the Section 5 meta-algorithm to arbitrary bounded proper losses (Chen et al., 23 Mar 2026).

The hardest barrier is negative certification. “Computational and Statistical Hardness of Calibration Distance” shows that exact computation becomes $$smCE(\mu) = \max_{\psi \in Lip}\mathbb{E}[\psi(p)(y-p)]$$8-hard once either uniformity or noiselessness is removed, and that the lower-bound side of the decision problem is co-NP-complete (Qiao, 19 Mar 2026). This means that positive bounded-calibration certificates are often easier than contesting them. A plausible implication is that real systems will rely more on approximate upper certificates, explicit benchmarks, or structured subclasses than on general lower-bound proofs.

The socio-technical papers add a distinct warning. Even a model with bounded calibration error may remain contestable because of predictive multiplicity, incorrect feature values, neglected overruling evidence, or disagreement over the value profile that should govern the system (Freiesleben et al., 15 May 2026, Mayer, 7 Jul 2025). This suggests that bounded calibration and contestability are complementary rather than interchangeable: calibration constrains the probabilistic quality of outputs, while contestability governs how those outputs may be challenged, revised, overridden, or institutionally bounded.