---
title: Omnipredictors for Loss-Oblivious Learning
url: https://www.emergentmind.com/topics/omnipredictors
type: topic
---

# Omnipredictors for Loss-Oblivious Learning

Searching arXiv for recent and foundational papers on omniprediction/omnipredictors to ground the article.
Omnipredictors are predictors designed for **loss-oblivious learning**: rather than training a separate model for each downstream objective, one learns a single predictor whose output can later be post-processed, by a loss-specific but data-independent rule, to compete with the best hypothesis in a reference class for many losses simultaneously [2109.05389]. In the binary setting, this predictor is typically a map \(p:X\to[0,1]\), interpreted as an estimate of a conditional probability, and the post-processing for a loss \(\ell\) is the Bayes-optimal action under a Bernoulli distribution with mean \(p(x)\) [2210.08649]. Subsequent work has recast this paradigm through multicalibration, outcome indistinguishability, calibrated multiaccuracy, proper calibration, and step calibration; extended it to fairness-constrained optimization, regression, multiclass prediction, online adversarial learning, performative prediction, and evolving graphs; and sharpened both sample-complexity and runtime guarantees [2210.08649] [2209.07463] [2307.08999] [2401.14645] [2501.17205] [2602.17577] [2606.20557].

## 1. Definition and basic formalism

In the binary-label formulation, a predictor is a function \(f:X\to[0,1]\), interpreted as estimating \(\Pr[y=1\mid x]\) [2109.05389]. Given a loss family \(\mathcal L\) and a hypothesis class \(\mathcal C=\{c:X\to\mathbb R\}\), \(f\) is an \((\mathcal L,\mathcal C,\delta)\)-omnipredictor if for every \(\ell\in\mathcal L\) there exists a univariate post-processing function \(k:[0,1]\to\mathbb R\) such that
\[
\ell_D(k\circ f)\le \min_{c\in\mathcal C}\ell_D(c)+\delta,
\qquad
\ell_D(h):=\mathbb E_{(x,y)\sim D}[\ell(y,h(x))].
\]
The central requirement is that learning takes place **without knowing \(\ell\)**; the loss only determines the final transformation \(k\) [2109.05389].

A closely related formulation defines, for each loss \(\ell\),
\[
k_\ell(p)\in \arg\min_t\ \mathbb E_{y\sim \mathrm{Ber}(p)}[\ell(y,t)].
\]
Then \(p\) is an \((L,C,\delta)\)-omnipredictor if for every \(\ell\in L\),
\[
\mathbb E_{(x,y^*)\sim D}\big[\ell(y^*,k_\ell(p(x)))\big]
\le
\min_{c\in C}\mathbb E_{(x,y^*)\sim D}\big[\ell(y^*,c(x))\big]+\delta.
\]
This makes explicit that the predictor is fixed once and for all, while the post-processing \(k_\ell\circ p\) is loss-specific [2210.08649].

The post-processing step is exactly the one induced by treating the predictor’s output as a probability estimate. For binary labels, the “ideal” loss-optimal post-processing is
\[
k_\ell^*(p)=\arg\min_{t\in\mathbb R}\; \mathbb E_{y\sim \mathrm{Ber}(p)}[\ell(y,t)].
\]
Examples given in the literature include squared loss, for which \(k_{\ell_2}^*(p)=p\), and \(\ell_1\) loss, for which \(k_{\ell_1}^*(p)=\mathbf 1[p\ge 1/2]\) [2109.05389].

The same perspective extends beyond binary outcomes. In the more general setting where labels lie in \(Y\subseteq\mathbb R\), the predictor outputs a distribution \(f:X\to P(Y)\), and the post-processing becomes \(k:P(Y)\to\mathbb R\) [2109.05389]. For regression with continuous labels \(y\in[0,1]\), later work formulates omnipredictors as predictors \(p(x)\) together with post-processings \(k\) such that, for every loss \(\ell\in\mathcal L\),
\[
\mathbb E_{D^*}\big[\ell(y^*,k(p(x)))\big]
\le
\min_{c\in\mathcal C}\mathbb E_{D^*}\big[\ell(y^*,c(x))\big]+\varepsilon
\]
[2401.14645].

## 2. Multicalibration, outcome indistinguishability, and structural characterizations

The original structural route to omniprediction is through **multicalibration**. In the binary case, if a partition \(S\) is \(\alpha\)-approximately multicalibrated for \(\mathcal C\), then the associated canonical predictor \(f^S\) is an \((\mathcal L,\mathcal C,2\alpha B+\varepsilon)\)-omnipredictor for the family \(L(B,\varepsilon)\) of \((B,\varepsilon)\)-nice convex losses [2109.05389]. The covariance-based multicalibration condition used there is
\[
\mathbb E_{i\sim D}\Big[\operatorname{Cov}_{D_i}(c(x),y)\Big]\le \alpha
\qquad \forall c\in\mathcal C.
\]
That formulation was chosen because it extends naturally to real-valued \(c\), multi-class labels, and real-valued outcomes, and it is closed under linear combinations [2109.05389].

A later conceptual reformulation introduces **Loss Outcome Indistinguishability**. For each \(\ell\in L\) and \(c\in C\), define
\[
u_{\ell,c}(y,p(x),x)=\ell(y,c(x))-\ell(y,k_\ell(p(x))).
\]
A predictor \(p\) is \((L,C,\delta)\)-loss OI if
\[
\left|
\mathbb E_{(x,y^*)\sim D}[u_{\ell,c}(y^*,p(x),x)]
-
\mathbb E_{(x,y)\sim D(p)}[u_{\ell,c}(y,p(x),x)]
\right|
\le \delta
\]
for all \(\ell\in L\) and \(c\in C\), where \(D(p)\) is the simulated world obtained by sampling \(x\sim D_X\) and then \(y\sim\mathrm{Ber}(p(x))\) [2210.08649]. By construction,
\[
(L,C,\delta)\text{-loss OI} \Longrightarrow (L,C,\delta)\text{-omnipredictor},
\]
while the converse does not hold [2210.08649].

Loss OI decomposes into **hypothesis OI** and **decision OI**. The paper introduces the discrete derivative
\[
\partial \ell(t):=\ell(1,t)-\ell(0,t),
\]
together with the identity
\[
\mathbb E[\ell(y,t)]-\mathbb E[\ell(y',t)] = \mathbb E[(y-y')\,\partial \ell(t)].
\]
This converts indistinguishability statements into correlation conditions. Hypothesis OI becomes multiaccuracy for the derived class
\[
\partial L\circ C := \{\partial \ell \circ c : \ell\in L,\ c\in C\},
\]
and decision OI becomes a calibration condition for the class \(\{\partial \ell\circ k_\ell:\ell\in L\}\) [2210.08649]. The generic recipe is therefore
\[
\boxed{\text{calibration}+\text{multiaccuracy for }\partial L\circ C\Rightarrow \text{Loss OI}\Rightarrow \text{omniprediction}.}
\]

This perspective led to **calibrated multiaccuracy**, defined as calibration plus multiaccuracy. It is positioned between multiaccuracy and multicalibration:
\[
MA(\alpha)\supseteq calMA(\alpha)\supseteq MC(\alpha).
\]
For generalized linear model losses of the form
\[
\ell_g(y,t)=g(t)-yt,
\]
one has
\[
\partial \ell_g(t)=\ell_g(1,t)-\ell_g(0,t)=-t,
\]
so the derived class collapses to \(C\). Hence, for GLM losses, calibration plus \(C\)-multiaccuracy is enough for Loss OI and therefore omniprediction [2210.08649].

A different but related structural simplification appears in later work on **proper calibration**. There, Decision OI is identified with a weighted calibration error
\[
\big|\mathbb E[\Delta\ell(k_\ell(p(x)))(y-p(x))]\big|,
\qquad
\Delta\ell(v):=\ell(v,1)-\ell(v,0),
\]
and proper calibration is defined by
\[
\sup_{\ell\in L}\left|\mathbb E\big[\Delta\ell(p(x))(y-p(x))\big]\right|\le \varepsilon.
\]
For the class \(L_{\mathrm{all}}\), proper calibration is equivalent, up to constant factors, to threshold-weighted calibration \(W_{\mathrm{Th}}\)-calibration [2501.17205]. More recent work further reduces omniprediction and panprediction to **step calibration**, which controls correlations on predictor sublevel sets and hypothesis sublevel sets inside groups [2510.27638].

## 3. Algorithmic learning and complexity

The first end-to-end omnipredictor construction in the batch setting is based on approximately multicalibrated partitions together with weak agnostic learning [2109.05389]. In the multi-class case, the paper gives iterative **Split** and **Merge** procedures, and states an explicit partition-size bound
\[
m \le 2l\left(\frac{12}{\alpha\, w(\alpha/2)^2}\right)^{l-1}.
\]
The resulting runtime is polynomial in \(1/\alpha\) and in the weak learner’s parameters [2109.05389].

The loss-OI formulation yields a simpler calibrated-multiaccuracy algorithm. The main idea is to alternate between a multiaccuracy update using a weak learner for \(C\) and a recalibration step, both of which reduce the potential
\[
\mathbb E\big[(p^*(x)-p(x))^2\big].
\]
The number of weak-learner calls is
\[
O(1/\sigma^2),
\]
which is comparable to standard multiaccuracy and significantly better than the multicalibration bound quoted there as roughly
\[
O(1/\alpha^2\sigma^4)
\]
[2210.08649].

A subsequent line of work shows that the sample and regret cost of omniprediction can be near that of ordinary prediction. In the online setting, one paper gives an oracle-efficient algorithm with \(\tilde O(\sqrt{T \log |\mathcal{H}|})\) regret for any class of Lipschitz loss functions \(\mathcal L\subseteq\mathcal L_{\mathrm{Lip}}\), and an offline learning algorithm for bounded-variation losses \(\mathcal L_{\mathrm{BV}}\) whose error scales near-linearly in the Rademacher complexity of \(\mathrm{Th}\circ \mathcal H\) [2501.17205]. In that framework, finite \(H\) and finite \(L\) yield an online regret bound
\[
O\!\left(\sqrt{T\log(|H||L|T)}\right),
\]
and, more generally,
\[
O\!\left(\sqrt{T\log T}+\mathrm{OracleReg}_{\Delta L\circ H}(T)\right)
\]
given an online weak agnostic learner for \(\Delta L\circ H\) [2501.17205].

The online adversarial setting was also developed through online multicalibration. One construction reduces online multicalibration to online squared-loss regression over \(F\), achieves contextual swap regret of roughly
\[
m\,r_A(T/m,F) + O(T/m) + O(m),
\]
and thereby derives the first efficient online omnipredictor for Lipschitz convex losses [2307.08999]. For linear predictors, combining the reduction with Azoury–Warmuth regression and choosing \(m=T^{1/4}\) gives
\[
sK_2(\pi_{1:T},F_B)=\tilde O(T^{-1/4}),
\qquad
O(\pi_{1:T},L,F_B)=\tilde O(T^{-1/8})
\]
[2307.08999].

The currently strongest online rates separate oracle efficiency from information-theoretic optimality. One paper proves an oracle-efficient online learning algorithm with \(\tilde O(\sqrt{T \log |\mathcal{H}|})\) regret for omniprediction [2501.17205], while another earlier paper showed that an oracle-efficient multicalibration route naturally yields **swap-omniprediction**, for which \(O(\sqrt{T})\) regret is impossible online, and established a lower bound of \(\Omega(T^{0.528})\) via online calibration [2307.08999]. That same paper also presents a non-oracle-efficient algorithm achieving the optimal \(\tilde O(\sqrt T)\) omniprediction rate for suitable finite Boolean classes \(F\), yielding an information-theoretic separation between omniprediction and multicalibration-based solution concepts [2307.08999].

A recent batch result sharpens deterministic sample complexity. It gives a deterministic multicalibration algorithm with sample complexity
\[
\widetilde O(\varepsilon^{-3})
\]
and a deterministic finite-test OI algorithm with sample complexity
\[
\widetilde O(\varepsilon^{-2}),
\]
then derives deterministic omnipredictors and panpredictors with optimal sample complexity [2606.20557]. The paper explicitly states that randomization is not statistically necessary for optimal offline multicalibration or omniprediction [2606.20557].

## 4. Fairness, constraints, and group-conditional guarantees

A major theme in the literature is that omnipredictors are not limited to unconstrained ERM. In **multi-group loss minimization**, if a partition is approximately multicalibrated for a product class \(\mathcal T\times\mathcal P\), then for each subgroup \(T\in\mathcal T\), the canonical predictor remains an omnipredictor for \(\mathcal P\) on the subpopulation \(T\) [2109.05389]. The induced partition on \(D' = D\mid x\in T\) retains approximate multicalibration with controlled degradation, and consequently each group can be treated optimally, for any loss in the family, without retraining [2109.05389].

This multi-group perspective was extended to **fairness-constrained optimization**. One formulation studies false-positive fairness constraints
\[
w_g \, \left| \rho_g(h)-\rho(h)\right| \le \gamma \quad \forall g\in G,
\]
with
\[
\rho_g(h)=\Pr[h(x)\neq y\mid y=0, g(x)=1],
\qquad
w_g=\Pr[g(x)=1, y=0].
\]
Starting from a multicalibrated regression function \(\hat f\), the paper shows that a no-regret primal-dual post-processing, using only unlabeled data and \(\hat f\), yields a classifier \(\bar h\) satisfying
\[
err(\bar h) \le err(h^*) + \alpha(5+2\sqrt{1/\alpha}) + 2\sqrt{\alpha},
\]
and
\[
w_g\left|\rho_g(\bar h)-\rho(\bar h)\right|
\le
w_g\left|\rho_g(h^*)-\rho(h^*)\right| + w_g\alpha
\]
for every \(g\in G\) [2209.07312]. The post-processing threshold depends on the group-membership vector through a linear form in the dual variables, allowing the method to handle intersecting groups without enumerating all intersections [2209.07312].

A parallel development defines **omnipredictors for constrained optimization** more generally. The downstream task is
\[
\min_{c:X\to A}\ \mathbb E[f_0(x,c(x),y)]
\quad\text{s.t.}\quad
\mathbb E[f_j(x,c(x),y)]\le 0 \ \forall j\in J,
\]
and the learner is assumed to know in advance the groups \(S_1,\dots,S_t\) that will define future constraints [2209.07463]. The key device is the simulated distribution \(\mathcal D_p\), obtained by sampling \(x\) from the marginal and then \(y'\sim\mathrm{Bernoulli}(p(x))\). Appropriate variants of group multiaccuracy, group multicalibration, group calibration, and group level-set multiaccuracy imply that solving the constrained task on \(\mathcal D_p\) transfers back to a near-optimal solution on the true distribution \(\mathcal D\) [2209.07463].

For convex and “special” group objectives and constraints, a predictor satisfying group multiaccuracy and group calibration is enough [2209.07463]. The “special” condition is
\[
\partial f(x,a):=f(x,a,1)-f(x,a,0)=\tau_1(g(x))+\tau_2(g(x))a,
\]
which covers \(\ell_1\) loss, squared loss, generalized linear model losses after scaling, and linear constraints including statistical parity, equal opportunity, and equalized odds [2209.07463]. For arbitrary group objectives and constraints with bounded differences, stronger group level-set multiaccuracy is required [2209.07463].

A plausible implication is that omniprediction functions as a reusable representation not only for uncertain losses but also for uncertain regulatory or fairness regimes, provided the relevant subgroup structure is specified in advance.

## 5. Extensions beyond the original binary batch setting

The regression extension reframes omniprediction through **sufficient statistics for loss minimization**. A family \(S=\{s_i:[0,1]\to[-1,1]\}_{i=0}^d\), with \(s_0\equiv 1\), gives \((d,\lambda,\delta)\)-uniform approximations to \(\mathcal L\) if for every \(\ell\in\mathcal L\) there are coefficient functions \(r_i^\ell\) such that
\[
\left| \ell(y,t)-\sum_{i=0}^d r_i^\ell(t)s_i(y) \right|\le \delta,
\qquad
\sum_{i=0}^d |r_i^\ell(t)|\le \lambda.
\]
If a predictor is multiaccurate on the corresponding statistic class and calibrated, then it is an \((\mathcal L,\mathcal C,\varepsilon)\)-omnipredictor with
\[
\varepsilon = 3(d\alpha+\lambda\beta+\delta)
\]
[2401.14645]. The same paper proves that the \(\delta\)-approximate dimension of bounded, convex, \(1\)-Lipschitz functions on \([0,1]\) satisfies
\[
\Omega\!\left(\frac{1}{\delta^{2/3}}\right)
\le
\dim_\delta(F)
\le
O\!\left(\frac{1}{\delta^{2/3}\log^3(1/\delta)}\right),
\]
which leads to faster omnipredictor learning for convex Lipschitz losses than a naive discretization-based approach [2401.14645].

For **single-index models**, omniprediction has become a route to agnostic learning with unknown activations. One paper uses calibrated multiaccuracy to construct omnipredictors that minimize matching losses, then converts matching-loss control into squared-error guarantees for SIMs and GLMs through Bregman-divergence inequalities [2306.10615]. A later paper gives a simpler Isotron-based omnipredictor construction for SIMs, defines the omnigap
\[
OG(p;\sigma,w) := \mathbb{E}\Big[(p(x)-y)\big(\sigma^{-1}(p(x))-w\cdot x\big)\Big],
\]
and proves that \(OG(p)\le \varepsilon\) implies \(p\) is an \(\varepsilon\)-omnipredictor [2411.13083]. Its finite-sample algorithm `Omnitron` requires
\[
n = \widetilde{O}\!\left(\min\left\{\frac{\beta^2 L^2 R^2}{\varepsilon^4},\;\frac{\beta L^2R^2}{\varepsilon^3}+\frac{dL^2R^2}{\varepsilon^2}\right\}\right)
\]
samples for Lipschitz links, with improvement to roughly \(\widetilde O(\varepsilon^{-2})\) in the bi-Lipschitz case, and outputs a structured multi-index model with \(\Theta(\varepsilon^{-2})\) heads [2411.13083].

The multiclass setting requires substantially different machinery. Recent work defines a multiclass omnipredictor \(p(x)\in\Delta^k\) through the ex ante optimal action map
\[
k^\star_\ell(p) := \arg\min_{k^\star \in \Omega} E_{i \sim p}\big[\ell(k^\star, e_i)\big]
\]
and proves omniprediction with horizon or sample complexity approximately \(\varepsilon^{-(k+1)}\) for \(k\)-class problems [2602.17577]. Its main technical contribution is a framework for **simultaneous Blackwell approachability**, developed to combine calibration and multiaccuracy constraints in the multiclass simplex [2602.17577].

The outcome-indistinguishability perspective has also been transferred to **performative prediction**, where deployed actions affect the distribution of outcomes. Under **outcome performativity**, Nature is modeled by
\[
p^* : \mathcal X \times \mathcal Y \to [0,1],
\qquad
y^* \sim \mathrm{Bernoulli}(p^*(x,y)).
\]
A performative omnipredictor is a predictor \(q:\mathcal X\times\mathcal Y\to [0,1]\) such that, for each loss \(\ell\), the induced decision rule
\[
f_\ell(x)\in \arg\min_{y\in\mathcal Y} \mathbb E_{y^*\sim q(x,y)}[\ell(x,y,y^*)]
\]
is nearly optimal under the true performative environment [2210.01745]. The paper proves that performative POI plus performative DOI imply performative omniprediction [2210.01745].

In evolving graphs, an online kernel-based framework based on a modified \(K29^*\) algorithm yields outcome indistinguishability and omniprediction guarantees for rich, possibly infinite distinguisher classes over node pairs and neighborhoods [2411.17582]. The resulting graph predictors support multicalibration across pairs of demographic groups, tests based on embeddedness
\[
\mathrm{Em}(u)=|\Gamma_G(i)\cap \Gamma_G(j)|,
\]
and competition against finite sets of bounded graph predictors, including graph neural networks [2411.17582].

## 6. Statistical optimality, proper losses, and the current frontier

A recent development argues that multicalibration is stronger than necessary for omniprediction. For binary proper losses
\[
\mathcal L_0=\left\{\ell:[0,1]\times[0,1]\to[0,1]\mid \forall p\in[0,1],\ p\in\arg\min_{a\in[0,1]}\mathbb E_{Y'\sim \mathrm{Ber}(p)}[\ell(a,Y')] \right\},
\]
one paper proves lower bounds showing that calibrated multiaccuracy and multicalibration are strictly harder objectives than omniprediction [2510.12769]. In particular, for \(\mathcal X=\mathbb R\) and \(\mathcal G=\{x\mapsto x\}\),
\[
\inf_{\hat p}\sup_{P_{XY}}
\mathbb E\!\left[\max\{\mathrm{MA}(\hat p;\mathcal G),\mathrm{ECE}(\hat p)\}\right]
\ge c\,n^{-2/5},
\]
whereas omniprediction over proper losses can achieve the standard VC rate
\[
\tilde O\!\left(\sqrt{\frac{\mathrm{VC}(\mathcal F)}{n}}\right)
\]
[2510.12769].

The same work exploits the fact that every left-continuous proper loss decomposes as a mixture of weighted \(0\!-\!1\) losses,
\[
\ell_\theta(p,y)=\theta\,\mathbf 1\{p>\theta,y=0\}+(1-\theta)\,\mathbf 1\{p\le \theta,y=1\},
\]
and establishes both a sample-efficient randomized online-to-batch algorithm and a direct deterministic unrandomized algorithm, each attaining
\[
\tilde O_P\!\left(\sqrt{\frac{\mathrm{VC}(\mathcal F)}{n}}\right)
\]
for omniprediction over left-continuous proper losses [2510.12769]. This suggests that, at least for proper losses, omniprediction can match ordinary statistical learning rates without the calibration overhead inherited from earlier constructions.

The same theme appears in a broader generalization called **panprediction**, which unifies omniprediction and multi-group learning. A predictor \(p^*:X\to[0,1]\) is a \((L,G,H,\varepsilon)\)-panpredictor if, for every loss \(\ell\in L\) and every group \(g\in G\),
\[
\mathbb{E}\bigl[\ell(k_\ell(p^*(x)),y)\mid g(x)=1\bigr]
\le
\min_{h\in H}\mathbb{E}\bigl[\ell(h(x),y)\mid g(x)=1\bigr]+\text{(small error)}.
\]
Recent work gives deterministic and randomized step-calibration algorithms with sample complexities \(\tilde O(1/\varepsilon^3)\) and \(\tilde O(1/\varepsilon^2)\), respectively, and shows that under bounded-variation losses, many-loss many-group prediction can be statistically as easy as ordinary learning [2510.27638].

The most recent deterministic results close a longstanding gap. Optimal deterministic multicalibration and finite-test outcome indistinguishability imply optimal deterministic omnipredictors and panpredictors, resolving open problems about whether randomized output predictors are necessary [2606.20557]. In this sense, the present frontier is less about existence than about the strongest possible combination of generality, oracle efficiency, and structural simplicity.

A common misconception is that omnipredictors are merely calibrated probability estimators. The literature instead treats calibration, multicalibration, proper calibration, calibrated multiaccuracy, step calibration, and various forms of outcome indistinguishability as **solution concepts** or sufficient conditions for a stronger objective: a single learned representation that supports many downstream decision rules through explicit Bayes-act post-processing [2210.08649] [2501.17205] [2510.27638]. Another misconception is that omniprediction is intrinsically tied to convex losses. Foundational results focused on nice convex losses [2109.05389], but later work covers non-convex losses through Loss OI [2210.08649], proper losses via threshold decompositions [2510.12769], bounded-variation losses [2501.17205], and constrained or group-conditional objectives [2209.07463] [2510.27638].

Taken together, these developments position omnipredictors as a unifying object at the intersection of calibration, fairness, robust post-processing, online learning, and statistical decision theory: a single predictor intended to preserve enough predictive information that many loss-minimization problems can be solved after training, rather than during it.

Source: https://www.emergentmind.com/topics/omnipredictors