Distributional Certificates

• Distributional Certificates are formal guarantees that bound worst-case expected losses or fairness violations over sets of probability distributions rather than single inputs.
• They apply optimization techniques over uncertainty sets like Wasserstein or Hellinger balls to certify properties in domains such as adversarial robustness and individual fairness.
• Constructive methods include interval bound propagation, dual formulations, and concave relaxations that yield practical, though sometimes conservative, safety guarantees.

Searching arXiv for the cited papers and closely related work on distributional certificates. First, locating the focal paper on distributional individual fairness. Now searching for complementary papers on distributional certificates in robustness, fairness, and formal verification. Distributional certificates are formal guarantees for properties evaluated not at a single input or along a single trajectory, but over a set of admissible probability distributions. In machine learning and formal verification, they typically take the form of a computable upper bound on a worst-case expected loss, violation, or specification value over an uncertainty set such as a Wasserstein ball or a Hellinger ball. A central instance is the certification of distributional individual fairness, where one seeks an upper bound on the worst-case expected local fairness violation under all distributions within a prescribed Wasserstein radius of an empirical distribution (Wicker et al., 2023). Closely related uses include robustness certificates for smoothed classifiers (Yang et al., 2020), fairness certificates under Hellinger-bounded shift (Kang et al., 2022), PAC-Bayesian certificates for posterior-averaged adversarial risk (Sabanayagam et al., 20 Feb 2025), sound-and-complete proof rules for distributional $\omega$-regular specifications in MDPs (Akshay et al., 6 Jul 2025), tight Wasserstein robustness certificates for deep networks (Le et al., 11 Oct 2025), and concave certificates based on least concave majorants of growth-rate functions (Chu, 4 Jan 2026).

1. General notion and scope

In the broader machine-learning literature, a distributional certificate is typically an upper bound on some worst-case risk or property over a set of distributions, often a Wasserstein or $f$-divergence ball. What varies across subfields is the certified functional: worst-case expected loss, worst-case misclassification probability, expected local fairness violation, posterior-averaged adversarial risk, or satisfaction of a temporal specification over a sequence of distributions (Wicker et al., 2023).

Setting	Certified object	Representative paper
Neural-network individual fairness	$$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$	(Wicker et al., 2023)
Smoothed classifiers	Worst-case adversarial population loss over input distributions	(Yang et al., 2020)
Group-fair deployment shift	Worst-case loss over fair distributions within Hellinger distance $\rho$	(Kang et al., 2022)
Bayesian linear regression	PAC-Bayesian bounds on $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$	(Sabanayagam et al., 20 Feb 2025)
Distributional MDP verification	Existence of ranking/invariant proof objects for $\omega$-regular specifications	(Akshay et al., 6 Jul 2025)
Wasserstein DRO for deep nets	Certified upper bound on worst-case expected loss over $\Omega_1$	(Le et al., 11 Oct 2025)
General DRO geometry	Concave upper bound $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$	(Chu, 4 Jan 2026)

A recurrent misconception is to identify a distributional certificate with a global worst-case guarantee over the entire input domain. The fairness literature makes the distinction explicit: global individual fairness requires $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$, whereas distributional certification only constrains the expected violation over distributions inside a Wasserstein neighborhood. This is weaker than global certification, but substantially more scalable and more tightly tied to likely deployment shifts (Wicker et al., 2023).

Another important distinction concerns what is being certified. Some papers certify a deterministic predictor under distribution shift, whereas PAC-Bayesian work certifies a randomized predictor by bounding posterior-averaged risk. In that setting, the guarantee is on $\mathbb{E}_{\theta\sim\rho}[R(\theta)]$ or $f$0, not on a single point estimate (Sabanayagam et al., 20 Feb 2025).

2. Mathematical formulations

A common starting point is a nominal distribution, usually the empirical distribution $f$1, and an uncertainty set around it. In the fairness and robustness literature, the uncertainty set is frequently a Wasserstein ball,

$f$2

or, for robustness of smoothed classifiers,

$f$3

where $f$4 models Gaussian smoothing (Wicker et al., 2023, Yang et al., 2020).

For empirical measures, Wasserstein DRO often reduces to finite-dimensional perturbation variables. In distributional individual fairness, any $f$5 can be represented by moving each atom $f$6 to $f$7 under the constraint

$f$8

This turns the infinite-dimensional supremum over distributions into an optimization over sample-wise perturbations (Wicker et al., 2023).

The property of interest is then inserted into the expectation. For individual fairness, the local fairness map is

$f$9

and distributional individual fairness requires

$$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$0

A distributional certificate is therefore a computable upper bound $$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$1 on that left-hand side (Wicker et al., 2023).

Other domains instantiate the same pattern differently. In certified fairness with subpopulation decomposition, the uncertainty set is defined by a Hellinger ball together with a fairness constraint that the shifted distribution $$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$2 is a fair base-rate distribution: $$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$3 In Bayesian linear regression, the certified quantity is the posterior-averaged adversarial risk

$$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$4

bounded by PAC-Bayesian terms involving empirical loss, $$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$5, and sub-gamma parameters of the adversarial loss (Kang et al., 2022, Sabanayagam et al., 20 Feb 2025).

A broader geometric abstraction replaces Lipschitz constants by growth-rate functions. For loss $$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$6, the individual growth rate is

$$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$7

and the central certificate is the least concave majorant of the maximal empirical rate. This yields a bound of the form

$$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$8

which applies even to non-Lipschitz and non-differentiable losses (Chu, 4 Jan 2026).

3. Distributional individual fairness

The paper “Certification of Distributional Individual Fairness” formalizes the fairness case around a metric-based notion of individual fairness for neural networks. With input space $$\sup_{P^\star \in \mathcal{B}_\gamma(\hat P^0)} \mathbb{E}_{x\sim P^\star}[\mathcal{I}(f^\theta,x,\delta)]$$9, output space $\rho$0, network $\rho$1, and fair distance metric $\rho$2, local $\rho$3–$\rho$4 individual fairness at input $\rho$5 is defined by

$\rho$6

The quantity $\rho$7 is the smallest such $\rho$8, so $\rho$9 corresponds to perfect local fairness (Wicker et al., 2023).

Distributional individual fairness moves from pointwise control to distributional control under shift. Let $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$0 be the data-generating distribution and $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$1 the Wasserstein ball of radius $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$2. Then $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$3 is distributionally $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$4–$\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$5–$\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$6 individually fair if

$\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$7

Because $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$8 is unknown, the paper replaces it by the empirical distribution $\mathbb{E}_{\theta\sim\rho}[R_\epsilon(\theta)]$9 and certifies the resulting empirical Wasserstein-ball property (Wicker et al., 2023).

In this setting, a certificate is a sound upper bound

$\omega$0

The exact optimum $\omega$1 is given by

$\omega$2

but this is intractable. The paper therefore computes a lower bound $\omega$3 and a certified upper bound $\omega$4, with the latter constituting the actual distributional certificate (Wicker et al., 2023).

Conceptually, this work positions distributional fairness certification between two older extremes. Pointwise IF certificates upper-bound $\omega$5 at a single input, while global IF certificates attempt to prove $\omega$6 over the entire domain. The former are too local for deployment guarantees; the latter, especially in MILP formulations associated with Benussi et al. and Khedr et al., do not scale beyond very small networks. Distributional certificates sacrifice full-domain worst-case coverage in exchange for guarantees over likely distribution shifts (Wicker et al., 2023).

4. Constructive techniques

A striking feature of the literature is the variety of proof mechanisms used to make distributional certificates computable. In distributional individual fairness, the key relaxation is an orthotope over-approximation of a Mahalanobis fair ball: $\omega$7 where $\omega$8. This turns an ellipsoidal fairness region into an axis-aligned box, after which interval bound propagation yields a sound local bound

$\omega$9

The distributional upper bound is then obtained by quasi-convex optimization over radius variables $\Omega_1$0: $\Omega_1$1 The same framework is also used as a training-time regularizer through F-IBP, L-DIF, and U-DIF objectives (Wicker et al., 2023).

In smoothed classification robustness, the dominant construction is dual and penalized rather than box-based. The central quantity

$\Omega_1$2

is upper-bounded by

$\Omega_1$3

This certificate leads to the Noisy Adversarial Learning procedure, whose inner optimization becomes strongly concave when the transportation cost is strongly convex and the smoothed loss is $\Omega_1$4-smooth (Yang et al., 2020).

In fairness under Hellinger-bounded shift, tractability comes from subpopulation decomposition. Writing the data as a mixture over subpopulations indexed by $\Omega_1$5 yields an exact Hellinger decomposition

$\Omega_1$6

Under sensitive shifting, where $\Omega_1$7, the certificate reduces to a tight convex program in mixture weights $\Omega_1$8 and $\Omega_1$9. Under general shifting, the paper combines this decomposition with the Weber et al. bound, a change of variables, and a grid-based convex relaxation to obtain a non-tight but tractable certificate (Kang et al., 2022).

For Wasserstein DRO in deep networks, a different line of work uses exact Lipschitz geometry. For ReLU networks, the certificate is

$\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$0

with matching lower bounds based on explicit adversarial distributions and a Wasserstein Distributional Attack that constructs a candidate worst-case distribution in $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$1 (Le et al., 11 Oct 2025). The 2026 concave-certificate framework generalizes the same ambition: instead of a global Lipschitz constant, it uses the least concave majorant of the growth-rate function and introduces adversarial scores as tractable layer-wise relaxations for neural networks (Chu, 4 Jan 2026).

Outside statistical learning, distributional certificates may be symbolic proof objects rather than numerical risk bounds. For MDPs under the distribution-transformer semantics, a certificate is a pair $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$2 consisting of a distributional Büchi ranking function and a distributional invariant over the product of the MDP dynamics and a nondeterministic Büchi automaton. The ranking decreases on non-accepting automaton states and stays nonnegative on accepting states, yielding sound-and-complete proof rules for distributional $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$3-regular specifications (Akshay et al., 6 Jul 2025).

5. Guarantees, tightness, and empirical behavior

The central desideratum is soundness: a certificate must never under-estimate the worst-case quantity it claims to bound. The fairness paper proves that the orthotope relaxation and IBP yield a sound upper bound on local individual fairness, that the empirical Wasserstein-ball supremum is equivalent to the perturbation optimization over $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$4, and that the quasi-convex radius optimization produces $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$5. It also establishes Hölder continuity for the aggregated objective under a Lipschitz or Hölder condition and gives finite-sample guarantees via Hoeffding-type inequalities, so that with $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$6 the empirical certificate deviates from the population value by at most $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$7 with probability at least $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$8 (Wicker et al., 2023).

Tightness varies substantially across constructions. In the sensitive-shifting fairness setting, the subpopulation-decomposition program is both convex and tight: its optimum equals the supremum in the original certification problem. In the general-shifting setting, by contrast, the certificate is explicitly non-tight because it inherits slack from the Weber per-subpopulation bound and the grid relaxation, though it remains non-trivial in experiments on six real-world datasets (Kang et al., 2022).

For smoothed classifiers, the upper bound based on $\mathcal{C}_{\Delta_\theta^{\max}}(\epsilon)$9 is accompanied by two important comparison results: adding Gaussian noise yields a worst-case distributional loss no larger than that of the base classifier, and the smoothed surrogate certificate is no worse than the standard base-classifier Wasserstein DRO certificate derived in the style of Sinha et al. (Yang et al., 2020). In the Bayesian linear-regression setting, the guarantees are rigorous PAC-Bayesian generalization certificates for standard and adversarial risks of both the standard posterior and the adversarially robust posterior, expressed through explicit data-fit terms, KL terms, and sub-gamma tail parameters (Sabanayagam et al., 20 Feb 2025).

The 2025 WDRO paper sharpens the notion of tightness by proving two-sided bounds

$\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$0

and then giving sufficient conditions under which $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$1, so that the Lipschitz certificate is exact (Le et al., 11 Oct 2025). The 2026 concave-certificate framework makes a related claim at a more abstract level: the least concave majorant produces a tight bound on DR risk that remains applicable when the loss is non-Lipschitz and non-differentiable (Chu, 4 Jan 2026).

Empirically, the fairness literature emphasizes that soundness need not imply vacuity. Distributional IF certificates were evaluated under geographic and temporal shifts on Folktables data, where the lower bound tracked the observed worst-case empirical DIF and the upper bound was conservative but typically within a factor of order $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$2 above the empirical worst-case, especially for models trained with DIF regularization (Wicker et al., 2023). The Hellinger-based fairness framework reported that sensitive-shifting certificates closely tracked the upper envelope of sampled fair distributions, while general-shifting certificates stayed clearly below trivial bounds and tightened further under non-skewness constraints (Kang et al., 2022).

6. Limitations and open directions

A recurring limitation is conservatism induced by tractable relaxations. In distributional individual fairness, the orthotope may be much larger than the true fair ball, especially in high dimension, and IBP is known to loosen on deep networks; as a result, $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$3 and $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$4 can be conservative for large $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$5 or deep models, even though soundness is preserved (Wicker et al., 2023). In the Hellinger-based framework, the general-shifting certificate is not tight, and the number of convex subproblems grows as $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$6, which becomes costly beyond the common binary-sensitive, binary-label setting (Kang et al., 2022).

Another limitation is that certificates are usually scenario-specific. Distributional IF addresses individual fairness, not group fairness; extending distributional robustness guarantees to group-fairness metrics is identified as an open path. The Hellinger-based fairness work restricts attention to fair base-rate distributions and does not guarantee performance under arbitrary shifts. Smoothed-classifier certificates depend on choosing $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$7 and $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$8, and their practical implementation uses approximate inner maximization. PAC-Bayesian adversarial certificates are derived only for Bayesian linear regression under a Gaussian model with $\sup_x \mathcal{I}(f^\theta,x,\delta)\le \epsilon$9 perturbations. The MDP verification framework currently targets distributionally memoryless strategies and affine templates. The exact-Lipschitz WDRO analysis for deep networks is limited in practice by the difficulty of exploring large numbers of reachable ReLU activation patterns. The concave-certificate framework still requires tractable upper bounds on the growth-rate function for large deep models (Yang et al., 2020, Sabanayagam et al., 20 Feb 2025, Akshay et al., 6 Jul 2025, Le et al., 11 Oct 2025, Chu, 4 Jan 2026).

The uncertainty model itself is also an open design choice. Wasserstein distance dominates much of the literature, but alternatives already appear. The fairness certification framework based on subpopulation decomposition uses Hellinger distance and notes implications for total variation. The distributional IF paper explicitly mentions that only Wasserstein distance is considered and that other distances, including $\mathbb{E}_{\theta\sim\rho}[R(\theta)]$0-divergences, optimal transport with different costs, or learned transport costs, could define alternative uncertainty sets. The 2026 framework is deliberately stated for general measurable costs $\mathbb{E}_{\theta\sim\rho}[R(\theta)]$1, which suggests a route toward broader transport-based certificates, though the paper does not claim a universal computational method (Kang et al., 2022, Wicker et al., 2023, Chu, 4 Jan 2026).

Taken together, these works support a precise view of distributional certificates as a family of proof techniques for uncertainty over distributions rather than points. In one branch, they certify fairness, robustness, or posterior risk by upper-bounding a worst-case expectation over a distributional ambiguity set. In another, they certify temporal properties of systems that transform distributions over time. The unifying theme is not a single algorithm, but a common shift in semantics: robustness, fairness, and verification are framed as properties of distributional perturbations or distributional trajectories, and the certificate is the sound object that upper-bounds, or otherwise proves, the desired property under that distributional view (Wicker et al., 2023, Akshay et al., 6 Jul 2025).