Distributional Stability in Statistical Models

• Distributional stability is a measure of system robustness that quantifies how outcomes or parameters resist degradation when the underlying data distribution is altered.
• It is applied across fields like robust machine learning, stochastic optimization, and reinforcement learning to ensure performance consistency under adversarial shifts.
• Methodologies include using divergence metrics, optimal transport discrepancies, and integer programming to compute minimal perturbations required for significant risk increases.

Distributional stability refers to the robustness of system outcomes, model parameters, or solution concepts when the underlying probability distribution is perturbed or subject to structural constraints. In contemporary research, the term appears across diverse areas including statistical inference, robust machine learning, stochastic optimization, control, reinforcement learning, matching markets, and economic dynamics. The common thread is the quantitative characterization of how solutions or performance guarantees degrade—or resist degradation—when the data-generating law, or distributional structure, is no longer fixed but allowed to shift within prescribed sets or balls (often defined by divergences, Wasserstein distances, or sub-population mixtures).

1. Formalizations of Distributional Stability

The precise formalization of distributional stability depends on the context:

• Distributional parameter perturbation (statistical/learning): For a parameter $\theta(P)$ defined on measures $P\in\mathcal P$, distributional stability is often given as the minimal "distance" (with respect to $f$-divergence, Wasserstein metric, or total variation) required to effect a qualitative change in $\theta$ or degrade performance by a prescribed level. For example, the $s$-value framework defines

$$s(\theta,P_0) = \exp\left( -\min_{P:\,\theta(P)=0} D_{KL}(P\|P_0) \right)$$

where small $s$ indicates high robustness of the sign of $\theta$ to distributional shift (Gupta et al., 2021, Rothenhäusler et al., 2022). A related line quantifies stability against directional or variable-specific marginal shifts by further constraining the conditional structure of $P$.

• Minimal perturbation for risk increase: In robust optimization and risk evaluation, distributional stability is defined as the minimal perturbation—measured in OT (optimal transport) discrepancy or other metrics—needed to increase a prediction risk by $\Delta$:

$$S_\Delta(P, \ell, f) = \inf_{Q: \mathbb{E}_Q[\ell(f;Z)] - \mathbb{E}_P[\ell(f;Z)] \ge \Delta} \text{Dist}(Q, P)$$

This perspective can be adapted to quantify separately the effect of support perturbations (data corruptions) and density reweighting (sub-population shifts) (Blanchet et al., 2024).

• Matching under distributional constraints: In two-sided matching, stability under distributional (type, quota) constraints requires matchings to respect group-specific lower and upper quotas. A matching is distributionally stable if it admits no blocking pairs and satisfies all quota requirements. Exact formulations may be encoded as linear (integer) programs (Ágoston et al., 2017).
• Evolutionary game dynamics: An equilibrium is distributionally stable if all nearby full-type distributions (not just the aggregate) converge back to it under the dynamic, requiring robustness to persistent heterogeneity (Zusai, 2018).
• Reinforcement learning and stochastic programming: In distributional RL, stability is characterized by the contraction properties of Bellman operators with respect to return distributions in Wasserstein metrics, with implications for algorithmic convergence and robustness (Bellemare et al., 2017, Zhou et al., 22 Jan 2025). In stochastic programming, stability of the optimal value or solutions is analyzed with respect to the epi-convergence of problem data under weak or strong probability metrics (Tian et al., 21 Jul 2025).
• Estimator stability: Recent work establishes explicit local Lipschitz bounds on the pushforward distribution of statistical estimators (e.g., sparse precision matrices) as a function of perturbations in the data-generating law measured in transportation-type metrics (Chen et al., 2024).

2. Core Metrics and Notions

Several canonical metrics underlie analyses of distributional stability:

• Kullback–Leibler (KL) Divergence: Used widely for defining divergence balls in robust statistics and distributional perturbation analyses. Ability to flip a parameter within a given KL ball defines $s$-value-type stability (Gupta et al., 2021, Rothenhäusler et al., 2022, Namkoong et al., 2022).
• Wasserstein (Kantorovich) Distances: Provide a geometry-aware metric for quantifying the minimal cost to redistribute probability mass, central in DRO and distributional RL (Bellemare et al., 2017, Eskenazis et al., 2023, Chen et al., 2024, Blanchet et al., 2024).
• Sub-population/mixture representations: Stability to adversarially chosen sub-populations down to mass $\alpha_0$ (partial mixture decompositions) leads to the $\mathrm{DS}_{\alpha_0}$ criterion, quantifying the maximal Kullback–Leibler divergence of sub-group conditionals to the overall (Liu et al., 2022).

3. Algorithmic and Optimization Formulations

Distributional stability requirements frequently translate to tractable optimization formulations:

• Integer and mixed-integer programming: Employed for matching under complex quota constraints, with the objective of feasibility and minimization of blockings or rank (Ágoston et al., 2017).
• Saddle-point and bi-level games: Used in stable adversarial learning (SAL) and stable risk minimization (SRM) to differentiate between stable and unstable covariates, or to minimize empirical risk subject to sub-population conditional stability (Liu et al., 2020, Liu et al., 2022, Liu et al., 2021).
• Convex dual programs: Dual strong form, especially for OT-type stability measures, enables computation via low-dimensional optimization even