Weak Harm Probability Bound

• Weak harm probability bound is a nonparametric interval that quantifies the probability that an intervention causes harm by comparing outcomes under treatment and control.
• It leverages sharp Fréchet and Tian–Pearl bounds to establish robust limits on harm probability without relying on untestable assumptions like monotonicity.
• Applications in medical studies, policy evaluation, and AI safety utilize proxy-based tightening and stabilized one-step inference to reliably assess risk under unmeasured confounding.

A weak harm probability bound is a nonparametric, assumption-lean interval for the probability that an exposure or intervention causes harm, defined as the probability that a subject’s outcome under treatment is worse than under control. Weak bounds are typically “sharp” in the sense that no tighter bound can be obtained from observed data without introducing untestable assumptions such as monotonicity or the absence of unmeasured confounding. These bounds play a central role in causal inference, partial identification, sensitivity analysis, and safety verification, particularly in heterogeneous populations and in the presence of unmeasured confounding.

1. Formal Definition and Notation

Let $X\in\{x,x'\}$ denote a binary exposure, $Y\in\{y,y'\}$ a (typically binary or tiered) outcome, and $U$ an arbitrary unmeasured confounder. The potential outcomes are $Y_x$ (the outcome under $X=x$) and $Y_{x'}$ (the outcome under $X=x'$). Harm occurs for an individual if $Y_{x'}=y$ (would be "good" under $x'$) but $Y_x=y'$ ("bad" under $x$). The harm probability is

$p(\text{harm}) = P(Y_{x'}=y,\, Y_x=y').$

For ordered categorical or continuous outcomes partitioned into $K$ tiers, tiered analogues define harm as $Y^1 < Y^0$ and parameterize the harm probability as $\pi_h(x) = P(Y^1 < Y^0 \mid X=x)$ (Aguas et al., 14 Feb 2025).

2. Sharp Weak Bounds without Monotonicity

In the absence of assumptions beyond observed data (ignorability and positivity, or none at all in the presence of unmeasured confounding), $p(\text{harm})$ is not point-identified. Instead, it is bounded by sharp Fréchet or Tian–Pearl bounds:

$0 \leq p(\text{harm}) \leq p(x,y') + p(x',y).$

For arbitrary observed joint distributions, these form the “basic bound” (Peña, 2023).

Sensitivity analysis parameterizes possible heterogeneity induced by $U$ with parameters

$$m_x = \min_u P(Y=y \mid X=x, U=u), \quad M_x = \max_u P(Y=y \mid X=x, U=u),$$

and similarly for $x'$. The sharp lower and upper bounds become

$$\begin{aligned} \text{Lower:}\;\;\; & \max \Bigl\{ 0,\, p(x',y) + p(x)m_{x'} - [p(x,y) + p(x')M_x],\, p(x',y) - p(x')M_x,\, p(x)m_{x'} - p(x,y) \Bigr\},\ \text{Upper:}\;\;\; & \min \Bigl\{ p(x',y)+p(x)M_{x'},\, 1-p(x,y)-p(x')m_x,\, p(x,y')+p(x',y),\, p(x')+p(x)M_{x'} - p(x')m_x \Bigr\}. \end{aligned}$$

These are “sharp” for any fixed sensitivity region $(m_x, M_x, m_{x'}, M_{x'})$—no smaller interval for $p(\text{harm})$ is compatible with the observed margins and these parameters (Peña, 2023). For tiered outcomes, setting appropriate conditionals and applying Fréchet bounds for each stratum yields lower and upper bounds

$\Lambda(x) \leq \pi_h(x) \leq \Upsilon(x),$

with $\Lambda(x)$ and $\Upsilon(x)$ built from observed and modelled conditional margins (Aguas et al., 14 Feb 2025).

3. Role of Assumptions and Point-Identification

Strong (non-harm) monotonicity, $P(Y^1 \geq Y^0 | X=x)=1$, eliminates the possibility of harm: $\pi_h(x)=0$ (Aguas et al., 14 Feb 2025). However, even imposing strong monotonicity does not generally point-identify the benefit or harm probability when $K \geq 3$ (for $K$ ordered outcome tiers), as the system of joint cell-probabilities is underdetermined. Sharp weak bounds remain optimal in this context.

Proxy-based tightening is possible when a measured proxy $W$ for $U$ satisfies a nondifferentiality assumption $W \perp\!\!\!\perp (X,Y)\mid U$; under monotonicity testable in $W$, observed marginals involving $W$ serve to bound the cross-world terms and yield strictly tighter bounds (Peña, 2023).

4. Sensitivity Analysis and Visualization

Weak harm probability bounds depend on expert-elicited or empirically motivated ranges for $(m_x, M_x, m_{x'}, M_{x'})$. The feasible region for these parameters is typically constrained by observed $P(y \mid x)$, $P(y \mid x')$. Plotting the upper bound as a function of these parameters produces a 2D contour map. This is recommended practice, as it transparently communicates how plausible ranges for sensitivity parameters map to uncertainty in $p(\text{harm})$ (Peña, 2023).

An example for binary outcomes: plot $m_{x'}$ (worst-case probability of “good” outcome under $x'$, any $u$) and $M_x$ (best-case probability under $x$, any $u$) on axes; color the region by the computed upper bound.

5. Statistical Inference and Implementation

For weak probability bounds that are functionals of observed data (e.g., $\Lambda(x)$, $\Upsilon(x)$ in tiered settings), plug-in and naive one-step estimators may be nonregular due to boundary or "exceptional law" issues. The stabilized one-step correction (S1S) strategy achieves valid $\sqrt{n}$ inference even under nonregularity:

• Target the vector $[\Lambda(x), \Upsilon(x)]^\top$ with a stabilizing matrix derived from the estimated influence-function covariance.
• Apply an online, stratified update scheme to ensure a martingale structure and reliable coverage.
• Construct confidence intervals from the corrected estimator and asymptotic variance.

Nuisance functions (conditional margins, propensity scores) should be estimated flexibly, e.g., via SuperLearner or targeted maximum likelihood estimation (TMLE) (Aguas et al., 14 Feb 2025).

6. Interpretability, Limitations, and Recommendations

The interval $[\Lambda(x), \Upsilon(x)]$ or $[\text{LB}_\text{harm}, \text{UB}_\text{harm}]$ is only as informative as the expert or empirical constraint on the sensitivity parameters. Proxy-based intervals, when available and justified by the testable properties of $W$, are typically contained within sensitivity-parameter intervals, and both are much tighter than the no-assumption “basic bound.” Presenting both intervals assists in honest uncertainty quantification regarding harm probabilities in the presence of unmeasured confounding (Peña, 2023).

In practice, one should display both plots—sensitivity-region and proxy-based bounds—along with reported monotonicity-test $p$-values, to communicate the robustness of harm probability conclusions transparently.

7. Applications and Broader Significance

Weak harm bounds are indispensable in observational causal inference (medical, economic, policy), safety verification in AI (where individual “harm” may correspond to a safety violation under uncertain context), and model selection or policy evaluation when the potential for subpopulation-specific harm precludes strong identification. Recent extensions to runtime, Bayesian AI “guardrails” involve bounding harm via posterior-maximizing plausible models, providing decision-theoretic guarantees even when world models are uncertain and potentially non-i.i.d. (Bengio et al., 9 Aug 2024). These methods yield uniform-in-time and asymptotic bounds, though practical implementation remains challenging due to tractability, posterior computation, and conversion of soft safety specifications to testable probabilistic predicates.

In summary, weak harm probability bounds provide rigorous, transparent, and assumption-lean quantification of the risk of harm from interventions or actions in the presence of confounding and heterogeneity, underpinning a wide range of robust inference and safety methodologies in statistics and AI.