---
title: Harm Propagation Prediction
url: https://www.emergentmind.com/topics/harm-propagation-prediction
type: topic
---

# Harm Propagation Prediction

Harm propagation prediction denotes the estimation of how an initiating failure, perturbation, or extreme event produces downstream harm across time, network distance, execution traces, or causal pathways. In the cited literature, the propagated object may be a future tail event in a multivariate time series, a cascade of line outages in a power system, citation-based harm from retracted papers, execution of a failed automated decision, amplification across multi-agent LLM traces, or the emergence of harmful behavior within a reasoning chain [2106.13564] [2410.19179] [2501.00473] [2602.18986] [2605.27489] [2604.19001]. The field is therefore not a single model class but a family of quantitative formalisms that link an identifiable source event to a structured notion of downstream impact.

## 1. Formal definitions of propagated harm

Courgeau and Veraart formulate propagation from an extreme “cause” margin to future “impact” events in a $d$-dimensional stationary time series $X_t=(X_t^1,\dots,X_t^d)$. For cause margin $i$, the cause event is $C_t^{(i)}=\{X_t^i>\mu_i\}$, while the impact over the next $k$ steps is defined as $I_t(v,w,k)=\{w^\top X_{t+1}(k)>v\}$ for a weight vector $w\in[0,\infty)^{kd}$. Their framework then evaluates factual and counterfactual tail probabilities,
$$
p_+^{(i)}(v;w,k)=P(w^\top X_1(k)>v\mid X_0^i>\mu_i),\qquad
p_-^{(i)}(v;w,k)=P(w^\top X_1(k)>v\mid X_0^i\le \mu_i),
$$
and converts them into the probabilities of necessity (PN), sufficiency (PS), and necessity-and-sufficiency (PNS) following Hannart & Pearl 2016 [2106.13564].

In citation-based studies of retracted research, harm is defined by shortfall in citations relative to a comparator set. If $D(p)=\{q\in P\mid venue(q)=venue(p),\,year(q)\in\{year(p)-1,year(p),year(p)+1\},\,field(q)\cap field(p)\neq\emptyset\}$, then total-citation harm is
$$
H_0(p)=1-\frac{cite\_count(p)}{\frac{1}{n_D(p)}\sum_{q\in D(p)} cite\_count(q)},
$$
with year-specific variants $H_k(p)$ for $k=1,\dots,10$. Propagation is indexed by citation distance: $C_1$ cites a retracted paper directly, and $C_d$ for $d\ge2$ cites at least one paper in $C_{d-1}$ [2501.00473].

In high-automation AI systems, Srivastava & Sah decompose expected loss per decision into technical failure risk, deployment risk, and consequence severity:
$$
\mathbb{E}[\text{Loss}]
=
P(F)\times P(H\mid F,A)\times \mathbb{E}[S\mid H].
$$
Here $F$ is “system failure,” $H$ is “harm occurs,” $A\in[0,1]$ is the automation level, and $S$ is severity conditional on harm. Their framework isolates $P(H\mid F,A)$ as the harm-propagation probability, and under the conditions $H\Rightarrow U$, $U\Rightarrow F$, and $P(H\mid U,F,A)=1$, proves the equivalence $P(H\mid F,A)=P(U\mid F,A)$, linking harm propagation to execution controls rather than model accuracy alone [2602.18986].

In multi-agent LLM systems, HARP defines local harm as deviation over targeted agents or corrupted channels, global harm as deviation over the full trace, and harm amplification as
$$
\mathrm{HA}(\tau^K,\tau^0)=
\frac{H_{\mathrm{global}}(\tau^K,\tau^0)}
{H_{\mathrm{local}}(\tau^K,\tau^0)}
\qquad (H_{\mathrm{local}}>0).
$$
A paired clean/perturbed execution therefore yields a direct operational measure of how orchestration spreads a bounded local perturbation into system-level harm [2605.27489].

For multi-turn conversations, HarmAmp introduces a turn-wise propagation probability $P(h_t=1\mid x_{1:t},y_{1:t-1})$, where $h_t$ is the increment in harmful content caused by the $t$th reply. HarmThoughts instead treats reasoning as a time series of discrete sentence-level behaviors $b_t$ drawn from 16 classes and represents propagation by a first-order Markov chain
$$
T_{i,j}=P(b_{t+1}=j\mid b_t=i)=\frac{N_{i\to j}}{\sum_k N_{i\to k}}.
$$
This shifts attention from final harmful outputs to the intermediate sequence by which harm emerges [2606.02423] [2604.19001].

## 2. Propagation mechanisms and model classes

A prominent line of work models propagation through tails, copulas, and counterfactual dependence. The Extreme Event Propagation framework uses extreme value theory for upper tails, with a semiparametric probability integral transform and generalized Pareto distributions above threshold $\mu_i$, while a stationary vine copula captures joint extremal behavior. One stationary vine copula is fitted to $(U_t)_t$ with Markov order $p$ selected by AIC/BIC, pair-copula families drawn from independence, Clayton, Gumbel, Frank, Joe, and rotated variants, and conditional sampling based on vine-conditioning and the Rosenblatt transform is used to estimate $p_+$ and $p_-$ when data are scarce at high $v$ [2106.13564].

A second family models propagation as network percolation or epidemic diffusion. In error propagation with non-uniform failure, edges are partitioned into types $i=1,\dots,n$ and each type independently transmits onward with probability $p_i$. The generating-function machinery yields a branching factor $\kappa=G_1^{occ\,\prime}(1)$ in the occupied network, and $\kappa>1$ is the necessary and sufficient condition for a nonzero probability of an unbounded cascade. The same formalism gives the fixed-point equation $u=G_1^{occ}(u)$ and final giant-component size $S=1-G_0^{occ}(u)$ [1604.03558]. In electric power cyber-physical systems, percolation is extended to a dual-layer dependency network with directed inter-layer links, survival fractions $\mu_t^c$ and $\mu_t^p$, and a critical attack threshold $\phi_{cps}$ determined by a survival-function analysis [1811.04191].

Diffusion models in power systems also appear in learned contagion form. The hyperparametric Information Cascades model represents transmission-line failures on a directed influence graph and assumes that each edge $(i\to j)$ carries a contagion probability
$$
p_{ij}=H(x_i,x_j;\theta)=\frac{1}{1+\exp(-\theta^\top x_{ij})},
$$
where $x_{ij}$ is derived from line features and $\theta\in[-B,B]^d$ is global. Given newly active parents $N_t(j)$, the independent-cascade activation probability is $1-\prod_{i\in N_t(j)}(1-p_{ij})$ [2406.08522].

Causal-inference approaches replace physical topology with a learned latent graph. In cascading-failure prediction for transmission networks, nodes represent lines, directed edges encode cause-effect relations, and the linear structural causal model
$$
S_i[t]=\sum_{k\in\pa(i)} B_{ik}S_k[t]+\epsilon_i[t]
$$
is learned from observational anomalies. The propagation probability $P(F_j\mid do(F_i))$ is then proportional to a path-sum over all directed paths from $i$ to $j$, with products of edge weights along each path. This explicitly captures nonlocal dependencies that the physical topology need not exhibit [2410.19179].

Economic cascade models take a different form. In the trade-network model for global cascading financial failure, each node has capacity $C_i=GDP_i+|net\ exports_i|$ and state $x_i(t)\in[0,1]$, the fractional loss of capacity. Propagation is governed by the transfer function
$$
T(x_i(t);\lambda)=\frac{\lambda x_i(t)}{\lambda+2x_i(t)-1},
$$
the edge transfer $\Delta_{i\to j}(t)=A_{ij}(t)\times T(x_i(t);\lambda)$, and the update
$$
x_j(t+1)=x_j(t)+\frac{1}{C_j}\sum_{i\to j\in E}\Delta_{i\to j}(t),
$$
with cap $x_j(t+1)\le1$ and a breadth-first sweep over newly activated nodes [2502.12980].

## 3. Learning, calibration, and inference procedures

The inference pipeline depends strongly on the modeling choice. In Extreme Event Propagation, marginal thresholds $\mu_i$ are chosen by sequential GoF tests controlling FDR, generalized Pareto parameters $(\gamma_i,\sigma_i)$ are fitted above threshold by POT MLE, semiparametric PIT values $U_t^i=F_i(X_t^i)$ are computed, a stationary vine is fitted to $\{U_{t-p},\dots,U_{t+k}\}$, and conditional samples for $U_{t+1}(k)$ are drawn under both $U_t^i>q$ and $U_t^i\le q$. The objective is then to maximize a chosen counterfactual probability of causation over $w\ge0$ with $\sum_l w_l=1$, using Differential Evolution globally and L-BFGS-B with multistart locally, optionally with $\ell_1$ or $\ell_2$ penalties [2106.13564].

In hyperparametric diffusion for power grids, training data are extracted from observed cascades as positive samples $S^+_{c,t}=\{(i\to j)\mid i\in \mathbf{V}_t,\, j\in \mathbf{V}_{t+1}\}$ and negative samples $S^-_{c,t}=\{(i\to j)\mid i\in \mathbf{V}_t,\, j\notin\cup_{\tau\le t+1}\mathbf{V}_\tau\}$. For a sample $s=(\mathcal{U}\to j)$, the success probability is $P(\theta\mid x,s)=1-\prod_{i\in\mathcal{U}}(1-H(x_i,x_j;\theta))$, and the average log-likelihood $L_S(\theta)$ is maximized over $\theta\in[-B,B]^d$ by L-BFGS-B with analytic gradients. The paper also provides a PAC-style sample-complexity guarantee of order
$$
\widetilde{O}\!\left(
\frac{V^2\log^2(1/\lambda)\left(d\log(BV\log(1/\lambda)/\epsilon)+\log(1/\delta)\right)}{\epsilon^2}
\right)
$$
for excess expected risk at most $\epsilon$ with probability at least $1-\delta$ [2406.08522].

Causal-inference models learn the propagation operator differently. The latent coefficient matrix $\mathbf{B}$ is recovered from steady-state anomalies $\bar{\mathbf{S}}_{\rm train}$ by a cyclic-LiNGAM procedure: sparse ICA estimates a mixing matrix $\mathbf{W}$, permutation and scaling normalize its diagonal entries, and the causal coefficient matrix is formed as $\mathbf{B}=\mathbf{I}-\mathbf{W}$. Once learned, prediction of the next failures requires only matrix updates and path-sums [2410.19179].

Financial cascade models calibrate a single free parameter. The propagation intensity $\lambda$ is fit to the Great Recession by minimizing quantile-regression pinball loss
$$
L_q(y,\hat y)=\frac{1}{N}\sum_{i=1}^N \left[q\max(y_i-\hat y_i,0)+(1-q)\max(\hat y_i-y_i,0)\right]
$$
for $q\in\{0.25,0.5,0.75\}$ using SciPy’s scalar minimizer. This produces a 50% prediction interval rather than a single best fit [2502.12980].

Run-time prediction in autonomous systems uses yet another inference mode. For trajectory predictors in autonomous vehicles, the monitor samples $M$ futures from the prediction network, pushes them through planner cost to obtain a cost distribution, compares the realized cost $c^*_{t+\tau}$ to the sampled order statistic $\hat c^{(M-n)}_{t+\tau}$, and declares a harmful prediction failure when $c^*_{t+\tau}\ge \hat c^{(M-n)}_{t+\tau}$. The false-positive and false-negative bounds are explicit binomial-tail sums, which permits data-free calibration to a desired upper bound on one error rate [2207.12380].

Sequence forecasting in LLM safety is correspondingly trace-centric. TrajSafe models the monitor as a policy $\pi_\theta$ over conversation histories and trains it by supervised fine-tuning followed by tree-based reinforcement learning with a composite reward $R(\tau)=(1-\lambda)\cdot(1-H(y_{1:t_{\rm end}}))+\lambda\cdot s_{fb}$, with $\lambda=0.5$ [2606.02423]. HarmThoughts, by contrast, explicitly positions its 16-class sentence labels as supervision for recurrent models, Transformers, conditional random fields, or hazard models that forecast the onset of harmful execution, such as the first occurrence of Domain Knowledge Synthesis [2604.19001].

## 4. Metrics and empirical regularities

The literature measures propagation in multiple, domain-specific ways: counterfactual probabilities of causation, giant-component size, regret, citation shortfall, AUROC, harm amplification, intervention rate, and total expected loss. Several empirical regularities recur: delayed manifestation, amplification across generations or turns, sensitivity to network position, and strong dependence on the intervention layer rather than only the source perturbation.

| Domain | Metric or reported result | Citation |
|---|---|---|
| Retracted research | For early years $(k\le4)$, $H_k(p)$ for papers in $C_1$ is small, with median $\approx10\%-20\%$, then steadily rises over $k=5,\dots,10$; for $IF<10$, $H_{1,k}^{low}-H_{1,k}^{high}\approx10$–$15$ percentage points for $k\le5$, with $p<0.01$ | [2501.00473] |
| Power-system causal inference | For $\kappa=25\%$, C-Path achieves $\approx45\%$ precision on 14/39 bus, versus $\approx25\%$ by influence-graph and GNN baselines; for $M=4$, $d=100$, regret drops below $0.2$ for $\kappa\approx30\%$ | [2410.19179] |
| Polypharmacy side-effect prediction | TIP-sum reports AUPRC $0.890$, AUROC $0.914$, AP@50 $0.890$, with $\sim83\times$ speed-up and $\sim3\times$ memory reduction relative to Decagon | [2001.10516] |
| Autonomous-vehicle monitoring | QAD reports AUROC $=0.946$; data-free calibration at target quantile $p=5\%$ yields empirical FPR $=4.2\%$, FNR $=13.9\%$ when calibrated for $\overline{\rm FPR}\le5\%$ | [2207.12380] |
| Multi-turn LLM harm | On Llama-3.1-8B, HarmAmp reports multi-turn harm score $86.36\%$ for the vanilla model; TrajSafe reduces it to $9.85\%$, with over-refusal $11.64\%$ and intervention turns $40.35\%$ | [2606.02423] |
| Multi-agent LLM harm amplification | IntegrityGuard achieves the lowest ASR and lowest HA, with HA $\approx1.2$–$2.9$ and NTC rising $\approx1.2\times$–$1.8\times$; prompt-only defense leaves HA $\sim3$–$4$ in single-point attacks | [2605.27489] |

Additional numerical findings sharpen the diversity of use cases. In the trade-network model, the median-fit $\lambda_{med}$ reproduces country-by-country Great Recession losses with $R^2\approx0.66$ on the linear scale, and the hypothetical India–Pakistan nuclear conflict produces a median global loss of $\approx\$0.812$ trillion under $\lambda_{med}$ [2502.12980]. In the CPS seepage-probability model, the IEEE 30-bus plus 150-node BA case remains intact up to $\phi\approx0.39$ under random attack and breaks at $\phi\approx0.35$ under targeted attack, with complete collapse by $\phi\approx0.46$ and $\phi\approx0.41$, respectively [1811.04191]. In HarmThoughts, the dataset contains 56,931 sentences from 1,018 reasoning traces, with harmful labels comprising 68.90% of sentences and Domain Knowledge Synthesis alone accounting for 39.19%, while current detectors reach at best Macro F1 $0.562$ for Gemini-2.5-pro in few-shot black-box classification and roughly $0.494$–$0.501$ for the best white-box linear probes [2604.19001].

## 5. Intervention, optimization, and decision support

Prediction frameworks are frequently paired with explicit control rules. In Extreme Event Propagation, the optimization variable is the nonnegative weight vector $w$ over future margins and lags, constrained to sum to one. The objective is to maximize a selected counterfactual causation probability, such as PNS$(v;w,k)$, and the output is $(w^\*,PC^\*)$ together with a ranking of marginals by $w^\*$ [2106.13564].

Automation-risk analysis makes intervention the central object. Srivastava & Sah define total cost as
$$
\mathrm{TC}(A)=C_{\mathrm{auto}}(A)+C_{\mathrm{oversight}}(A)+P(F)P(H\mid F,A)\mathbb{E}[S\mid H],
$$
derive the first-order condition for the optimal automation level $A^\*$, and state that if all three cost-and-risk functions are convex in $A$, then $A^\*$ is the unique global minimum. For fixed risk-management budget $B$, they further show an equalized marginal ROI rule: resources should be split between reducing $P(F)$ and reducing $P(H\mid F,A)$ until marginal expected-loss reduction per dollar is equal across the two channels [2602.18986].

In power grids, once the hyperparametric independent-cascade probabilities $p_{ij}$ are learned, the model supports influence-maximization-style strengthening decisions. The paper formulates selection of $k$ critical lines as minimizing expected spread $\sigma(S)$ and notes that greedy or CELF-style routines exploit submodularity; on the IEEE-300 network, strengthening the top-$k$ lines recommended by the model can reduce the frequency of large cascades by tens of percent across all size regimes [2406.08522].

LLM systems introduce trajectory-level control. TrajSafe organizes interventions into five families—Engage, Probe, Shape, Divert, and Hard Refuse—and selects among them according to predicted risk along the conversation trajectory [2606.02423]. HARP recommends trace-first paired evaluation, decomposed deviations for each output component, monitoring of the amplification ratio $\mathrm{HA}=H_{\mathrm{global}}/H_{\mathrm{local}}$, and defense selection by minimizing a weighted objective over ASR, HA, benign utility, latency, and token cost:
$$
J_\lambda(m)=
\lambda_1\frac{\mathrm{ASR}_m}{100}
+\lambda_2\mathrm{HA}_m
+\lambda_3\bigl(1-\mathrm{BU}_m/100\bigr)
+\lambda_4\mathrm{Lat}^{\mathrm{norm}}_m
+\lambda_5\mathrm{Tok}^{\mathrm{norm}}_m.
$$
The emphasis is not only on blocking the initial perturbation, but on suppressing downstream amplification [2605.27489].

A longer-horizon version of intervention appears in risk-aware alignment via simulation. The event-trajectory search builds a breadth-first causal event graph from a prompt-response pair, ranks salient events by likelihood and impact, expands affected population strata, elicits group-specific feedback, and then refines the original response or transfers the preference signal by DPO. On the 100-example indirect harm classification task, this approach reaches roughly 75% accuracy, compared with roughly 60% for Chain-of-Thought and roughly 58% for Best-of-N, while also achieving an average win rate exceeding 70% on existing safety benchmarks [2506.20949].

## 6. Assumptions, limitations, and research directions

The strongest limitations are model-specific and often explicit. In Extreme Event Propagation, conditional vine sampling assumes no unobserved confounders; the vine copula may mis-specify tail dependence if the pair-copula families lack flexibility; high-dimensional $w$ may overfit; the impact threshold $v$ must be high enough for EVT but low enough for stable estimates; and stationarity must either hold or be modeled through time-varying margins [2106.13564]. In the trade-network cascade model, the mechanism is not microfounded on individual supply-chain or agent expectations, contains no gains or resilience beyond the single quantified transfer function, assumes static topology, and is deterministic rather than stochastic [2502.12980].

LLM-focused work identifies complementary failure modes. HarmThoughts argues that no existing benchmark had previously captured harm emergence at sentence-level granularity in reasoning traces, and its empirical comparisons show that existing detectors struggle with fine-grained behavior detection, particularly within harm emergence and execution [2604.19001]. Long-horizon simulation for alignment acknowledges coarse likelihood variables, lack of full probability distributions over event trajectories, and reduced performance when harmful intents are embedded in long adversarial narratives [2506.20949]. HARP shows that prompt-only defenses preserve benign utility but leave high success and stealth, while stronger trace-level defenses reduce global harm at utility, latency, and token-cost trade-offs [2605.27489].

Several strands of the literature also stop short of full forecasting and explicitly frame prediction as a next step. The retracted-research study computes $H_k(p)$ and documents monotonic increases across years and citation generations, but states that no parametric model was fitted; it instead suggests that harm vectors, impact-factor group, field, and citation distance could serve as features for supervised learning or graph-neural-network-based predictors [2501.00473]. HarmThoughts similarly presents recurrent, CRF, and hazard formulations as natural modeling directions for forecasting drift points such as transitions into Task Decomposition or Domain Knowledge Synthesis [2604.19001].

Taken together, these results suggest that effective harm propagation prediction depends less on a single universal propagation law than on preserving the intermediate structure of the process: conditional tails in extremes, path effects in causal graphs, order statistics in planner cost, citation generations, memory and routing events in LLM traces, or sentence-level behavioral states in reasoning. That inference is consistent with the recurring movement in the literature from end-state damage estimation toward process-level monitoring, counterfactual analysis, and intervention-aware prediction [2106.13564] [2410.19179] [2207.12380] [2605.27489].

Source: https://www.emergentmind.com/topics/harm-propagation-prediction