---
title: Execution Likelihood in Machine Learning
url: https://www.emergentmind.com/topics/execution-likelihood
type: topic
---

# Execution Likelihood in Machine Learning

Execution Likelihood is a formal construct describing the conditional probability that a given output or instruction by a machine learning system—typically a large language model (LLM)—will be operationalized or carried through to completion by an external actor, such as a user. Distinct from severity-focused risk metrics, execution likelihood explicitly quantifies the real-world plausibility that a harmful (or beneficial) outcome materializes, given the model's response. This conditional perspective is crucial for robust safety evaluation, for understanding long-horizon task performance, and for modeling how single-step errors accumulate into emergent system-level behaviors.

## 1. Formal Definitions and Metrics

Execution Likelihood is defined as the probability that a given model output is executed by a user:
$$
P_\mathrm{exec} \equiv \Pr(\mathrm{Execution}\mid\mathrm{Model\ Response})
$$
This probability is typically modeled as a (monotonically decreasing) function of a discrete variable, Execution Cost $C$, where higher $C$ reflects greater barriers to operationalization. No closed-form for $f(C)$ is universally assumed, but the qualitative assumption is that $C \uparrow \Rightarrow P_\mathrm{exec}(C) \downarrow$, acknowledging that higher expertise, rare equipment, prohibitive legality, or complex infrastructure lower the probability of execution [2602.01600].

Execution Likelihood is foundational to the "Expected Harm" ($\mathrm{EH}$) metric, where realized risk is decomposed as:
$$
\mathrm{EH} = \mathrm{Severity} \times P_\mathrm{exec}
$$
Here, "Severity" is a discrete harm score (1–5) reflecting downstream consequences if execution occurs [2602.01600]. This product formalism captures the intuition that even highly severe outputs pose limited real-world risk if practically impossible to execute, whereas low-severity but highly executable outputs may still warrant scrutiny and mitigation.

In multi-step task execution, as studied in long-horizon LLM evaluation, execution likelihood is often expressed in terms of stepwise accuracy $p$ and the probability of completing a chain of $H$ steps without error:
$$
P(\mathrm{success}\mid H) = p^H
$$
The horizon length $H_s(p)$, the largest number of steps for which the task succeeds with probability at least $s$, is therefore governed by the exponential relationship between per-step accuracy and overall execution likelihood [2509.09677].

## 2. Estimation of Execution Cost and Empirical Proxying

Execution Cost ($C$) operationalizes the friction or barriers to a user successfully carrying out a model's response. In safety-oriented evaluations, $C$ is commonly scored on a five-point scale:
- 1 = Very easy (minimal barriers)
- 5 = Very difficult (extreme barriers)

Key factors include required domain expertise, specialized equipment, legality of the action, and requisite infrastructure. Automated labeling (e.g., via a specialized prompt to gpt-oss-120b) is commonly utilized, with labels validated on human-annotated subsets for accuracy. In practice, achieved cost-prediction accuracies reach approximately 68% exact, with 96% off-by-one, and mean absolute error (MAE) near 0.36 [2602.01600].

Within the Expected Harm framework, the discrete cost label $C$ is used directly as a proxy for $P_\mathrm{exec}$; no further regression or calibration is imposed, reflecting both practical constraints and calibration to human judgments.

## 3. Execution Likelihood in LLM Safety and Risk Calibration

The integration of execution likelihood into LLM safety analysis highlights critical misalignments in refusal strategies. Empirical evaluation reveals that real-world toxic prompts cluster at low cost (mean ≈ 1.2), while synthetic benchmarks are 1.47× higher in cost on average (Figure 2 in [2602.01600]). Models demonstrate "Inverse Risk Calibration," in which refusal behaviors are disproportionately strong for high-cost, low-execution-likelihood threats, and unduly weak for low-cost, high-likelihood queries—the very region where real user behavior concentrates.

Attack success rate (ASR) thus exhibits a heatmap pattern: high vulnerability (high ASR) at $C=1$, robust refusal at $C=5$ [2602.01600]. This mismatch exposes a structural vulnerability. Empirically, strategic exploitation of this calibration error can double the success rate of jailbreak attacks.

## 4. Execution Capability and Sequential Task Performance

Execution likelihood also governs how reliably LLMs can perform long-horizon, multi-step tasks, even when supplied with explicit knowledge and plans. Here, stepwise correctness (step accuracy $p$) is critical, as even small deviations compounds rapidly. The probability of a flawless $H$-turn execution (assuming no self-correction) is $p^H$. The number of steps $H_{0.5}$ a model can execute reliably with 50% success rate is:
$$
H_{0.5}(p) \approx -\frac{\ln 2}{\ln p} \approx \frac{\ln2}{1-p} \text{ for } p \to 1
$$
Even marginal improvements in single-step accuracy yield super-exponential gains in horizon length; for near-perfect $p$, the sensitivity $\frac{dH_{0.5}}{dp}$ grows as $\sim 1/(1-p)^2$. This magnifies the economic and practical value of even small scaling gains [2509.09677].

Empirically, per-turn accuracy in multi-step settings degrades with depth: models that start at 100% at turn 1 often fall to ≈ 80% by turn 20 and below 50% by turn 50. This trend is found across medium- and large-scale systems (Qwen3, Gemma3 8B–32B), with the largest variants failing after more turns but still subject to degradation [2509.09677].

## 5. Compounding Errors, Self-Conditioning, and Robustness Strategies

A critical dynamic affecting execution likelihood involves the interaction of past errors with future performance, a phenomenon termed "self-conditioning." When LLMs are conditioned on their own erroneous outputs (even with constant context length), per-step accuracy at later turns drops further than baseline long-context effects would predict [2509.09677]. Scale mitigates only long-context drift, not error compounding: very large models (200B+) retain high accuracy in error-free contexts but remain vulnerable to self-conditioned declines.

Distinctly, "thinking models"—LLMs trained via RL to generate and then discard chain-of-thought traces—maintain stable performance regardless of the error rate in conditioning history. Sequential test-time reasoning, by regenerating CoTs afresh each turn, interrupts the feedback loop and prevents error propagation [2509.09677]. This suggests that sequence-level execution robustness requires not merely scale but architectural or procedural interventions to break cycles of error amplification.

## 6. Implications for Model Evaluation and Safety Mitigation

Systematic "cost blindness," i.e., models' inability to represent or utilize execution cost in refusal decisions, drives suboptimal safety outcomes. Linear probing reveals that latent states in current models robustly encode severity but lack any monotonic or interpretable representation of execution cost—activation in hidden layers correlates strongly with severity but only bimodally with cost (high at extremes, baseline at midrange), confirming the absence of an internal "execution likelihood" dimension [2602.01600].

Key recommendations include:

- Integrating execution likelihood into safety taxonomies and evaluation metrics—adopting Expected Harm (EH) over Attack Success Rate (ASR) alone.
- Augmenting safety training with low-cost, high-likelihood harmful examples to recalibrate model refusal behavior.
- Developing compositional or sequential guardrails capable of recognizing and neutralizing benign subtasks that may compose into harmful wholes.
- Auditing real-world prompt distributions to align defense calibrations with the true landscape of executable threats [2602.01600].

A plausible implication is that further advances in execution likelihood modeling and error-controlled long-horizon reasoning are prerequisites for both safer and more performant LLM deployments.

## 7. Summary Table: Metrics and Failure Modes

| Term/Metric           | Definition / Measurement                                  | Source           |
|-----------------------|----------------------------------------------------------|------------------|
| $P_\mathrm{exec}$     | $\Pr(\mathrm{Execution}\mid\mathrm{Model\ Response})$    | [2602.01600]     |
| Execution Cost $C$    | Discrete (1–5), difficulty of operationalization         | [2602.01600]     |
| Expected Harm (EH)    | Severity $\times P_\mathrm{exec}$                        | [2602.01600]     |
| Step Accuracy $p$     | Probability of correct step update                       | [2509.09677]     |
| $H_{0.5}$             | Steps to 50% task success, $H_{0.5} \approx \ln2/(1-p)$ | [2509.09677]     |
| Inverse Risk Calibration | Strongest refusal at low $P_\mathrm{exec}$, weak at high | [2602.01600]  |
| Self-conditioning     | Error propagation via autoregressive context             | [2509.09677]     |

The recognition, measurement, and mitigation of execution likelihood is central to both advancing LLM safety and extending practical, reliable automation to long-horizon, real-world workflows. Recent benchmarks and theoretical analysis underscore that even incremental improvements in per-step reliability can yield exponential benefits in sequential task capacity, but also that robust execution safety demands explicit attention to the real-world feasibility of model outputs.

Source: https://www.emergentmind.com/topics/execution-likelihood