---
title: Controllable vs Uncontrollable Failure Attribution
url: https://www.emergentmind.com/topics/controllable-versus-uncontrollable-failure-attribution
type: topic
---

# Controllable vs Uncontrollable Failure Attribution

Controllable versus uncontrollable failure attribution refers to the systematic distinction between failures that could be prevented or mitigated via feasible interventions, and those that derive from factors fundamentally outside the scope of intervention. This distinction is central to debugging complex systems, interpreting predictive models in high-stakes domains such as medicine and public health, and operationalizing causal reasoning over multi-agent interactions. Methodologically, attributions of failure or risk can be partitioned along the axis of controllability, guiding both research on causal inference and the design of actionable, transparent decision-support tools.

## 1. Formal Definitions and Conceptual Foundations

In automated decision processes, machine learning, and multi-agent systems, the notion of controllability underpins the ability to identify where and how a system’s undesirable outcome could have been avoided via feasible changes. Two primary categorizations emerge:

- **Controllable failures**: Adverse outcomes that would have been averted by a feasible, localized intervention (e.g., correcting an action, changing a policy, or modifying a controllable input).
- **Uncontrollable failures**: Outcomes impervious to any such localized or practical intervention—arising from immutable features (age, genetic background), exogenous noise, or factors outside the modeled environment.

In feature attribution contexts, controllability demarcates the set of input features that an actor or policy-maker can in principle adjust (controllable features, $F_c$) from those which cannot be changed in practice (uncontrollable features, $F_u$) [2203.12701]. In causal sequential decision settings, controllability is assessed by whether single-step (or, more generally, minimal) interventions suffice to avert failure as verified through counterfactual reasoning [2509.10401].

## 2. Causal Frameworks and Mathematical Formalism

### Multi-Agent Trajectory Models

In multi-agent systems, a trajectory is defined as
$$
\tau = [(s_0, a_0), (s_1, a_1), \ldots, (s_T, a_T)],
$$
where $s_t$ is the global state at time $t$ and $a_t$ is the action taken. State transitions follow the structural equation
$$
s_{t+1} = f(s_t, a_t, \epsilon_t),
$$
with $\epsilon_t$ drawn from a distribution of unobserved exogenous factors (knowledge gaps, misperceptions, etc.).

The outcome function $Z(\tau) \in \{0,1\}$ indicates success or failure, $F \equiv [Z(\tau)=1]$. Interventions are formalized using Pearl’s do-operator: the trajectory after an intervention at time $t$ is denoted $\tau^* = do(a_t \leftarrow a_t^*)$, and system evolution proceeds as
$$
s_{t+1}^* = f(s_t, a_t^*, \epsilon_t).
$$
[2509.10401]

### Feature Attribution Models

Given an input vector $x \in \mathbb{R}^m$ with index set $\{1, ..., m\} = F_c \uplus F_u$, controllable and uncontrollable features are defined respectively as those variables which can and cannot be altered by intervention. Traditional model-agnostic attributions (e.g., SHAP, LIME) do not distinguish between these sets, treating all inputs uniformly [2203.12701].

## 3. Methodological Approaches for Attribution

### Abduct–Act–Predict (A2P) Scaffolding

A2P is a causal inference framework for failure attribution in multi-agent conversational systems [2509.10401], operationalized as follows:

1. **Abduction**: For a candidate failure step $t$, infer the most plausible hidden cause $\epsilon_t$ via posterior estimation:
   $$
   \hat{\epsilon}_t = \arg\max_{\epsilon} P(\epsilon | s_{0:t}, a_t, Z(\tau)=1)
   $$
2. **Action**: Identify a minimal corrective intervention $a_t^*$ targeting $\hat{\epsilon}_t$:
   $$
   a_t^* = \arg\min_{a'} \Delta(a', a_t) \text{ subject to } P(F|do(a_t \leftarrow a')) < \theta
   $$
   where $\Delta$ is an action-distance metric.
3. **Prediction**: Simulate the outcome trajectory post-intervention, estimate $P_{\text{success}} = P(Z(\tau^*)=0\,|\,do(a_t \leftarrow a_t^*))$. If $P_{\text{success}} > 1-\theta$, declare the failure at $t$ controllable.

If no intervention at any $t$ achieves $P_{\text{success}} > 1-\theta$, the failure is deemed uncontrollable.

### Controllable fActor Feature Attribution (CAFA)

CAFA partitions input features into $F_c$ and $F_u$ and produces an attribution vector $\Phi \in \mathbb{R}^m$ with the key constraint $\Phi_j = 0$ for all $j \in F_u$, reflecting only the influence of the controllable features [2203.12701]. CAFA proceeds through:

- **Selective perturbation**: Only features in $F_c$ are perturbed around the input $x$ within a threshold $\pi_x$, with $x_j' = x_j$ fixed for all $j \in F_u$.
- **Surrogate modeling**: Fit a high-capacity surrogate $g$ (e.g., random forest) on the perturbed dataset $D_x$.
- **Global-for-local SHAP explanation**: Compute attributions using the SHAP method on $g$, guaranteeing uncontrollable features are assigned zero attribution due to invariance in $D_x$.

## 4. Empirical Evaluation and Case Studies

### Multi-Agent Failure Attribution

On the WhoWhen benchmark for multi-agent dialogue failures, A2P achieves a step-level failure attribution accuracy of $47.46\%$ on the Algorithm-Generated subset, a $2.85\times$ improvement over the $16.67\%$ baseline. On the more complex Hand-Crafted subset, A2P attains $29.31\%$ (a $2.43\times$ gain over the $12.07\%$ baseline) [2509.10401]. This demonstrates robust operationalization of controllable versus uncontrollable failures in sequential decision settings.

### Feature Attribution in Medical and Public Health Applications

CAFA was evaluated on Simulacrum lung cancer and UCI breast-cancer datasets [2203.12701]. In both cases, CAFA suppresses attributions to uncontrollable features such as age or sex (Pearson correlation between CAFA and SHAP on controllable features $0.96$–$0.99$), and highlights controllable features (e.g., chemotherapy regimen, BMI). Applied to UK COVID-19 intervention data, CAFA attributed zero importance to uncontrollable indicators (cases, fatalities), and prioritized actionable intervention policies (e.g., public gathering bans, venue restrictions), consistent with established transmission risk factors.

## 5. Implications for Decision-Making and System Design

Partitioning failure attributions along controllability axes underpins actionable recommendations and root-cause analysis:

- **In medicine**, the separation identifies which aspects of a risk profile or treatment outcome are potentially modifiable, shaping clinical intervention strategies.
- **In public health**, the framework isolates policy levers (e.g., targeted restrictions) with maximal marginal effect, excluding immutable demographic or environmental variables from action planning.
- **In multi-agent and conversational AI**, rigorous counterfactual analysis determines whether error remediation is feasible at the step/action level or if system/environmental limitations preclude prevention.

A clear delineation channels operational focus toward interventions that are expected to yield improvements, and avoids misplaced emphasis on intractable sources of failure.

## 6. Limitations and Open Questions

Several constraints and unresolved issues pertain to controllable vs. uncontrollable failure attribution:

- Partitioning $F_c/F_u$ generally relies on a priori domain knowledge and may be nontrivial, especially as the set of “controllable” variables can evolve with technological advances (e.g., gene therapy potentially rendering genetic factors controllable) [2203.12701].
- The CAFA algorithm’s dependence on local perturbations and surrogate modeling may not extend straightforwardly to regression tasks or imbalance-heavy datasets. SHAP’s computational and statistical limitations (feature dependence, cost) are inherited by CAFA.
- In A2P, the fidelity of predictive counterfactual simulations is bounded by the capabilities of the large language model, and the threshold $\theta$ for declaring controllability is heuristic [2509.10401].
- Single-step or localized interventions are assumed; genuinely uncontrollable failures or those requiring coordinated multi-step interventions may be underrepresented.
- Neither method quantifies the dominance of uncontrollable factors in the original model prediction—a possible direction for further risk decomposition [2203.12701].
- External, unmodeled exogenous factors may render some failures undiagnosable via language or feature-based approaches, motivating integration with external monitoring or simulation [2509.10401].

## 7. Comparative Summary Table

| Aspect                       | CAFA Approach [2203.12701]                        | A2P Scaffolding [2509.10401]                     |
|------------------------------|---------------------------------------------------|--------------------------------------------------|
| Domain                       | Predictive modeling, XAI, feature attribution     | Multi-agent sequential decision, dialogue agents |
| Core notion of controllability| Partition of input features (controllable/uncontrollable) | Success probability under local action intervention |
| Method                       | Selective feature perturbation, SHAP-based attribution | Abduct–Act–Predict counterfactual routine        |
| Output                       | Attribution vector zeroed for uncontrollable features | Step/action labeled as controllable/uncontrollable failure |
| Illustrative application     | Medicine, public health (cancer, COVID-19 policy) | Root-cause error localization in dialogue systems|

Both CAFA and A2P provide rigorous methodological frameworks for distinguishing controllable from uncontrollable attributions, enhancing transparency and verifiability in automated decision-support and failure analysis across scientific and engineering domains.

Source: https://www.emergentmind.com/topics/controllable-versus-uncontrollable-failure-attribution