Causal Action Influence Score (CAIS)

• CAIS is a distribution-based measure that quantifies causal influence by computing the conditional mutual information between actions and outcome factors in MDPs.
• It integrates structural causal modeling, information theory, and deep learning to robustly assess agency, drive intrinsic rewards, and facilitate counterfactual data augmentation.
• Empirical studies in robotics, social systems, and reinforcement learning highlight CAIS's effectiveness in improving model fidelity and intervention outcomes.

The Causal Action Influence Score (CAIS) is a principled, distribution-based measure for quantifying the causal impact of actions on outcomes in sequential decision-making systems and complex environments. It has emerged as a unifying framework for robust agency detection, intrinsic motivation, counterfactual data augmentation, and influence quantification, leveraging advances in structural causal modeling, information theory, and deep learning.

1. Mathematical Definition and Causal Foundations

Formally, the Causal Action Influence Score quantifies, for each factor $S'_j$ of the next state in a Markov Decision Process (MDP), the conditional mutual information between that factor and the action, given the current state $S=s$: $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$ Alternatively,

$$\text{CAIS}_j(s) = H(S'_j | S=s) - \mathbb{E}_{a \sim \pi} [ H(S'_j | S=s, A=a) ].$$

This formulation directly quantifies the information gained about $S'_j$ by knowing the action $A$, beyond what is already known from $S$. Equivalent definitions via Kullback-Leibler divergence appear in both the structural causal modeling tradition (Janzing et al., 2012, Urpí et al., 2024, Yuan et al., 2 Feb 2025) and in alternative divergences such as Wasserstein distance for metric spaces (Xu et al., 20 Jul 2025).

Conceptually, a zero CAIS indicates absence of any direct causal arrow $A \rightarrow S'_j$ in the local structural causal model (SCM), consistent with the rigorous postulates for causal strength: nullity under independence, nonnegativity, locality, and proper quantitative Markov bounds (Janzing et al., 2012). CAIS majorizes any observed conditional dependence, and generalizes to arbitrary discrete or continuous domains.

2. Estimation Methodologies

Offline Trajectory Estimation

In offline RL and data augmentation, empirical estimation of CAIS proceeds by learning probabilistic one-step models for each state factor:

• Fit, for each $j$, a conditional model $\hat{P}_j(s'_j|s,a)$, typically Gaussian, on static trajectories $S=s$0.
• Approximate the marginal $S=s$1 via Monte Carlo sampling $S=s$2 and empirical mixture.
• Evaluate:

$S=s$3

No rollout or unrolling beyond one timestep is required (Urpí et al., 2024).

Alternative Divergence Formulations

For visually rich or metric observation spaces, CAIS can be framed via the 1-Wasserstein distance, quantifying the geometric shift between the outcome distribution conditional on an action and the marginal outcome distribution: $S=s$4 where $S=s$5 indexes latent sensory representations (Xu et al., 20 Jul 2025).

Causal Influence in Sequential and Social Systems

For settings where actions correspond to interventions in time series or social processes, CAIS can be operationalized as the (counterfactual) average treatment effect: $S=s$6 where $S=s$7 and $S=s$8 denote factual and counterfactually perturbed treatments (e.g., signal exposure, posting activity) (Tian et al., 25 May 2025).

CAIS estimation in these settings involves fitting joint sequential models to the treatment and outcome processes, generating counterfactual predictions under explicit treatment modifications, and reporting the causal difference in expected outcomes.

3. Applications and Empirical Significance

Counterfactual Data Augmentation

By thresholding CAIS values (e.g., declare factor $S=s$9 “uncontrollable” at state $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$0 if $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$1), one can algorithmically identify state dimensions unaffected by action. This enables principled counterfactual swaps of transition data along such invariant directions, generating high-likelihood synthetic trajectories that expand the support of offline datasets and directly improve out-of-distribution RL generalization (Urpí et al., 2024).

Intrinsic Reward for Exploration and Skill Acquisition

CAIS serves as a reward signal for intrinsic motivation, driving agents to select actions that maximize causal influence over entities or latent visual state. Hierarchical RL frameworks have leveraged CAIS-based intrinsic rewards—estimated with composite physics-informed and learned models—for sample-efficient acquisition of non-prehensile manipulation skills, whole-body object pushing, and robust transfer to real hardware (Yuan et al., 2 Feb 2025, Xu et al., 20 Jul 2025). CAIS is robust against sensor and environmental noise, outperforming correlation-based alternatives.

Causal Influence in Social and Temporal Systems

In digital social systems, CAIS-equipped joint treatment-outcome models quantify the true influence of exposure or user actions on engagement, outperforming heuristic baselines and standard deep learning methods. In empirical case studies, CAIS aligns more strongly (Spearman $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$2, Kendall $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$3) with expert-curated measures of influence, compared to aggregate or followership-based metrics (Tian et al., 25 May 2025).

4. Theoretical Properties and Comparison to Related Measures

CAIS fulfills a set of foundational postulates for quantifying causal strength (Janzing et al., 2012):

• Nonnegativity (P0): $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$4, with equality if and only if $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$5.
• Exactness in Simple Cases (P1): On two-node structures, CAIS reduces to ordinary mutual information.
• Locality (P2): CAIS depends only on relevant conditionals and local joint distributions, ensuring it is not confounded by upstream network structure.
• Quantitative Markov Bound (P3): Always upper bounds the observed conditional dependence.
• Heredity (P4): Zero CAIS on a set of edges propagates to all subsets.

Unlike average causal effect or variance-based methods, CAIS fully captures nonlinear and non-mean-shift dependencies. It is not susceptible to under- or over-estimation due to confounding, which plague mutual information, conditional mutual information, transfer entropy, and related proposals. CAIS diverges for deterministic, surjective interventions and does not misreport influence as zero in the presence of strong structure (Janzing et al., 2012, Urpí et al., 2024).

5. Implementation Details and Best Practices

• Model Specification: Probabilistic models for $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$6 are typically Gaussian, with learned mean and covariance. For high-dimensional latent observation settings, quantile regression networks for marginal and conditional distributions (with Quantile Huber loss) are preferred (Xu et al., 20 Jul 2025).
• Action Sampling: Actions are drawn either uniformly or from a clipped neighborhood of the policy distribution, reflecting the physical realizability constraints (Yuan et al., 2 Feb 2025).
• KL and Wasserstein Computation: KL divergences between mixtures can be stably approximated via canonical methods (e.g., Durrieu et al., 2012), and Wasserstein distances via quantile function differences (Urpí et al., 2024, Xu et al., 20 Jul 2025).
• Scalability: CAIS estimation requires no multi-step model unrolling, only a single forward pass per action and state factor, making it tractable for high-dimensional problems (Urpí et al., 2024, Yuan et al., 2 Feb 2025).

6. Empirical Evidence and Case Studies

Empirical validation of CAIS spans robotics, agency detection, and large-scale social systems:

• In manipulation and kitchen robotics, CAIS-based factor splitting precisely matches human-interpretable object manipulation events, with ROC AUC $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$7 for causal detection and dramatic gains in out-of-distribution task success (e.g., $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$8 vs $$\text{CAIS}_j(s) := I(S'_j; A \mid S=s) = \mathbb{E}_{a \sim \pi(\cdot | s)} \left[ D_{\mathrm{KL}}\left( P(S'_j | S=s, A=a)\,\|\,P(S'_j | S=s) \right) \right].$$9) (Urpí et al., 2024).
• In simulated infant-mobile paradigms, CAIS yields robust detection of contingencies and drives extinction-burst phenomena, reproducible only when leveraging a high-fidelity causal model (Xu et al., 20 Jul 2025).
• Hierarchical control for legged manipulation demonstrates superior sample efficiency and reliable sim-to-real transfer when CAIS is used as intrinsic motivation, attaining up to $$\text{CAIS}_j(s) = H(S'_j | S=s) - \mathbb{E}_{a \sim \pi} [ H(S'_j | S=s, A=a) ].$$0 success in challenging environments absent reward shaping (Yuan et al., 2 Feb 2025).
• In social engagement, CAIS-driven models realize $$\text{CAIS}_j(s) = H(S'_j | S=s) - \mathbb{E}_{a \sim \pi} [ H(S'_j | S=s, A=a) ].$$1--$$\text{CAIS}_j(s) = H(S'_j | S=s) - \mathbb{E}_{a \sim \pi} [ H(S'_j | S=s, A=a) ].$$2 lower RMSE in engagement forecasting across diverse counterfactual scenarios and deliver influence estimates strongly concordant with gold-standard empirical rankings (Tian et al., 25 May 2025).

7. Generalizations, Limitations, and Research Directions

CAIS admits generalization to:

• Multivariate sets of arrows or factors,
• Arbitrary metric spaces via general OT divergences,
• Multi-step and counterfactual interventional analysis.

Key limitations include reliance on a well-specified conditional model, data efficiency under high-dimensionality, and scalability for long temporal dependencies. Extending CAIS frameworks with continuous action spaces, end-to-end representation learning, and alternative divergences (e.g., Sinkhorn regularization, kernel MMD) is under active exploration (Xu et al., 20 Jul 2025).

CAIS establishes a rigorous, information-theoretic paradigm for action influence, with wide applicability across autonomous systems, robust RL, causal reinforcement learning, and causal analysis of sequential interventions.