Local Causal Attribution of Chain-of-Thought Reasoning

Abstract: Understanding the causal structure of a LLM's thought process is a problem of significant importance for both transparency and safety. In this work, we take a local approach toward this goal by analyzing the causal relationships among individual components, termed units, of a given, specific chain-of-thought trace. We construct a structural causal model on these units and relate each unit to the log probability of generating (subsequent) output units. Our algorithm, termed AttriCoT, is a black-box method that performs attribution by estimating importance parameters in the structural causal model using $O(U)$ forward passes through the model, where $U$ is the number of units. Evaluation of perturbation curves across 5 datasets and 4 reasoning models shows that AttriCoT produces attributions that are more faithful to the model's behavior than alternative methods. The attribution results also reveal notable differences in thought structure between models and domains.

• The paper introduces AttriCoT, a causal attribution framework that decomposes chain-of-thought into units and quantifies pairwise causal contributions.
• It employs deletion-based interventions with O(U) forward passes to measure the impact of each reasoning unit, achieving higher faithfulness than prior methods.
• Experimental results on datasets like GSM8K demonstrate significant AUPC gains and reveal detailed influence patterns and decay across reasoning steps.

Local Causal Attribution of Chain-of-Thought Reasoning

Introduction

This paper introduces AttriCoT, a black-box causal attribution algorithm for analyzing the intra-trajectory causal structure of chain-of-thought (CoT) reasoning in LLMs and large reasoning models (LRMs) (2606.21821). Unlike prior approaches focused on faithfulness of verbalized CoT or distributional effects across samples, AttriCoT focuses on "local causality": it decomposes a single CoT instance into "units" (segments of input or output), and estimates the pairwise causal contributions between these units along the trajectory. The primary contribution is a computationally efficient structural causal modeling approach that provides fine-grained, local attributions using only forward passes through the LLM, without white-box access.

Methodological Framework

AttriCoT constructs a structural causal model (SCM) wherein each output unit (step, sentence, or segment) in a CoT is treated as a node whose generation is (potentially) causally dependent on prior units. The quantification of causal effect is operationalized by performing interventions: units are sequentially removed (leave-one-out or higher-order deletions), and the effect on the target unit is measured as the change in its conditional log probability under the autoregressive LLM.

The measured log-probs across interventions and units are then linearly regressed on indicator variables corresponding to the presence/absence of each potential cause, producing a matrix of "importance scores." This maps each pair of units (i, j) to an estimated causal effect of unit i on unit j, resulting in a pairwise attribution matrix over the CoT.

The computational cost is $O(U)$ forward passes for $U$ units (given basic leave-one-out interventions)—a key result enabling application to long traces. Unlike previous methods such as Thought Anchors—which use KL divergence over token distributions—AttriCoT's attributions directly correspond to the locally realized log-probabilities of the generated output.

Comparison with Related Work

The proposed method advances prior approaches across several axes:

• Causal vs. Correlational Attribution: Earlier perturbation/ablation methods (e.g., hint injection or SHAP for CoT, e.g., [Turpin et al., 2023]) identify correlations, not causal dependencies. AttriCoT inherits the causal intervention ethos but operates in a fully black-box, input/output only regime.
• Granularity and Locality: White-box approaches (e.g., circuit-based interventions in [Zhao et al., 2026], [Ye et al., 2026]) require access to internal states and often operate at coarser, answer-level scales or across sample distributions. AttriCoT provides fine-grained, local attribution per specific output unit within a trace, for the actual generated units seen by the user.
• Computational Tractability: By operating with $O(U)$ forward passes, AttriCoT avoids the quadratic or resampling cost of previous CoT resampling or mediation analysis methods (e.g., [Macar et al., 2026], [Paul et al., 2024]) that require regenerating full traces under interventions.

Experimental Results

AttriCoT's effectiveness was benchmarked across four state-of-the-art reasoning models and five datasets spanning mathematical (GSM8K, MATH500), logic (ZebraLogic), and multi-domain knowledge reasoning (MMLU-Pro, GPQA-Diamond). The core performance metric is AUPC (area under perturbation curve): for each model and sample, units are removed in order of attributed importance, and the cumulative log-prob drop for the target output unit is aggregated.

Key empirical findings:

• Faithfulness: AttriCoT produces attributions that are consistently more behaviorally faithful (higher AUPC) than both the sentence masking baseline (Thought Anchors) and prompting-based methods. For example, on GSM8K, AttriCoT yields up to 15-30% AUPC gain over Thought Anchors and 70-165% over prompting.
• Computation-Efficiency Tradeoff: Doubling the intervention coverage (e.g., leave-two-out) produces marginal improvements, indicating most causality is recovered via single-unit ablations, validating linear cost design.
• Attention-Based Baselines: Exploratory white-box variants using direct attention modulations were less faithful than main black-box causal methods, but offer a possible direction for further efficiency gains where internals are available.

Analytical Insights and Implications

The pairwise attribution matrices produced by AttriCoT support secondary quantification of CoT structure:

• Influence Patterns: The average influence of CoT units on future steps, and input-to-output ratios, reveal that the first output units are more input-driven, after which the model relies on prior reasoning steps, occasionally returning to input units as answers are finalized.
• Decay of Influence: The causal effect of a reasoning unit decays rapidly with increased structural distance along the CoT, flattening after a small number of steps.
• Model and Task Variability: Differences in the density and entropy of attributions reveal that more complex datasets (e.g., ZebraLogic) induce more diffuse attributive dependencies; model family and training strategies (e.g., DeepSeek distillation modalities) also yield systematic differences in CoT structure.

Practically, this granular, local causal decomposition enables practitioners to identify which specific reasoning steps actually contribute to each answer segment, surfacing unfaithful or spurious justifications—a central open problem in CoT alignment and interpretability research. Theoretically, it provides an empirical tool to study LLMs' reasoning mechanisms, challenge hypotheses of coherence, and diagnose phenomena such as backtracking or input recursivity.

Limitations and Future Directions

The present approach is constrained by the expressivity of linear SCMs and the stylized nature of deletion-based interventions ("removals" may induce non-local distributional shifts or grammar violations). The authors suggest possible white-box acceleration for models with internal access (e.g., attention weight patching), and the potential for more expressive, non-linear or hierarchical structural models.

Further, extending AttriCoT to link attribution profiles with trace correctness, reasoning step types, and model scale will elucidate how faithfulness and causal structure change with scale and prompt/task type.

Conclusion

AttriCoT establishes a robust, computationally efficient framework for local causal analysis of LLM chain-of-thought reasoning. Empirical evidence demonstrates superior behavioral faithfulness and enables fine-grained mechanistic analysis of LLMs' reasoning traces. This provides a principled and practical tool for CoT auditing, model debugging, and the empirical study of attribution dynamics in neural reasoning systems (2606.21821).