Estimating Causal Effects from Data Generated by Stochastic Algorithms

Abstract: Recommendation systems and chatbots present content to users, typically using stochastic algorithms that select the content based on user characteristics or context. Examples of content include chat responses, videos, or items available for purchase. Scientists and application developers are often interested in whether characteristics of content increase outcomes such as user engagement. Estimates of such causal effects may guide content providers to generate content that emphasize desirable features. However, in settings with a large content library or where content is generated uniquely for a given user, it can be difficult to use observational data to learn the causal effect of content features, because the content a user sees is tailored to that user, and because content varies in many dimensions. This paper proposes a new method for estimating the impact of content features using observational data, when the algorithm that determines user exposure incorporates some randomization, and when two additional data elements are logged for each user: $(i)$ the identity of at least one item that could have been exposed to the user, but was not (the unexposed item); $(ii)$ an estimate of the ratio of the probability that the unexposed item would have been shown to the probability that the exposed item was shown. We show that causal effects of features are identified in this setting, even in the presence of unobserved confounders that affect both user preferences and the identity of the considered pair of items (exposed and unexposed). Our estimator differs from prior approaches in terms of what data is used and how the estimator is constructed.

• The paper introduces the LCE framework to robustly estimate causal effects from observational logs in stochastic, algorithm-driven environments.
• It develops the novel Conditional Treatment-Averaged Causal Effect (C-TACE) methodology, enabling precise within-pair comparisons in high-cardinality settings.
• The approach mitigates unmeasured confounding and informs policy-relevant interventions in personalized recommendation systems and LLM outputs.

Estimating Causal Effects from Data Generated by Stochastic Algorithms: A Technical Survey

Introduction and Motivation

This paper addresses the identification and estimation of content-feature effects—such as stylistic choices in LLM outputs or item attributes in recommendation systems—using observational data generated by inherently stochastic selection algorithms (2607.05792). Modern content-serving algorithms provide unique challenges for causal inference: they individualize exposure to content, operate in extremely high cardinality or even infinite content spaces, and often exhibit feature entanglement, where the feature of interest co-varies nontrivially with other, often unobserved, content attributes. The traditional unconfoundedness assumptions or marginal treatment assignment models break down in these settings due to persistent, algorithm-induced and user-preference confounding.

The key observation of the paper is that stochasticity present in current-generation algorithms naturally creates micro-randomization—each user interaction is a potential micro-experiment, randomizing the selection among plausible alternatives. By leveraging minimal augmentation of logs—specifically (1) identity and features of at least one counterfactually plausible but unexposed item, and (2) the ratio of exposure probabilities between realized and unrealized alternatives—the authors introduce the concept of Logged Counterfactual Exposures (LCE) datasets.

The Logged Counterfactual Exposures (LCE) Framework

LCE fundamentally augments standard logged bandit feedback with two additional elements for each user-algorithm interaction: (i) logging a potential but non-shown item; and (ii) logging (or estimating through replay) the relative probability that the alternative would have been shown compared to the realized exposure. This framework is particularly suited for large action spaces (such as LLM completions) where the probability of any specific item is vanishingly small, rendering classical importance sampling highly unstable and fundamentally incomplete for causal effect estimation.

The major virtue of LCE data is its ability to identify causal effects of content features even in the presence of unmeasured confounding that influences both the pool of considered content and user outcomes. Unlike prior off-policy evaluation methods, which rely on logged marginal propensity scores and unconfoundedness conditional on observed context, LCEs restore identification via the randomization in the selection process itself. The paper’s estimator leverages the within-pair (or within-set, for larger treatment sets) contrasts and is constructed using careful weighting schemes that directly exploit the recorded stochastic assignment probabilities.

Figure 1: Dropping the pairs with equal $v_c$. Estimate against the share of tied pairs $\pi_0$: dropping ties holds the effect at $\beta_c$, keeping them under the natural weight dilutes the estimate toward zero, and under the alternative weight drives it below zero.

Conditional Treatment-Averaged Causal Effects (C-TACE)

A pivotal methodological innovation is the development and formalization of the Conditional Treatment-Averaged Causal Effect (C-TACE). Unlike the classical potential outcomes framework, which relies on well-defined, population-level treatments, C-TACE is adapted for settings where both the set of potential exposures and the feature of interest are idiosyncratic to each user interaction.

The C-TACE is defined as the average effect of exposing users to higher-feature content, where the averaging is performed both across users and across all stochastic pairs the algorithm could have generated for each user. Importantly, the C-TACE captures the causal effect as it manifests in the deployed algorithm’s output distribution, unavoidably bundled with any features that systematically co-vary with the target attribute; thus, it represents a bundled, policy-relevant effect.

Figure 2: Robustness to selection. Probability limits of the estimators as the recommender sorts more strongly on concreteness; only the naive regression drifts, by $\eta_c b$, while C-TACE remains at $\beta_c$.

Figure 3: An unobserved trait the analyst cannot control for. As the recommender sorts on an unobserved trait, unconfoundedness-based regression drifts away from $\beta_c$, while within-pair estimation (C-TACE) is robust.

Estimation, Policy Identification, and Robustness

The paper provides both identification theory and practical estimation procedures:

• Pseudo-Experiments and Weighting: Micro-experiments are constructed by treating each (observed, unobserved) pair as a local randomization; the estimator forms a pseudo-treatment indicator (±1/0), and model-based or replay-based weights are used when assignment probabilities are unequal.
• Handling Large Action Spaces and Feature Entanglement: The method eschews the need for a global propensity score estimation and avoids the parametric or semi-parametric regression risks that plague high-cardinality settings. Instead, it uses only local relative assignment probabilities.
• Trimming and Overlap: Tied pairs (i.e., those without a feature contrast) are dropped from the estimand, as they contain no experimental variation (Figure 1). Similarly, units where the assignment probability of one arm is near zero or one are trimmed to control variance and define a well-supported estimand.
• Replay Estimation and Inverse Sampling: If assignment probabilities are not logged, the required probabilities are consistently estimable by replaying the selection mechanism, with sequential inverse binomial sampling allowing unbiased estimation of inverse-propensities (Theorem in Section on Inverse Sampling).
• Robustness: Simulations and identification results show that C-TACE is robust to selection on both observed and unobserved user features. Any method conditioning solely on realized exposure features (e.g., standard difference-in-means on $V$) is susceptible to severe selection bias unless the pairing is properly accounted for.

  Figure 4: Real versus spurious heterogeneity. Without proper controls, feature effect heterogeneity can be conflated with selection-induced confounding, whereas C-TACE recovers the genuine heterogeneity.

Comparison with Existing Approaches

The LCE paradigm is contrasted to:

• Off-Policy Evaluation (OPE): Classical OPE methods with importance weighting are dismissed as infeasible in high-cardinality (LLM, streaming) settings and are shown to be fragile to feature confounding and incomplete logging.
• Unconfoundedness Regression: Covariate adjustment and regression-based analyses that condition on observed exposure features fail under unmeasured confounding or insufficient observation of the content allocation mechanism, leading to index bias as demonstrated in both theoretical examples and placebo analyses.
• Negative Sampling: Methods using catalog-level negative sampling do not identify causal effects, as they fail to mimic the critical aspect of algorithmic randomization among plausible local alternatives.

Extensions, Placebo Tests, and Multi-Feature Settings

• The framework generalizes to multiple treatment-characterizing functions (TCFs), supporting joint estimation of bundled effects when randomization allows identification of local ceteris-paribus contrasts.
• Placebo regressions using features of the non-shown or counterfactual item provide diagnostics for unaddressed selection bias: any apparent effect of the unexposed item's feature is a signal of residual confounding (Figure 5).
• The methodology is suited to adaptive and evolving algorithms, batch or streaming settings, and both ex ante logging and ex post replay settings.

  Figure 6: Joint identification with two pseudo-treatments. The identification of causal effects requires that the feature of interest varies independently; otherwise, only bundled effects are identified.

  

  Figure 5: A placebo on the non-shown pitch. Estimated effects of the non-shown feature grow with selection but vanish if the selection is fully controlled.

Practical and Theoretical Implications

From a practical viewpoint, the LCE + C-TACE approach enables firms and researchers to estimate interpretable, actionable feature effects from routinely generated logs, including those from LLMs where content spaces are effectively infinite and user contexts unique. The method is computationally tractable, requiring only minimal logging augmentation or judicious post hoc replay. Most importantly, the resulting estimand aligns with the real-world policy question: what will be the effect on outcomes if the algorithm is modified to favor some feature value, holding the algorithm’s generative process constant?

Theoretically, the work extends the conceptual reach of causal inference into nonstandard settings, bridging the gap between the causal inference and machine learning communities. It refines our understanding of what is learnable (policy effects on algorithm-induced exposure distributions) when full ignorability is neither realistic nor testable.

Future Directions

Ongoing work should address:

• Efficient joint estimation for settings with multiple co-varying content features, optimizing over bundled and ceteris-paribus effect bounds.
• Systematic diagnostics and empirical benchmarks for the necessary exchangeability and independence assumptions underlying replay-based identification.
• Theoretical characterization of variance and convergence rates for sequential and batched replay-based estimators in large-scale deployments.
• Integration with adaptive policy learning and recommendation system optimization frameworks to automate experimental design at deployment time.

Conclusion

This paper establishes a new framework for causal inference in highly personalized, algorithm-driven content selection environments, centered on logged counterfactual exposures and the conditional treatment-averaged causal effect estimand. The methodology is robust to unobserved confounding, supports generalization to multi-feature and high-cardinality spaces, and is practically deployable on modern algorithmic systems, including LLMs and personalized recommenders. Its main impact is to enable the estimation of policy-relevant, interpretable feature effects using everyday observational data, providing a valuable bridge between observational log analysis and real-world algorithmic optimization (2607.05792).