Propensity Inference Methods

• Propensity Inference is a framework for estimating latent probabilities that drive actions by modeling conditional probabilities given observed covariates.
• It underpins causal effect estimation using methods like weighting, matching, and calibration to minimize bias in observational and AI behavior studies.
• Recent innovations integrate Bayesian and frequentist approaches with advanced calibration and high-dimensional covariate selection to enhance robustness and accuracy.

Propensity Inference is a methodological paradigm for quantifying and analyzing latent probabilities (“propensities”) that govern actions or behaviors arising under complex observational or environmental regimes, especially when those propensities are not directly observed but must be inferred from data. Propensity inference spans classical causal-inference domains—where the propensity score is pivotal for removing confounding bias in non-randomized studies—and recent extensions targeting the measurement of behavioral propensities in artificial intelligence systems, such as LLMs. Across domains, propensity inference centers on designing, estimating, and leveraging models for conditional probabilities given observed covariates, then using these as the foundation for causal effect estimation, risk analysis, and policy evaluation.

1. Core Definitions and Frameworks

In conventional statistical terminology, the archetype of propensity inference is the propensity score, defined as the conditional probability of assignment to a particular group or treatment given observed covariates. In particular, with treatment indicator $Z$ and covariates $X$, the propensity score is $e(X) = \Pr(Z=1 \mid X)$ or in the multi-level case, $e_j(X) = \Pr(Z=j \mid X)$. The balancing property—$X \perp Z \mid e(X)$—underpins most adjustment strategies in causal inference (Wijayatunga, 2018, Guo et al., 2015).

In behavioral AI contexts, propensity is generalized to denote the conditional probability that an agent takes a designated action (e.g., an unsanctioned action) given a vector of environmental factors. E.g., for $Y \in \{0,1\}$ indicating such an action and environmental vector $F=(F_1,\dots,F_{12})$, propensity is $\Pr(Y=1 \mid F)$ (Järviniemi et al., 22 Apr 2026). Propensity thus refers broadly to any structurally meaningful latent probability that encodes a system’s inclination toward specific observable outcomes, conditional on available covariates, and is treated as an estimand of inferential interest.

2. Propensity Score Construction and Role in Causal Inference

Propensity scores are central to identification and estimation of causal effects in observational studies. Under the assumptions of strong ignorability—i.e., unconfoundedness and positivity—the propensity score allows for unbiased estimation of treatment effects via stratification, weighting, or matching (Guo et al., 2015). In multi-treatment or continuous-treatment scenarios, the concept generalizes to a vector of generalized propensity scores (GPS) $e_j(X)$, $j=1,\ldots,J$ (Li, 2018). Propensity inference in these settings enables the formulation of target estimands as population contrasts over “tilted” distributions, with weighting schemes (e.g., ATE, ATT, overlap, matching, trimming) determined by the choice of tilting function $X$0—see the class of balancing weights and their special cases (Li, 2018).

For environmental-behavioral analysis, as in LLM safety research, propensity inference involves explicit modeling of outcome rates (e.g., rate of unsanctioned behavior) as a function of controlled environmental manipulations ($X$1), operationalized in a Bayesian GLM framework and interpreted as reflecting latent conditional probabilities of interest (Järviniemi et al., 22 Apr 2026).

3. Methodological Innovations and Estimation Procedures

3.1 Propensity-Score Weighting and Calibration

Modern propensity inference incorporates advanced weighting schemes to address bias and variance. The “balancing weights” framework defines weights $X$2, unifying inverse probability, ATT, overlap, and other weighting methods. The “generalized overlap weights” (GOW), with $X$3, are designed to optimize asymptotic variance and mitigate the impact of extreme propensities, yielding bounded weights and stable finite-sample properties. The corresponding sandwich estimator of variance accounts for uncertainty in GPS estimation (Li, 2018).

Calibration and post-processing are essential when propensity score models are estimated using flexible learners. Poor probabilistic calibration of predicted e(X) values—common with tree-based or neural models—leads to bias in causal effect estimation. Simple post-calibration (e.g., Platt scaling) achieves calibration and reduces estimation error, often without corresponding improvement on covariate-balance diagnostics. Routine assessment and calibration of propensity scores is now recommended for all propensity-based inference (Gutman et al., 2022).

3.2 Bayesian and Frequentist Inference

Bayesian approaches to propensity inference, such as general Bayesian updating or Gibbs posteriors with covariate-balancing losses, subsume classical likelihood-based and robust penalized methods. These procedures yield posterior distributions for causal effect parameters that account for both sampling variability and parameter uncertainty, with frequentist-valid coverage achieved via calibrated learning rates (Orihara et al., 2024). Approximate Bayesian inference can also be used in missing data settings via reparameterization of estimating equations, yielding posteriors that asymptotically match frequentist intervals (Sang et al., 2017).

Posterior predictive p-values using the propensity score yield a unifying frequentist-Bayesian solution for hypothesis testing (strong or weak nulls), especially when leveraging doubly-robust estimators. Re-randomization under the posterior predictive distribution over assignment mechanisms achieves valid inference, even in regimes with extreme propensities (Ding et al., 2022).

3.3 Covariate Selection, Heterogeneity, and High-Dimensionality

Propensity-score adapted covariate selection ensures inclusion of confounders and outcome predictors while excluding instruments and spurious covariates, increasing efficiency without bias. Data-adaptive penalized regression methods (e.g., IPW adaptive Lasso) achieve this under the “oracle property” with robust consistency guarantees (Zhou et al., 2021).

For effect heterogeneity, nonparametric “propensity score regression” (PSR) decomposes the estimation task via two-stage local linear regression (over effect modifiers $X$4 and e(X)), allowing efficient estimation of conditional treatment effects in high dimensions and with extreme PS values (Wu et al., 2021).

4. Extensions: Complex Designs, Clustering, Selection, and Interference

Propensity inference extends to non-standard designs including clustered sampling, web survey response, big-data selection, and interference/network effects.

• Clustered Data and Unmeasured Cluster-Level Confounding: Calibration techniques enforce balance both at the global and cluster-specific level, yielding unbiased estimators robust to misspecification of the PS model and unobserved cluster effects (Yang, 2016).
• Big Data and Non-Probability Samples: Integrated pseudo-likelihoods using auxiliary probability samples yield optimal propensity estimates for inclusion, facilitating correct population inference under complex integration settings (Ang et al., 7 Jan 2025).
• Web and Multi-Phase Sample Response: Specialized implicit logistic regression methods fit the “true” selection likelihood, outperforming pseudo-likelihood and weighted regression, especially when selection is a multi-phase process (Beresovsky, 2019).
• Interference and Networks: Bayesian generalized propensity scores for interconnected units allow for simultaneous adjustment for individual and neighborhood-level confounding, using penalized spline regression and modularized Bayesian estimation (Forastiere et al., 2018).

5. Propensity Inference for LLM and AI Behavior Analysis

In AI safety and interpretability research, propensity inference is repurposed to measure models’ conditional likelihood of undesired (unsanctioned) behavior, operationalized via controlled ablation experiments and hierarchical Bayesian GLMs. The method decomposes the environment into “strategic” and “non-strategic” factors, quantifies effect sizes on the log-odds or odds-ratio scales, and separately identifies the contributions of each factor class to misbehavior rates (Järviniemi et al., 22 Apr 2026). Key empirical findings include parity in the explanatory contribution of both factor classes across a spectrum of model capabilities, emergent sensitivity to goal conflict in higher-capability models, and the absence of a monotonic trend towards increased “strategic” behavior with capability. The framework is extensible to richer cognitive models, higher-order interactions, and synthetic environment generation for large-scale empirical testing.

6. Statistical Properties, Robustness, and Practical Considerations

Propensity inference methods for causal effect estimation achieve (under standard conditions):

• Consistent estimation and asymptotically valid (often doubly robust) inference for target estimands, conditional on correct specification of either the propensity or outcome model.
• Minimization of asymptotic variance for pairwise contrasts under specialized weighting schemes (optimal tilting/GOW) (Li, 2018).
• Robustness to model misspecification via calibration, covariate balancing, or explicit use of design-based uncertainty (e.g., via propagation) (Orihara et al., 2024, Heng et al., 19 Jan 2026).
• Explicit diagnostics and practical guidance regarding overlap, balance, and weight extremity; routine calibration and pre/post-matching balance checks are essential.
• In design-based inference with unknown e(X), “propensity score propagation”—regenerating e(X) from the estimated uncertainty and pooling over standard design-based CIs—restores nominal coverage even under nonparametric PS estimation, outperforming both plug-in and matching-based approaches in settings with many and/or continuous covariates (Heng et al., 19 Jan 2026).

7. Outlook and Expanding Domain

Propensity inference has evolved into a unifying analytic and inferential approach, spanning observational study design, complex surveys, non-probability/big data settings, dependent data, and AI behavior analysis. Key open directions include:

• Formalizing cognitive models of AI systems, extending statistical propensity frameworks to accommodate structured latent belief and planning variables (Järviniemi et al., 22 Apr 2026).
• Development and empirical validation of robust, scalable calibration and weighting procedures for ultra-high-dimensional or weak-overlap regimes.
• Integration with robust, selectivity-aware variable selection, and selective inference ensuring post-selection validity (Ninomiya et al., 2021).
• Extending design-based inference with explicit uncertainty propagation to non-standard assignment, missingness, and networked data (Heng et al., 19 Jan 2026).

Propensity inference thus remains a central and rapidly-evolving component of methodological and applied statistics, causal inference, and adaptive behavioral modeling.