Cross-Head Attention Uplift Network with Inverse Propensity Score under Unobserved Confounding

Abstract: Uplift modeling, crucial for estimating individual treatment effects (ITE), faces dual challenges: flexibly leveraging inter-group similarity to enhance discriminative power and debiasing under unobserved confounding scenarios. In this paper, we propose the Cross-Head Attention Uplift Network (CHAUN) and Robust Adversarial Inverse Propensity Score (RA-IPS) method to address these limitations. CHAUN employs shared feature embeddings and cross-head attention mechanisms to dynamically integrate treatment-specific and control-specific representations, enhancing inter-group correlation modeling. Theoretically, we prove that access to the true propensity scores ensures ITE identifiability even with unobserved confounders. For practical scenarios lacking true propensity scores, RA-IPS adversarially optimizes propensity weights within constrained uncertainty sets to mitigate bias from unobserved variables. Experiments on public datasets (CRITEO-UPLIFT, LAZADA) and a production e-commerce dataset demonstrate CHAUN's superiority over state-of-the-art uplift models, achieving relative improvements of up to 25.6% in QINI scores. RA-IPS further enhances robustness, outperforming standard IPS by 5.4% under unobserved confounding. The results validate the effectiveness of our proposed methods in real-world causal inference tasks.

• The paper proposes a novel Cross-Head Attention Uplift Network (CHAUN) coupled with a Robust Adversarial Inverse Propensity Score (RA-IPS) estimator to address unobserved confounding in ITE estimation.
• It employs dynamic attention mechanisms to fuse treatment and control representations, enhancing inter-group similarity exploitation and predictive accuracy.
• Empirical results demonstrate significant gains, with up to a 25.6% improvement in QINI scores and a 5.4% increase from RA-IPS over standard IPS estimators.

Cross-Head Attention Uplift Network with Inverse Propensity Score under Unobserved Confounding

Overview

The paper "Cross-Head Attention Uplift Network with Inverse Propensity Score under Unobserved Confounding" (2606.27114) introduces a robust methodology for individual treatment effect (ITE) estimation in uplift modeling, addressing both the exploitation of inter-group similarity and adaptation to unobserved confounders. The principal innovations are (1) the Cross-Head Attention Uplift Network (CHAUN), which dynamically fuses treatment/control representations via attention mechanisms, and (2) the Robust Adversarial Inverse Propensity Score (RA-IPS) estimator, which enhances debiasing under unobserved confounding via adversarial optimization within theoretically constrained uncertainty sets. The empirical evaluation demonstrates CHAUN's substantial improvement over established baselines and confirms the added robustness of RA-IPS in realistic, confounded environments.

Problem Formulation and Theoretical Guarantees

Uplift modeling is operationalized in the Neyman-Rubin potential outcomes framework: given a dataset with observed covariates $x$, unobserved confounders $u$, treatment assignment $t$, and observed outcome $y$, the goal is to estimate the ITE $\mathbb{E}(y(1)-y(0)|x)$. Standard uplift architectures assume ignorability (no unobserved confounders), but practical situations—e.g., online advertising, recommendation systems—violate this assumption, resulting in biased estimates unless the causal assignment mechanism is adequately characterized.

The paper's theoretical contributions demonstrate that unbiased ITE estimation is possible given knowledge of the true propensity scores $\pi(x,u) = P(t=1|x,u)$. Specifically, the ITE can be identified via

$$ITE(x) = \mathbb{E}\left( \frac{ty}{\pi(x,u)} - \frac{(1-t)y}{1 - \pi(x,u)} \Big| x \right).$$

In practical settings where only nominal propensity scores $\tilde{\pi}(x)=P(t=1|x)$ are available, the authors provide a sensitivity analysis that constrains the feasible set of true propensity scores given observed propensity and uncertainty bounds.

CHAUN Architecture

CHAUN is constructed with three primary building blocks: Shared Feature Embedding Layer, Propensity Learner, and Outcome Learner.

• Feature Embedding: Observed features are processed via parallel continuous and categorical embeddings, concatenated into a unified representation.
• Propensity Learner: An MLP outputs propensity score predictions calibrated through cross-entropy minimization and global regularization enforcing the overlap condition.
• Outcome Learner: Two parallel MLPs encode treatment-specific and control-specific latent representations. These are fused adaptively by a cross-head attention mechanism, computing attention weights via learned projections and weighted summations. The fused representation is used for outcome prediction, enabling the model to dynamically exploit inter-group similarity and treatment-induced discriminative patterns.

  Figure 1: CHAUN’s predicted outcomes align with true stratification and enhance discriminative power via attention-weighted treatment/control fusion.

The IPS-based loss function is regularized to prevent pathological solutions (e.g., degenerate overlap), ensuring stability for large-scale training.

Robust Adversarial Inverse Propensity Score (RA-IPS)

To address the inherent bias in IPS estimators under unobserved confounding, the paper advances the RA-IPS approach. The uncertainty set for true propensity scores is characteristically tight, derived from bounding the influence of confounders on logit-transformed propensity scores. The adversarial optimization maximizes the loss over feasible weight configurations while enforcing global regularization, ensuring predicted potential outcomes are robust to the worst-case confounder impact.

Mathematically, propensity score perturbations are bounded by hyperparameter $\Gamma$, and feasible weighting solutions respect global constraints:

$$\mathcal{W} = \{W \in \mathbb{R}_+^N : a(x_i, t_i) \le w_i \le b(x_i, t_i),\ \frac{1}{N}\sum_{i=1}^N(w_i - \tilde{w}_i) \leq \epsilon_N \}$$

RA-IPS loss is computed with the worst-case weight configuration within $u$0, yielding a model robust against latent assignment mechanisms.

Empirical Results

Extensive benchmarking on CRITEO-UPLIFT, LAZADA, and a proprietary large-scale e-commerce dataset validate the methodological advances. CHAUN achieves top or second-best performance across all ranking metrics (LIFT@30, AUUC, QINI, PUC), surpassing alternatives including S/T-learners, CFRNet, DragonNet, CEVAE, FlexTENet, EUEN, DESCN, and EFIN. Notably, CHAUN realizes up to 25.6% improvement in QINI scores compared to DragonNet.

Ablation studies confirm the criticality of cross-head attention: replacing with conventional mixture-of-experts (MMoE) gates or removing attention modules noticeably degrades performance, particularly in high-dimensional settings, underscoring the value of dynamic inter-group exploitation.

RA-IPS consistently improves over standard IPS and RD-IPS (which lacks a theoretically grounded uncertainty set), yielding up to 5.4% QINI increase under simulated unobserved confounding.

Sensitivity analyses of the $u$1 hyperparameter demonstrate robust improvements for moderate settings; excessive values induce instability and diminished gains, indicating practical guidance for operational deployment.

Practical and Theoretical Implications

The methodological advances fundamentally address two longstanding challenges in uplift modeling: dynamic inter-group similarity modeling and robust debiasing under unobserved confounding. The CHAUN architecture is applicable to a wide array of causal inference tasks involving stratified interventions, enabling precise, individualized effect estimation and improved group ranking. RA-IPS provides a principled framework for sensitivity to latent assignment pathways, which is practically critical for large-scale observational applications—advertising, recommender systems, and open-market interventions—where confounding is often structurally unavoidable.

The established theoretical guarantees (identifiability, unbiased estimation, generalization bounds) are broadly applicable, and the empirical benchmarks validate practical viability.

Future Directions

Future work can extend CHAUN and RA-IPS methodologies to multi-treatment settings, more granular outcome spaces, and structured covariate architectures (e.g., graph-based or temporal embeddings). Integrating structural assumptions (instrumental or proxy variables) in the adversarial IPS estimation could further fortify robustness in complex causal landscapes. Real-world deployment will necessitate automated hyperparameter calibration, scalable architectures, and integration with operational feedback loops for continuous adaptation.

Conclusion

This work introduces CHAUN and RA-IPS, advancing uplift modeling with attention-based dynamic fusion and adversarial debiasing against unobserved confounding. The methodology is theoretically grounded, empirically validated, and widely applicable, providing principled, robust ITE estimation for precision intervention in observational causal inference (2606.27114).