Doubly Robust Mean Embedding (DR-ME)

• Doubly Robust Mean Embedding (DR-ME) is a framework that unifies kernel mean embedding with doubly robust estimation to capture complete counterfactual outcome distributions.
• It constructs DR pseudo-outcomes by leveraging robust nuisance models, ensuring estimator consistency when either the propensity or outcome model is correctly specified.
• DR-ME facilitates nonparametric inference, hypothesis testing, and interpretable localization of heterogeneous treatment effects in high-dimensional settings.

Doubly Robust Mean Embedding (DR-ME) unifies kernel mean embedding with doubly robust semiparametric estimation, providing a principled framework for inference on counterfactual distributions under observational or off-policy data. DR-ME methods represent full interventional or counterfactual outcome distributions in a reproducing kernel Hilbert space (RKHS) and construct estimators or tests that remain consistent when either of two nuisance models (propensity or outcome) are correctly specified. Unlike mean-based approaches, DR-ME captures the entire (potentially heterogeneous, multimodal, or heavy-tailed) distribution, supporting nonparametric inference, hypothesis testing, and interpretable localization of distributional effects.

1. Counterfactual Mean Embedding in RKHS

Let $\mathcal{Y}$ denote an outcome space and $\phi: \mathcal{Y} \to \mathcal{H}$ a feature map into a RKHS $\mathcal{H}$ with characteristic kernel $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$. The mean embedding of a random variable $Y$ is the element $\mu_Y := \mathbb{E}[\phi(Y)] \in \mathcal{H}$, uniquely representing the law of $Y$.

For causal or policy evaluation settings, the counterfactual mean embedding under a treatment or target policy—denoted $\mu_{Y^t|V}$ or $\chi(\pi)$—is the RKHS embedding of the conditional or interventional distribution of $Y$ given treatment or policy assignment, possibly conditioned on covariates or summary features $\phi: \mathcal{Y} \to \mathcal{H}$0 (Anancharoenkij et al., 4 Feb 2026, Zenati et al., 3 Jun 2025, Zenati et al., 8 May 2026). In arithmetic, for $\phi: \mathcal{Y} \to \mathcal{H}$1,

$\phi: \mathcal{Y} \to \mathcal{H}$2

which generalizes standard mean estimation to full nonparametric distributional representation.

2. Doubly Robust Estimation and Orthogonalized Scores

DR-ME estimators are derived by constructing augmented inverse-propensity features (pseudo-outcomes) that are orthogonal and doubly robust:

• Nuisance models: Propensity score $\phi: \mathcal{Y} \to \mathcal{H}$3 and outcome regression (in $\phi: \mathcal{Y} \to \mathcal{H}$4) $\phi: \mathcal{Y} \to \mathcal{H}$5 (or $\phi: \mathcal{Y} \to \mathcal{H}$6 in the policy setting).
• DR pseudo-outcome: For unit $\phi: \mathcal{Y} \to \mathcal{H}$7, the pseudo-outcome for target $\phi: \mathcal{Y} \to \mathcal{H}$8 is:

$\phi: \mathcal{Y} \to \mathcal{H}$9

• Key property: The population regression $\mathcal{H}$0 is unbiased for $\mathcal{H}$1 provided either the propensity or the outcome embedding model is correctly specified (Anancharoenkij et al., 4 Feb 2026, Zenati et al., 3 Jun 2025, Zenati et al., 8 May 2026).

For fixed-location vector-valued witnesses in hypothesis testing, analogous orthogonal features are constructed for greater statistical efficiency and interpretability.

3. DR-ME Estimation Algorithms

The typical DR-ME procedure consists of the following steps:

1. Nuisance model estimation: Fit $\mathcal{H}$2 and $\mathcal{H}$3 via cross-fitting or sample splitting to avoid overfitting and ensure orthogonality (Anancharoenkij et al., 4 Feb 2026, Zenati et al., 8 May 2026).
2. Pseudo-outcome construction: Evaluate DR pseudoresiduals or features for each observation.
3. Second-stage regression: Regress the pseudo-outcome in $\mathcal{H}$4 on the conditioning variable $\mathcal{H}$5 (or, for off-policy settings, solve linear equations involving conditional mean operators) (Anancharoenkij et al., 4 Feb 2026, Zenati et al., 3 Jun 2025).
4. Estimator forms: DR-ME encompasses several functional forms:
   • Kernel Ridge Regression (KRR): Closed-form RKHS ridge regression of pseudo-outcomes.
   • Deep Feature and Neural Kernel estimators: Learn representations (e.g., via neural networks) mapping $\mathcal{H}$6 to finite-dimensional or RKHS-valued embeddings, then regress to the mean embedding target (Anancharoenkij et al., 4 Feb 2026).
   • Plug-in and One-Step (DR) Policy Embedding: Separate estimation of conditional mean operator and policy embedding ($\mathcal{H}$7), with DR correction via the efficient influence function ($\mathcal{H}$8) (Zenati et al., 3 Jun 2025).
5. Test statistics: For testing, construct DR mean-embedding witnesses at selected locations or form global MMD-based tests using the empirical efficient influence function (Zenati et al., 8 May 2026, Zenati et al., 3 Jun 2025).

The following table summarizes primary DR-ME estimator types:

Estimator	Stage 1 (Nuisance)	Stage 2 (Pseudo-outcome Regression)
Kernel Ridge Regression	$\mathcal{H}$9	RKHS-valued ridge regression
Deep Feature	$$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$0	Neural net feature + linear map
Neural Kernel	$$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$1	Parameterized kernel representation

4. Theoretical Guarantees: Double Robustness and Convergence Rates

DR-ME offers double robustness: at the population level, the estimator for $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$2 or $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$3 is consistent if either the propensity model or the outcome embedding model is correct. This property extends to the test statistics and finite-location discrepancy vectors.

Key rates include:

• Plug-in estimators: Converge at problem-dependent rates, e.g., $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$4 in the best-case smoothness scenario (Zenati et al., 3 Jun 2025).
• Doubly robust (one-step) estimators: Achieve $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$5 root-n rates when both nuisances are estimated consistently at $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$6 rate (Zenati et al., 3 Jun 2025, Anancharoenkij et al., 4 Feb 2026).
• Sample splitting and cross-fitting are critical for preserving orthogonality and ensuring valid inference and calibration.

Under regularity, test statistics (e.g., Hotelling’s $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$7 for fixed-location DR-ME) are $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$8-calibrated, with noncentrality governed by the local-power geometry for alternatives converging to the null at $$k(y, y') = \langle \phi(y), \phi(y') \rangle_{\mathcal{H}}$$9 (Zenati et al., 8 May 2026).

5. DR-ME Hypothesis Testing and Interpretable Localization

DR-ME provides two principal testing paradigms:

• Global distributional testing: Tests $Y$0 using DR-efficient mean-embedding test statistics, such as cross-fitted MMD-based kernel tests equipped with the DR influence function (Zenati et al., 3 Jun 2025).
• Finite-location localization: Projects the distributional change onto kernel evaluations at selected locations $Y$1, constructing a vector of causal discrepancy coordinates:

$Y$2

The DR-ME Hotelling statistic $Y$3 is chi-square calibrated and interpretable at the finite set of locations (Zenati et al., 8 May 2026).

For localization, a data-driven criterion selects outcome locations $Y$4 to maximize local detection power, i.e.,

$Y$5

with sample splitting to preserve validity. These locations identify interpretable points at which the effect is maximally detectable, supporting hypothesis testing with post-selection coverage (Zenati et al., 8 May 2026).

6. Practical Implementations and Empirical Performance

The DR-ME framework supports practical algorithms with the following features:

• Kernel choice: RBF and Matérn kernels on $Y$6 and $Y$7 are standard; bandwidths are set via median heuristic or cross-validation.
• Regularization: Tuning parameter $Y$8 is selected via (generalized) cross-validation.
• Nuisance estimation: Propensity scores and conditional mean embeddings are estimated via logistic regression, machine learning (e.g., random forests, boosting), or large-scale kernel regression and random features (Zenati et al., 3 Jun 2025, Anancharoenkij et al., 4 Feb 2026).
• Sample splitting and cross-fitting: Essential for valid inference; splitting into folds for nuisance, location learning, and testing is standard (Zenati et al., 8 May 2026).
• Sampling from embeddings: Deterministic kernel herding draws are available for generating samples from the estimated counterfactual law, converging in MMD to the oracle distribution (Zenati et al., 3 Jun 2025).

Empirically, DR-ME estimators outperform plug-in and classical IPW or doubly robust estimators for distributional treatment effects. DR-ME tests exhibit near-nominal type-I error and superior power, while finite-location DR-ME enables interpretable effect localization in high-dimensional settings, such as medical imaging (Zenati et al., 8 May 2026, Anancharoenkij et al., 4 Feb 2026, Zenati et al., 3 Jun 2025).

7. Applications and Scope

The DR-ME methodology subsumes a wide range of tasks:

• Counterfactual estimation: Full nonparametric recovery of counterfactual or interventional outcome distributions.
• Policy evaluation: Distributional off-policy evaluation in recommendation, healthcare, and advertisement (Zenati et al., 3 Jun 2025).
• Causal distributional analysis: Hypothesis testing for arbitrary difference in distribution (not just means) and quantifying where effects differ across outcome space (Zenati et al., 8 May 2026).
• Structured and high-dimensional data: Applicability to scarlar, vector, image-valued, or general structured outcomes, as long as a universal kernel is available.

DR-ME extends classical scalar doubly robust approaches to the rigor of infinite-dimensional, nonparametric distributional inference, enabling robust and interpretable causal analysis of heterogeneous treatment effects (Anancharoenkij et al., 4 Feb 2026, Zenati et al., 3 Jun 2025, Zenati et al., 8 May 2026).