Nonparametric inference for sublevel-set probabilities of conditional average treatment effect functions

Abstract: The average treatment effect can obscure important heterogeneity when individuals respond differently to a treatment. While the conditional average treatment effect (CATE) function captures such heterogeneity, it is difficult to communicate when it depends on many covariates. Sublevels sets of a multivariate CATE function are equally complicated objects, but the probability of a sublevel set of a CATE function is a single number with a simple interpretation as the proportion of individuals whose expected treatment effect does not exceed a prespecified threshold. By varying the threshold, a univariate monotone curve appears which can be used to visualize the overall type and degree of heterogeneity in a population. We formalize this curve as a target parameter and show that it is not pathwise differentiable under a nonparametric model. To address this nonstandard estimation problem, we leverage recent advances in monotone function estimation and develop a Grenander-type estimator that incorporates machine learning. We also show that the best piecewise linear approximation to the curve of interest is a pathwise differentiable parameter, and we develop a debiased machine learning estimator of this approximation. We investigate our proposed estimators' finite sample performance in a sequence of numerical studies based on data synthesized from a randomized trial. The methods are illustrated in data from a randomized trial on diabetes medication.

• The paper establishes that pointwise CATE sublevel-set probabilities are generally not pathwise differentiable, limiting regular influence-function-based inference for the full curve.
• The paper introduces a Grenander-type estimator with an n^{1/3}-rate Chernoff limit and an efficient spline approximation estimator that supports pointwise intervals and simultaneous bands under product-rate conditions.
• The paper finds that debiasing reduces bias but often increases variance, while monotonized splines can improve mean squared error; in the LEADER trial, wide intervals showed substantial uncertainty about who benefits from treatment.

Motivation and target parameter

The conditional average treatment effect (CATE) function captures treatment effect heterogeneity, but as a function of many covariates it is difficult to summarize or visualize. Munch and Gerds propose a univariate summary: the sublevel function $\gamma(\alpha) = P(\tau(W) \leq \alpha)$, the marginal probability that the CATE function $\tau$ does not exceed level $\alpha$. Equivalently, $\gamma$ is the cumulative distribution function of the expected treatment effects in the population. Each value has a direct interpretation — e.g., $\gamma(0)$ is the proportion of the population for which the treatment is not expected to be beneficial — and varying $\alpha$ traces a monotone curve that encodes both the type and degree of heterogeneity (degenerate point mass at the ATE under no interactions; plateaus for discrete-covariate interactions; continuous curves when a continuous covariate interacts with treatment). The interpretation depends on which covariates define the CATE, and the authors are explicit about this relativity.

The setting is the Neyman–Rubin framework with binary outcome and treatment, identification via coarsening at random plus a uniform positivity assumption on the propensity score $\pi$, and a fully nonparametric observed-data model.

Non-pathwise differentiability

A central theoretical result is that pointwise evaluation $\gamma(\alpha)$ is not pathwise differentiable at any $P$ in a smooth submodel whenever the $\alpha$-level set of $\tau$0 contains either a non-critical point or an open region where $\tau$1 is locally constant. The proof constructs explicit perturbation paths: in the first case the Gateaux derivative exists but cannot be represented as an inner product with any fixed element of $\tau$2; in the second case the map is discontinuous along paths. The practical consequence is that influence-function-based semiparametric efficiency tools do not directly apply to $\tau$3, so regular asymptotically linear estimation of the curve itself is generally unavailable. The theorem does not cover saddle points of $\tau$4 (the authors conjecture non-differentiability there too), nor models where $\tau$5 takes finitely many values, where pathwise differentiability may hold at some levels.

Estimation strategies

All three proposed estimators use cross-fitting over $\tau$6 folds and plug-in estimates of the sublevel-set indicator $\tau$7 derived from a CATE learner.

Cross-fitted plug-in estimator: the empirical distribution function of cross-fitted predicted effects $\tau$8. Its error decomposes into a sampling term converging at rate $\tau$9 and a bias term governed by the disagreement probability between $\alpha$0 and $\alpha$1, so it inherits the sup-norm convergence rate of the chosen CATE learner; no tractable asymptotic distribution is available in general.

Grenander-type estimator: the key move is lifting to the antiderivative $\alpha$2. A von Mises expansion shows that $\alpha$3 is pathwise differentiable wherever $\alpha$4 is continuous, with efficient influence function built from the indicator $\alpha$5, the augmented inverse-probability-weighted pseudo-outcome combining $\alpha$6 and $\alpha$7, and the offset $\alpha$8. A cross-fitted one-step estimator of $\alpha$9 is formed, and $\gamma$0 is estimated by the derivative of the greatest convex minorant of this estimator. Under differentiability of $\gamma$1 near $\gamma$2, the estimator converges at rate $\gamma$3 to a scaled Chernoff distribution, enabling pointwise confidence intervals using tabulated Chernoff quantiles ($\gamma$4), with scale factors estimated from "level-one" data via nearest-neighbor and kernel density methods. Notably, one technical condition (bounding a localized empirical process term) is imported from Westling's unified theory without simple verifiable sufficient conditions; a crude analysis requires $\gamma$5, under which the plug-in estimator would actually converge faster — though the Grenander limit distribution would still be valuable for inference.

Spline approximation estimator: the best piecewise linear approximation $\gamma$6 of $\gamma$7 over fixed knots, measured in $\gamma$8, is shown to be pathwise differentiable. Its coefficients depend only on evaluations and integrals of $\gamma$9, each of which admits an efficient influence function obtained by integrating $\gamma(0)$0. Plugging the cross-fitted one-step estimator into these expressions yields a locally efficient, regular, asymptotically linear estimator under product-rate conditions ($\gamma(0)$1, $\gamma(0)$2) that permit slower-than-parametric nuisance rates; boundary conditions are trivially satisfied when approximating over all of $\gamma(0)$3. Delta-method pointwise intervals and simulation-based $\gamma(0)$4-sup simultaneous bands follow. Since $\gamma(0)$5 need not be monotone or bounded in $\gamma(0)$6, a monotonized variant is constructed via Chernozhukov et al.'s rearrangement.

Numerical performance

Simulations calibrated to the LEADER trial data-generating structure (with weak- and strong-heterogeneity variants) compare the four estimators at $\gamma(0)$7 across $\gamma(0)$8, 1000 replications. The debiased estimators substantially reduce bias relative to the plug-in estimator, but at a variance cost large enough that the plug-in estimator retains the lowest MSE in most settings — an honest negative result for the debiasing machinery. Monotonizing the spline improves MSE enough to beat the plug-in under both weak and strong heterogeneity. Coverage of pointwise intervals is close to nominal for large samples when heterogeneity is present, but anti-conservative even at $\gamma(0)$9 in the weak-heterogeneity setting. Simultaneous spline bands are anti-conservative (raw) and conservative (monotonized) in most cases, and neither achieves nominal coverage against the true $\alpha$0 — expected, since they target the spline approximation rather than $\alpha$1 itself.

Application to the LEADER trial

Applied to 8,563 randomized patients (26 censored before three years were dropped, solely for illustration) with a binary composite cardiovascular endpoint within three years, super learner–based outcome regression, and a randomized propensity score, all estimators place expected effects close to the ATE of roughly 1%. The constrained Grenander estimate of $\alpha$2 is near zero, but its 95% confidence interval is $\alpha$3; the interval for $\alpha$4 is $\alpha$5. These wide intervals indicate that the trial data cannot rule out substantial fractions of non-benefited patients, and the confidence bands remain compatible with an abruptly changing $\alpha$6 concentrated near the ATE.

Limitations and open questions

The paper concedes several points plainly. The minimax difficulty of estimating $\alpha$7 is unresolved; the authors conjecture an $\alpha$8 rate when $\alpha$9 is differentiable and CATE-level difficulty when $\pi$0 may be discontinuous. Assumption (b) behind the Grenander limit distribution lacks easily checkable sufficient conditions. All estimators perform poorly under the null of no heterogeneity, where $\pi$1 is degenerate and discontinuous, and require large samples plus genuine heterogeneity to work well. The analysis also discards censoring rather than modeling it, and relies on plug-in construction of $\pi$2 from a single CATE learner rather than dedicated classification approaches or orthogonal statistical learning with isotonic regression, both flagged as alternatives not explored here.

Conclusion

The paper formalizes the CDF of expected treatment effects as a function-valued causal parameter, establishes its non-pathwise differentiability, and supplies two debiased machine learning estimators — a Grenander-type estimator with a Chernoff limit law and an efficient spline approximation estimator — together with valid inference procedures. The empirical evidence supports bias reduction but not uniform MSE dominance over the naive plug-in, making the contribution primarily one of principled inference for a well-motivated heterogeneity summary rather than a uniformly superior point estimator.