Total Treatment Heterogeneity (TTH)

• Total Treatment Heterogeneity (TTH) is a family of functionals that quantify the dispersion of individual treatment effects across different population strata.
• Variance-based formulations like VTE summarize heterogeneity by measuring the variance of the conditional average treatment effect, emphasizing both explainable and residual components.
• Recent methods decompose TTH into explained and idiosyncratic parts and employ efficient estimation procedures, such as CV-TMLE, under standard causal assumptions.

Total Treatment Heterogeneity (TTH) denotes the overall extent to which treatment effects vary across a population. The term itself is not standardized across the recent causal-inference and HTE literature. In practice, closely related work has defined global heterogeneity through the variance of the conditional average treatment effect, through the variance of individual treatment effects, through proportions of individuals who benefit or are harmed, through subgroup contrasts, and through the gain obtainable from individualized treatment assignment. This suggests that TTH is best understood as a family of population-level heterogeneity functionals rather than a single universally accepted estimand (Levy et al., 2018, Lipkovich et al., 2023).

1. Conceptual scope and causal objects

The basic causal object underlying TTH is the individual treatment effect, usually written as

$\Delta_i = Y_i(1)-Y_i(0),$

with the conditional average treatment effect

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$

serving as the practically estimable heterogeneous-effect surface. Recent reviews distinguish sharply between variation in $\Delta(x)$, which is explainable by observed covariates, and variation in $Y(1)-Y(0)$, which is a latent individual-level quantity. That distinction is central: one may study heterogeneity across observed strata without identifying the full distribution of individual causal effects (Lipkovich et al., 2023).

A foundational semiparametric formulation defines the stratum-specific treatment effect, or blip function,

$$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$

Its mean is the average treatment effect (ATE), while its variance is the variance of treatment effect (VTE). In that framework, the global heterogeneity estimand is

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$

so that

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$

The same work stresses that VTE is

$$\mathrm{Var}\!\left(\mathbb{E}[Y_1-Y_0\mid W]\right),$$

not

$\mathrm{Var}(Y_1-Y_0).$

Accordingly, variance-based TTH in this sense measures heterogeneity across observed covariate strata, not latent within-stratum heterogeneity (Levy et al., 2018).

2. Variance-based formulations

The most explicit scalar candidate for TTH in the cited literature is VTE. In the binary-treatment observed-data model $O=(W,A,Y)$, with

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$0

the blip is

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$1

and the corresponding variance

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$2

summarizes the overall dispersion of treatment effects across the population distribution of $\Delta(x)=E(Y(1)-Y(0)\mid X=x)$3. This formulation is attractive because it is causally interpretable under consistency, no unmeasured confounding, and positivity, and because it is a direct functional of the CATE surface. At the same time, the original paper emphasizes that variance is only one scalar summary: two treatment-effect distributions can have the same variance but different shapes, tails, skewness, modality, or proportions harmed versus benefited, and VTE does not separately highlight sign heterogeneity except through dispersion (Levy et al., 2018).

An analogous construction appears for multivariate continuous exposures. With

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$4

the paper on multivariate continuous treatments defines

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$5

and describes it as “the overall amount of heterogeneity of the treatment effect.” This is an integrated variance of conditional treatment effects across covariate profiles, averaged over the treatment distribution. It is therefore a direct multivariate-exposure analogue of variance-based TTH (Shin et al., 2024).

A closely related large-scale experimentation paper uses the label Total Treatment Effect Variation and writes

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$6

with decomposition

$\Delta(x)=E(Y(1)-Y(0)\mid X=x)$7

Here $\Delta(x)=E(Y(1)-Y(0)\mid X=x)$8 is the explained component and $\Delta(x)=E(Y(1)-Y(0)\mid X=x)$9 is idiosyncratic variation. This is conceptually closer to a latent unit-level notion of TTH than VTE, although the paper also treats the idiosyncratic part as only partially identified absent additional assumptions (Cai et al., 2022).

3. Identification and efficient estimation

Variance-based TTH is identified in the observed-data setting under the standard causal assumptions. With i.i.d. data

$\Delta(x)$0

binary treatment $\Delta(x)$1, and counterfactual outcomes $\Delta(x)$2, the identifying conditions are consistency, exchangeability

$\Delta(x)$3

and positivity

$\Delta(x)$4

Under these assumptions,

$\Delta(x)$5

identifies the ATE/VTE pair from observed data (Levy et al., 2018).

The same paper derives the efficient influence curve for jointly estimating ATE and VTE. For the VTE coordinate, the influence function contains both a residual correction term and a plug-in variance term: $\Delta(x)$6 This leads naturally to plug-in estimators based on the empirical variance of estimated blip values, but the paper develops TMLE and especially CV-TMLE to support efficient inference. The practical message is explicit: CV-TMLE provides simultaneous plug-in estimates and inference for ATE and VTE, and it achieves asymptotic efficiency “under one less condition than TMLE,” namely without the Donsker condition (Levy et al., 2018).

The difficulty is the second-order remainder. For VTE, the remainder contains

$\Delta(x)$7

so the blip must be estimated at $\Delta(x)$8 in $\Delta(x)$9, even if the treatment mechanism is known. The paper states this directly: VTE estimation is not doubly robust. In consequence, randomized treatment or correctly specified propensity scores do not rescue poor outcome-model estimation. The same paper also stresses a practical boundary issue: VTE is bounded below by $Y(1)-Y(0)$0, and when the true VTE is small, normal approximations may fail. It reports a rule of thumb that sample sizes around $Y(1)-Y(0)$1 or more may be needed even to hope for reliable estimation when true VTE is around $Y(1)-Y(0)$2 (Levy et al., 2018).

4. Decomposition of total heterogeneity

Once TTH is expressed as a variance of the CATE, several papers decompose that variance into explained and residual components. One targeted-learning paper defines

$Y(1)-Y(0)$3

then introduces

$Y(1)-Y(0)$4

and shows the equivalent forms

$Y(1)-Y(0)$5

and

$Y(1)-Y(0)$6

Its scaled version,

$Y(1)-Y(0)$7

functions as an $Y(1)-Y(0)$8-type proportion of total heterogeneity attributable to a covariate subset. In variance-based TTH terms, this paper turns total heterogeneity into an estimable decomposition problem (Li et al., 2023).

A different decomposition arises when the analyzed treatment is itself heterogeneous. In the aggregated-treatment setting, the observed subgroup gap is not purely effect heterogeneity. One paper decomposes unadjusted and adjusted group contrasts as

$Y(1)-Y(0)$9

and

$$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$0

Here $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$1 is the clean effect-heterogeneity term, while $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$2 is the pure treatment-composition term. The paper explicitly states that the closest single “pure treatment heterogeneity” term is $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$3, whereas the broader “total treatment-heterogeneity contribution” is the sum of all treatment-heterogeneity-driven terms. This decomposition is especially relevant when treatment is a bundle of modules, doses, or versions rather than a homogeneous intervention (Heiler et al., 2 Jul 2025).

A closely related earlier paper expresses the same idea at the level of conditional mean effects. Let $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$4 be the natural binary-treatment effect under the observed mixture of underlying treatment versions and $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$5 the corresponding effect under a fixed population-level version mixture. Then

$$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$6

In this decomposition, $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$7 captures heterogeneity in response to comparable treatment-version composition, while $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$8 captures heterogeneity induced by differences in treatment-version composition across covariate groups. For TTH, this shows that total observed heterogeneity under a coarse treatment indicator may conflate response heterogeneity and treatment heterogeneity (Heiler et al., 2021).

5. Detection, validation, and decision relevance

Several papers address not the magnitude of TTH itself, but whether heterogeneity is present, whether a learned score captures it, and whether it is useful for action. In randomized trials, sequential validation evaluates a candidate CATE score $$b_P(W)=\mathbb{E}_{P}[Y\mid A=1,W]-\mathbb{E}_{P}[Y\mid A=0,W].$$9 through fold-specific statistics $$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$0 based on metrics such as BLP, GATES, QINI curves, and RATEs. The key point is conceptual: these metrics do not measure total heterogeneity directly. They test whether a learned score is aligned with genuine treatment-effect variation, or whether it is useful for subgroup separation or ranking. Sequential validation replaces invalid naive fold aggregation with a martingale-based procedure that preserves validity while pooling evidence across folds (Wager, 2024).

A complementary nonparametric paper tests the null

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$1

across finitely many pre-specified strata using multi-sample U-statistics. Its omnibus statistic

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$2

is a test statistic, not a calibrated TTH estimand. It is therefore best interpreted as a method for between-stratum heterogeneity detection rather than a full measure of total heterogeneity across individuals (Dai et al., 2020).

A still more decision-oriented notion appears in work on actionable heterogeneity. There the primary estimand is the conditional expected out-of-sample loss difference

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$3

comparing an unrestricted HTE model with a constant-effect restricted model. The associated $$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$4-value is the smallest significance level at which the confidence interval excludes zero. This quantity is explicitly model-, loss-, and sample-dependent. It measures practical gain from modeling heterogeneity, not the population’s total heterogeneity. A small $$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$5 can coexist with substantial TTH if the heterogeneity is weak, noisy, or hard to learn; conversely, modest heterogeneity can yield meaningful predictive gain if it is simple and predictable (Ashouri et al., 6 Mar 2025).

6. Partial identification, benefit–harm summaries, and fundamental limits

If TTH is interpreted as the full distribution of individual treatment effects, nonidentification becomes unavoidable. A partial-identification paper makes this point directly: the marginals

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$6

do not determine the joint law of $$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$7, hence they do not determine the distribution of

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$8

The paper develops nonparametrically sharp bounds for subgroup treatment effects, positive and negative parts of treatment effects, and subgroup winner/loser fractions such as

$$\Psi(P)=\big(\mathbb{E}_P b_P(W),\ \mathrm{var}_P(b_P(W))\big),$$9

and notes that its results extend to Makarov-type bounds on

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$0

In this view, TTH is fundamentally a dependence problem: observed marginals reveal only partial information about the total heterogeneity of individual treatment responses (Kaji et al., 2023).

A latent-variable identification paper addresses this problem through treatment benefit and harm rates. For binary outcomes it defines

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$1

and for continuous outcomes with threshold $$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$2,

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$3

These are directional heterogeneity summaries. Their sums,

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$4

and

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$5

are natural thresholded notions of total heterogeneity. The paper identifies them under the assumption

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$6

within non-separable generalized linear mixed models. This route delivers interpretable total-benefit and total-harm summaries, but only under structural assumptions stronger than randomization alone (Yin et al., 2016).

A final limitation arises in predictive patient-specific modeling. Contextualized tuberculosis models represent heterogeneity through patient-specific coefficients in

$$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$7

so that the distribution of $$\text{ATE}=\mathbb{E}[b_0(W)], \qquad \text{VTE}=\mathrm{Var}(b_0(W)).$$8 across patients is a natural proxy for individualized treatment heterogeneity. Yet the same paper uses observational data and does not present an explicit causal identification strategy. It therefore illustrates an important boundary for TTH work: predictive heterogeneity surfaces can be rich and clinically suggestive without being causally identified measures of treatment-effect variation (Wu et al., 2024).

In sum, TTH has no single canonical definition across the current literature. The most rigorous observed-data scalar measure is the variance of the CATE, usually VTE, while broader formulations extend to integrated variance functionals for multivariate treatments, explained-versus-residual decompositions, treatment-version decompositions, benefit–harm rates, and policy-value gains. The decisive conceptual fault line is between heterogeneity across observed covariate strata and heterogeneity in latent individual causal effects. The former is often estimable with semiparametric efficiency; the latter is generally only partially identified unless stronger structural assumptions are imposed (Levy et al., 2018).