Surrogate Index in Causal Inference

• Surrogate index is a method that condenses multiple short-term surrogate measures into a composite estimator for long-term treatment effects.
• It employs techniques like penalized regression, tree-based models, and ensemble methods to manage high-dimensional surrogate data effectively.
• Extensions include proximal and weighted approaches to address unobserved confounding, improve decision-making in experiments, and enable trial-level meta-analysis.

A surrogate index is a construction that compresses surrogate information into an estimand, predictor, or composite marker used in place of an unavailable target. In causal inference, the canonical surrogate index is the conditional mean of a long-term outcome given short-term surrogates and pre-treatment covariates, estimated in one sample and transported to another (Athey et al., 2016). Recent work extends this formulation to settings with unobserved confounding through the proximal surrogate index, defined as an outcome bridge involving proxy variables (Hung et al., 25 Jan 2026), and to sensitivity analysis through Weighted Surrogate Indices that relax full mediation via a conditional copula (Fan et al., 28 Feb 2026). In high-dimensional clinical-trial meta-analysis, a surrogate index may instead be a rank-based composite signature formed from promising trial-level markers (Hughes et al., 5 May 2026). In surrogate modeling, the phrase is also used for low-dimensional projections or admissible multi-index sets that organize emulator structure and adaptive refinement (Li et al., 2023, Kent et al., 4 Jul 2025).

1. Classical surrogate index for long-term treatment effects

The foundational causal-inference formulation considers two samples. The experimental sample contains treatment $W\in\{0,1\}$ and short-term surrogates $S\in\mathbb R^d$ but no long-term outcome $Y$, while the observational sample contains $S$ and $Y$ but no $W$. The surrogate index is defined as

$\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$

with $P\in\{E,O\}$ denoting the sample indicator. Under comparability, $\mu(s,x,E)=\mu(s,x,O)$, so $\mu(s,x,O)$ estimated in the observational sample can be used to impute long-term outcomes in the experiment (Athey et al., 2016).

Identification relies on three core conditions. First, unconfoundedness in the experiment requires $S\in\mathbb R^d$0 with overlap. Second, Prentice-type surrogacy requires $S\in\mathbb R^d$1 together with overlap in $S\in\mathbb R^d$2. Third, comparability requires $S\in\mathbb R^d$3 and support inclusion of $S\in\mathbb R^d$4 in $S\in\mathbb R^d$5. These conditions imply that treatment affects the long-term outcome only through the observed surrogates once $S\in\mathbb R^d$6 are held fixed, and that the conditional law of $S\in\mathbb R^d$7 given $S\in\mathbb R^d$8 is transportable across the two samples (Athey et al., 2016).

Under these assumptions, the average treatment effect in the experiment,

$S\in\mathbb R^d$9

can be written as

$Y$0

An equivalent inverse-probability-weighted representation is

$Y$1

where $Y$2. Operationally, the estimator is therefore a difference in means on an imputed long-term outcome (Athey et al., 2016).

A central motivation is that multiple surrogates may collectively satisfy the statistical surrogacy criterion even when no individual proxy does so alone. This is particularly relevant in modern settings with large numbers of intermediate outcomes on or near the causal pathway between treatment and a long-term endpoint (Athey et al., 2016).

2. Estimation with multiple and high-dimensional surrogates

When $Y$3 is high-dimensional, the surrogate index is estimated by regressing $Y$4 on $Y$5 in the observational sample. The methods explicitly listed for this step include penalized linear estimators such as Lasso, elastic net, and ridge; tree-based estimators such as random forests, boosted trees, and Bayesian additive regression trees; and “Super-Learner” ensembles. A generic procedure is: fit $Y$6 in the observational sample, compute $Y$7 in the experimental sample, and estimate the treatment effect by difference in means, weighted regression, or IPW with $Y$8 (Athey et al., 2016).

Cross-fitting may be used to remove over-fitting bias. The influence-function approach yields a “double-robust” estimator that remains $Y$9-consistent if either $S$0 or $S$1 is well-estimated. This places surrogate-index estimation within the general semiparametric program of nuisance-robust estimation with flexible first-stage learners (Athey et al., 2016).

The California GAIN application illustrates the approach empirically. There, the primary outcome is average quarterly employment rate or earnings over 36 quarters; the surrogates are employment, earnings, and AFDC receipt in the first $S$2 quarters; and the observational sample consists of other sites while the experimental sample is the Riverside site. A surrogate index based on $S$3 quarters reproduces the 36-quarter ATE on employment, reported as approximately $S$4 percentage points, and does so about three years earlier. Standard errors shrink by roughly $S$5, with the example given as $S$6 percentage points to $S$7 percentage points. By contrast, naive short-window estimators remain biased until about $S$8, and tests for surrogacy and comparability indicate that $S$9 quarters suffice to remove statistically significant direct $Y$0–$Y$1 associations (Athey et al., 2016).

These results indicate that the surrogate index is not merely a dimensionality-reduction device. It is an identification-driven summary: the projection of a possibly high-dimensional surrogate vector onto the conditional expectation most relevant for the long-term outcome.

3. Experimentation and decision-making with auto-surrogate indices

In large-scale online experimentation, the surrogate index has been used as a decision tool rather than only as an ATE estimator. A Netflix study evaluates linear “auto-surrogate” models that use the shorter-term observations of the long-term outcome itself. For a user $Y$2, the target long-term quantity is

$Y$3

and the linear auto-surrogate predictor based on the first $Y$4 days is

$Y$5

The coefficients are estimated by OLS (Zhang et al., 2023).

The empirical setting contains 200 A/B tests and 1,098 non-control treatment arms in Netflix’s personalization space, with treatment arms and controls randomly assigned by “single-shot” allocation and typically tens or hundreds of thousands of users per arm. Two training regimes are considered for $Y$6: a pre-test regime that trains OLS on each user’s 63-day history immediately prior to allocation, and a similar-test regime that trains on a different but similar A/B test in the same product area (Zhang et al., 2023).

The decision rule maps each arm to $Y$7 or $Y$8, depending on whether the estimated treatment effect is positive and statistically significant at level $Y$9, negative and significant, or otherwise nonsignificant. The principal evaluation criterion is consistency,

$W$0

together with precision and recall for positive launches (Zhang et al., 2023).

Using 14 days of data to infer day-63 effects, the reported overall decision consistency is approximately $W$1. The surrogate produces nonsignificant reads more often, with $W$2 versus $W$3. Restricting attention to the set of tests that would be launched according to the 63-day directly measured treatment effects, the surrogate-based decision achieves precision of approximately $W$4 and recall of approximately $W$5. The report also states that there are no false-positive launches: the surrogate never indicates “launch” when the direct estimate is significantly negative. Under additivity and no marginal cost, shifting from 63-day to 14-day decisions could increase experiment capacity by up to $W$6, while $W$7 recall implies that about $W$8 more 14-day experiments would be needed to match the gains of the 63-day cycle (Zhang et al., 2023).

The assumptions are explicit: the short-term metrics must fully mediate the treatment effect on the long-term outcome, there must be no unmeasured confounders linking treatment, $W$9, and $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$0, and the linear model with i.i.d. errors must be correctly specified. The same study emphasizes model misspecification, training-data drift, and fat-tailed treatment-effect distributions as practical risks, and recommends periodic validation on holdout long-term tests (Zhang et al., 2023).

4. Failures of surrogacy, proximal identification, and sensitivity analysis

The classical surrogate-index approach is vulnerable when Prentice surrogacy fails. If $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$1, then the surrogate-index estimator converges not to the true ATE but to

$\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$2

where $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$3. The bias depends on the residual dependence of $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$4 on $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$5 given $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$6 and on $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$7. The same analysis notes that the bias is small if $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$8 explains little of the variation in $\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],$9 or if $P\in\{E,O\}$0 explains a large variation in $P\in\{E,O\}$1; for binary $P\in\{E,O\}$2, a sharp bound is obtained by replacing the residual term with its maximal possible value $P\in\{E,O\}$3 or $P\in\{E,O\}$4 (Athey et al., 2016).

The proximal surrogate index addresses a different failure mode: unobserved confounding. The data combine an experimental sample $P\in\{E,O\}$5, where $P\in\{E,O\}$6 are observed but $P\in\{E,O\}$7 is missing, with an observational sample $P\in\{E,O\}$8, where $P\in\{E,O\}$9 are observed but $\mu(s,x,E)=\mu(s,x,O)$0 is missing. An unobserved confounder $\mu(s,x,E)=\mu(s,x,O)$1 affects $\mu(s,x,E)=\mu(s,x,O)$2. The proxy variables satisfy

$\mu(s,x,E)=\mu(s,x,O)$3

and the target is

$\mu(s,x,E)=\mu(s,x,O)$4

Under no direct effect of $\mu(s,x,E)=\mu(s,x,O)$5 on $\mu(s,x,E)=\mu(s,x,O)$6 beyond $\mu(s,x,E)=\mu(s,x,O)$7, transportability, overlap, and completeness conditions, one seeks an outcome bridge $\mu(s,x,E)=\mu(s,x,O)$8 satisfying

$\mu(s,x,E)=\mu(s,x,O)$9

This function is called the proximal surrogate index. It identifies $\mu(s,x,O)$0 through

$\mu(s,x,O)$1

and hence identifies the experimental-population ATE (Hung et al., 25 Jan 2026).

The same framework derives a surrogate-bridge representation through functions $\mu(s,x,O)$2 and combines both views in a single doubly robust representation $\mu(s,x,O)$3. Estimation is multiply robust and cross-fitted: it estimates the experimental propensity $\mu(s,x,O)$4, the outcome bridge $\mu(s,x,O)$5, the regression of $\mu(s,x,O)$6 on $\mu(s,x,O)$7, and the surrogate bridges $\mu(s,x,O)$8, then averages held-out influence scores. Under stated $\mu(s,x,O)$9-error and product-rate conditions,

$S\in\mathbb R^d$00

with the empirical variance of the foldwise scores providing a consistent variance estimator (Hung et al., 25 Jan 2026).

In the Job Corps illustration, the treatment is program assignment; the long-term outcomes are 4-year weekly earnings and weeks employed; the surrogates are years 2–3 earnings and employment; and the proxies are GED possession $S\in\mathbb R^d$01 and English mother tongue $S\in\mathbb R^d$02, together with baseline $S\in\mathbb R^d$03. The reported finding is that naive surrogate-index methods underestimate the ATE, even when proxies are simply added as regressors, whereas the proximal estimator recovers the randomized benchmark, though with wider standard errors (Hung et al., 25 Jan 2026).

A complementary line of work treats surrogacy as a sensitivity parameter rather than a maintained assumption. The Weighted Surrogate Index for a known copula $S\in\mathbb R^d$04 is

$S\in\mathbb R^d$05

where $S\in\mathbb R^d$06. When $S\in\mathbb R^d$07 is the independence copula, $S\in\mathbb R^d$08 and $S\in\mathbb R^d$09, so the classical surrogate index is recovered. If the copula is known, the ATE on $S\in\mathbb R^d$10 is identified by the ATE on the WSI; if the copula is unknown but bounded in concordance order between $S\in\mathbb R^d$11 and $S\in\mathbb R^d$12, then the sharp identified set is $S\in\mathbb R^d$13. The Fréchet–Hoeffding bounds yield worst-case intervals, with the upper bound expressed through conditional $S\in\mathbb R^d$14 functionals (Fan et al., 28 Feb 2026).

This sensitivity framework also develops debiased, Neyman-orthogonal moments and cross-fitted estimators for both point-identified and partially identified cases. In the Pakistan poverty-alleviation application, even small departures from surrogacy, with $S\in\mathbb R^d$15, can render the long-term effect statistically significant and change its sign. The report further notes that the tight centering of $S\in\mathbb R^d$16 makes the results relatively insensitive to the copula family, so the dominant feature is the global strength of dependence summarized by Kendall’s tau (Fan et al., 28 Feb 2026).

5. Trial-level surrogate indices in high-dimensional meta-analysis

In multi-trial surrogate evaluation, the object of interest is not the individual-level imputation map $S\in\mathbb R^d$17 but trial-level surrogacy across studies. RISE-Meta addresses this problem when the candidate surrogates are high-dimensional, such as omics data. For each candidate marker $S\in\mathbb R^d$18 in trial $S\in\mathbb R^d$19, it computes a within-study nonparametric surrogacy estimator based on the indicator kernel

$S\in\mathbb R^d$20

Using treated–control pairs, it estimates the “area under the Mann–Whitney curve” for the primary endpoint and for the marker, denoted $S\in\mathbb R^d$21 and $S\in\mathbb R^d$22, and defines the trial-level bias

$S\in\mathbb R^d$23

If $S\in\mathbb R^d$24, the average superiority probability on the surrogate equals that on the primary endpoint, so the candidate is an unbiased trial-level surrogate (Hughes et al., 5 May 2026).

Across trials, RISE-Meta fits the hierarchical Gaussian model

$S\in\mathbb R^d$25

with $S\in\mathbb R^d$26 estimated by REML and inverse-variance weights $S\in\mathbb R^d$27. The pooled surrogate effect is

$S\in\mathbb R^d$28

and uncertainty is quantified by the Hartung–Knapp–Sidik–Jonkman variance estimator, yielding a $S\in\mathbb R^d$29 reference distribution (Hughes et al., 5 May 2026).

Operational validity is defined by equivalence, not by exact equality. For a pre-specified equivalence bound $S\in\mathbb R^d$30, trial-level surrogacy is declared when $S\in\mathbb R^d$31, tested through TOST: $S\in\mathbb R^d$32 Equivalence is declared at level $S\in\mathbb R^d$33 when the resulting $S\in\mathbb R^d$34-value satisfies $S\in\mathbb R^d$35, equivalently when the $S\in\mathbb R^d$36 confidence interval lies inside $S\in\mathbb R^d$37. When the number of markers $S\in\mathbb R^d$38 is large, $S\in\mathbb R^d$39-values are adjusted by Benjamini–Hochberg to give $S\in\mathbb R^d$40 (Hughes et al., 5 May 2026).

The framework then forms a composite surrogate signature from the promising markers $S\in\mathbb R^d$41. After standardizing each selected marker to $S\in\mathbb R^d$42, it defines

$S\in\mathbb R^d$43

and the weights

$S\in\mathbb R^d$44

The composite index is

$S\in\mathbb R^d$45

The paper states that, by construction, $S\in\mathbb R^d$46 concentrates information in markers with small $S\in\mathbb R^d$47 and low variance, improving overall surrogacy through $S\in\mathbb R^d$48 closer to zero, narrower confidence and prediction intervals, and higher concordance (Hughes et al., 5 May 2026).

The stated assumptions include independence of trials, i.i.d. sampling within trials, fully nonparametric within-study estimation, normality only for study-level $S\in\mathbb R^d$49, and clinically or power-motivated selection of $S\in\mathbb R^d$50. The paper notes that TOST is conservative when $S\in\mathbb R^d$51 is small, that Bayesian shrinkage for $S\in\mathbb R^d$52 could improve power, and that avoiding the surrogate paradox requires monotonicity and no residual treatment effect conditions. The method is reported to be directly applicable to both high-dimensional and classical low-dimensional meta-analyses, with agreement with the Buyse et al. approach when $S\in\mathbb R^d$53 is small; applications include gene expression as trial-level surrogate markers for antibody response to seasonal influenza vaccination (Hughes et al., 5 May 2026).

6. Other technical meanings and terminological distinctions

Outside causal-inference applications, the phrase “surrogate index” appears in surrogate modeling with a different meaning. In the Additive Multi-Index Gaussian process model, a surrogate index is a low-dimensional linear combination of a high-dimensional input $S\in\mathbb R^d$54,

$S\in\mathbb R^d$55

that captures one “mode” or “physics” of variation in the response. The surrogate model is additive,

$S\in\mathbb R^d$56

with independent GP priors on the $S\in\mathbb R^d$57, Laplace priors on entries of $S\in\mathbb R^d$58, and a variational approximation over inducing variables and index matrices. The stated purpose is to reduce dimensionality, avoid the curse of dimensionality of a full $S\in\mathbb R^d$59-dimensional GP, and separate distinct physical mechanisms. In the quark–gluon plasma application, the learned indices are reported to align with known physics, with one index selecting only the three initial-condition inputs and another selecting only viscosity-related parameters (Li et al., 2023).

In Multi-Index Stochastic Collocation, the surrogate index is not a scalar prediction map at all. It is an admissible downward-closed set $S\in\mathbb R^d$60 of multi-indices $S\in\mathbb R^d$61 that simultaneously tracks solver-fidelity levels and polynomial or interpolation levels. The MISC surrogate is

$S\in\mathbb R^d$62

and an adaptive profit-driven algorithm expands $S\in\mathbb R^d$63 while enforcing admissibility. The PlateauMISC refinement monitors the spectral envelope $S\in\mathbb R^d$64, detects solver-noise plateaux by a piecewise-linear fit in $S\in\mathbb R^d$65, and blocks refinements whose contributions lie entirely beyond the detected plateau degree. This use of “surrogate index” refers to the combinatorial scaffold of a multi-fidelity surrogate, not to a surrogate endpoint or a causal bridge (Kent et al., 4 Jul 2025).

These usages are structurally related only at a high level. In each case, the index is a summary object that organizes surrogate information, but the inferential target changes: long-term treatment effects in the classical and proximal formulations, trial-level surrogacy across studies in RISE-Meta, low-dimensional latent structure in additive GP surrogates, and admissible refinement structure in MISC. A plausible implication is that the term should always be interpreted relative to its surrounding methodological framework rather than as a domain-independent object.