---
title: Surrogate Index in Causal Inference
url: https://www.emergentmind.com/topics/surrogate-index
type: topic
---

# Surrogate Index in Causal Inference

A surrogate index is a construction that compresses surrogate information into an estimand, predictor, or composite marker used in place of an unavailable target. In causal inference, the canonical surrogate index is the conditional mean of a long-term outcome given short-term surrogates and pre-treatment covariates, estimated in one sample and transported to another [1603.09326]. Recent work extends this formulation to settings with unobserved confounding through the proximal surrogate index, defined as an outcome bridge involving proxy variables [2601.17712], and to sensitivity analysis through Weighted Surrogate Indices that relax full mediation via a conditional copula [2603.00580]. In high-dimensional clinical-trial meta-analysis, a surrogate index may instead be a rank-based composite signature formed from promising trial-level markers [2605.03819]. In surrogate modeling, the phrase is also used for low-dimensional projections or admissible multi-index sets that organize emulator structure and adaptive refinement [2306.07299], [2507.03691].

## 1. Classical surrogate index for long-term treatment effects

The foundational causal-inference formulation considers two samples. The experimental sample contains treatment \(W\in\{0,1\}\) and short-term surrogates \(S\in\mathbb R^d\) but no long-term outcome \(Y\), while the observational sample contains \(S\) and \(Y\) but no \(W\). The surrogate index is defined as
\[
\mu(s,x,p)\;:=\;E[Y\mid S=s,X=x,P=p],
\]
with \(P\in\{E,O\}\) denoting the sample indicator. Under comparability, \(\mu(s,x,E)=\mu(s,x,O)\), so \(\mu(s,x,O)\) estimated in the observational sample can be used to impute long-term outcomes in the experiment [1603.09326].

Identification relies on three core conditions. First, unconfoundedness in the experiment requires \((Y(0),Y(1),S(0),S(1))\perp\!\!\!\perp W\mid X,P=E\) with overlap. Second, Prentice-type surrogacy requires \(Y\perp\!\!\!\perp W\mid S,X,P=E\) together with overlap in \(\Pr[W=1\mid S,X,P=E]\). Third, comparability requires \(Y\perp\!\!\!\perp P\mid S,X\) and support inclusion of \((S,X\mid P=E)\) in \((S,X\mid P=O)\). These conditions imply that treatment affects the long-term outcome only through the observed surrogates once \((S,X)\) are held fixed, and that the conditional law of \(Y\) given \((S,X)\) is transportable across the two samples [1603.09326].

Under these assumptions, the average treatment effect in the experiment,
\[
\tau \equiv E[Y(1)-Y(0)\mid P=E],
\]
can be written as
\[
\tau
= E[\mu(S,X)\mid W=1,P=E]-E[\mu(S,X)\mid W=0,P=E].
\]
An equivalent inverse-probability-weighted representation is
\[
\tau = E\!\left[\frac{W\,\mu(S,X)}{e(X)}-\frac{(1-W)\,\mu(S,X)}{1-e(X)}\right],
\]
where \(e(x)=\Pr[W=1\mid X=x,P=E]\). Operationally, the estimator is therefore a difference in means on an imputed long-term outcome [1603.09326].

A central motivation is that multiple surrogates may collectively satisfy the statistical surrogacy criterion even when no individual proxy does so alone. This is particularly relevant in modern settings with large numbers of intermediate outcomes on or near the causal pathway between treatment and a long-term endpoint [1603.09326].

## 2. Estimation with multiple and high-dimensional surrogates

When \(S\) is high-dimensional, the surrogate index is estimated by regressing \(Y\) on \((S,X)\) in the observational sample. The methods explicitly listed for this step include penalized linear estimators such as Lasso, elastic net, and ridge; tree-based estimators such as random forests, boosted trees, and Bayesian additive regression trees; and “Super-Learner” ensembles. A generic procedure is: fit \(\hat\mu(s,x)\) in the observational sample, compute \(\hat Y_i=\hat\mu(S_i,X_i)\) in the experimental sample, and estimate the treatment effect by difference in means, weighted regression, or IPW with \(\hat e(X)\) [1603.09326].

Cross-fitting may be used to remove over-fitting bias. The influence-function approach yields a “double-robust” estimator that remains \(\sqrt n\)-consistent if either \(\mu\) or \(e\) is well-estimated. This places surrogate-index estimation within the general semiparametric program of nuisance-robust estimation with flexible first-stage learners [1603.09326].

The California GAIN application illustrates the approach empirically. There, the primary outcome is average quarterly employment rate or earnings over 36 quarters; the surrogates are employment, earnings, and AFDC receipt in the first \(t\) quarters; and the observational sample consists of other sites while the experimental sample is the Riverside site. A surrogate index based on \(t=6\) quarters reproduces the 36-quarter ATE on employment, reported as approximately \(6.4\) percentage points, and does so about three years earlier. Standard errors shrink by roughly \(30\text{–}40\%\), with the example given as \(1.2\) percentage points to \(0.7\) percentage points. By contrast, naive short-window estimators remain biased until about \(t=24\), and tests for surrogacy and comparability indicate that \(t=6\text{–}8\) quarters suffice to remove statistically significant direct \(W\)–\(Y\) associations [1603.09326].

These results indicate that the surrogate index is not merely a dimensionality-reduction device. It is an identification-driven summary: the projection of a possibly high-dimensional surrogate vector onto the conditional expectation most relevant for the long-term outcome.

## 3. Experimentation and decision-making with auto-surrogate indices

In large-scale online experimentation, the surrogate index has been used as a decision tool rather than only as an ATE estimator. A Netflix study evaluates linear “auto-surrogate” models that use the shorter-term observations of the long-term outcome itself. For a user \(i\), the target long-term quantity is
\[
\mu_i=\frac{1}{63}\sum_{t=1}^{63}Y_{it},
\]
and the linear auto-surrogate predictor based on the first \(T\) days is
\[
\hat\mu_i=\hat\beta_0+\sum_{t=1}^T\hat\beta_t\,Y_{it},
\qquad
\hat\tau_T
=\frac1{|N_a|}\sum_{i\in N_a}\hat\mu_i-\frac1{|N_0|}\sum_{i\in N_0}\hat\mu_i.
\]
The coefficients are estimated by OLS [2311.11922].

The empirical setting contains 200 A/B tests and 1,098 non-control treatment arms in Netflix’s personalization space, with treatment arms and controls randomly assigned by “single-shot” allocation and typically tens or hundreds of thousands of users per arm. Two training regimes are considered for \(T=14\): a pre-test regime that trains OLS on each user’s 63-day history immediately prior to allocation, and a similar-test regime that trains on a different but similar A/B test in the same product area [2311.11922].

The decision rule maps each arm to \(d_{63}\in\{+1,0,-1\}\) or \(d_T\in\{+1,0,-1\}\), depending on whether the estimated treatment effect is positive and statistically significant at level \(\alpha=0.05\), negative and significant, or otherwise nonsignificant. The principal evaluation criterion is consistency,
\[
\mathbb P(d_T=d_{63}),
\]
together with precision and recall for positive launches [2311.11922].

Using 14 days of data to infer day-63 effects, the reported overall decision consistency is approximately \(95\%\). The surrogate produces nonsignificant reads more often, with \(86.5\%\) versus \(79\%\). Restricting attention to the set of tests that would be launched according to the 63-day directly measured treatment effects, the surrogate-based decision achieves precision of approximately \(79\%\) and recall of approximately \(65\%\). The report also states that there are no false-positive launches: the surrogate never indicates “launch” when the direct estimate is significantly negative. Under additivity and no marginal cost, shifting from 63-day to 14-day decisions could increase experiment capacity by up to \(300\%\), while \(65\%\) recall implies that about \(53\%\) more 14-day experiments would be needed to match the gains of the 63-day cycle [2311.11922].

The assumptions are explicit: the short-term metrics must fully mediate the treatment effect on the long-term outcome, there must be no unmeasured confounders linking treatment, \(Y_{it}\), and \(\mu_i\), and the linear model with i.i.d. errors must be correctly specified. The same study emphasizes model misspecification, training-data drift, and fat-tailed treatment-effect distributions as practical risks, and recommends periodic validation on holdout long-term tests [2311.11922].

## 4. Failures of surrogacy, proximal identification, and sensitivity analysis

The classical surrogate-index approach is vulnerable when Prentice surrogacy fails. If \(Y\not\!\perp\!\!\!\perp W\mid S,X,P=E\), then the surrogate-index estimator converges not to the true ATE but to
\[
\tau^{SI}=E[\mu(S,X,1,E)-\mu(S,X,0,E)],
\]
where \(\mu(s,x,w,E)=E[Y\mid S=s,X=x,W=w,P=E]\). The bias depends on the residual dependence of \(Y\) on \(W\) given \((S,X)\) and on \(r(s,x)=\Pr[W=1\mid S=s,X=x,P=E]\). The same analysis notes that the bias is small if \(W\) explains little of the variation in \(S\) or if \(S\) explains a large variation in \(Y\); for binary \(Y\), a sharp bound is obtained by replacing the residual term with its maximal possible value \(1\) or \(-1\) [1603.09326].

The proximal surrogate index addresses a different failure mode: unobserved confounding. The data combine an experimental sample \(G=E\), where \(A,S,X,W\) are observed but \(Y\) is missing, with an observational sample \(G=O\), where \(Y,S,X,Z,W\) are observed but \(A\) is missing. An unobserved confounder \(U\) affects \((A,S,Y)\). The proxy variables satisfy
\[
Z \perp\!\!\!\perp Y\mid(U,S,X,G),\qquad
W \perp\!\!\!\perp (A,S,Z)\mid(U,X,G),
\]
and the target is
\[
\tau_0=E[Y(1)-Y(0)\mid G=E].
\]
Under no direct effect of \(A\) on \(Y\) beyond \(S\), transportability, overlap, and completeness conditions, one seeks an outcome bridge \(h_0(W,S,X)\) satisfying
\[
E[Y\mid Z,S,X,G=O]=E[h_0(W,S,X)\mid Z,S,X,G=O].
\]
This function is called the proximal surrogate index. It identifies \(E[Y(a)\mid X,G=E]\) through
\[
E[Y(a)\mid X,G=E]=E[h_0(W,S,X)\mid A=a,X,G=E],
\]
and hence identifies the experimental-population ATE [2601.17712].

The same framework derives a surrogate-bridge representation through functions \(q_{a,0}(Z,S,X)\) and combines both views in a single doubly robust representation \(E[\phi(D;\eta)]=\tau_0\). Estimation is multiply robust and cross-fitted: it estimates the experimental propensity \(e(X)\), the outcome bridge \(h\), the regression of \(h\) on \((A,X)\), and the surrogate bridges \(q_0,q_1\), then averages held-out influence scores. Under stated \(L_2\)-error and product-rate conditions,
\[
\sqrt N(\hat\tau_{MR}-\tau_0)\to^d N(0,V_{eff}),
\]
with the empirical variance of the foldwise scores providing a consistent variance estimator [2601.17712].

In the Job Corps illustration, the treatment is program assignment; the long-term outcomes are 4-year weekly earnings and weeks employed; the surrogates are years 2–3 earnings and employment; and the proxies are GED possession \(W\) and English mother tongue \(Z\), together with baseline \(X\). The reported finding is that naive surrogate-index methods underestimate the ATE, even when proxies are simply added as regressors, whereas the proximal estimator recovers the randomized benchmark, though with wider standard errors [2601.17712].

A complementary line of work treats surrogacy as a sensitivity parameter rather than a maintained assumption. The Weighted Surrogate Index for a known copula \(C_o\) is
\[
\mu_{C_o,w}(s,x)
=\int_0^1 F_Y^{-1}(u\mid s,x)\,
\sigma_{C_o,w}\!\bigl(u;\alpha(s,x)\bigr)\,du,
\qquad
\alpha(s,x)=1-\rho(s,x),
\]
where \(\rho(s,x)=\Pr(W=1\mid S=s,X=x)\). When \(C_o\) is the independence copula, \(\sigma_{C_o,w}\equiv 1\) and \(\mu_{C_o,w}=\mu\), so the classical surrogate index is recovered. If the copula is known, the ATE on \(Y\) is identified by the ATE on the WSI; if the copula is unknown but bounded in concordance order between \(C_L\) and \(C_U\), then the sharp identified set is \([\tau_{C_L},\tau_{C_U}]\). The Fréchet–Hoeffding bounds yield worst-case intervals, with the upper bound expressed through conditional \(AVaR\) functionals [2603.00580].

This sensitivity framework also develops debiased, Neyman-orthogonal moments and cross-fitted estimators for both point-identified and partially identified cases. In the Pakistan poverty-alleviation application, even small departures from surrogacy, with \(|\varrho_K|\approx .03\), can render the long-term effect statistically significant and change its sign. The report further notes that the tight centering of \(\rho(s,x)\approx .5\) makes the results relatively insensitive to the copula family, so the dominant feature is the global strength of dependence summarized by Kendall’s tau [2603.00580].

## 5. Trial-level surrogate indices in high-dimensional meta-analysis

In multi-trial surrogate evaluation, the object of interest is not the individual-level imputation map \(E[Y\mid S,X]\) but trial-level surrogacy across studies. RISE-Meta addresses this problem when the candidate surrogates are high-dimensional, such as omics data. For each candidate marker \(S_j\) in trial \(m\), it computes a within-study nonparametric surrogacy estimator based on the indicator kernel
\[
G(x,y)=1_{x>y}+\tfrac12\,1_{x=y}.
\]
Using treated–control pairs, it estimates the “area under the Mann–Whitney curve” for the primary endpoint and for the marker, denoted \(\hat U_{Y,m}\) and \(\hat U_{S,j,m}\), and defines the trial-level bias
\[
\hat\delta_{j,m}=\hat U_{Y,m}-\hat U_{S,j,m}.
\]
If \(\delta_{j,m}=0\), the average superiority probability on the surrogate equals that on the primary endpoint, so the candidate is an unbiased trial-level surrogate [2605.03819].

Across trials, RISE-Meta fits the hierarchical Gaussian model
\[
\delta_{j,m}\sim N(\mu_j,\tau_j^2),\qquad
\hat\delta_{j,m}\mid \delta_{j,m}\sim N(\delta_{j,m},\sigma_{j,m}^2),
\]
with \(\tau_j^2\) estimated by REML and inverse-variance weights \(w_{j,m}=(\sigma_{j,m}^2+\tau_j^2)^{-1}\). The pooled surrogate effect is
\[
\hat\mu_j=
\frac{\sum_{m=1}^M w_{j,m}\hat\delta_{j,m}}
{\sum_{m=1}^M w_{j,m}},
\]
and uncertainty is quantified by the Hartung–Knapp–Sidik–Jonkman variance estimator, yielding a \(t_{M-1}\) reference distribution [2605.03819].

Operational validity is defined by equivalence, not by exact equality. For a pre-specified equivalence bound \(\epsilon>0\), trial-level surrogacy is declared when \(\mu_j\in(-\epsilon,\epsilon)\), tested through TOST:
\[
H_0^L:\mu_j\le -\epsilon,\qquad
H_0^U:\mu_j\ge +\epsilon.
\]
Equivalence is declared at level \(\alpha\) when the resulting \(p\)-value satisfies \(p_j<\alpha\), equivalently when the \((1-2\alpha)\cdot 100\%\) confidence interval lies inside \((-\epsilon,\epsilon)\). When the number of markers \(J\) is large, \(p\)-values are adjusted by Benjamini–Hochberg to give \(p_j^*\) [2605.03819].

The framework then forms a composite surrogate signature from the promising markers \(\Gamma=\{j:p_j^*<\alpha\}\). After standardizing each selected marker to \(\bar S_j\), it defines
\[
a_j=(\epsilon-|\hat\mu_j|)/\epsilon \quad \text{("strength")},
\qquad
b_j=\sum_{m=1}^M w_{j,m}\quad \text{("precision")},
\]
and the weights
\[
\hat\lambda_j=\frac{a_j b_j}{\max_{k\in\Gamma}(a_k b_k)}.
\]
The composite index is
\[
\gamma_\Gamma=\sum_{j\in\Gamma}\hat\lambda_j \bar S_j.
\]
The paper states that, by construction, \(\gamma_\Gamma\) concentrates information in markers with small \(|\mu_j|\) and low variance, improving overall surrogacy through \(|\mu_\gamma|\) closer to zero, narrower confidence and prediction intervals, and higher concordance [2605.03819].

The stated assumptions include independence of trials, i.i.d. sampling within trials, fully nonparametric within-study estimation, normality only for study-level \(\hat\delta_{j,m}\), and clinically or power-motivated selection of \(\epsilon\). The paper notes that TOST is conservative when \(M\) is small, that Bayesian shrinkage for \(\tau_j^2\) could improve power, and that avoiding the surrogate paradox requires monotonicity and no residual treatment effect conditions. The method is reported to be directly applicable to both high-dimensional and classical low-dimensional meta-analyses, with agreement with the Buyse et al. approach when \(J\) is small; applications include gene expression as trial-level surrogate markers for antibody response to seasonal influenza vaccination [2605.03819].

## 6. Other technical meanings and terminological distinctions

Outside causal-inference applications, the phrase “surrogate index” appears in surrogate modeling with a different meaning. In the Additive Multi-Index Gaussian process model, a surrogate index is a low-dimensional linear combination of a high-dimensional input \(x\in\mathbb R^D\),
\[
z_m=A_m^\top x,\qquad m=1,\dots,M,
\]
that captures one “mode” or “physics” of variation in the response. The surrogate model is additive,
\[
\hat f(x)=\sum_{m=1}^M g_m(A_m^\top x),
\]
with independent GP priors on the \(g_m\), Laplace priors on entries of \(A_m\), and a variational approximation over inducing variables and index matrices. The stated purpose is to reduce dimensionality, avoid the curse of dimensionality of a full \(D\)-dimensional GP, and separate distinct physical mechanisms. In the quark–gluon plasma application, the learned indices are reported to align with known physics, with one index selecting only the three initial-condition inputs and another selecting only viscosity-related parameters [2306.07299].

In Multi-Index Stochastic Collocation, the surrogate index is not a scalar prediction map at all. It is an admissible downward-closed set \(I\subset\mathbb N^d\) of multi-indices \(\vec i=(i_1,\dots,i_d)\) that simultaneously tracks solver-fidelity levels and polynomial or interpolation levels. The MISC surrogate is
\[
u_I(\vec y)=\sum_{\vec i\in I}\Delta^{\vec i}[u_h^{\vec m(\vec p)}](\vec y),
\]
and an adaptive profit-driven algorithm expands \(I\) while enforcing admissibility. The PlateauMISC refinement monitors the spectral envelope \(e(\ell)\), detects solver-noise plateaux by a piecewise-linear fit in \((\ell,\log_{10} e(\ell))\), and blocks refinements whose contributions lie entirely beyond the detected plateau degree. This use of “surrogate index” refers to the combinatorial scaffold of a multi-fidelity surrogate, not to a surrogate endpoint or a causal bridge [2507.03691].

These usages are structurally related only at a high level. In each case, the index is a summary object that organizes surrogate information, but the inferential target changes: long-term treatment effects in the classical and proximal formulations, trial-level surrogacy across studies in RISE-Meta, low-dimensional latent structure in additive GP surrogates, and admissible refinement structure in MISC. A plausible implication is that the term should always be interpreted relative to its surrounding methodological framework rather than as a domain-independent object.

Source: https://www.emergentmind.com/topics/surrogate-index