---
title: Triple Lasso Estimator
url: https://www.emergentmind.com/topics/triple-lasso-estimator
type: topic
---

# Triple Lasso Estimator

Searching arXiv for the specified papers to ground the article in the cited literature.
arxiv_search(query="2403.03240 Triple/Debiased Lasso for Statistical Inference of Conditional Average Treatment Effects", max_results=5)
The **Triple Lasso estimator** denotes two related high-dimensional inference constructions that combine Lasso-based nuisance estimation with explicit bias correction. In "Triple/Debiased Lasso for Statistical Inference of Conditional Average Treatment Effects" [2403.03240], the term refers to estimation and inference for conditional average treatment effects under a sparse-differences structure, using doubly robust orthogonalization, cross-fitting, and debiased Lasso. In "Triple/Double-Debiased Lasso" [2603.20134], the term refers to inference on a low-dimensional parameter in a high-dimensional linear regression model using a moment function that satisfies both first- and second-order Neyman orthogonality conditions.

## 1. Terminological scope and distinguishing uses

The label has two distinct uses in the recent literature. One use is **causal**: it targets
\[
\tau(x)=E[Y(1)-Y(0)\mid X=x]
\]
in observational settings with binary treatment and high-dimensional covariates. The other is **regression-based**: it targets a scalar coefficient \(\beta_0\) in a high-dimensional linear model with many controls. In both uses, the central idea is to combine orthogonalization and desparsification so that valid inference remains feasible when \(p \gg n\) [2403.03240] [2603.20134].

| Formulation | Target | Meaning of “triple” |
|---|---|---|
| Triple/Debiased Lasso | CATE \( \tau(x)=x'\beta_\Delta \) | Doubly robust orthogonalization, cross-fitting DML, and debiased Lasso |
| Triple/Double-Debiased Lasso | Low-dimensional \(\beta_0\) | First-order debiasing plus a second-order correction term |

A recurring source of confusion is the relation to **“triple selection” Lasso**. The causal paper states that “Triple/Debiased Lasso” refers to combining **(1) doubly robust orthogonalization, (2) cross-fitting (DML), and (3) debiased Lasso**, and that it differs from “triple selection” terminology in other contexts [2403.03240]. The 2026 paper uses “triple” and “double-debiased” interchangeably to emphasize elimination of both leading and second-order regularization bias [2603.20134].

## 2. CATE formulation: sparse differences and orthogonal pseudo-outcomes

In the CATE formulation, observations are i.i.d. copies of \((Y,D,X)\), with binary treatment \(D\in\{0,1\}\), real-valued outcome \(Y\), and \(p\)-dimensional covariates \(X\in\mathbb{R}^p\). Potential outcomes \(Y(1),Y(0)\) are bounded, and the observed outcome satisfies
\[
Y=D\cdot Y(1)+(1-D)\cdot Y(0).
\]
Identification uses **unconfoundedness**,
\[
(Y(1),Y(0))\perp D\mid X,
\]
and **overlap**, requiring \(\phi<P(D=1\mid X)<1-\phi\) almost surely for some \(\phi\in(0,1/2)\) [2403.03240].

With propensity score \(\pi(X)=P(D=1\mid X)\) and outcome regressions \(m_d(X)=E[Y\mid D=d,X]\), the target is
\[
\tau(x)=E[Y(1)-Y(0)\mid X=x].
\]
Under the linear high-dimensional outcome model \(E[Y\mid D=d,X]=X'\beta_d\), the CATE becomes
\[
\tau(x)=x'\beta_\Delta,\qquad \beta_\Delta:=\beta_1-\beta_0.
\]
The key structural assumption is **sparsity-in-differences**: \(\beta_\Delta\) is sparse or approximately sparse even if \(\beta_0\) and \(\beta_1\) are individually dense. This is the condition that enables high-dimensional inference for \(\tau(x)\) under \(p\gg n\) [2403.03240].

The construction begins from the AIPW or doubly robust pseudo-outcome
\[
\phi_i=m_1(X_i)-m_0(X_i)+\frac{D_i\,[Y_i-m_1(X_i)]}{\pi(X_i)}-\frac{(1-D_i)\,[Y_i-m_0(X_i)]}{1-\pi(X_i)}.
\]
Its conditional mean equals the CATE if either \(m_d\) is correctly specified or \(\pi\) is correctly specified. The same pseudo-outcome is Neyman orthogonal in the sense that the Gateaux derivative of \(E[\phi_i-\tau(X_i)]\) with respect to small perturbations of the nuisance functions is zero at the truth. The paper states that this orthogonality mitigates the impact of first-stage estimation errors and enables \(\sqrt{n}\)-inference under mild product-rate conditions via cross-fitting [2403.03240].

## 3. Weighted DR-Lasso and the debiasing step for CATE inference

Under the linear CATE specification, the pseudo-outcome is projected onto the covariates:
\[
\phi_i\approx X_i'\theta,\qquad \theta=\beta_\Delta.
\]
An unweighted Lasso estimator is defined by
\[
\hat{\theta}^{lasso}=\arg\min_{\theta}\frac{1}{n}\sum_{i=1}^n(\phi_i-X_i'\theta)^2+\lambda\|\theta\|_1,
\]
with theoretical tuning \(\lambda\asymp c\sqrt{\log p/n}\), while cross-validation is common in practice. Estimation relies on restricted eigenvalue or compatibility conditions on \(E[XX']\) or its sample analog [2403.03240].

Because the doubly robust pseudo-outcome has covariate-dependent conditional variance,
\[
\sigma^2_{\bar\varepsilon}(x)=\frac{\sigma^2_{\varepsilon}(1)(x)}{\pi(x)}+\frac{\sigma^2_{\varepsilon}(0)(x)}{1-\pi(x)},
\]
the method introduces a weighted least squares refinement. Both \(\phi_i\) and \(X_i\) are scaled by \(1/\hat{\sigma}_{\bar\varepsilon}(X_i)\), producing the weighted design
\[
W_i:=\frac{X_i}{\hat{\sigma}_{\bar\varepsilon}(X_i)},\qquad \tilde y_i:=\frac{\hat\phi_i}{\hat{\sigma}_{\bar\varepsilon}(X_i)}.
\]
The weighted DR-Lasso estimator is
\[
\hat\beta^{WDML}\in\arg\min_{\beta}\frac{1}{n}\sum_{i=1}^n\big(\tilde y_i-W_i'\beta\big)^2+\lambda\|\beta\|_1.
\]

Cross-fitting is essential. The sample is split into \(K\) folds; for each fold, nuisance estimators \((\hat m_0,\hat m_1,\hat\pi)\) are fit on the complement data, and out-of-fold pseudo-outcomes are computed:
\[
\hat\phi_i=\hat m_{1,-\ell(i)}(X_i)-\hat m_{0,-\ell(i)}(X_i)+\frac{D_i\,[Y_i-\hat m_{1,-\ell(i)}(X_i)]}{\hat\pi_{-\ell(i)}(X_i)}-\frac{(1-D_i)\,[Y_i-\hat m_{0,-\ell(i)}(X_i)]}{1-\hat\pi_{-\ell(i)}(X_i)}.
\]

The final desparsification step uses an approximate inverse \(M\approx \hat\Sigma^{-1}\), where \(\hat\Sigma=(1/n)W'W\), and forms
\[
\tilde\beta=\hat\beta^{WDML}+M\cdot \frac{1}{n}\sum_{i=1}^nW_i\big(\tilde y_i-W_i'\hat\beta^{WDML}\big).
\]
In practice, \(M\) is built by nodewise Lasso: for each column \(j\), \(W_j\) is regressed on \(W_{-j}\), producing \(\hat\gamma_j\) and \(\hat\tau_j^2\), and then \(M=\hat T^{-2}\hat C\) with \(\hat T^{-2}=\mathrm{diag}(\hat\tau_1^2,\dots,\hat\tau_p^2)\) and \(\hat C\) having \(1\) on the diagonal and \(-\hat\gamma_j\) off-diagonals. The paper terms the resulting estimator **Triple/Debiased Lasso** because it combines **(i) doubly robust orthogonalization (AIPW), (ii) cross-fitting DML, and (iii) debiased Lasso** [2403.03240].

## 4. CATE theory, inference, practice, and extensions

The CATE paper states consistency of the weighted DR-Lasso under high-dimensional design conditions, overlap, bounded potential outcomes, and nuisance estimates that are \(L_2\)-consistent and satisfy the standard DML product-rate condition
\[
\|\hat m_d-m_d\|_2\to 0,\qquad \|\hat\pi-\pi\|_2\to 0,\qquad \|\hat m_d-m_d\|_2\cdot \|\hat\pi-\pi\|_2=o_P(n^{-1/2}).
\]
When \(s_0=|\mathrm{supp}(\beta_\Delta)|\) satisfies \(s_0\log p/n\to 0\), the weighted DR-Lasso has \(\ell_1\)-error and prediction error bounds of the stated \(o_P\)-orders. The debiased estimator
\[
\hat b^{WTDL}:=\hat\beta^{WDML}+\hat\Theta\,\frac{1}{n}W'(\tilde y-W\hat\beta^{WDML})
\]
is asymptotically linear, and the paper gives coordinate-wise asymptotic normality as well as inference for linear functionals and pointwise CATEs [2403.03240].

For any \(x\in\mathbb{R}^p\), the estimator and plug-in variance are
\[
\hat\tau(x)=x'\hat b^{WTDL},\qquad \hat V(x)=x'(\hat\Theta\,\hat\Sigma\,\hat\Theta')x,
\]
which yield the pointwise confidence interval
\[
CI_{1-\alpha}(x):\ \hat\tau(x)\pm z_{1-\alpha/2}\sqrt{\hat V(x)/n}.
\]
The paper summarizes the main takeaway as follows: under standard high-dimensional and DML conditions—compatibility, sub-Gaussian design, overlap, and the nuisance product-rate condition—TDL is \(\sqrt{n}\)-consistent and asymptotically normal for coordinates of \(\beta_\Delta\) and for linear functionals such as \(\tau(x)=x'\beta_\Delta\) [2403.03240].

Practical guidance is stated in operational terms. For cross-fitting, **\(K=5\) or \(10\) folds** are suggested, and repeat-splitting and averaging can stabilize. Cross-validation is common for \(\lambda\) and \(\lambda_j\), although \(\lambda\asymp c\sqrt{\log p/n}\) is a default. Standardizing \(X\), checking overlap through the distribution of \(\hat\pi(X)\), monitoring AIPW residuals, and comparing weighted with unweighted variants are recommended. The weighted step is described as improving efficiency but requiring estimation of \(\hat\sigma^2_{\bar\varepsilon}\); if unstable, the paper suggests starting with unweighted TDL and comparing [2403.03240].

The reported simulations vary \(n\), \(p\), and sparsity \(s_0\), with \(\beta_\Delta\) sparse while \(\beta_0\) and \(\beta_1\) need not be. The key findings are that **WDML-CATELasso is consistent**, **WTDL achieves near-nominal coverage for coordinates of \(\beta_\Delta\) and for linear functionals \(x'\beta_\Delta\)**, and weighting by \(\hat\sigma^2_{\bar\varepsilon}\) improves efficiency through **smaller standard errors** relative to unweighted debiased Lasso on \(\hat\phi\). The same source also states that WTDL has **reduced RMSE relative to unweighted variants and DR-learners that ignore heteroskedasticity** [2403.03240].

The framework is also presented as extensible. The paper explicitly discusses replacing \(X\) by a high-dimensional feature map \(g(X)\) for nonlinear CATEs, generalizing the orthogonal score to multi-valued or continuous treatments, building analogous scores for binary or count outcomes, and using group Lasso or structured penalties for structured sparsity. These are presented as extensions for which formal theory would require corresponding conditions [2403.03240].

## 5. Triple/Double-Debiased Lasso in high-dimensional linear regression

The 2026 formulation considers the high-dimensional linear regression model
\[
Y=D\,\beta_0+X^\top\theta_0+\varepsilon,\qquad \mathbb{E}[\varepsilon\mid D,X]=0,
\]
where \(Y\in\mathbb{R}\), \(D\in\mathbb{R}\) is the regressor of interest, \(X\in\mathbb{R}^p\) are controls with \(p\gg n\) allowed, \(\beta_0\in\mathbb{R}\) is the target parameter, and \(\theta_0\) is an approximately sparse nuisance vector. It also uses the first-stage and reduced-form regressions
\[
D=X^\top\gamma_0+\nu,\qquad \mathbb{E}[\nu X]=0_p,
\]
\[
Y=X^\top\phi_0+e,\qquad \mathbb{E}[eX]=0_p,
\]
with \(\phi_0:=\theta_0+\beta_0\gamma_0\) and \(e:=\varepsilon+\beta_0\nu\) [2603.20134].

The benchmark **double Lasso** score is
\[
\psi^{DL}(\beta,\phi,\gamma):=\mathbb{E}\Big[(Y-X^\top\phi-\beta(D-X^\top\gamma))(D-X^\top\gamma)\Big].
\]
At the truth, this moment satisfies first-order orthogonality with respect to \(\phi\) and \(\gamma\), but not second-order orthogonality. The triple or double-debiased construction adds an adjustment term,
\[
\psi^{ADJ}(\beta,\phi,\gamma,\Theta):=\mathbb{E}\Big[(Y-X^\top\phi-\beta(D-X^\top\gamma))X^\top\,\Theta\,\tilde X\,(\tilde D-\tilde X^\top\gamma)\Big],
\]
where \((\tilde X,\tilde D)\) is an independent copy of \((X,D)\). The resulting moment is
\[
\psi^{TL}(\beta,\eta):=\psi^{DL}(\beta,\phi,\gamma)-\psi^{ADJ}(\beta,\phi,\gamma,\Theta),
\qquad \eta:=(\phi^\top,\gamma^\top,\mathrm{vec}(\Theta)^\top)^\top.
\]
Using independence, the paper rewrites this as
\[
\psi^{TL}(\beta,\eta)
=\mathbb{E}[(Y-X^\top\phi-\beta(D-X^\top\gamma))(D-X^\top\gamma)]
-\mathbb{E}[(Y-X^\top\phi)X^\top]\Theta\,\mathbb{E}[X(D-X^\top\gamma)].
\]

The central lemma states that if \(\Sigma_0=\mathbb{E}[XX^\top]\) is invertible and \(Y,D,X\) have finite second moments, then
\[
\psi^{TL}(\beta_0,\eta_0)=0,\qquad
\nabla_\eta \psi^{TL}(\beta_0,\eta)\big|_{\eta=\eta_0}=0,\qquad
\nabla^2_{\eta\eta^\top}\psi^{TL}(\beta_0,\eta)\big|_{\eta=\eta_0}=0.
\]
Thus, \(\psi^{TL}\) is both first- and second-order Neyman orthogonal at the truth. The paper presents this as the mechanism that eliminates both the leading bias and the second-order bias induced by regularization [2603.20134].

Estimation uses cross-fitting and Lasso throughout. In each fold, \(\phi_0\) and \(\gamma_0\) are estimated by Lasso on the training sample; \(\Theta_0=\Sigma_0^{-1}\) is estimated row-by-row by node-wise Lasso; and the node-wise matrix is then **row-sparsified** by zeroing rows not selected by the first stage. On the holdout fold, residuals
\[
u_i:=Y_i-X_i^\top\hat\phi_k,\qquad v_i:=D_i-X_i^\top\hat\gamma_k
\]
are used to construct fold-specific quantities
\[
\hat A_k:=\mathbb{E}_{I(k)}[u_i v_i]-\mathbb{E}_{I(k)}[u_i X_i^\top]\hat\Theta_k\,\mathbb{E}_{I(k)}[X_i v_i],
\]
\[
\hat B_k:=\mathbb{E}_{I(k)}[v_i^2]-\mathbb{E}_{I(k)}[v_i X_i^\top]\hat\Theta_k\,\mathbb{E}_{I(k)}[X_i v_i],
\]
yielding the closed-form fold estimator
\[
\hat\beta_k=\frac{\hat A_k}{\hat B_k},
\qquad
\hat\beta^{TL}=\frac{1}{K}\sum_{k=1}^K\hat\beta_k.
\]
The paper recommends penalties of order \(\lambda\asymp c\sqrt{\log p/n}\), small fixed \(K\), and emphasizes row-sparsification of \(\hat\Theta_k\) for variance control in high dimension [2603.20134].

## 6. Higher-order orthogonality, asymptotics, simulations, and relation to adjacent methods

For the 2026 formulation, the main asymptotic result is an asymptotic linear representation in which the remainder terms are **never larger and are often smaller in order** than the corresponding remainder for cross-fitted double Lasso. Under bounded fourth moments for \(\varepsilon\) and \(\nu\), invertibility of \(\Sigma_0\), sparse eigenvalue conditions, and standard approximate sparsity and node-wise Lasso rate conditions, the paper shows
\[
\sqrt{n}(\hat\beta^{TL}-\beta_0)\to_d N(0,V),\qquad
V=\frac{\mathbb{E}[(\varepsilon\nu)^2]}{(\mathbb{E}[\nu^2])^2}.
\]
A natural variance estimator uses fold-specific residuals
\[
\hat\nu_{ik}:=D_i-X_i^\top\hat\gamma_k,\qquad
\hat\theta_k:=\hat\phi_k-\hat\beta_k\hat\gamma_k,
\]
\[
\hat\varepsilon_{ik}:=Y_i-D_i\hat\beta_k-X_i^\top\hat\theta_k,
\]
and constructs
\[
\hat V_k:=\frac{\mathbb{E}_{I(k)}[(\hat\varepsilon_{ik}\hat\nu_{ik})^2]}{(\mathbb{E}_{I(k)}[\hat\nu_{ik}^2])^2},
\qquad
\hat\sigma^2:=\frac{1}{K}\sum_{k=1}^K\hat V_k,
\]
leading to
\[
\mathrm{CI}_{1-\alpha}:\ \hat\beta^{TL}\pm z_{1-\alpha/2}\frac{\hat\sigma}{\sqrt n}.
\]
The same source contrasts this with cross-fitted double Lasso, whose remainder is of order \(\sqrt{s_\gamma s_\phi \log p/n}\), and states that the triple Lasso remainder terms are often strictly smaller [2603.20134].

The Monte Carlo study uses Gaussian designs with \(X\sim N(0,\Sigma_0(\rho))\), Toeplitz covariance, \(p=n/2\), \(n\in\{500,1000,2000\}\), \(\rho\in\{0,0.2,0.4,0.6,0.8\}\), approximately sparse \(\theta_0\), and both exact and approximate sparsity designs for \(\gamma_0\). Performance metrics are **squared bias, variance, MSE, empirical 95\% CI coverage, CI length, and kernel densities of studentized statistics**. The reported numerical comparisons are explicit: with \((n,p,\rho)=(1000,500,0)\) under approximate sparsity, squared bias drops from **0.00216** under double Lasso to **0.00025** under triple Lasso; with \((2000,1000,0)\), it drops from **0.00082** to **0.00009**. In the same difficult design \((1000,500,0)\) under approximate sparsity, empirical coverage rises from **0.672** to **0.923**, with only slightly longer intervals. The paper summarizes these results as showing that studentized statistics under triple Lasso are better centered and closer to \(N(0,1)\), consistent with reduced second-order bias [2603.20134].

The 2026 paper further embeds the construction in a **general recursive formula for constructing higher-order Neyman orthogonal moment functions in Z-estimation problems**. For \(k=2\), the recursion produces
\[
\tilde F(\beta,\tilde\eta)=F(\beta,\eta)-\frac{1}{2}U(\beta,\eta)^\top B\,U(\beta,\eta),
\]
and the paper states that the triple Lasso score is the \(k=2\) instance of this general construction. This places the method in a broader line of work on orthogonal scores, double/debiased Lasso, double machine learning, and higher-order influence-function methods, while retaining a specifically regression-based construction with ordinary derivatives [2603.20134].

Two objective clarifications follow from the paired literature. First, **triple lasso is not a single estimator with one universal definition**; the phrase identifies related but non-identical estimators. Second, in both uses the added “triple” layer is a bias-control device beyond standard first-order orthogonalization: in the CATE setting it is the combination of AIPW orthogonalization, cross-fitting, and debiased Lasso, whereas in the low-dimensional linear regression setting it is a second-order orthogonal correction to the standard double Lasso score [2403.03240] [2603.20134].

Source: https://www.emergentmind.com/topics/triple-lasso-estimator