---
title: Post-Double-Selection LASSO
url: https://www.emergentmind.com/topics/post-double-selection-lasso-pds-lasso
type: topic
---

# Post-Double-Selection LASSO

Post-Double-Selection LASSO (PDS-LASSO) is a variable-selection and estimation framework designed for high-dimensional linear models, particularly when inference about a low-dimensional parameter is required in the presence of potentially many nuisance covariates. The method augments standard LASSO selection by conducting variable selection for both the outcome and the regressor of interest, followed by unpenalized estimation on the union of selected controls. This double selection, paired with Neyman-orthogonal moments, yields √n-consistent, asymptotically normal inference and mitigates omitted variable bias in settings where the number of covariates may exceed the number of observations [2505.14282, 2511.21257, 1905.02107].

## 1. Model Framework and Identifying Conditions

PDS-LASSO is posed within a high-dimensional linear regression or partially linear model where inference is required for a low-dimensional structural parameter (e.g., a treatment effect or a coefficient on a key input) in the presence of high-dimensional controls:

- **Standard partially linear model**:
  $$
  y_i = d_i\,\theta + x_i^\top\beta^y + \varepsilon_i,\\
  d_i = x_i^\top\gamma + \eta_i,
  $$
  where $y_i$ is the scalar outcome, $d_i$ is the regressor of interest, $x_i \in \mathbb{R}^p$ is the high-dimensional control vector, and $\theta$ is the low-dimensional parameter of interest [2511.21257]. In stochastic-frontier settings, the model generalizes to 
  $$
  y_i = \beta_0 + x_i^\top\beta + z_i^\top\delta + \varepsilon_i, \quad \varepsilon_i = v_i - u_i,
  $$
  where $x_i$ are primary inputs and $z_i$ are high-dimensional environmental (inefficiency-shifting) covariates [2505.14282].

- **Assumptions**:
  1. **Approximate sparsity:** Only a small subset of regressors are relevant: $\|\beta^y\|_0 + \|\gamma\|_0 \ll n$, or more generally, $\delta_0$ is $s$-sparse in $z$ variables ($s \ll n$).
  2. **Restricted Eigenvalue (RE)/Compatibility:** The Gram matrix of controls must satisfy compatibility to ensure identifiability and control the LASSO error [2511.21257].
  3. **Exogeneity:** Errors have mean zero conditional on controls; for stochastic-frontier, $E[v_i|x_i,z_i]=0$ and $E[u_i|x_i,z_i]=E[u_i]$ [2505.14282].
  4. **Moments and Tails:** Variables and errors are sub-Gaussian/bounded moments of sufficiently high order.
  5. **Orthogonality/Neyman-Orthogonality:** The score function used in final estimation must be first-order insensitive to small estimation error in nuisance parameters [2505.14282].

## 2. The PDS-LASSO Algorithm

PDS-LASSO proceeds as follows:

1. **First Stage (LASSO for $d$):**
   $$
   \hat{\gamma} = \arg\min_{\gamma} \frac{1}{2n}\sum_{i=1}^n (d_i - x_i^\top\gamma)^2 + \lambda \|\gamma\|_1
   $$
   Select $\hat S_d = \{j : \hat{\gamma}_j \neq 0\}$ [2511.21257].

2. **Second Stage (LASSO for $y$):**
   $$
   \hat{\alpha} = \arg\min_{\alpha} \frac{1}{2n}\sum_{i=1}^n (y_i - x_i^\top\alpha)^2 + \lambda \|\alpha\|_1
   $$
   Select $\hat S_y = \{j : \hat{\alpha}_j \neq 0\}$.

3. **Union of Selected Controls:**
   $$
   \hat S = \hat S_d \cup \hat S_y
   $$
   This ensures that any variable affecting either the regressor of interest or the outcome is included.

4. **Final Estimation (Post-LASSO OLS/COLS):**
   $$
   (\hat{\theta}, \hat{\beta}_{\hat S}) = \arg\min_{\theta, \beta_{\hat S}} \frac{1}{2n}\sum_{i=1}^n \left(y_i - d_i\theta - x_{i, \hat S}^\top \beta_{\hat S}\right)^2
   $$
   For stochastic-frontier models, analogous three-block selection and estimation steps are performed, including cross-fitting or sample splitting to ensure independence of selection and estimation phases [2505.14282].

A table summarizing the three main stages:

| Stage                    | Operation                   | Selected Set      |
|--------------------------|-----------------------------|-------------------|
| 1. LASSO for $d$         | $d \sim x$                  | $\hat{S}_d$       |
| 2. LASSO for $y$         | $y \sim x$                  | $\hat{S}_y$       |
| 3. Final OLS             | $y \sim d + x_{\hat{S}}$    | $\hat{S}_d \cup \hat{S}_y$ |

## 3. Neyman-Orthogonality and Bias Elimination

The principal advantage of PDS-LASSO lies in the orthogonal construction of the final estimating equation:

- **Orthogonal Score:** The resulting moment condition is first-order robust to small errors in the estimation of nuisance parameters. For the partially linear model, Neyman-orthogonality (partialling out $x$ in both $y$ and $d$) ensures that specification mistakes in variable selection do not introduce leading-order bias in inference about $\theta$ [2511.21257, 2505.14282].
- **In stochastic-frontier models:** Construction of a Neyman-orthogonal moment via partialling out $z$ from $y$ and $x$ precedes estimation of the frontier parameters. This approach restores √n-consistency even when $d \gg n$ [2505.14282].
- **Mathematical statement:** Neyman-orthogonality holds if
  $$
  \left. \frac{\partial}{\partial \eta} E[\psi(W_i;\theta_0,\eta)] \right|_{\eta=\eta_0} = 0
  $$
  Such moment conditions yield final estimators whose asymptotic law does not depend on first-stage selection mistakes of order $o_P(1)$.

- **Implication:** Single-selection LASSO (i.e., stepwise inclusion only for $y$ or $d$) can leave bias of order $O_P(n^{-1/2+\delta})$ in the coefficient of interest, whereas PDS-LASSO eliminates this to $o_P(1)$ [2505.14282].

## 4. Theoretical Guarantees and Inference

Under approximate sparsity ($s_y,s_d \ll n/\log p$), Restricted Eigenvalue assumptions, and appropriate regularization ($\lambda \asymp \sqrt{(\log p)/n}$):

- **√n-Consistency and Asymptotic Normality:**
  $$
  \sqrt{n}(\hat{\theta} - \theta) \rightsquigarrow \mathcal{N}(0, \sigma_\theta^2),
  $$
  with $\sigma_\theta^2$ consistently estimated via plug-in residuals from the final OLS [2511.21257, 1905.02107].

- **Oracle Inequalities:** If the true support is contained in $\hat S$, the final estimator matches the performance (rate and limiting distribution) of the "oracle" estimator that knows the true support.
- **Uniformly Valid Inference:** As the final regression is Neyman-orthogonal, post-selection inference is valid without data-snooping corrections beyond robust/clustered standard errors as dictated by the error structure [2511.21257, 1905.02107].

- **Extensions to Clustering:** The method adapts to cluster settings by using cluster-robust standard errors, and by improving variance estimation to account for clustering in the design [1905.02107].

## 5. Algorithmic Implementation and Practical Considerations

- **Penalty selection:** The LASSO penalty parameter $\lambda$ plays a critical role. If set too high, important covariates may be omitted (inducing finite sample bias); too low, and the estimator becomes high variance due to over-selection [2511.21257].
- **Sample splitting/cross-fitting:** Used to ensure the independence between variable selection and final estimation, improving validity of inference [2505.14282].
- **Software:** PDS-LASSO can be implemented via standard LASSO routines, often with post-processing for variable selection and final OLS.

## 6. Applications, Empirical Performance, and Comparisons

- **Stochastic-Frontier (Efficiency) Analysis:** For estimating efficiency frontiers in settings with "big (wide) data," PDS-LASSO enables reliable measurement of inefficiency parameters by robustly selecting environmental covariates and avoiding spurious "no-inefficiency" artifacts seen with naive or single-selection approaches [2505.14282].
- **Empirical Evidence:** Monte Carlo simulations and empirical studies highlight PDS-LASSO’s small bias, accurate coverage, and robustness to high-dimensional control vectors, contrast to single-LASSO and naive methods [2505.14282, 1905.02107].
- **Limitations:** In finite samples, particularly with strongly negatively correlated controls or weak signal variables just below the LASSO threshold, the method may suffer omitted variable bias if relevant covariates are excluded in both selection steps [2511.21257]. Other selection methods (e.g., Post-Double-Autometrics) have been proposed to address these circumstances.
- **Comparison to 2SLS:** In many-instrument settings, PDS-LASSO avoids the weak-instrument and over-ID bias typical of traditional two-stage least squares, while maintaining inferential validity under high dimensionality [2505.14282].

## 7. Methodological Developments and Alternatives

- **Post-Double-Autometrics:** As an inference-based alternative, Post-Double-Autometrics utilizes classical t-tests in general-to-specific algorithms to ensure higher "potency" (fewer missed confounders), with superior finite-sample performance and smaller RMSE demonstrated in empirical and Monte Carlo comparisons [2511.21257].
- **Post-Double Selection under Multi-way Clustering:** The framework generalizes to arbitrary multi-way clustered sampling, provided cluster-robust variance estimation is employed [1905.02107].

PDS-LASSO constitutes a core methodology for valid inference in high-dimensional linear and efficiency models under approximate sparsity, with theoretical properties and practical performance established across diverse econometric and statistical settings [2505.14282, 2511.21257, 1905.02107].

Source: https://www.emergentmind.com/topics/post-double-selection-lasso-pds-lasso