---
title: Instrumental Variable Least Squares
url: https://www.emergentmind.com/topics/instrumental-variable-least-squares-iv-ls
type: topic
---

# Instrumental Variable Least Squares

Instrumental Variable Least Squares (IV-LS), commonly referred to as two-stage least squares (2SLS), is the foundational method for consistent estimation of linear models with endogenous regressors when valid instrumental variables (IVs) are available. Recent innovations extend IV-LS to high-dimensional, functional, nonparametric, nonlinear, and online contexts, focusing on regularization, bias reduction, robustness, and efficiency.

## 1. Classical Formulation and Methodology

The classical IV-LS (2SLS) framework considers a structural model
$$
y = X\beta + \varepsilon, \quad X = Z\Gamma + u,
$$
where $y \in \mathbb{R}^n$ is the outcome, $X \in \mathbb{R}^{n \times k}$ possibly includes endogenous regressors, and $Z \in \mathbb{R}^{n \times m}$ ($m \ge k$) are full-rank instruments. The standard two-stage procedure involves:
- **Stage 1**: Regress the endogenous columns of $X$ on $Z$ to obtain fitted values $\hat X = P_Z X$ (where $P_Z$ is the projection onto $\operatorname{col}(Z)$).
- **Stage 2**: Regress $y$ on $\hat X$ using OLS, yielding the point estimator
$$
\hat\beta_\mathrm{2SLS} = (X' P_Z X)^{-1} X' P_Z y.
$$
Under instrument exogeneity ($\mathbb{E}[Z'\varepsilon]=0$), relevance ($\operatorname{rank}(\mathbb{E}[Z'X])=k$), and homoskedasticity/full rank, $\hat\beta_\mathrm{2SLS}$ is consistent while OLS is generally biased if $X$ is endogenous [1504.03381, 1602.01051].

The population moment condition is:
$$
\mathbb{E}[Z^\top(y - X\beta)] = 0,
$$
corresponding to the minimization of the IV quadratic objective function
$$
J_0(\beta) = \frac{1}{2n} (y-X\beta)' P_Z (y-X\beta).
$$

## 2. Bias, Efficiency, and Regularization

While 2SLS eliminates bias from endogeneity under valid IVs, it often exhibits higher variance than OLS, especially with weak instruments.

- **Convex Least Squares (CLS)**: Combines OLS and 2SLS by minimizing the mean squared error (MSE) of $\hat\beta_\mathrm{CLS}(k) = k \hat\beta_\mathrm{OLS} + (1-k)\hat\beta_\mathrm{2SLS}$ over $k \in [0,1]$ [1504.03381]. The optimal $k^*$ is data-driven and unique unless OLS and 2SLS have identical MSEs. Finite-sample and simulation evidence shows CLS outperforms pure 2SLS, especially with weak instruments.

- **Ridge-Path Estimation**: To mitigate poor-precision or nearly-flat IV criteria, a penalized objective adds a ridge penalty towards a prior $\beta^p$:
$$
J_n(\beta;\lambda) = \frac{1}{2n}(y-X\beta)'P_Z(y-X\beta) + \frac{\lambda}{2}(\beta-\beta^p)'(\beta-\beta^p).
$$
Sample-splitting selects the regularization parameter $\hat\lambda$ by minimizing the unpenalized IV criterion on a test block, yielding the estimator
$$
\hat\beta = (\hat\Sigma + \hat\lambda I)^{-1} [\hat\xi + \hat\lambda \beta^p],
$$
where $\hat\Sigma = X_\mathrm{train}'P_{Z_\mathrm{train}}X_\mathrm{train}/n_\mathrm{train}$, $\hat\xi = X_\mathrm{train}'P_{Z_\mathrm{train}}y_\mathrm{train}/n_\mathrm{train}$ [1908.09237]. Ridge-path provides large MSE reductions in small or weak-instrument samples. The joint limit law of $(\hat\beta,\hat\lambda)$ is a nonstandard mixture with positive mass at $\hat\lambda=0$.

- **James-Stein First Stage Shrinkage**: For scalar $x$ and $\ell \ge 4$ instruments, shrinking the first-stage OLS estimator of $x$ toward zero (using a James–Stein factor) strictly reduces finite-sample bias relative to standard 2SLS, at fixed invariance properties [1708.06443]. The final estimate is obtained using a control-function approach in the second stage.

## 3. Extensions: Nonlinear, Functional, and Modern IV-LS

Recent work generalizes IV-LS to various contemporary settings.

- **Kernel IV Regression (KIV)**: Extends 2SLS to nonlinear settings using reproducing kernel Hilbert spaces (RKHS). Stage 1 learns a conditional mean embedding $\hat\mu(\cdot)$ from $Z$ to the RKHS on $X$. Stage 2 regresses $y$ onto the fitted RKHS features, using kernel ridge regression [1906.00232]. KIV is minimax-optimal and outperforms classical IV when the structural relation is nonlinear.

- **Functional IV-LS**: For scalar-on-function regression models with measurement error and functional instruments, basis expansion reduces the infinite-dimensional functional regression to a finite-dimensional two-stage IV-LS problem [2509.12122]. Both pointwise and multivariate TSLS methods are considered, providing substantial bias and mean integrated squared error (AIMSE) reductions over naive or regression-calibration approaches.

- **System Identification and Time Series**: IV-LS can debias least-squares identification of dynamical systems even when external instruments are unavailable, by sample splitting and filtering to create “internal” instruments [2511.09024]. Regularization ensures L^p consistency at O($n^{-1/2}$) rate, with dramatic reductions in bias and root MSE (RMSE) on nonlinear time series models.

## 4. Robustness and Double-Robust Estimation

Model misspecification can inflate 2SLS variance or bias.

- **Double-Robust G-Estimators**: Double-robust G-methods achieve consistency if either the instrument-assignment model $f(Z|C)$ or the outcome model $m_y(C; \theta)$ is correctly specified. The semiparametric efficient estimator within this class uses an index $e_\mathrm{opt}(Z,C)$ involving conditional variance and conditional means. Efficiency-maximized and bias-reduced G-estimators further improve finite-sample performance under local misspecification [1510.01770].

- **Control Function and Pretest Approaches**: In nonlinear models, the control-function (CF) estimator augments the second-stage regression with functions of first-stage residuals. CF and 2SLS are equivalent in linear contexts, but diverge otherwise. Hausman-type pretests can adaptively choose between CF and 2SLS, optimizing for worst-case MSE under unknown validity of augmented instruments [1602.01051].

## 5. IV-LS and Treatment Effect Heterogeneity

Addressing structural and causal heterogeneity requires extended IV-LS frameworks.

- **Interacted 2SLS for Heterogeneity**: The interacted two-stage least squares (2SLS) incorporates treatment-covariate interaction terms and their corresponding instrument interactions. Under either (i) linearity of instrument-covariate interactions or (ii) correct linear outcome modeling, the estimator delivers consistent projections of heterogeneous local average treatment effects (LATE) among compliers [2502.00251]. If these fail, stratification by the IV-propensity score approximates the heterogeneity structure.

- **Weighting Interpretation**: The weights in interacted 2SLS admit a formal connection to the Abadie (2003) weighting scheme for compliers; necessary and sufficient conditions for identification highlight the relevance of instrument-covariate linearity assumptions.

## 6. Online, Adaptive, and High-Dimensional IV-LS

Adaptation to online data streams and large dimensions drives new IV-LS algorithms.

- **Online 2SLS (O2SLS)**: In sequential settings, O2SLS recursively updates both first- and second-stage sufficient statistics, maintaining adaptively regularized Gram matrices. The method achieves $\mathcal O(d_x d_z \log^2 T)$ identification regret and $\widetilde{\mathcal O}(\gamma \sqrt{d_z T})$ oracle regret, where $\gamma$ quantifies endogeneity [2302.09357]. When embedded in an optimistic linear bandit (OFUL-IV), the regret bound matches the exogenous lower bound.

- **Bias-variance Tradeoffs and High-dimensionality**: IV-LS extensions now routinely incorporate regularization (ridge, lasso, group penalties), cross-validation for tuning parameter selection, and shrinkage, all underpinned by nonstandard asymptotic theory when the tuning itself is estimated from the data [1908.09237].

## 7. Applications and Practical Considerations

IV-LS methods have been empirically validated in a range of settings:
- In education economics, CLS and 2SLS provide consistent but different estimates for the returns to schooling, with CLS exhibiting reduced MSE for weak instruments [1504.03381].
- In functional regression, IV-LS corrections are essential for unbiased inference where scalar outcomes depend on error-prone functional predictors [2509.12122].
- In nonparametric or nonlinear settings, KIV and related methods consistently outperform linear IV-LS when the underlying relationships depart from linearity [1906.00232].

Key practical recommendations include:
- Careful instrument selection and evaluation of their strength and exogeneity.
- Employing regularized or shrinkage-based variants in low-signal or high-dimensional settings.
- Using bootstrap or robust (sandwich) variance estimators to account for tuning parameter estimation and model uncertainty.

## References

- "The Ridge Path Estimator for Linear Instrumental Variables" [1908.09237]
- "Convex Combination of Ordinary Least Squares and Two-stage Least Squares Estimators" [1504.03381]
- "Control Function Instrumental Variable Estimation of Nonlinear Causal Effect Models" [1602.01051]
- "Instrumental variables system identification with $L^p$ consistency" [2511.09024]
- "Interacted two-stage least squares with treatment effect heterogeneity" [2502.00251]
- "Kernel Instrumental Variable Regression" [1906.00232]
- "Least squares-based methods to bias adjustment in scalar-on-function regression model using a functional instrumental variable" [2509.12122]
- "Robustness and efficiency of covariate adjusted linear instrumental variable estimators" [1510.01770]
- "Stochastic Online Instrumental Variable Regression: Regrets for Endogeneity and Bandit Feedback" [2302.09357]
- "Bias Reduction in Instrumental Variable Estimation through First-Stage Shrinkage" [1708.06443]

Source: https://www.emergentmind.com/topics/instrumental-variable-least-squares-iv-ls