Two-Step Estimation Procedure Overview

• Two-step estimation procedure is a method that sequentially estimates nuisance parameters and then structural parameters to simplify complex inferential problems.
• It decouples model estimation into manageable stages, enhancing computational tractability and allowing for bias correction in high-dimensional settings.
• The approach is widely applied across econometrics, biostatistics, and machine learning to achieve robust and efficient parameter estimates.

A two-step estimation procedure is a composite estimation framework in which core model parameters are estimated in two sequential stages, each targeting a distinct sub-component of the inferential problem, often for reasons of feasibility, computational tractability, or to leverage regularity structure. This framework is fundamental in a variety of complex statistical models where a one-step (joint) estimator is computationally burdensome, model components are amenable to isolated estimation, or the sample structure and identifiability conditions support staged inference.

1. Foundational Concept and Motivating Examples

A two-step estimation procedure divides the estimation of a semiparametric or structured statistical model into two consecutive sub-problems, typically: 1. First Step: Estimation of so-called nuisance or intermediate parameters, which may include nonparametric components, latent variables, or high-dimensional selections.

1. Second Step: Estimation of the primary (structural) parameters of interest, using the first-step results as inputs or plug-in values.

Prominent examples include:

• High-dimensional additive models (Kato, 2012): First step—group Lasso for variable selection; second step—smoothing by penalized least squares with Sobolev penalties.
• Nonlinear SEMs: First step—fit the measurement model; second step—fix measurement parameters and fit the structural model (Kuha et al., 2023).
• Copula-based survival models: First step—estimation of terminal event margin; second step—joint estimation of nonterminal event margin and copula parameter (Arachchige et al., 2023).
• Generalized partially linear models: Pilot estimation of parametric and additive components with undersmoothing; refinement of each function after plugging in pilot values (Ma, 2013).
• Decision-theoretic parameter estimation: Data compression via summary statistics, then parameter mapping by regression or classification (Lakshminarayanan et al., 2022).
• In econometrics: First, estimating propensity scores or control functions; second, plugging into GMM or structural equations (Cattaneo et al., 2018).

2. Theoretical Structure and Asymptotics

Let the full parameter vector be θ = (θ₁, θ₂), with θ₁ referred to as "nuisance" and θ₂ as the "structural" parameter. The canonical two-step estimator is

$$\hat{\theta}_1 = \arg\max_\psi \mathcal{L}_1(\psi; \text{data}),$$

$$\hat{\theta}_2 = \arg\max_\phi \mathcal{L}_2(\phi; \hat{\theta}_1, \text{data}),$$

where $\mathcal{L}_1$ and $\mathcal{L}_2$ denote log-likelihoods or objective functions for the first and second steps, respectively.

Under regularity, two-step estimators are consistent and asymptotically normal:

• If the first-step estimator $\hat{\theta}_1$ attains $\sqrt{n}$-consistency and the mapping $$(\theta_1, \theta_2) \mapsto L_2(\theta_2; \theta_1)$$ is sufficiently smooth, then $\hat{\theta}_2$ is also $\sqrt{n}$-consistent, with asymptotic variance augmented by the propagation of error from step 1.
• The asymptotic covariance is typically of "sandwich" form,

$$\operatorname{Var}(\hat{\theta}_2) = I_{22}^{-1} + I_{22}^{-1} I_{12}' E_{11} I_{12} I_{22}^{-1},$$

where $I_{22}$ is the second-step Fisher information, $I_{12}$ the cross-derivative, $E_{11}$ is the asymptotic variance of $\hat{\theta}_1$ (Mari et al., 22 Jul 2025).

• In complex frameworks (e.g., with selection in the first-step), post-selection error can enter the limit law as bias or variance inflation (Cattaneo et al., 2018).

3. Structure of the Two Steps

First Step: Nuisance or Screening Estimation

• Variable Selection or Screening: Group Lasso in high-dimensional models (Kato, 2012), clustering for latent group assignment (Beyhum et al., 10 Dec 2024).
• Nonparametric Estimation: B-spline regression for function approximation in ODEs (Bhaumik et al., 2014), kernel regression for conditional probabilities (Doudchenko et al., 2020).
• Likelihood-based Estimation: Maximize partial or marginal likelihoods (e.g., for one margin in copula-based survival models) (Arachchige et al., 2023).
• Compression: Summarize large data (e.g., quantiles/order statistics) for parameter mapping (Lakshminarayanan et al., 2022).

Second Step: Structural/Parameter Estimation

• Plug-in Estimation: Substitute step-1 estimates (as fixed) in the estimation of main parameters—either unconstrained optimization or with further penalization/smoothing (Kato, 2012, Kuha et al., 2023).
• Penalized or Sieve Estimation: Smooth estimates over a chosen sieve or regularization space (Dasgupta et al., 2017).
• Moment-Based Estimation: GMM estimation after plugging in prediction/model quantities from step 1 (Cattaneo et al., 2018, Doudchenko et al., 2020).
• Maximum Likelihood or Pseudo-likelihood: Apply full-likelihood-based estimation with fixed nuisance parameters (Arachchige et al., 2023).
• Bias Correction: Use resampling (e.g., jackknife) or cross-fitting to correct for first-step estimation error (Cattaneo et al., 2018, Beyhum et al., 10 Dec 2024).

4. Theoretical Guarantees, Bias, and Variance Calculation

Two-step estimators often face the following challenges:

• Propagation of Estimation Error: The variability of the nuisance estimator must be properly propagated into the inference for the target parameter (augmented variance).
• Bias from Selection or Model Misspecification: In settings where the first step is used for variable screening or selection, bias may be induced via model selection mistakes or overfitting (Kato, 2012, Cattaneo et al., 2018). As shown in (Cattaneo et al., 2018), overfitting or including too many covariates in step 1 leads to a $k/\sqrt{n}$ bias that must be bias-corrected for valid inference.
• Variance Estimation: Standard formulas involve evaluation or approximation of cross-derivative (information) matrices. Simulation-based variance estimators sample the first-step parameter from its estimated sampling distribution and rerun the second-step estimator to estimate variance components (Mari et al., 22 Jul 2025).

A representative calculation: suppose the two-step estimator is $(\hat{e}_1, \hat{e}_2)$ where $\hat{e}_1$ has variance $E_{11}$, and the conditional variance of $\hat{e}_2$ at $e_1$ is $V_2$. The variance is then estimated as

$$\operatorname{var}(\hat{e}_2) \approx E[V_2] + \operatorname{var}[E_2 | e_1],$$

where the second term captures the extra variability from plugging in an estimated $e_1$ (Mari et al., 22 Jul 2025).

5. Empirical and Computational Properties

• Computation: Two-step estimation is often adopted for its computational decoupling—fitting large measurement or screening models can be computationally separated from structural or outcome modeling (Kuha et al., 2023, Mari et al., 22 Jul 2025).
• Adaptation to High Dimension: By reducing the parameter space or isolating the relevant dimensions, two-step estimators scale to problems where full joint estimation is intractable (Kato, 2012, Cattaneo et al., 2018).
• Simulation Studies: Simulation evidence in domains such as latent trait modeling, mixture cure models, nonlinear SEMs, and high-dimensional additive models consistently demonstrates that two-step estimators match or outperform naive plug-in/three-step approaches, and, in many cases, nearly match the efficiency of joint (one-step) estimation (Kuha et al., 2023, Musta et al., 2022, Mari et al., 22 Jul 2025).
• Handling Nonstandard Objectives: In models where the likelihood is not tractable but simulation is feasible, two-step decision-theoretic estimators—mapping summaries (e.g., quantiles) to parameters—achieve reliable results without requiring explicit likelihoods (Lakshminarayanan et al., 2022).

6. Advantages, Limitations, and Variants

Advantages

• Robustness: By isolating steps, the estimator can be more robust to misspecification or instability in any one component (Mari et al., 22 Jul 2025).
• Conceptual Clarity: Allows for a clean separation (e.g., of measurement and structure), avoiding interpretational confounding in latent variable models (Kuha et al., 2023).
• Oracle Properties: Under suitable conditions, two-step estimators can attain "oracle efficiency," matching the rate and distribution of estimators that know the nuisance parameters (Ma, 2013).
• Bias Correction and Valid Inference: Explicit methods for first-step bias correction exist (e.g., jackknife, cross-fitting) to ensure valid confidence intervals (Cattaneo et al., 2018, Beyhum et al., 10 Dec 2024).

Limitations and Remedies

• Step 1 Misspecification: If the first-step is misspecified or inconsistent, the second-step estimator is generally biased or inconsistent.
• Incorrect Variance Estimation: Naive variance computation ignoring first-step variability underestimates true uncertainty.
• Computational Cost: In some models, step 2 must be repeated many times with perturbed nuisance parameters for variance estimation (Mari et al., 22 Jul 2025), although this is typically less onerous than analytic derivation.

Extensions and Variants

• Simulation-Based Variance: Simulation-based approaches for variance estimation simplify the process for complex or non-differentiable models (Mari et al., 22 Jul 2025).
• Double Machine Learning and Neyman Orthogonality: Bias-reducing moment functions can be constructed to yield robustness to first-step estimation error (especially when utilizing machine learning tools with potential overfitting) (Beyhum et al., 10 Dec 2024).
• Multi-Step Extensions: Some algorithms allow for multi-step refinement or correction (not only two steps), mimicking iterative Fisher-scoring or Newton-Raphson corrections in an online/sample splitting context (Kutoyants et al., 2016).

7. Applications Across Domains

The two-step framework is central in:

• High-dimensional regression and variable selection: group Lasso with post-selection smoothing (Kato, 2012).
• Semiparametric panel and time series models: e.g., two-step estimation under non-zero-median innovations in heavy-tailed AR models (She et al., 13 Jun 2025), and double machine learning in panel models with time-varying unobserved heterogeneity (Beyhum et al., 10 Dec 2024).
• Latent variable models: two-step estimation is routine in IRT and latent class models (Kuha et al., 2023, Mari et al., 22 Jul 2025).
• Mixture cure survival models: presmoothing and projection for cure incidence (Musta et al., 2022).
• Nonlinear SEM with splines or mixtures: plug-in estimation for complex functional covariates (Holst et al., 2018).
• Discrete choice and demand models: first-step nonparametric machine learning for choice probabilities; second-step GMM for structural parameters (Doudchenko et al., 2020).
• Biostatistics and causal inference for semicontinuous outcomes: hTMLE and other targeted learning strategies (Williams et al., 8 Jan 2024).
• Econometrics: bias-corrected two-step estimation with many nuisance covariates (Cattaneo et al., 2018).

8. Summary Table: Key Elements of Two-Step Estimation

Domain	Step 1	Step 2
High-dim additive models	Group Lasso selection	Penalized LS (Sobolev) smoothing
Latent trait/SEM	Measurement model estimation	Structural model estimation, fixed θ₁
Mixture cure survival	Nonpar. (Beran) presmoothing	Logistic projection (parametric)
Copula survival	MLE for margin (e.g., D)	Pseudo-MLE for margin + copula
Panel data (heterogeneity)	K-means clustering	OLS regression with grouped FE
Decision theory (risk/Loss)	Data compression (e.g., quantiles)	Linear/nonlinear mapping to parameter

• Two-step estimation of high dimensional additive models (Kato, 2012)
• Two-step spline estimating equations for generalized additive partially linear models (Ma, 2013)
• A two-stage estimation procedure for non-linear structural equation models (Holst et al., 2018)
• Two-step estimation and inference with possibly many included covariates (Cattaneo et al., 2018)
• Two-step estimation of latent trait models (Kuha et al., 2023)
• Estimating the variance-covariance matrix of two-step estimates of latent variable models: a general simulation-based approach (Mari et al., 22 Jul 2025)
• A 2-step estimation procedure for semiparametric mixture cure models (Musta et al., 2022)
• Two-step estimation of a multivariate Lévy process (Esmaeili et al., 2013)
• A Two-step Heuristic for the Periodic Demand Estimation Problem (Laage et al., 2021)
• Two-step estimation of ergodic Lévy driven SDE (Masuda et al., 2015)
• Inference after discretizing time-varying unobserved heterogeneity (Beyhum et al., 10 Dec 2024)
• Two-Step Targeted Minimum-Loss Based Estimation for Non-Negative Two-Part Outcomes (Williams et al., 8 Jan 2024)
• Bayesian two-step estimation in differential equation models (Bhaumik et al., 2014)
• A Statistical Decision-Theoretical Perspective on the Two-Stage Approach to Parameter Estimation (Lakshminarayanan et al., 2022)
• Estimation of Discrete Choice Models: A Machine Learning Approach (Doudchenko et al., 2020)
• Two-Stage Pseudo Maximum Likelihood Estimation of Semiparametric Copula-based Regression Models for Semi-Competing Risks Data (Arachchige et al., 2023)
• A Two-step Estimating Approach for Heavy-tailed AR Models with General Non-zero Median Noises (She et al., 13 Jun 2025)

---

The two-step estimation paradigm continues to be highly influential, evolving with advances in machine learning, high-dimensional statistics, and causal inference. Its capacity to modularize inference, allow for targeted bias correction, and facilitate computational tractability ensures centrality across modern statistical practice and theory.