Two-Stage Maximum Likelihood Estimator

• Two-stage MLE is a methodology that splits parameter estimation into sequential stages, reducing complexity and improving tractability.
• It enhances robustness by isolating well-behaved parameters in the first stage and refining estimates through plug-in or pseudo-likelihood optimization in the second stage.
• Widely applied in semiparametric survival analysis, spatial processes, and deep learning, TS-MLE delivers asymptotically efficient and stable estimators in challenging models.

A two-stage maximum likelihood estimator (TS-MLE) is a methodology wherein parameter estimation is split into two sequential components. The first stage typically reduces data complexity or isolates parameters that are more tractable, while the second stage uses these intermediate estimates within a constrained or joint likelihood optimization to recover remaining parameters or refine estimates. TS-MLE approaches are encountered in diverse statistical settings, including semiparametric survival analysis with copulas, composite likelihoods for spatial processes, multivariate mixture models, fractional Brownian motion inference, algebraic statistics, and deep learning frameworks. The core motivation is computational tractability, robustness, and often improved identification or finite-sample properties relative to one-stage MLE.

1. Theoretical Underpinnings and Decision-Theoretic Context

The TS-MLE can be rigorously formalized within the statistical decision-theory framework (Lakshminarayanan et al., 2022). Any estimation procedure can be represented as a decision rule $\delta:\mathcal{Y}\rightarrow\Theta$ that aims to minimize risk $R(\theta, \delta)$ for data $y$ drawn from $P(y|\theta)$. The classical Bayes and minimax optimality criteria require minimization of expected or worst-case risk: $$R_{Bayes}(\delta) = \int_\Theta R(\theta,\delta)\,dP(\theta),\qquad \delta^*_{minimax} = \arg\min_\delta\max_{\theta\in\Theta} R(\theta,\delta)$$ TS-MLE decomposes $\delta$ into two components: a compression stage $h_N:\mathcal{Y}\rightarrow\mathbb{R}^n$ (with $n\ll N$) and a mapping $g_N:\mathbb{R}^n\rightarrow\Theta$, so that $\delta_N=g_N\circ h_N$. For independent and identically distributed (i.i.d.) data, $h_N$ often comprises sample quantiles or other order statistics, concentrating sufficient information about $\theta$ while overcoming measure concentration phenomena. The second stage $g_N$—often structured as a convex or linear mapping—translates this compressed representation into an estimate, enabling efficient and stable optimization.

2. Canonical Examples and Methodological Structure

TS-MLE procedures are found across multiple domains:

A. Semi-Competing Risks and Copula Regression

In semiparametric copula-based regression models for semi-competing risks data (Arachchige et al., 2023), the first stage estimates the marginal survival function parameters for the terminal event (typically not subject to dependent censoring), via direct likelihood maximization using only these outcomes. The second stage plugs in the first-stage estimates into the pseudo-joint likelihood of non-terminal and terminal events, then jointly estimates the non-terminal event's marginal parameters and copula (dependence) parameter. Both stages rely on semiparametric transformation models and a parametric copula (e.g., Clayton, Gumbel, Frank, elliptical families), with extensive analytic backing for consistency and asymptotic normality. The approach demonstrably improves computational efficiency and robustness relative to one-stage MLE and other two-stage alternatives.

B. Composite Likelihood for Spatial Point Processes

The generalized maximum composite likelihood estimator (GMCLE) for determinantal point processes (Fujimori et al., 2019) uses a two-step strategy: first estimating the intensity parameter by maximizing a normalized quasi-likelihood, followed by conditional estimation of interaction/correlation parameters via higher-order composite likelihood functions exploiting explicit determinant forms of joint intensities. Under stationarity and regularity conditions, the two-step GMCLE achieves strong moment convergence properties, facilitating the derivation of bias-corrected information criteria (e.g., second-order AIC-type) for robust model selection.

C. Stochastic Process Models with Mixed Noise

In models mixing two independent fractional Brownian motions (Mishura et al., 2015, Mishura, 2015), TS-MLE is realized via a transformation-filtering and likelihood stage. The first stage solves an integral Fredholm equation of the second kind, involving a weakly singular kernel factored into a bounded continuous component and a weak singular factor. This produces a filter function $h_T$ that projects the drift-containing process onto the observed composite process. The second stage uses $h_T$ in a classical likelihood ratio formula: $$\hat{\theta}(T) = \frac{N(T)}{\delta_{H_1}\langle N \rangle(T)},\;\; N(T) = \int_0^T h_T(t)dX(t)$$ Existence, uniqueness, and asymptotic normality are proven via operator theory and spectral methods.

D. Algebraic Statistics Dualization

In the setting of algebraic statistical models with discrete data (Rodriguez, 2014), TS-MLE is supported by dual varieties and conormal varieties. Stage one solves the dual likelihood equations on the dual model $X^*$ (often more tractable than the primal model $X$), yielding a solution set $B_X(u)$. Stage two reconstructs the critical points of the original likelihood by a coordinate-wise mapping $p_i = u_i / b_i$. This split drastically simplifies solution workflows in high-degree or tensor models.

3. Computational Considerations and Practical Implementation

TS-MLE methods are consistently motivated by computational constraints and identifiability. By isolating well-behaved components in the first stage (e.g., marginal distributions not subject to dependent censoring (Arachchige et al., 2023), or intensity parameters in spatial models (Fujimori et al., 2019)), the high-dimensionality and intricacy of joint or full likelihood optimization in complex models are alleviated. Constrained, plug-in, or pseudo-likelihood maximization in the second stage yields estimators with asymptotic properties demonstrable through analytic (sandwich-type) variance estimators or operator-theoretic proofs.

Software implementations, such as the R package PMLE4SCR for copula-based semi-competing risk models (Arachchige et al., 2023), demonstrate that TS-MLE leads to consistent estimation and robust finite-sample behavior. Simulation studies in these domains systematically report reductions in computational cost, improved numerical stability, and favorable robustness against misspecified dependencies when compared with one-stage MLE alternatives.

4. Extensions and Variants

TS-MLE handles nonstandard estimation situations:

• In time series and pseudo-likelihood contexts (Buchweitz et al., 2020), regularization utilizing the distribution of an initial estimator, combined with prior information (instead of direct penalization of the pseudo-likelihood function), yields improved estimators particularly sensitive to hidden correlation and heteroscedasticity.
• For mixture models (Manole et al., 2020), although explicit two-stage procedures are not developed, analytical findings imply that preliminary estimation followed by local refinement exploiting polynomial constraints on parameter increments can approach minimax optimal convergence rates, especially under non-regulary exponential loss structures.
• In deep learning-based structure-from-motion (Xiao et al., 2022), two-stage MLE manifests as correlation volume computation and uncertainty parameter prediction, followed by iterative deep gradient-based optimization, maximizing a mixture Gaussian-uniform likelihood function. This makes MLE robust to noise and outliers while preserving interpretability.

5. Statistical Properties and Asymptotic Theory

The estimators constructed using TS-MLE exhibit provably strong statistical properties under regularity conditions. Consistency and asymptotic normality prevail in complex semiparametric models, evidenced by explicit analytic variance estimators and limit theorems (Arachchige et al., 2023, Fujimori et al., 2019). Under stationarity and mixing conditions, GMCLE achieves moment convergence, permitting the derivation of model selection criteria that are both statistically efficient and computationally scalable.

In models with operator-theoretic complexity, TS-MLE leverages compactness and spectral gaps in integral operators to guarantee existence and uniqueness (Mishura et al., 2015). In dual geometric settings, the TS-MLE inherits bijectivity (Corollary 1) between dual and primal solutions, conferring a powerful alternative when direct likelihood equations are intractable (Rodriguez, 2014).

6. Implications, Strengths, and Limitations

TS-MLE approaches offer substantial improvements in identification, computational tractability, and robustness in multi-component, censored, or nonstandard models. A key implication is that TS separation often preserves asymptotic efficiency while reducing numerical instability and complexity. The plug-in or pseudo-likelihood structure in the second stage is typically robust to marginal misspecification; however, its performance may be sensitive to dependence mechanisms or stage-one estimation errors. In some settings (e.g., nonstationary spatial or highly misspecified copula settings), empirical performance may require further theoretical scrutiny. Comparisons to strictly one-stage approaches emphasize that the two-stage separation can provide both practical and theoretical advantages, especially for large-scale or noisy datasets.

7. Summary Table: Key TS-MLE Applications

Domain	Stage 1	Stage 2
Semi-Competing Risks (Arachchige et al., 2023)	Marginal MLE for terminal event	Joint pseudo-likelihood for non-terminal and copula
Spatial DPPs (Fujimori et al., 2019)	Intensity estimation via quasi-lik.	Composite likelihood for interaction params
Fractional Brownian Motion (Mishura et al., 2015)	Solve Fredholm eqn for filter	Likelihood max using innovation martingale
Algebraic Geometry (Rodriguez, 2014)	Solve dual variety ML equations	Map solutions to primal via coordinate-wise inversion

These exemplars illustrate the fundamental strategy and adaptability of TS-MLE across complex contemporary statistical models, providing both rigorous theoretical footing and practical utility for high-dimensional or semiparametric inference.