---
title: Overspecified Two-Component Mixed Linear Regression
url: https://www.emergentmind.com/topics/overspecified-two-component-mixed-linear-regression-2mlr
type: topic
---

# Overspecified Two-Component Mixed Linear Regression

Searching arXiv for recent and foundational papers on overspecified two-component mixed linear regression and related convex/EM formulations.
Overspecified two-component mixed linear regression (2MLR) denotes settings in which a two-component mixture of linear regressions is fitted with more latent structure than the data-generating mechanism contains, or in which the optimization is made deliberately redundant. In the literature summarized here, this includes fitting a two-component mixture to data generated by a single linear regressor, introducing one vector variable per observation even though only two regression vectors exist, and semiparametric two-component formulations in which one component is treated as known [1810.05752, 2508.10154, 1612.06067, 1301.3473]. The resulting theory is heterogeneous: some overspecified formulations admit exact recovery, some exhibit global EM convergence, and some reveal sharp slowdowns or statistical floors that depend on symmetry, initialization, or high-dimensional sparsity.

## 1. Definitions and forms of overspecification

A standard 2MLR model observes \((x_i,y_i)\) and latent labels \(z_i\in\{0,1\}\) such that
\[
y_i = z_i\cdot\langle x_i,\beta_1^*\rangle + (1-z_i)\cdot\langle x_i,\beta_2^*\rangle + e_i,\qquad i=1,\dots,n,
\]
with two unknown regressors \(\beta_1^*,\beta_2^*\in\mathbb R^p\). Estimation is naturally measured up to label swap through
\[
\rho((\hat\beta_1,\hat\beta_2),(\beta_1^*,\beta_2^*))=
\min\{\|\hat\beta_1-\beta_1^*\|_2+\|\hat\beta_2-\beta_2^*\|_2,\;
\|\hat\beta_1-\beta_2^*\|_2+\|\hat\beta_2-\beta_1^*\|_2\},
\]
which isolates the intrinsic non-identifiability of component labels [1312.7006].

Within this baseline, the literature uses “overspecified” in several distinct ways. One is optimization-level overspecification: a convex program introduces one vector \(z_i\in\mathbb R^d\) per data point and then uses an \(\ell_1\)-style fusion penalty to force these variables to collapse to the true regressors [1612.06067]. A second is model misspecification: the data are generated by a single linear model, but EM is asked to fit a two-component mixture anyway [1810.05752, 2508.10154]. A third is semiparametric overspecification, where one component is fully known and the other is unknown, so the two-component model contains a built-in asymmetry that changes identifiability and estimation strategy [1301.3473].

| Form of overspecification | Representative formulation | Main guarantee |
|---|---|---|
| Per-observation lifting | one vector variable \(z_i\in\mathbb R^d\) for each data point | unique SOCP solution \(z_i=\beta_{\ell_i}\) under well-separation, balance, and full span |
| Symmetric overfitting of a single regressor | fit \(\pm \beta\) with equal weights | EM converges from random initialization |
| Overfitting with unknown weights | fit \((\beta_1,\beta_2,w)\) to a single-component model | linear convergence if initial weights are unbalanced; sublinear if perfectly balanced |
| Known/unknown semiparametric mixture | one component known, the other unknown | method-of-moments estimation in \(O(n)\) time |

A recurring theme is that overspecification is not synonymous with non-identifiability. In several regimes, redundancy is absorbed either by geometric conditions, by symmetry-breaking in initialization, or by regularization that collapses superfluous degrees of freedom.

## 2. Convex overspecification by per-observation lifting

In the noiseless formulation of mixed linear regression over \(d\) features, one observes \((a_i,b_i)\in\mathbb R^d\times\mathbb R\), \(i=1,\dots,m\), with two unknown regression vectors \(\beta_1,\beta_2\in\mathbb R^d\) and an unknown partition \(S_1\cup S_2=\{1,\dots,m\}\) such that
\[
a_i^\top \beta_p=b_i \quad \text{whenever } i\in S_p,\; p=1,2.
\]
The convex construction introduces one vector variable \(z_i\in\mathbb R^d\) per observation and solves the SOCP
\[
\begin{aligned}
\min_{\{z_i\},\{t_{ij}\}} \quad &\sum_{1\le i<j\le m} t_{ij}\\
\text{s.t.}\quad &\|z_i-z_j\|_2\le t_{ij}\qquad \forall\,1\le i<j\le m,\\
&a_i^\top z_i=b_i\qquad \forall\,i=1,\dots,m,
\end{aligned}
\]
equivalently,
\[
\min_{z_1,\dots,z_m}\;\sum_{i,j=1}^m\|z_i-z_j\|_2
\quad\text{s.t.}\quad a_i^\top z_i=b_i\;\;(\forall i).
\]
This is an extreme overspecification of the two-component model, because the parameterization contains \(m\) vector variables rather than two regressors [1612.06067].

Exact recovery for \(k=2\) is proved under three structural conditions. First, the measurements in each class must satisfy a well-separation inequality:
\[
\max_{p=1,2}\; \max_{i\in S_p}
\frac{\|P_{v_p}^\perp a_i\|_2}{\|P_{v_p}a_i\|_2}
<
\frac12\,\min_{p=1,2}\frac{|S_p|}{m},
\]
where \(v_{12}=(\beta_1-\beta_2)/\|\beta_1-\beta_2\|_2\), \(v_{21}=-v_{12}\), \(P_{v_p}=v_{p,q}v_{p,q}^\top\), and \(P_{v_p}^\perp=I-P_{v_p}\). Second, the data must satisfy the balance condition
\[
\sum_{i\in S_p}
\operatorname{sign}(v_p^\top a_i)\,
\frac{P_{v_p}^\perp a_i}{\|P_{v_p}a_i\|_2}=0,
\qquad p=1,2.
\]
Third, each class span must be full:
\[
\mathrm{span}\{a_i:i\in S_p\}=\mathbb R^d,\qquad p=1,2.
\]

Under these assumptions, the unique solution of the SOCP is
\[
z_i=\beta_{\ell_i}\qquad \text{for all } i.
\]
The sample requirement is correspondingly geometric: each class must have at least \(d\) independent measurements, hence \(n_p=|S_p|\ge d\), and in total one needs at least \(2d\) measurements. The overspecified variable set does not increase this bound, because the convex program still forces only two distinct \(z_i\)’s. After solving the SOCP, one may run \(k'\)-means with any \(k'\ge 2\); the empirical \(z_i\)’s lie in just two locations, so the true two clusters are recovered automatically.

The implementation route is iteratively reweighted least squares (IRLS). Since \(\sum_{i<j}\|z_i-z_j\|_2\) is nonsmooth, iteration \(t\) forms
\[
w_{ij}^{(t)}=\bigl(\|z_i^{(t)}-z_j^{(t)}\|_2^2+\delta\bigr)^{-1/2},
\]
and solves
\[
\min_{z_1,\dots,z_m}
\sum_{i<j} w_{ij}^{(t)}\|z_i-z_j\|_2^2
\quad\text{s.t.}\quad a_i^\top z_i=b_i\;\;(\forall i).
\]
This makes the overspecified formulation computationally tractable while preserving the exact-recovery statement in the noiseless regime.

## 3. EM and global convergence in the symmetric overspecified model

A second overspecified regime arises when one fits a symmetric two-component mixture even though “one might have hoped only for a single linear fit.” In the analyzed model,
\[
y_i=z_i\,\beta^{*T}x_i+e_i,\qquad z_i\in\{+1,-1\},\;\Pr(z_i=+1)=\tfrac12,
\]
with isotropic Gaussian covariates \(x_i\sim N(0,I_d)\), Gaussian noise \(e_i\sim N(0,\sigma^2)\), and signal-to-noise ratio
\[
\eta=\|\beta^*\|/\sigma.
\]
The fitted parameter is a single vector \(\beta\in\mathbb R^d\), corresponding to symmetric components \(\pm\beta\). The paper emphasizes that EM behaves very differently in mixed linear regression from its behavior in Gaussian mixture models, and that it may fail to converge for three or more components, whereas the two-component case admits a global theory [1810.05752].

The EM iteration has closed form. The E-step computes
\[
w_i
=\Pr[z_i=+1\mid x_i,y_i;\beta]
=\frac{1}{1+\exp\bigl(-2\,y_i\,(x_i^T\beta)/\sigma^2\bigr)}
=\tfrac12\Bigl(1+\tanh\!\bigl(y_i\,x_i^T\beta/\sigma^2\bigr)\Bigr),
\]
and the M-step becomes
\[
\beta\leftarrow
\Bigl(\tfrac1n\sum_{i=1}^n x_ix_i^T\Bigr)^{-1}
\Bigl(\tfrac1n\sum_{i=1}^n\tanh\!\bigl(y_i\,x_i^T\beta/\sigma^2\bigr)\,y_i\,x_i\Bigr).
\]
An “easy-EM” variant replaces \(\tfrac1n\sum x_ix_i^T\) by the identity.

The population dynamics have a particularly rigid geometry. All iterates remain in the plane spanned by \(\beta\) and \(\beta^*\), and the population map has only five critical points: aside from the trivial zero, the only local maxima are \(\beta=\pm\beta^*\); the origin is a local minimum, and there are two saddles orthogonal to \(\beta^*\). A common concern is that overspecification necessarily creates spurious local maxima. In this symmetric 2MLR model, that concern is contradicted by the critical-point structure: there is no spurious local maximum to trap EM.

The proof strategy tracks the angle between \(\beta_t\) and \(\beta^*\). The angle strictly decreases at every step unless the iterate is already perfectly aligned or orthogonal. Starting from random \(\beta_0\), in \(O(\max\{1,\eta^{-2}\}\log d)\) population-EM steps one reaches angle \(<60^\circ\), then in \(O(\max\{1,\eta^{-2}\})\) more steps one reaches angle \(<22.5^\circ\), after which classical local \(\ell_2\)-contraction yields exponential convergence in norm. Overall,
\[
T=O\bigl(\max(1,\eta^{-2})[\log d+\log(1/\epsilon)]\bigr)
\]
suffices for \(\|\beta_T-\beta^*\|\le \epsilon\|\beta^*\|\).

The finite-sample analysis uses sample splitting. With fresh \(\tilde O\bigl(\max(1,\eta^{-2})\,d/\epsilon^2\bigr)\) samples per iteration, sample-splitting EM achieves \(\|\tilde\beta_T-\beta^*\|\le\epsilon\|\beta^*\|\) in the same number of iterations, with high probability. In the high-SNR regime \((\eta\gg 1)\), one obtains \(\ell_2\)-error \(O\bigl(\sigma\sqrt{d/n}\bigr)\), independent of \(\|\beta^*\|\). The practical conclusion drawn in the paper is explicit: overspecification is not fatal.

## 4. Balance-sensitive overspecification with unknown mixing weights

A later overspecified analysis considers the true single-component model
\[
x_1,\dots,x_n\sim N(0,I_d),\qquad y_i=x_i^T\beta^*+\epsilon_i,\qquad \epsilon_i\sim N(0,\sigma^2),
\]
but fits the full two-component Gaussian 2MLR model with parameters \(\theta=(\beta_1,\beta_2)\in\mathbb R^d\times\mathbb R^d\) and mixing weight \(w\in(0,1)\). The observed log-likelihood is
\[
\ell(\theta,w)=\sum_{i=1}^n\log\!\Big[w\,\phi(y_i;x_i^T\beta_1,\sigma^2)+(1-w)\,\phi(y_i;x_i^T\beta_2,\sigma^2)\Big],
\]
and EM updates the responsibilities
\[
\gamma_i(\beta_1,\beta_2,w)
=
\frac{w\,\phi(y_i;x_i^T\beta_1,\sigma^2)}
{w\,\phi(y_i;x_i^T\beta_1,\sigma^2)+(1-w)\phi(y_i;x_i^T\beta_2,\sigma^2)}
\]
followed by the closed-form M-step
\[
\beta_1^{(t+1)}=
\Big[\sum_i \gamma_i^{(t)}x_ix_i^T\Big]^{-1}\Big[\sum_i \gamma_i^{(t)}y_i x_i\Big],
\]
\[
\beta_2^{(t+1)}=
\Big[\sum_i (1-\gamma_i^{(t)})x_ix_i^T\Big]^{-1}\Big[\sum_i (1-\gamma_i^{(t)})y_i x_i\Big],
\]
\[
w^{(t+1)}=\frac1n\sum_i \gamma_i^{(t)}.
\]
The paper studies the overspecified-no-separation regime by normalizing by \(\sigma\) and setting \(\beta^*=0\) [2508.10154].

The population theorem exhibits a sharp dependence on initialization through the imbalance variable \(\beta^t:=w^{(t)}-\tfrac12\). If the initial mixture is unbalanced, \(|\beta^0|>0\), then after \(T=O(\log(1/\epsilon))\) EM iterations one has \(\alpha^T\le \epsilon\), where \(\alpha^t=\|\beta_1^{(t)}/\sigma\|=\|\beta_2^{(t)}/\sigma\|\). If the initial guess is perfectly balanced, \(\beta^0=0\), then EM converges sublinearly, with
\[
T=O(\epsilon^{-2})
\]
required to drive \(\alpha^T\le \epsilon\). The proof sketch isolates two different recurrences: a linear-rate contraction \(\alpha^{t+1}\le (1-c\,\beta^\infty{}^2)\alpha^t\) in the unbalanced case, and a cubic drop \(\alpha^{t+1}\approx \alpha^t-C\alpha_t^3\) in the balanced case, which leads to \(\alpha^t=O(t^{-1/2})\).

The finite-sample theorem fixes the mixing weights \(\pi^t\equiv \pi^0\) and again splits the theory by balance. For
\[
n\gtrsim d\vee \log(1/\delta)\vee \log^3(1/\delta'),
\]
the sufficiently unbalanced regime
\[
\|\pi^0-\tfrac12\|_1 \gtrsim \Big[(d\vee \log(1/\delta))/n\Big]^{1/4}
\]
yields
\[
T=O\Big(\|\pi^0-\tfrac12\|_1^{-2}\cdot\log\!\big(n/(d\vee \log(1/\delta))\big)\Big)
\]
and final accuracy
\[
\|\beta_1^{(T)}/\sigma\|=O\Big(\|\pi^0-\tfrac12\|_1^{-1}\cdot[(d\vee \log(1/\delta))/n]^{1/2}\Big).
\]
In the sufficiently balanced regime
\[
\|\pi^0-\tfrac12\|_1 \lesssim \Big[(d\vee \log(1/\delta))/n\Big]^{1/4},
\]
the algorithm takes
\[
T=O\Big((n/(d\vee \log(1/\delta)))^{1/2}\Big)
\]
steps and reaches accuracy
\[
\|\beta_1^{(T)}/\sigma\|=O\Big([(d\vee \log(1/\delta))/n]^{1/4}\Big).
\]

This is one of the clearest statements in the overspecified literature that balance is not a benign symmetry. A sufficiently unbalanced initialization preserves the usual \(\sqrt{d/n}\)-type statistical scale, whereas exact balance slows the optimization to sublinear convergence and lowers the finite-sample accuracy to a \((d/n)^{1/4}\) floor. The same paper also extends the population analysis to low SNR by perturbing the EM updates around \(\eta=0\).

## 5. Noisy convex relaxations and minimax rates for 2MLR

Overspecified 2MLR is most naturally interpreted against the broader theory of noisy two-component mixed regression. A foundational convex approach lifts the two unknown regressors into
\[
K^*=\frac{\beta_1^*\beta_2^{*T}+\beta_2^*\beta_1^{*T}}{2},\qquad
g^*=\frac{\beta_1^*+\beta_2^*}{2},
\]
and then estimates \((K,g)\) rather than \((\beta_1,\beta_2)\) directly. Under arbitrary noise, the estimator solves
\[
\min_{K,g}\;\|K\|_*
\quad\text{s.t.}\quad
\sum_{i=1}^n
\Bigl|
-\langle x_ix_i^\top,K\rangle
+2y_i\langle x_i,g\rangle
-y_i^2
\Bigr|
\le \eta,
\]
while under stochastic sub-Gaussian noise with known \(\sigma^2\), it solves
\[
\min_{K,g}\;
\sum_{i=1}^n
\bigl(
-\langle x_ix_i^\top,K\rangle
+2y_i\langle x_i,g\rangle
-(y_i^2-\sigma^2)
\bigr)^2
+\lambda\|K\|_*.
\]
The regression vectors are then reconstructed from
\[
\hat J=\hat g\hat g^\top-\hat K,
\quad \text{take top eigenpair }(\hat\lambda,\hat u),
\quad \hat\beta_{1,2}=\hat g\pm \sqrt{\hat\lambda}\,\hat u
\]
[1312.7006].

The guarantees are sharp. Let
\[
\alpha=\frac{\|\beta_1^*-\beta_2^*\|_2^2}{\|\beta_1^*\|_2^2+\|\beta_2^*\|_2^2}\in(0,2],
\]
and assume \(n_1,n_2\gtrsim p/\alpha\) with \(n_1/n_2=\Theta(1)\). Under arbitrary noise, if \(\|e\|_2\le c_5\sqrt{\alpha n}(\|\beta_1^*\|_2+\|\beta_2^*\|_2)\) and \(\eta\asymp c_4\sqrt n\|e\|_2\|\beta_1^*-\beta_2^*\|_2\), then with probability \(1-O(e^{-c_2n})\),
\[
\|\hat K-K^*\|_F\le C\,\frac{\eta}{\sqrt{\alpha}\,n},\qquad
\|\hat g-g^*\|_2\le C\,\frac{\eta}{\sqrt{\alpha}\,n(\|\beta_1^*\|_2+\|\beta_2^*\|_2)},
\]
and consequently
\[
\rho((\hat\beta_1,\hat\beta_2),(\beta_1^*,\beta_2^*))
\le C'\,\frac{\|e\|_2}{\sqrt{\alpha n}}.
\]
Exact recovery in the noiseless case follows by setting \(e=0\), \(\eta=0\).

The same paper supplies minimax lower bounds up to constants or log factors. If \(n\gtrsim p\), then for any estimator and any true \((\beta_1,\beta_2)\) separated by at least \(\gamma\), with arbitrary noise satisfying \(\|e\|_2\le \epsilon\),
\[
\inf_{\hat\beta}\sup_{(\beta^*,e)}\rho(\hat\beta,\beta^*)\gtrsim \frac{\epsilon}{\sqrt n}.
\]
Under stochastic Gaussian designs and noise, the upper bounds display three SNR-driven phases, and the lower bounds match these phases:
1. if \(\gamma\gtrsim \sigma\), any estimator has risk \(\gtrsim \sigma\sqrt{p/n}\);
2. if \(c_1\sigma(p/n)^{1/4}\lesssim \gamma\lesssim \sigma\), the risk is \(\gtrsim (\sigma^2/\gamma)\sqrt{p/n}\);
3. if \(\gamma\lesssim \sigma(p/n)^{1/4}\), the risk is \(\gtrsim \sigma(p/n)^{1/4}\).

These results are not an overspecification theorem in the same sense as the EM papers, but they provide the principal noisy 2MLR benchmark. They also identify several limitations and open directions: if \(n<p\), identifiability requires additional structure such as sparsity; extension to \(K>2\) is open; direct convex extensions may blow up in dimensionality; and heavy-tailed designs would require robustification such as truncation or \(M\)-estimation.

## 6. High-dimensional inference, sparse regimes, and alternative objectives

In high-dimensional 2MLR, Zhang–Ma–Cai–Li study the model
\[
y_i = z_i\,(x_i^\top \beta_1^*+\epsilon_i^{(1)}) + (1-z_i)\,(x_i^\top \beta_2^*+\epsilon_i^{(2)}),
\]
with \(z_i\sim \mathrm{Bernoulli}(\omega^*)\), \(x_i\sim N_p(0,\Sigma^*)\), Gaussian noise of variance \(\sigma^2\), unknown nondegenerate \(\Sigma^*\), and both \(\beta_1^*,\beta_2^*\) \(s\)-sparse. Because direct likelihood maximization is nonconvex and ill-posed when \(p\gg n\), the method uses an \(\ell_1\)-penalized EM algorithm:
\[
\beta_\ell^{(t+1)}
=\arg\min_\beta \frac1{2n}\sum_{i=1}^n w_{i,\ell}^{(t)}(y_i-x_i^\top\beta)^2+\lambda_n^{(t+1)}\|\beta\|_1,\qquad \ell=1,2,
\]
with \(\omega^{(t+1)}=\frac1n\sum_i w_i^{(t)}\). Under good initialization, a separation condition \((\beta_1^*-\beta_2^*)^\top{\Sigma^*}^{-1}(\beta_1^*-\beta_2^*)\ge c_s>0\), bounded eigenvalues of \(\Sigma^*\), and sparsity \(s\,\log p\,\log n/n=o(1)\), after \(T\gtrsim \log n\) iterations one has, with probability at least \(1-p^{-1}\),
\[
\|\beta_1^{(T)}-\beta_1^*\|_2+\|\beta_2^{(T)}-\beta_2^*\|_2
\lesssim \sqrt{\frac{s\,\log p\,\log n}{n}}.
\]
The same work constructs debiased estimators, proves asymptotic normality for individual coordinates, gives confidence intervals, and proposes a large-scale testing procedure for \(H_{0j}:\beta_{1,j}^*=\beta_{2,j}^*=0\) with asymptotic FDR control [2011.03598].

A complementary sparse-regime analysis examines two \(k\)-sparse signals \(\beta_1,\beta_2\in\mathbb R^p\), \(k=o(p)\), under Gaussian design, Gaussian noise, and latent Bernoulli labels. The paper identifies a large statistical-computational gap in the symmetric balanced regime
\[
\phi=\tfrac12,\qquad \beta_1=-\beta_2,
\]
denoted SB-MLR. Information-theoretically, exact support recovery requires
\[
n_{\mathrm{IT}}=\Theta\!\Big(\frac{k\log(p/k)}{\log(1+\mathrm{SNR})}\Big),
\]
and the scaling \(n=\Theta(k/\mathrm{SNR}^2)\) appears as the minimax threshold up to log factors. Algorithmically, however, low-degree-polynomial hardness shows that if
\[
n\ll k^2/\mathrm{SNR}^2,
\]
then any polynomial-time algorithm fails, and slightly below that threshold super-polynomial runtime \(\exp\!\tilde{}\!\bigl(k^2/(n\mathrm{SNR}^2)\bigr)\) is required. Outside SB-MLR, a simple thresholding algorithm,
\[
T_j:=|\langle X_j,y\rangle|/\|y\|_2,
\]
recovers the joint support \(S_1\cup S_2\) with high probability whenever
\[
n\gtrsim (k(\mathrm{SNR}+1)/\mathrm{SNR})\cdot \log p,
\]
in \(O(np)\) time; after support recovery, one can use any dense two-component mixed-regression solver to recover \(\beta_1,\beta_2\) [2303.02118]. This suggests that the hardest overspecified and symmetric cases are concentrated in a narrow regime rather than being generic across mixed regression.

An alternative objective comes from optimal transport. Lin et al. formulate Wasserstein Mixed Linear Regression (WMLR) as a nonconvex–strongly-concave saddle-point problem,
\[
\min_{\beta_{[2]}}\max_{\gamma_1,\gamma_2} L(\beta,\gamma_1,\gamma_2),
\]
where the discriminator family is
\[
\psi_{\gamma_1,\gamma_2}(x,y)
=
\log\bigl(e^{y\,\gamma_1^\top x}+e^{-y\,\gamma_1^\top x}\bigr)
-
\log\bigl(e^{y\,\gamma_2^\top x}+e^{-y\,\gamma_2^\top x}\bigr),
\]
and a quadratic penalty regularizes the \(\gamma\)-variables. Gradient descent ascent with
\[
\eta_\gamma=\frac1L,\qquad \eta_\beta=\frac1{\kappa^2L}
\]
finds an \(\epsilon\)-minimax-stationary point in
\[
\mathcal O\Big(\frac{\kappa^2L\Delta+\kappa L^2(2\eta/\lambda)^2}{\epsilon^2}\Big)
\]
iterations. In the symmetric ground-truth case \(\beta_2^*=-\beta_1^*\), any stationary minimax solution whose reference vector \(\tilde\gamma\) is sufficiently correlated with \(\beta^*\) is actually global, so \(\hat\beta=\pm\beta^*\). The sample complexity is linear in dimension, with \(n=O(d/\epsilon^2)\) for gradient-uniform error \(\epsilon\). The same framework also permits over-parameterizing beyond the true \(k\): extra \(\gamma_i\)-blocks are shrunk back to the reference \(\tilde\gamma\) unless a true component is activated by the data [2106.07537].

A different semiparametric overspecified variant assumes one component is fully known. After recentering, the model is
\[
Y=(1-Z)\,\varepsilon^*+Z(\alpha_0+\beta_0X+\varepsilon),\qquad Z\sim \mathrm{Bernoulli}(\pi_0),
\]
where \(\pi_0\in(0,1]\), \(\beta_0\neq 0\), and the support of \(X\) contains at least three points spanning \(\{1,x,x^2\}\). Identifiability follows from regressions for \(\mathbb E[Y\mid X]\) and \(\mathbb E[Y^2\mid X]\), and estimation is based on empirical moments
\[
\overline{X^pY^q}=\frac1n\sum_{i=1}^n X_i^pY_i^q\qquad (p,q\le 2).
\]
The resulting estimators \((\alpha_n,\beta_n,\pi_n)\) are strongly consistent and asymptotically normal, all required sample moments are computable in \(O(n)\), and the unknown error c.d.f. admits a plug-in estimator \(F_n\) with \(\sqrt n\)-consistency in \(\ell^\infty(\mathbb R)\). A weighted bootstrap then yields asymptotically valid uniform confidence bands [1301.3473].

Taken together, these strands show that overspecified 2MLR is not a single phenomenon but a family of regimes. In some, redundancy can be neutralized by convex fusion, by symmetry-breaking, or by shrinkage in a minimax objective. In others, especially balanced sparse mixtures, the same symmetry produces genuine statistical-computational barriers. The unresolved issues identified across the literature include extension to more than two components, robustification beyond sub-Gaussian or Gaussian designs, and a sharper understanding of misspecification in the low-SNR and high-dimensional settings.

Source: https://www.emergentmind.com/topics/overspecified-two-component-mixed-linear-regression-2mlr