Two-Step Convex Relaxation and Recovery (TS-CRR)

• TS-CRR is a design principle that replaces nonconvex estimation with a convex surrogate followed by an explicit recovery step.
• It is demonstrated in applications like sparse polynomial regression, phase retrieval, and super-resolution, improving interpolation and extrapolation accuracy.
• Empirical results show that TS-CRR effectively balances bias reduction and recovery accuracy, using metrics such as mean MSE in practical datasets.

Two-Step Convex Relaxation and Recovery (TS-CRR) denotes a methodological pattern in which a nonconvex, combinatorial, or otherwise intractable estimation problem is first replaced by a convex surrogate and then mapped back to the original object by an explicit recovery step. The label appears explicitly in sparse polynomial regression with anomalous data filtering, where a convex relaxation of a fractional program is followed by coefficient recovery on the selected monomial support (Abolpour et al., 25 Aug 2025). Earlier papers do not generally use the name, but they instantiate closely related architectures: lifted semidefinite relaxation followed by factor extraction in phase retrieval (Candes et al., 2011), autocorrelation relaxation followed by KL-based deautocorrelation for pairwise interactions (Bandegi et al., 2015), total-variation relaxation followed by support extraction from a dual polynomial in off-the-grid super-resolution (Valiulahi et al., 2017), and staged weighted $L_1$ refinement for sparse feature selection (Zhang, 2011).

1. Conceptual scope and historical placement

In the literature represented here, TS-CRR is not a single fixed algorithm but a family of architectures organized around a common division of labor. The relaxation stage enlarges the feasible set or replaces a discrete structural prior by a convex proxy; the recovery stage then extracts support, amplitudes, factors, or exact consistency constraints from the relaxed solution. This suggests that TS-CRR is best understood as a design principle rather than as a single model class.

The explicit 2025 formulation for sparse polynomial regression is the first paper in this set to use the term “Two-Step Convex Relaxation and Recovery” directly (Abolpour et al., 25 Aug 2025). Closely related antecedents appear under other names. “PhaseLift” lifts a quadratic inverse problem into a rank-one matrix problem and then extracts the signal by eigendecomposition (Candes et al., 2011). “Relax, Compensate and then Recover” (RCR) is a three-phase graphical-model formalism in which relaxed equivalence constraints are selectively restored to tighten a dual-decomposition approximation (Choi et al., 2015). “Multi-stage convex relaxation” in sparse regression and structured low-rank recovery goes beyond a literal two-step procedure, but its first stage is an ordinary convex relaxation and its later stages act as recovery or debiasing refinements [(Zhang, 2011); (Bi et al., 2017)].

A recurrent distinction across these papers is whether recovery is explicit, implicit, or internalized. In some cases the recovery stage is operationally separate, as in support localization from a dual trigonometric polynomial in two-dimensional super-resolution (Valiulahi et al., 2017) or KL-based deautocorrelation in pairwise interaction energies (Bandegi et al., 2015). In others, the convex optimizer already equals the target structured object, so the second step reduces to reading off a support or a rank-one factor, as in planted dense subgraph recovery (Ames, 2013). A further variant appears in convex relaxations of shallow convolutional networks, where a randomized perturbation selects the desired optimizer inside the convex program itself rather than through a separate post-processing stage (Bartan et al., 2018).

2. Explicit TS-CRR in sparse polynomial regression under anomalous data

The most explicit TS-CRR formulation in this corpus is the sparse polynomial regression model with anomalous data filtering (Abolpour et al., 25 Aug 2025). The data are $\{x^{(k)}\}_{k=1}^N$ and $\{y^{(k)}\}_{k=1}^N$, with $x^{(k)}\in\mathbb R^n$, and the regression model is

$$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$

where

$$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$

The baseline fitting criterion is minimax: $$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$

Sparsity and anomaly filtering are encoded jointly through binary monomial-selection variables $s_\alpha\in\{0,1\}$ and sample-inclusion variables $b_k\in\{0,1\}$. The resulting model imposes

$$-M(1-b_k)+\left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,$$

$\{x^{(k)}\}_{k=1}^N$0

together with the budgets

$\{x^{(k)}\}_{k=1}^N$1

Thus $\{x^{(k)}\}_{k=1}^N$2 is the anomaly budget and $\{x^{(k)}\}_{k=1}^N$3 is the monomial budget.

The paper then passes through a continuous QCQP reformulation and an equivalent fractional program (FP). Its central structural result is that the FP has a convex objective and convex constraints except for the single scalar nonconvex constraint

$\{x^{(k)}\}_{k=1}^N$4

provided

$\{x^{(k)}\}_{k=1}^N$5

The TS-CRR algorithm relaxes that remaining nonconvexity by the linear inequality

$\{x^{(k)}\}_{k=1}^N$6

The paper proves that this linear relaxation is valid, tighter than the SDC-based relaxation, and therefore also tighter than the SOC-based relaxation (Abolpour et al., 25 Aug 2025).

The two stages are then explicit. In step 1, the linear-based convex relaxation of the FP is solved, yielding $\{x^{(k)}\}_{k=1}^N$7. These are mapped back through $\{x^{(k)}\}_{k=1}^N$8 to produce $\{x^{(k)}\}_{k=1}^N$9, which the paper describes as identifying the sparse monomial structure and anomaly-filtering pattern. In step 2, the coefficients are recovered by solving the LP

$\{y^{(k)}\}_{k=1}^N$0

An exactness condition is also stated: if

$\{y^{(k)}\}_{k=1}^N$1

then the linear relaxation solves the original FP exactly.

The empirical evaluation uses two datasets. For Nordic electricity prices, the reported TS-CRR parameters are

$\{y^{(k)}\}_{k=1}^N$2

and the paper reports mean interpolation MSE $\{y^{(k)}\}_{k=1}^N$3, mean extrapolation MSE $\{y^{(k)}\}_{k=1}^N$4, and max extrapolation MSE $\{y^{(k)}\}_{k=1}^N$5, compared with linear-regression values $\{y^{(k)}\}_{k=1}^N$6, $\{y^{(k)}\}_{k=1}^N$7, $\{y^{(k)}\}_{k=1}^N$8 and polynomial-regression values $\{y^{(k)}\}_{k=1}^N$9, $x^{(k)}\in\mathbb R^n$0, $x^{(k)}\in\mathbb R^n$1 (Abolpour et al., 25 Aug 2025). For temperature forecasting, the reported parameters are

$x^{(k)}\in\mathbb R^n$2

The paper describes the method as more balanced in interpolation and extrapolation than several standard regression and AI baselines.

3. Convex-relaxation architectures associated with TS-CRR

The convex stage in TS-CRR varies substantially across domains, but the recurring pattern is replacement of a hard structural prior by a tractable convex surrogate.

In phase retrieval, lifting replaces a quadratic inverse problem in $x^{(k)}\in\mathbb R^n$3 by a linear inverse problem in

$x^{(k)}\in\mathbb R^n$4

with measurements

$x^{(k)}\in\mathbb R^n$5

The corresponding relaxation is the semidefinite program

$x^{(k)}\in\mathbb R^n$6

or its noisy version with $x^{(k)}\in\mathbb R^n$7 (Candes et al., 2011). In sparse quadratic recovery, the lifted variable is again $x^{(k)}\in\mathbb R^n$8, but sparsity is added through the mixed objective

$x^{(k)}\in\mathbb R^n$9

which the paper interprets as a sparse variant of PhaseLift (Li et al., 2012).

In off-the-grid super-resolution, the convex stage is infinite-dimensional but still classical: total-variation minimization over measures,

$$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$0

with a dual formulation in terms of a bounded low-pass trigonometric polynomial (Valiulahi et al., 2017). In planted dense subgraph recovery, the nonconvex rank-and-cardinality problem is relaxed to

$$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$1

so the convex stage is a low-rank-plus-sparse program in the style of robust PCA (Ames, 2013).

Other TS-CRR-related relaxations operate on still different surrogate spaces. Pairwise interaction minimization is reformulated over autocorrelations $$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$2, and the exact but nonconvex autocorrelation set is enlarged to a convex cone-slice $$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$3, leading to the linear conic program

$$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$4

with positivity, Fourier-sign, symmetry, and mass constraints (Bandegi et al., 2015). In shallow ReLU networks, the relaxation is an epigraph relaxation in the original parameter space,

$$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$5

augmented by a random perturbation $$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$6 because the naive relaxation has spurious optima (Bartan et al., 2018).

4. Recovery mechanisms

The recovery stage in TS-CRR is domain-dependent because the relaxed variable usually lives in a different space from the original object. In PhaseLift, the SDP returns a PSD matrix $$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$7, not a signal vector. Recovery therefore consists of rank-one factor extraction: $$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$8 In the exact case $$y^{(k)}=\sum_{\alpha \in \Gamma_d} c_\alpha \, \big(x^{(k)}\big)^\alpha + \epsilon^{(k)},\qquad k=1,\dots,N,$$9, this yields $$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$0 up to global phase; in noise, the top eigenpair provides the approximate signal (Candes et al., 2011).

In two-dimensional super-resolution, recovery is mediated by the dual polynomial

$$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$1

The support is localized from the rule

$$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$2

and amplitudes are then obtained from the Fourier fitting system

$$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$3

The paper emphasizes that the certificate proving optimality is also the practical support-localization device (Valiulahi et al., 2017).

In pairwise interaction problems, the relaxed variable is the autocorrelation $$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$4, and recovery seeks a realizable probability measure $$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$5 whose autocorrelation matches $$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$6. The criterion is relative entropy,

$$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$7

and the associated Schulz–Snyder iteration is

$$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$8

Here recovery is not a support readout but a deautocorrelation problem on the manifold of realizable autocorrelations (Bandegi et al., 2015).

Sparse polynomial regression provides a more classical support-first recovery pattern. After the linear-based convex relaxation identifies $$\Gamma_d=\left\{\alpha\in \mathbb{N}^n \mid \sum_{i=1}^n \alpha_i \le d\right\}.$$9, the recovery stage solves an LP on the selected monomial support, rather than attempting to estimate coefficients directly inside the relaxed model (Abolpour et al., 25 Aug 2025). In graphical-model inference, RCR interprets recovery as restoration of previously relaxed equivalence constraints $$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$0, which tightens the dual upper bound and can lead all the way back to exact inference (Choi et al., 2015).

A useful contrast appears in planted dense subgraph recovery. There, under the theorem assumptions, the convex optimizer itself is already

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$1

so the “recovery” step is essentially trivial: one reads off the planted vertex set from the support or rank-one factor of $$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$2 (Ames, 2013). This suggests that TS-CRR spans both explicit two-stage pipelines and one-shot convex programs whose solutions already encode the recovered object.

5. Exactness, sample complexity, and bias reduction

A central reason for the TS-CRR pattern is that the convex stage often has sharp certificates, while the recovery stage either preserves or improves them. In phase retrieval, if

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$3

then the trace-minimization SDP has the unique solution $$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$4 with probability at least

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$5

and in the noisy case the lifted estimator obeys

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$6

with corresponding vector-level stability after top-eigenvector extraction (Candes et al., 2011).

In sparse quadratic recovery, exact recovery by the mixed lifted convex program is proved when

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$7

equivalently

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$8

but the same paper also proves that for this class of naive convex relaxations one cannot expect exactness unless

$$\min_{c,\gamma}\ \gamma \quad\text{s.t.}\quad \left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,\ \forall k.$$9

for flat $s_\alpha\in\{0,1\}$0-sparse signals (Li et al., 2012). This negative result is especially important for TS-CRR because it motivates separating support identification from final signal recovery instead of enforcing all structure in a single lifted convex model.

In off-the-grid super-resolution, the exact TV-minimization theorem is certificate-based: if $s_\alpha\in\{0,1\}$1 and

$s_\alpha\in\{0,1\}$2

then the TV solution is unique (Valiulahi et al., 2017). The support-extraction stage is justified by the same dual polynomial that certifies exactness.

The sparse feature-selection paper makes the bias issue explicit. Standard Lasso has $s_\alpha\in\{0,1\}$3-error

$s_\alpha\in\{0,1\}$4

and thresholding therefore demands the stronger beta-min condition

$s_\alpha\in\{0,1\}$5

By contrast, the multi-stage weighted $s_\alpha\in\{0,1\}$6 scheme based on capped-$s_\alpha\in\{0,1\}$7 regularization removes penalty from coefficients above a threshold and achieves exact support recovery after $s_\alpha\in\{0,1\}$8 stages under sub-Gaussian noise and sparse-eigenvalue assumptions (Zhang, 2011). The paper explicitly states that a literal two-stage method does not remove the bias issue in general under RIP.

An analogous debiasing phenomenon appears in noisy structured low-rank matrix recovery. The first stage is exactly nuclear-norm minimization, while later stages solve

$s_\alpha\in\{0,1\}$9

where $b_k\in\{0,1\}$0 is built from previous singular structure. Under a restricted eigenvalue condition, the paper shows deterministic reduction of both estimation error and approximate-rank bounds in later stages, together with geometric convergence to a statistical floor (Bi et al., 2017). In TS-CRR language, stage 2 is a convex recovery correction that reduces nuclear-norm bias.

6. Limitations, misconceptions, and broader interpretation

A common misconception is that TS-CRR always means exactly two convex programs. The literature here is more heterogeneous. Some methods are strictly two-stage, such as sparse polynomial regression with linear-based relaxation followed by an LP (Abolpour et al., 25 Aug 2025). Others are multi-stage by design: sparse feature selection proves that more than two stages are generally needed for exact unbiased recovery under RIP, although only $b_k\in\{0,1\}$1 stages suffice (Zhang, 2011), and structured low-rank recovery derives an entire sequence of convex corrections from an exact-penalty reformulation (Bi et al., 2017).

Another misconception is that the recovery stage is always a rounding heuristic. In several papers it is theorem-level mathematics. The dual polynomial in super-resolution is simultaneously an optimality certificate and a support-localization mechanism (Valiulahi et al., 2017). The KL recovery stage in pairwise interactions comes with the sufficient condition $b_k\in\{0,1\}$2, which implies exact global optimality, and lattice Dirac solutions are recovered exactly when $b_k\in\{0,1\}$3 (Bandegi et al., 2015). Conversely, some papers show that recovery may be only implicit or even absent as a separate algorithmic object: planted dense subgraph recovery returns the planted rank-one matrix directly (Ames, 2013), while shallow CNN relaxation performs recovery through randomized tie-breaking inside the convex program rather than through a separate decoding stage (Bartan et al., 2018).

Computational cost is a persistent limitation. Lifted semidefinite models operate in $b_k\in\{0,1\}$4-scale matrix spaces in phase retrieval (Candes et al., 2011) and sparse quadratic recovery (Li et al., 2012). Two-dimensional super-resolution solves a dual SDP whose $b_k\in\{0,1\}$5 variable is of size $b_k\in\{0,1\}$6, and the paper explicitly notes poor scaling at large bandwidths (Valiulahi et al., 2017). The 2025 TS-CRR paper improves tractability by isolating the FP’s nonconvexity to a single scalar quadratic constraint, but it still requires choosing $b_k\in\{0,1\}$7, $b_k\in\{0,1\}$8, $b_k\in\{0,1\}$9, and a sufficiently large $$-M(1-b_k)+\left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,$$0, and it does not provide a sophisticated rounding rule beyond the inverse mapping $$-M(1-b_k)+\left|y^{(k)}-\sum_{\alpha\in\Gamma_d} c_\alpha (x^{(k)})^\alpha\right|\le \gamma,$$1 (Abolpour et al., 25 Aug 2025).

The broader lesson is that TS-CRR is most effective when the relaxation stage produces a surrogate object with clear structural semantics: a rank-one PSD matrix, an autocorrelation, a dual polynomial, a weighted sparse estimate, or a partially decomposed graphical model. The recovery stage then exploits that semantics rather than treating the convex solution as an endpoint. This suggests that the central design question in TS-CRR is not merely how to convexify, but how to choose a convex surrogate from which the target structure remains recoverable.