Quadratic Subspace Pursuit (QSP) Methods

• Quadratic Subspace Pursuit is a method that simplifies high-dimensional quadratic optimization by projecting problems onto carefully chosen lower-dimensional subspaces.
• It unifies various approaches, including subspace-based quadratic programming, Stiefel-manifold techniques, and derivative-free trust-region methods with quadratic models.
• The technique highlights ambiguity in naming, as similar acronyms appear in sparse recovery and quantum signal processing, yet it uniquely preserves quadratic structure for effective optimization.

Quadratic Subspace Pursuit (QSP) denotes a subspace-centered approach to quadratic optimization in which a high-dimensional problem is analyzed or solved through a sequence of lower-dimensional representations. Recent work suggests that the term is not standardized across the literature: the closest optimization meanings arise in equally constrained quadratic programming, where eigendecomposition of the quadratic term induces a linear mapping between a diagonalized formulation and the original problem (Yu, 2020); in Stiefel-manifold optimization, where a Sequential Subspace Method (SSM) solves quadratic objectives through repeatedly constructed low-dimensional surrogate problems (Chen et al., 2024); and in derivative-free optimization, where random subspace trust-region methods use quadratic interpolation models that are explicitly “$Q$-fully quadratic” on the chosen subspace (Chen et al., 2023). The same acronym, however, is also used for quantum signal processing, and nearby sparse-recovery formulations such as quadratically constrained basis pursuit belong to the same broad family of quadratic-constrained recovery problems without being QSP methods (Skelton, 2024, Yi, 2021).

1. Terminological scope

The literature does not present a single canonical algorithm under the exact name “Quadratic Subspace Pursuit.” Instead, it exhibits several technically distinct uses of “QSP” and several optimization methods that are subspace-pursuit-like in a quadratic setting. That ambiguity is substantive rather than cosmetic, because the associated mathematical objects, feasibility conditions, and computational goals differ sharply.

Usage	Mathematical setting	Relation to Quadratic Subspace Pursuit
Subspace-based quadratic optimization	Quadratic objectives reduced to low-dimensional subspaces	Closest optimization meaning
Quadratically Constrained Basis Pursuit	$\ell_1$-minimization with $\|y-Ax\|_2\le \eta$	Related sparse-recovery family, but not QSP
Quantum Signal Processing	Laurent polynomials and $SU(2)$ rotations	Different field sharing the acronym

In optimization-oriented usage, the shared idea is that the quadratic structure is exploited after a restriction, projection, or change of coordinates into a subspace. In the sparse-recovery paper on quadratically constrained basis pursuit, the method is explicitly described as an ADMM/operator-splitting algorithm and “not a paper about Quadratic Subspace Pursuit (QSP)” (Yi, 2021). In the quantum-computing paper, “QSP-processing” refers to classical preprocessing for quantum signal processing with Laurent polynomials and $SU(2)$ rotation matrices, which is conceptually unrelated to subspace pursuit in numerical optimization (Skelton, 2024).

2. Core optimization pattern

Across the optimization papers, the defining pattern is reduction of a quadratic problem to a smaller surrogate problem whose variables live in a carefully chosen subspace. The details vary by domain, but the structure is recurrent: construct a subspace from problem-relevant directions, solve a reduced problem there, map the solution back, and iterate.

For quadratic minimization on the Stiefel manifold, the subspace at iteration $k$ is

$$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$

where $V_g$ contains the ground eigenvectors of $A$, $X_k$ is the current iterate, $\ell_1$0 is the Euclidean gradient term, and $\ell_1$1 is a Newton/SQP direction. After QR factorization, the full problem is reduced to a core problem on $\ell_1$2 (Chen et al., 2024). In the derivative-free setting, the affine subspace is

$\ell_1$3

with $\ell_1$4, and the method builds a quadratic interpolation model $\ell_1$5 for the restricted objective $\ell_1$6 before solving a trust-region subproblem there (Chen et al., 2023).

This suggests that “quadratic subspace pursuit” is best understood as a methodological class rather than a single update rule. The common objective is not merely dimensionality reduction; it is the preservation of enough quadratic information inside the reduced space to support meaningful descent, stationarity tests, or exact recovery of optimizers.

3. Eigendecomposition and subspace transfer in equally constrained quadratic programming

A particularly direct subspace interpretation appears in the work on equally constrained quadratic programming. The central claim is that, when eigenvalue decomposition is applied to the quadratic term matrix $\ell_1$7 in a type of linear equally constrained quadratic programming (EQP), “there exists a linear mapping to project optimal solutions between the new EQP formulation where $\ell_1$8 is diagonalized and the original formulation” (Yu, 2020).

The result is conditional: the mapping “requires a particular type of equality constraints.” At the same time, the abstract states that the construction is “generalizable to some real problems such as efficient frontier for portfolio allocation and classification of Least Square Support Vector Machines (LSSVM)” (Yu, 2020). The same abstract further states that the established mapping “could be potentially useful to explore optimal solutions in subspace,” while also noting that this role is “not very clear to the author” (Yu, 2020).

The significance of this line of work lies in its exactness. The diagonalized problem is not presented merely as a preconditioned proxy; it is linked to the original EQP through a linear projection of optimal solutions. This suggests an interpretation of subspace pursuit in which eigenspaces of the quadratic form provide a coordinate system for exact optimal-solution transfer, provided the equality constraints have the required structure. The paper also places itself relative to earlier unconstrained analysis, stating that it was inspired by similar work proved on an unconstrained formulation and that “very few similar discussion appears in literature” (Yu, 2020).

4. Sequential subspace methods on the Stiefel manifold

The most explicit optimization framework of a subspace-pursuit type is the Sequential Subspace Method for minimizing a quadratic over the Stiefel manifold

$\ell_1$9

with objective

$\|y-Ax\|_2\le \eta$0

where $\|y-Ax\|_2\le \eta$1 is symmetric, $\|y-Ax\|_2\le \eta$2 is symmetric positive definite, and $\|y-Ax\|_2\le \eta$3 (Chen et al., 2024). The Euclidean gradient is

$\|y-Ax\|_2\le \eta$4

and the first-order stationarity condition is

$\|y-Ax\|_2\le \eta$5

for some symmetric multiplier matrix $\|y-Ax\|_2\le \eta$6 (Chen et al., 2024).

The theory centers on “qualified critical points.” A point $\|y-Ax\|_2\le \eta$7 is qualified if

$\|y-Ax\|_2\le \eta$8

where $\|y-Ax\|_2\le \eta$9 is the $SU(2)$0-th smallest eigenvalue of $SU(2)$1 (Chen et al., 2024). These points are important for two reasons stated in the paper: any global minimizer is qualified, and for the regularized problem introduced there, any qualified critical point is a global minimizer. The regularization modifies the first $SU(2)$2 eigenvalues of $SU(2)$3 so that the ground block becomes flat at $SU(2)$4,

$SU(2)$5

which yields a surrogate problem with stronger optimality properties (Chen et al., 2024).

Algorithmically, SSM reduces the $SU(2)$6 problem to a $SU(2)$7-dimensional core problem. If $SU(2)$8 is an isometric basis for $SU(2)$9, any feasible point in the subspace has the form $SU(2)$0 with $SU(2)$1, and the reduced problem is

$SU(2)$2

The method uses a surrogate regularized model $SU(2)$3 that majorizes the original objective and is tangent at the current iterate, giving strict decrease whenever the current point is not stationary (Chen et al., 2024).

The main convergence statement is correspondingly strong. The gradient norms satisfy

$SU(2)$4

and if $SU(2)$5 is nonsingular, then any limit point is a qualified critical point: $SU(2)$6 In this sense, the method is not only subspace-based but also spectrally informed: the subspace is anchored by the lowest eigenvectors of $SU(2)$7, corrected by first-order information, and accelerated by a Newton/SQP direction (Chen et al., 2024).

5. Random-subspace quadratic modeling in derivative-free optimization

A derivative-free analogue appears in the Quadratic Approximation Random Subspace Trust-region Algorithm (QARSTA). Here the objective is unconstrained optimization of a general function $SU(2)$8, but the algorithm works only on a low-dimensional affine subspace at each iteration (Chen et al., 2023). The subspace objective is

$SU(2)$9

and the quadratic model is

$k$0

The trust-region step solves, approximately,

$k$1

and the acceptance ratio is

$k$2

The distinctive theoretical contribution is the notion of a $k$3-fully quadratic model. A family $k$4 is $k$5-fully quadratic if, for all $k$6,

$k$7

$k$8

$k$9

The interpolation set is structured as

$$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$0

and the corresponding ambient-space formulation interpolates on

$$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$1

The geometry analysis explains why subspace updates are pursuit-like. If one appends a vector $$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$2 of fixed norm $$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$3 to an existing matrix $$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$4, then

$$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$5

with equality when $$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$6 is orthogonal to the existing columns. This justifies generating new directions orthogonal to the current subspace (Chen et al., 2023). The resulting method attains almost-sure global convergence,

$$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$7

and expected complexity with $$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$8 (Chen et al., 2023). A plausible implication is that QSP, in derivative-free form, is less about exact subspace optimality than about maintaining a well-conditioned, gradient-aligned quadratic model in a small active space.

6. Adjacent methods and recurrent misconceptions

Two nearby literatures are often conflated with Quadratic Subspace Pursuit. The first is quadratically constrained basis pursuit,

$$\mathcal{S}_k = \operatorname{span}\{V_g,\ X_k,\ AX_kC-B,\ Z_k\},$$9

which is solved in the cited work by ADMM with operator splitting, soft-thresholding for the $V_g$0 term, projection onto the $V_g$1-ball, and Euclidean projection onto the graph set $V_g$2 (Yi, 2021). Its relation to QSP is described there as “directly related in broad sparse recovery” but “not directly the same method,” because the method is convex optimization with proximal splitting rather than a greedy support-selection pursuit algorithm (Yi, 2021).

The second source of confusion is quantum signal processing. In that literature, QSP-processing starts from a target Laurent polynomial

$V_g$3

constructs a complementary polynomial $V_g$4 satisfying $V_g$5, and decomposes the resulting matrix-valued polynomial into a sequence of $V_g$6 rotations (Skelton, 2024). The cited work replaces optimization or root-finding with Fejér factorization via a Wilson / Newton-Raphson iteration, and its “QSP” terminology belongs to quantum algorithm design rather than numerical quadratic optimization (Skelton, 2024).

A common misconception is therefore to read “QSP” as a unique method across all fields. The literature instead supports a narrower conclusion: in optimization, the phrase is most coherent when it refers to quadratic problems solved or analyzed through iteratively constructed low-dimensional subspaces, while in sparse recovery and quantum computing the same initials designate different mathematical programs and different algorithmic traditions.