Subspace Projection Strategies

• Subspace Projection Strategies are methods that restrict computation to a targeted linear or affine subspace to solve estimation, optimization, filtering, and clustering problems.
• They utilize techniques such as orthogonal, weighted, and iterative projections to enhance signal preservation, drift invariance, and discriminative features.
• Their effectiveness relies on precise subspace construction, robust projection operators, and acceleration methods tailored to domain-specific challenges.

Searching arXiv for the cited papers and topic coverage to ground the article in published work. Subspace projection strategies comprise a broad class of methods that solve estimation, optimization, filtering, clustering, and representation problems by restricting computation to a selected linear or affine subspace, or by removing an identified subspace from the ambient space. Across domains, the central operation is either to project data, iterates, operators, or hidden representations onto a subspace chosen for geometric, statistical, or computational reasons. The selected subspace may be cluster-revealing, drift-invariant, discriminative, signal-preserving, reasoning-specific, low-rank, or feasibility-defining, and the projection may be exact, approximate, orthogonal, weighted, polynomially parameterized, or implemented iteratively. The literature shows that subspace projection is not a single technique but a family of strategies whose effectiveness depends on how the subspace is constructed, how the projection is computed, and how projection error interacts with the downstream task (Scrucca, 2021, Yi et al., 2018, Ginat, 2018).

1. Foundational formulations

A common abstract formulation is to choose a subset of vectors whose span maximizes the norm of the projection of a target vector. In a Hilbert space setting with ground set $X$, target vector $n$, and matroid $(X,\mathcal I)$, the optimization problem is

$$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$

where $P_n(E)$ is the orthogonal projection of $n$ onto $\operatorname{span}(E)$ (Zhang et al., 2015). The problem is generally NP-hard, and the principal algorithmic question becomes how to approximate the optimal subspace under structural constraints (Zhang et al., 2015).

A second foundational formulation arises in feasibility and best approximation. For closed affine subspaces $M_1,\dots,M_n\subseteq H$ with nonempty intersection $M=\bigcap_{i=1}^n M_i$, the problem is

$$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$

equivalently, computing the projection $n$0 (Tam, 2020). Here the subspace projection strategy is not to choose a low-dimensional representation, but to reach the intersection by repeated projections onto constituent subspaces (Ginat, 2018, Pang, 2014).

A third formulation treats projection as a feature-extraction or denoising primitive. In electronic-nose drift correction, one learns a projection matrix $n$1 mapping source and target data into a low-dimensional representation,

$n$2

with the goal of reducing source–target discrepancy while preserving class structure (Yi et al., 2018). In image denoising, NBNet constructs a basis matrix $n$3 and projects features via

$n$4

thereby treating denoising as image-adaptive projection into a learned signal subspace (Cheng et al., 2020).

These formulations already indicate three recurring roles of subspace projection: maximizing informative projection energy, enforcing feasibility through repeated orthogonal projection, and suppressing nuisance variation by restricting representation to a structured subspace.

2. Subspace construction principles

The literature distinguishes several ways to build the projection subspace, each tied to a different notion of relevance.

In projection-pursuit clustering, PPGMMGA seeks an orthogonal basis $n$5 for a $n$6-dimensional subspace $n$7, with $n$8, and projects data as

$n$9

The basis is optimized by maximizing a projection-pursuit criterion based on negentropy,

$(X,\mathcal I)$0

using Gaussian-mixture density estimation and genetic algorithms (Scrucca, 2021). The chosen subspace is therefore the one in which the projected distribution is most non-Gaussian, and the paper interprets such directions as cluster-revealing (Scrucca, 2021).

In discriminative anti-drift learning, D-DRCA extends DRCA by constructing a subspace that jointly reduces mean distribution discrepancy and imposes within-class compactness and between-class separation. Its final objective is

$(X,\mathcal I)$1

so the selected projection directions are simultaneously domain-invariant and class-discriminative (Yi et al., 2018).

In generalized difference subspace methods, the subspace is built from class subspaces. For class-specific projection matrices $(X,\mathcal I)$2, the sum matrix

$(X,\mathcal I)$3

is formed, and the generalized difference subspace is spanned by the eigenvectors of $(X,\mathcal I)$4 corresponding to its smallest eigenvalues (Fukui et al., 2019). The construction suppresses directions shared by many class subspaces and preserves low-energy directions that emphasize class differences (Fukui et al., 2019).

In HARP, the subspace is derived from the unembedding matrix of a LLM. With

$(X,\mathcal I)$5

the top singular directions define the semantic subspace and the remaining directions define the reasoning subspace,

$(X,\mathcal I)$6

using a low-rank approximation because exact zero singular values are rare (Hu et al., 15 Sep 2025). The selected subspace is therefore determined by the spectral action of the unembedding layer rather than by task labels alone (Hu et al., 15 Sep 2025).

In CoQuant, the high-precision subspace for mixed-precision quantization is selected from a mixed covariance

$(X,\mathcal I)$7

and the optimal basis is given by the top-$(X,\mathcal I)$8 eigenvectors of $(X,\mathcal I)$9 (Ding et al., 29 Apr 2026). This makes the subspace jointly sensitive to activation and weight perturbations rather than activation-only (Ding et al., 29 Apr 2026).

A plausible implication is that “subspace projection strategy” is best understood not by the act of projection alone, but by the principle used to define the subspace: distributional salience, class geometry, operator spectrum, perturbation sensitivity, or physical invariance.

3. Projection operators and computational mechanisms

Once a subspace has been chosen, the next design question is how projection is computed.

For standard linear subspaces, orthogonal projection is the basic operator. In decentralized sensor networks, the desired estimate is

$$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$0

with $$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$1 and $$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$2 (Romero et al., 2020). The challenge there is not the definition of $$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$3, but implementing it distributively through graph filters (Romero et al., 2020, Mollaebrahim et al., 2020).

In NBNet, the projection is explicitly non-orthonormality-aware: $$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$4 Because the learned basis vectors are not guaranteed to be orthogonal, the method uses the standard projection matrix for a non-orthogonal basis (Cheng et al., 2020).

In HARP, the projection onto the reasoning subspace is coordinate extraction in the right-singular basis,

$$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$5

where $$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$6 (Hu et al., 15 Sep 2025). The projection is therefore low-dimensional and feature-producing rather than reconstructive (Hu et al., 15 Sep 2025).

In low-rank signal estimation governed by generalized linear recurrence relations, the weighted orthogonal projection onto $$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$7 is

$$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$8

and the paper develops a stable algorithm for constructing the projection onto a subspace of time series satisfying a given GLRR (Zvonarev et al., 2021). The projection becomes a core routine in a variable-projection Gauss–Newton solver (Zvonarev et al., 2021).

In wideband DOA estimation, the object projected is itself a projector-valued function of frequency. The signal-subspace projector is

$$\max_{E \in \mathcal I} \; \left\| P_{n}(E) \right\|^{2},$$9

and the method approximates $P_n(E)$0 by a polynomial,

$P_n(E)$1

fit from sampled projector estimates (Selva, 2017). Because the polynomial fit is not exactly idempotent, it is then projected back onto the set of rank-$P_n(E)$2 orthogonal projectors by solving

$P_n(E)$3

(Selva, 2017).

These examples show that “projection” may refer to a direct orthogonal map, a weighted constrained projector, a spectral-coordinate restriction, or a projection onto the manifold of projectors.

4. Iterative projection methods for intersections of subspaces

A major branch of the field studies repeated projections onto multiple subspaces.

The method of alternating projections considers closed subspaces $P_n(E)$4, orthogonal projections $P_n(E)$5, an initial vector $P_n(E)$6, and iterates

$P_n(E)$7

When the order is periodic, Halperin’s theorem states that for

$P_n(E)$8

one has

$P_n(E)$9

(Ginat, 2018). For two subspaces, von Neumann’s theorem gives

$n$0

for any $n$1 (Ginat, 2018).

For affine subspaces, cyclic projection uses

$n$2

and converges to the best approximation point $n$3 (Tam, 2020). The standard convergence rate is bounded by

$n$4

with $n$5 determined by Friedrichs angles between the parallel linear subspaces (Tam, 2020).

Acceleration strategies modify the basic projection step. Gearhart–Koshy acceleration uses

$n$6

with exact line-search step size minimizing the squared distance to the solution (Tam, 2020). For affine subspaces, the paper derives the explicit formula

$n$7

which depends only on intermediate projection steps computed during one cyclic pass (Tam, 2020).

A different acceleration mechanism uses supporting hyperplanes. Each projection of $n$8 onto an affine subspace $n$9 produces $\operatorname{span}(E)$0, normal vector $\operatorname{span}(E)$1, and hyperplane

$\operatorname{span}(E)$2

which contains the intersection $\operatorname{span}(E)$3 (Pang, 2014). The accelerated schemes project not only onto the current subspace but onto intersections of previously generated hyperplanes,

$\operatorname{span}(E)$4

and converge strongly under explicit boundedness and selection conditions (Pang, 2014).

For two subspaces in $\operatorname{span}(E)$5, the method of simultaneous projections uses

$\operatorname{span}(E)$6

while alternating projections uses

$\operatorname{span}(E)$7

When the common starting point lies in $\operatorname{span}(E)$8, the paper proves

$\operatorname{span}(E)$9

and relates their Q-linear rates by

$M_1,\dots,M_n\subseteq H$0

so MAP is faster in that setting (Reich et al., 2023). However, the same paper also provides examples where simultaneous projections outperform alternating projections for specially chosen starting points (Reich et al., 2023).

A recurring misconception is that repeated orthogonal projection always yields norm convergence regardless of ordering. The Hilbert-space results are more precise: periodic and quasiperiodic orders yield norm convergence, arbitrary orders yield weak convergence, and sufficiently pathological orders can fail to converge in norm (Ginat, 2018).

5. Domain-specific strategies

Subspace projection strategies diverge sharply across application areas because the nuisance structure being removed or preserved is domain-specific.

In clustering, the strategy is “project first, cluster second.” PPGMMGA identifies a low-dimensional projection where mixture structure is pronounced, after which a modal EM algorithm estimates modes of a parsimonious Gaussian-mixture density on the projected data. The MEM update has closed form,

$M_1,\dots,M_n\subseteq H$1

with a damped iteration

$M_1,\dots,M_n\subseteq H$2

and cluster labels are assigned by domains of attraction of estimated modes (Scrucca, 2021).

In sensor drift correction, D-DRCA learns a projection that simultaneously aligns source and target domains and preserves source-label semantics. It is solved by a generalized eigenvalue problem,

$M_1,\dots,M_n\subseteq H$3

and then used as a feature extractor for classification (Yi et al., 2018).

In array radio telescopes, the direction of projection is reversed: the goal is to remove an RFI subspace. Under stationarity, one uses

$M_1,\dots,M_n\subseteq H$4

to project out the interference contribution (Hellbourg, 2018). But when the interferer moves during covariance estimation, the RFI covariance can smear across the whole data vector space, and the paper states that subspace projection is then “not relevant anymore” (Hellbourg, 2018). The recommended alternative is covariance subtraction using a lagged covariance estimate,

$M_1,\dots,M_n\subseteq H$5

with

$M_1,\dots,M_n\subseteq H$6

(Hellbourg, 2018).

In quantum Krylov subspace diagonalization, the projection is onto a quantum Krylov subspace generated by

$M_1,\dots,M_n\subseteq H$7

producing projected Hamiltonian and overlap matrices

$M_1,\dots,M_n\subseteq H$8

The practical difficulty is finite-sampling noise in these projected quantities, and the paper proposes two strategies, the shifting technique and coefficient splitting, to reduce measurement variance under a fixed shot budget (Lee et al., 2024).

In graph analytics on evolving graphs, the strategy is to update leading eigenvectors by Rayleigh–Ritz projection onto a restricted subspace $M_1,\dots,M_n\subseteq H$9 with basis $M=\bigcap_{i=1}^n M_i$0, solving the projected eigenproblem

$M=\bigcap_{i=1}^n M_i$1

The proposed subspace includes the padded old eigenspace together with projected update directions and a term derived explicitly from the new-node block $M=\bigcap_{i=1}^n M_i$2,

$M=\bigcap_{i=1}^n M_i$3

which addresses a limitation of first-order perturbation methods that ignore $M=\bigcap_{i=1}^n M_i$4-block interactions among newly added nodes (Eini et al., 19 Mar 2026).

These cases show that the projection operator may preserve a signal subspace, eliminate an interference subspace, regularize a density estimate, or compress dynamic eigenspace updates; what unifies them is subspace-restricted computation, not a single statistical objective.

6. Decentralized and operator-design perspectives

A distinct line of work treats subspace projection as an operator-synthesis problem under communication constraints.

For decentralized projection in sensor networks, one seeks a graph filter

$M=\bigcap_{i=1}^n M_i$5

that equals the desired projector $M=\bigcap_{i=1}^n M_i$6 while respecting the graph topology (Romero et al., 2020). Exact implementation requires polynomial feasibility: $M=\bigcap_{i=1}^n M_i$7 The key spectral condition is that the eigenvalues associated with the signal subspace block and the orthogonal-complement block of $M=\bigcap_{i=1}^n M_i$8 must be disjoint (Romero et al., 2020).

Because minimizing the number of distinct eigenvalues is nonconvex, the paper introduces a convex relaxation based on the nuclear norm identity

$M=\bigcap_{i=1}^n M_i$9

which encourages repeated eigenvalues and therefore lower-order filters (Romero et al., 2020). When exact implementation is impossible for a given topology, the paper formulates an approximate design that trades approximation error against the number of communication rounds (Romero et al., 2020).

For asymmetric directed networks, the same projection problem is addressed through a Schur-decomposition parameterization of the shift operator,

$$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$0

with $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$1 diagonal and $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$2 strictly upper triangular (Mollaebrahim et al., 2020). Exact projection again requires spectral separation between the “kept” and “discarded” eigenvalue sets, and the design is cast as a convex quadratic program with linear constraints and solved by ADMM (Mollaebrahim et al., 2020).

These works suggest a broader interpretation of subspace projection strategy: the subspace need not merely be selected; one may instead design a local operator whose polynomial action realizes the projector with finite communication.

7. Performance regimes, guarantees, and limitations

The literature repeatedly emphasizes that projection strategies are effective only under conditions matching the geometry or noise model of the task.

Greedy subspace selection under matroid constraints admits curvature-dependent guarantees. If the ground-set vectors are mutually orthogonal, forward regression and orthogonal matching pursuit are optimal for uniform matroids, and achieve at least $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$3-approximations for non-uniform matroids (Zhang et al., 2015). When vectors are not mutually orthogonal, the guarantees depend on forward and backward elemental curvatures and on a principal-angle quantity $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$4 (Zhang et al., 2015).

Alternating-projection methods are robust but not uniformly fast. Gearhart–Koshy acceleration improves the upper bound relative to plain cyclic projections by a multiplicative factor $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$5, but for $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$6 subspaces it can sometimes be slower than the unaccelerated method (Tam, 2020). Supporting-hyperplane accelerations can perform dramatically better in practice, but their strong convergence proofs require explicit boundedness and span conditions (Pang, 2014).

In RFI mitigation, the low-rank assumption behind orthogonal projection can break down under motion-induced smearing, at which point projection begins to remove source-of-interest components as well (Hellbourg, 2018). In QKSD, projection onto a Krylov subspace leads to an ill-conditioned generalized eigenvalue problem, so sampling error in projected matrix elements can dominate the final eigenvalue error unless measurement strategy is optimized (Lee et al., 2024).

In representation-learning settings, the dimension of the protected or projected subspace is critical. HARP reports that the reasoning-subspace dimension is approximately $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$7 of the original hidden size, and that performance peaks around 256 dimensions in its dimension study (Hu et al., 15 Sep 2025). NBNet reports that moderate basis counts such as $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$8 and $$\min_{x\in H}\ \|x-x_0\|^2 \quad \text{subject to } x\in M,$$9 work well, while $n$00 causes training not to converge (Cheng et al., 2020). CoQuant similarly preserves only $n$01 of channels in 8-bit while the rest are quantized to 4-bit in its mixed-precision setup, reflecting a structured high-precision subspace rather than uniform precision allocation (Ding et al., 29 Apr 2026).

A plausible implication is that the effectiveness of a subspace projection strategy is often controlled less by the existence of a projection formula than by whether the chosen subspace remains low-dimensional, identifiable, and well matched to the perturbations or structures that matter in the application.