Projection Onto Span Algorithm (POSA)

• Projection Onto Span Algorithm (POSA) is a strategy that projects current estimates onto a span formed by multiple measurement vectors rather than single coordinates.
• It is applied in diverse settings including coherent overdetermined systems, (min,+) dynamic programming, and wavelet-domain SAR processing for despeckling and superresolution.
• Recent approaches extend POSA through greedy span selection and sparse representation strategies to address combinatorial and non-convex optimization challenges.

Searching arXiv for recent and foundational papers related to POSA and projection-onto-span methods. Projection Onto Span Algorithm (POSA) denotes a projection-based design pattern in which an iterate, signal, or approximation is updated by projecting onto a span generated by selected measurement vectors, basis functions, or subband matrices, rather than by acting on a single coordinate or constraint at a time. In the available literature, this principle appears in several technically distinct forms: a POCS-type two-row solver for coherent overdetermined linear systems, projection onto a $(\min,+)$ subsemimodule for approximate dynamic programming, greedy maximization of projection norm under matroid constraints, and a wavelet-domain algorithm explicitly named “Projection Onto Span Algorithm (POSA)” for SAR despeckling and superresolution (Needell et al., 2012, Lakshminarayanan et al., 2014, Zhang et al., 2015, Mastriani, 2016).

1. Conceptual basis and mathematical scope

At its most general, POSA is a projection-onto-span procedure. For linear systems $Ax=b$, the span is generated by a selected block of rows of $A$; for value-function approximation, it is the $(\min,+)$ linear span of basis functions; for Hilbert-space subset selection, it is the subspace $\mathrm{span}(E)$ generated by a feasible set $E$; and for wavelet-domain SAR processing, it is the span of normalized subband matrices or low-resolution observations. This suggests that POSA is best understood as a family of projection constructions rather than a single universally fixed update rule.

The common operation is the replacement of a current object by its projection onto a low-dimensional model class. In conventional Euclidean notation, if $B$ collects basis vectors, the orthogonal projector onto $\mathrm{span}(B)$ is

$x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$

assuming $B^T B$ is invertible. Several of the cited works implement this idea implicitly rather than through an explicit matrix inverse. The linear-systems formulation uses Gram–Schmidt inside the span of two rows; the SAR formulation uses inner products of matrices with $Ax=b$0; and the $Ax=b$1 formulation replaces ordinary linear span by a subsemimodule and ordinary orthogonal projection by a pointwise minimal upper approximation.

Within this scope, POSA is closely related to POCS, block Kaczmarz, multi-subspace projection, and projection maximization. It is not identical to any one of them. The two-subspace linear-systems method is explicitly described as POCS-type; the wavelet-domain SAR algorithm is explicitly described as a linear projection algorithm; and the combinatorial selection formulation studies how to choose a span so that the projection of a target vector has maximum norm.

2. POSA for coherent overdetermined systems

For a consistent linear system

$Ax=b$2

with $Ax=b$3, $Ax=b$4 full rank, and standardized rows $Ax=b$5, the two-subspace projection method of Needell and Ward selects two distinct rows $Ax=b$6 and projects the current iterate onto the solution set defined by those two measurements. The paper does not use the name “Projection Onto Span Algorithm (POSA)” explicitly, but it introduces what is essentially a “projection onto span” strategy specialized to the case of two rows of $Ax=b$7 (Needell et al., 2012).

Given $Ax=b$8, the method picks $Ax=b$9 uniformly at random, with $A$0, defines the row correlation

$A$1

computes the intermediate projection

$A$2

constructs

$A$3

and then performs the second projection

$A$4

After these two projections, $A$5 lies in

$A$6

so each iteration is a projection onto the two-row affine subspace associated with $A$7.

The convergence theorem is stated in terms of the scaled condition number

$A$8

and the coherence parameters

$A$9

If $(\min,+)$0 denotes the estimate in the $(\min,+)$1-th iteration, then

$(\min,+)$2

where

$(\min,+)$3

The baseline factor $(\min,+)$4 is the rate corresponding to two standard randomized Kaczmarz steps; the additional term $(\min,+)$5 quantifies the acceleration obtained by exploiting row coherence. The paper’s interpretation is that standard Kaczmarz slows down when successive row hyperplanes are highly correlated, whereas the two-subspace projection disentangles those correlations by working in an orthonormal basis of the selected two-row span.

The same section of the literature also presents the direct two-row projector. If

$(\min,+)$6

then the orthogonal projection of $(\min,+)$7 onto $(\min,+)$8 is

$(\min,+)$9

This is the canonical block projection onto the span of two measurement vectors, written either as a pseudoinverse update or as two Kaczmarz-like scalar corrections after orthogonalization.

3. Projection onto span in $\mathrm{span}(E)$0 approximate dynamic programming

In infinite-horizon discounted reward MDPs, the paper on $\mathrm{span}(E)$1 approximate dynamic programming replaces the usual linear span of basis functions by a $\mathrm{span}(E)$2 linear span whose columns form a subsemimodule $\mathrm{span}(E)$3, where

$\mathrm{span}(E)$4

For $\mathrm{span}(E)$5 and $\mathrm{span}(E)$6,

$\mathrm{span}(E)$7

With basis matrix $\mathrm{span}(E)$8, the $\mathrm{span}(E)$9 span is

$E$0

so the approximant takes the form $E$1 (Lakshminarayanan et al., 2014).

The projection operator onto this span is not an orthogonal projector. It is defined by

$E$2

that is, the pointwise minimum among all elements of $E$3 that dominate $E$4. If

$E$5

then

$E$6

The paper states that, in the light of the discrete analogues of the Fenchel transform, this projection “is nothing but the min-Transform.”

The corresponding fixed-point equation is the $E$7 Projected Bellman Equation,

$E$8

or equivalently

$E$9

The optimization form used to compute $B$0 is

$B$1

for any $B$2 with strictly positive components. The associated Min-Plus Approximate Dynamic Programming (MPADP) algorithm starts from a feasible $B$3, computes

$B$4

and updates

$B$5

The paper proves finite termination for any $B$6 and gives the $B$7-error bound

$B$8

This formulation broadens the meaning of projection onto span. The span is not Euclidean, the projection is not orthogonal, and the geometry is governed by pointwise domination and idempotent algebra. A plausible implication is that POSA is not restricted to Hilbert-space orthogonal projection; it also includes algebraically compatible projection operators that preserve the structure of the ambient semiring.

4. Span selection, projection maximization, and greedy POSA variants

A different line of work formulates span selection itself as the optimization problem. Let $B$9 be a finite ground set of vectors in a Hilbert space $\mathrm{span}(B)$0, let $\mathrm{span}(B)$1 be a vector of interest, and let $\mathrm{span}(B)$2 be a family of feasible subsets. For any $\mathrm{span}(B)$3, if $\mathrm{span}(B)$4 denotes the orthogonal projection of $\mathrm{span}(B)$5 onto $\mathrm{span}(B)$6, the objective is

$\mathrm{span}(B)$7

The paper states that this problem is generally NP-hard, and studies forward regression and orthogonal matching pursuit as heuristic algorithms under uniform and non-uniform matroid constraints (Zhang et al., 2015).

Forward regression begins with $\mathrm{span}(B)$8 and at each step chooses a feasible element $\mathrm{span}(B)$9 maximizing

$x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$0

Orthogonal matching pursuit begins with residual $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$1, chooses a feasible element maximizing $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$2, updates $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$3, and resets

$x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$4

The incremental gain formula

$x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$5

shows that each greedy step is determined by the component of a candidate vector orthogonal to the current span.

Because $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$6 is monotone non-decreasing but not submodular in general, the paper introduces forward elemental curvature $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$7, backward elemental curvature $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$8, and OMP elemental curvature $x_{\mathrm{proj}} = B(B^T B)^{-1}B^T x,$9, together with the principal angle $B^T B$0. For the uniform matroid case, if $B^T B$1 is the forward regression solution, $B^T B$2 is an optimal solution with $B^T B$3, and

$B^T B$4

then

$B^T B$5

If $B^T B$6 is the OMP solution, then

$B^T B$7

For non-uniform matroids, the paper gives

$B^T B$8

and

$B^T B$9

The orthogonal case is especially transparent. If the elements in the ground set are mutually orthogonal, then forward regression and OMP are optimal under a uniform matroid,

$Ax=b$00

and they achieve at least $Ax=b$01-approximations of the optimal solution under a non-uniform matroid. This establishes a direct POSA interpretation: the quality of a projection-onto-span method depends not only on the update rule, but also on how the span itself is selected under combinatorial structure.

5. The explicitly named POSA in wavelet-domain SAR processing

The paper “New wavelet-based superresolution algorithm for speckle reduction in SAR images” explicitly defines the Projection Onto Span Algorithm (POSA) as a linear projection algorithm operating in the wavelet domain (Mastriani, 2016). Images or subbands are treated as elements of an inner-product space $Ax=b$02 of real matrices with

$Ax=b$03

The 2D DWT of a speckled SAR image produces the four subbands $Ax=b$04, $Ax=b$05, $Ax=b$06, and $Ax=b$07. After normalization,

$Ax=b$08

POSA constructs despeckled detail subbands by

$Ax=b$09

$Ax=b$10

$Ax=b$11

The reconstructed image is obtained by inverse DWT from $Ax=b$12, $Ax=b$13, $Ax=b$14, and $Ax=b$15. The paper uses the term POSAshrink for this despeckling procedure.

The same article gives two superresolution constructions. With four low-resolution observations $Ax=b$16, after normalizing $Ax=b$17,

$Ax=b$18

$Ax=b$19

$Ax=b$20

$Ax=b$21

With a single low-resolution observation $Ax=b$22 and auxiliary matrices $Ax=b$23, after normalizing $Ax=b$24,

$Ax=b$25

$Ax=b$26

$Ax=b$27

$Ax=b$28

In both cases, inverse 2D DWT yields a high-resolution image.

The SAR paper also specifies the underlying speckle model,

$Ax=b$29

with $Ax=b$30 of mean $Ax=b$31. For single-look SAR, the paper states that $Ax=b$32 is Rayleigh distributed for amplitude images and negative exponential for intensity images; for multi-look SAR, $Ax=b$33 has a gamma distribution with mean $Ax=b$34. The additive reformulation $Ax=b$35 is used to avoid the log-transform.

The reported despeckling experiment uses an ERS SAR Precision Image of the Buenos Aires area, size $Ax=b$36 pixels, 65536 gray levels, with Daubechies 1 wavelet and one level of decomposition. The original noisy image has $Ax=b$37, $Ax=b$38, $Ax=b$39, and $Ax=b$40. POSAshrink yields $Ax=b$41, $Ax=b$42, $Ax=b$43, $Ax=b$44, and $Ax=b$45. In the superresolution experiment, a $Ax=b$46 high-resolution image from Sierra Grande, Patagonia, resolution $Ax=b$47, is degraded into four $Ax=b$48 low-resolution frames with $Ax=b$49; the interpolated HR image using four LR frames has $Ax=b$50, while the POSA-based HR reconstruction has $Ax=b$51.

This explicit POSA differs from iterative POCS. The paper characterizes it as non-iterative: apply DWT, project the subbands or observations onto a chosen span, and reconstruct by inverse DWT. A frequent misconception is therefore to equate POSA with generic iterative POCS. In this SAR formulation, the projector acts on a span of matrices in a single wavelet-domain pass.

6. Related projection operators, interpretive boundaries, and extensions

Projection-based algorithms that enforce additional structural constraints can be used as building blocks in POSA-like methods. One example is Euclidean projection onto the non-convex set

$Ax=b$52

which is the intersection of a simplex-like set and a hypersphere and corresponds to a fixed value of Hoyer’s normalized sparseness measure

$Ax=b$53

The projection has the form

$Ax=b$54

or equivalently

$Ax=b$55

for a uniquely defined $Ax=b$56. The paper gives a linear time and constant space algorithm for computing this projector and a corresponding efficient algorithm for computing the product of the gradient of the projection with an arbitrary vector (Thom et al., 2013).

Although this sparseness-enforcing projection is not itself presented under the POSA name, it is conceptually close to projection-based sparse representation methods. This suggests a broader interpretation in which POSA can be combined with non-convex structural projectors: one projection step selects or constructs a span, and another enforces a prescribed sparsity level on the coefficients that live in that span.

The literature also places clear boundaries around the term. In the linear-systems setting, the method is POCS-type and randomized, with rigorous exponential convergence in expectation for coherent overdetermined systems (Needell et al., 2012). In the SAR setting, POSA is a wavelet-domain linear projection algorithm with explicitly stated formulas for $Ax=b$57, $Ax=b$58, and $Ax=b$59, and with reported advantages in despeckling and superresolution metrics (Mastriani, 2016). In the $Ax=b$60 MDP setting, projection onto span means projection onto a subsemimodule and is tied to the MPPBE rather than to ordinary orthogonal least squares (Lakshminarayanan et al., 2014). In the Hilbert-space subset-selection setting, the principal object is the norm of the orthogonal projection onto $Ax=b$61, and the central difficulty is NP-hard combinatorial span selection under matroid constraints (Zhang et al., 2015).

Taken together, these results support a technically precise encyclopedia view: POSA is a projection-onto-span methodology whose concrete instantiation depends on the algebra, geometry, and constraints of the ambient problem. Its core promise is not a single formula, but the reuse of a common strategy—construct a meaningful span from available measurements or basis elements, and project onto that span in a way that exploits structure more effectively than one-at-a-time updates.