---
title: Convergent Manifold Alternating Projection Iterations
url: https://www.emergentmind.com/topics/convergent-manifold-alternating-projection-iterations
type: topic
---

# Convergent Manifold Alternating Projection Iterations

“Convergent manifold alternating projection iterations” (*Editor's term*) denotes the class of alternating-projection-type schemes on smooth manifolds, semi-algebraic sets, convex sets, and related geometric constraint sets for which rigorous convergence results are available. In its canonical Euclidean form, the method seeks to minimize the squared distance between two sets,
\[
\min_{x\in X,\;y\in Y}\;g(x,y)=\|x-y\|_2^2,
\]
by alternating nearest-point projections,
\[
x_{k+1}\in P_X(y_k),\qquad y_{k+1}\in P_Y(x_{k+1}),
\]
with \(P_V(u)=\arg\min_{v\in V}\|v-u\|^2\). Across the literature, convergence is established through several complementary mechanisms: semi-algebraic/Kurdyka–Łojasiewicz analysis for nonconvex sets, transversality and principal-angle arguments for smooth manifolds, linear-regularity estimates for convex sets, and tangent-space or relaxed variants that replace exact projections by cheaper local surrogates while preserving local linear convergence [1802.03889] [1402.0550] [2101.07286] [2605.17384].

## 1. Canonical formulations and problem classes

A standard formulation uses the extended-value function
\[
f(x,y)=\|x-y\|^2+\delta_X(x)+\delta_Y(y),
\]
where \(\delta_X,\delta_Y\) are indicator functions and \(\partial f\) is the limiting/Mordukhovich subdifferential. In this setting, convergence means that the stacked iterates \(z_k=(x_k,y_k)\) approach a critical point \(z^*\) satisfying \(0\in\partial f(z^*)\) [1802.03889].

For manifold feasibility problems, the same idea is expressed geometrically. In phase retrieval and ptychographic imaging, one alternates between a data-consistency manifold
\[
{\mathcal N}=\{z\in\mathbb C^M:\ |z|=a\}
\]
and a linear subspace
\[
\mathcal M=\mathrm{Range}(Q)\subset\mathbb C^M,
\]
using the map
\[
T=P_{\mathcal M}\circ P_{\mathcal N}.
\]
Here
\[
P_{\mathcal M}(z)=Q(Q^*Q)^{-1}Q^*z,
\qquad
[P_{\mathcal N}(z)]_j=a_j\,\frac{z_j}{|z_j|},
\]
with arbitrary phase assignment when \(z_j=0\) [1402.0550].

Generalized alternating projections introduce relaxed projections and a mixing parameter. For two manifolds \(\M_1,\M_2\subset\mathbb R^n\), the operator is
\[
S=(1-\alpha)I+\alpha\bigl(P_{\M_2}^{\alpha_2}\circ P_{\M_1}^{\alpha_1}\bigr),
\]
with
\[
P_C^\alpha(x)=(1-\alpha)x+\alpha P_C(x),\qquad \alpha\in(0,2],
\]
and iteration \(x_{k+1}=Sx_k\) [2101.07286].

| Setting | Basic iteration | Proven limit concept |
|---|---|---|
| Semi-algebraic sets \(X,Y\) | \(x_{k+1}\in P_X(y_k),\; y_{k+1}\in P_Y(x_{k+1})\) | Convergence to a critical point of \(f\) |
| Smooth manifolds \(\mathcal M,\mathcal N\) | \(z^{(k+1)}=P_{\mathcal M}(P_{\mathcal N}(z^{(k)}))\) | Local linear convergence to the intersection |
| Generalized AP on manifolds | \(x_{k+1}=Sx_k\) | Local R-linear convergence |
| Tangent-space AP | Projection to affine tangent plane, then re-projection | Local linear convergence |

These formulations share the same underlying structure: repeated correction relative to two constraint sets. The analytical differences arise from how local geometry is encoded.

## 2. Geometric regularity conditions

In the semi-algebraic framework, two local properties play a central role. The first is the **three-point property** on one set, say \(Y\): there exist \(\alpha>0\) and \(\delta_\alpha:Y\times Y\to\mathbb R_+\) with
\[
\delta_\alpha(y,y')\ge \alpha\|y-y'\|_2^2,
\]
such that for all relevant \(\widetilde x\in \overline X\), all \(y\in \overline Y\), and any minimizer \(\widetilde y\in\arg\min_{y'\in Y}g(\widetilde x,y')\),
\[
g(\widetilde x,y)-g(\widetilde x,\widetilde y)\ge \delta_\alpha(y,\widetilde y).
\]
The second is the **local contraction property** on the other set, say \(X\): there exist \(\beta>0\) and \(\varepsilon>0\) such that
\[
\|P_X(y)-P_X(y')\|\le \beta\|y-y'\|,
\qquad
\forall\,y,y'\in\overline Y,\;\|y-y'\|\le\varepsilon.
\]
Closed convex sets satisfy these conditions with \(\alpha=1\) and \(\beta=1\), while the unit sphere yields a sphere-type identity for the three-point property [1802.03889].

In smooth manifold analyses, the corresponding notion is transversality or nontangentiality. For phase retrieval, local convergence is proved under the hypothesis that the tangent spaces
\[
T_{\mathcal M}(z^*)=\mathrm{Range}(Q),\qquad
T_{\mathcal N}(z^*)=\{w:\Re(\overline{z^*}\odot w)=0\}
\]
intersect transversely, meaning \(T_{\mathcal M}(z^*)+T_{\mathcal N}(z^*)=\mathbb C^M\) [1402.0550]. In tangent-space-based low-rank approximation, the analogous condition is \(\sigma(P)<1\), where \(\sigma(P)=\cos\alpha(P)\in[0,1)\); such a point is called non-tangential [2009.03998]. For generalized alternating projections on manifolds, the assumptions are that \(\M_i\) are \(\C^k\)-smooth and
\[
T_{\M_1\cap \M_2}(\bar x)=T_{\M_1}(\bar x)\cap T_{\M_2}(\bar x),
\]
with the stronger condition
\[
T_{\M_1}(\bar x)+T_{\M_2}(\bar x)=\mathbb R^n
\]
implying a positive Friedrichs angle \(\theta_F>0\) [2101.07286].

A more recent framework replaces transversality by **clean intersection**. If two \(C^{2,1}\) embedded submanifolds \(\mathcal M_1,\mathcal M_2\subset\mathbb R^n\) intersect cleanly, the associated alternating map admits a local limiting map on the intersection manifold; if the manifolds are \(C^{3,1}\), that limiting map is second-order [2605.17384].

These conditions are not interchangeable, but they play analogous roles: each supplies a local mechanism preventing tangential stalling and ensuring that the composition of projections contracts in directions normal to the intersection.

## 3. Convergence mechanisms

For semi-algebraic sets, convergence is derived from a combination of descent, subgradient control, and the Kurdyka–Łojasiewicz property. Under boundedness of the alternating-projection sequence and the two local geometric properties above, one obtains the **partial sufficient decrease**
\[
f(z_{k-1})-f(z_k)\ge \alpha\|y_{k-1}-y_k\|^2,
\]
hence \(\|y_k-y_{k-1}\|\to 0\). A subgradient estimate is then built via
\[
d_k=(2(y_{k-1}-y_k),\,0)\in\partial f(z_k),
\]
so that
\[
{\rm dist}(0,\partial f(z_k))\le 2\|y_{k-1}-y_k\|.
\]
The KL inequality converts these facts into summability,
\[
\sum_k \|y_k-y_{k-1}\|<\infty,
\]
which yields the Cauchy property of both \(\{y_k\}\) and \(\{x_k\}\), and therefore convergence to a critical point \(z^*\) with \(0\in\partial f(z^*)\) [1802.03889].

For manifold intersections, the mechanism is local linearization. Near a solution \(z^*\), the manifolds are represented as graphs over their tangent spaces, and the projection operators satisfy first-order expansions. For phase retrieval,
\[
e_{k+1}
=
\bigl(P_{T_{\mathcal M}}P_{T_{\mathcal N}}\bigr)e_k + O(\|e_k\|^2),
\]
where \(e_k=z^{(k)}-z^*\). If the tangent spaces intersect transversely, the linear part contracts, forcing linear convergence. In that setting the following three statements are equivalent for starts in a neighborhood of the solution circle: convergence of \(z^{(k)}\) to \(\mathcal M\cap\mathcal N\), decay of \(\|(I-P_{\mathcal N})z^{(k)}\|\) to zero, and decay of \(\|(I-P_{\mathcal M})P_{\mathcal N}(z^{(k)})\|\) to zero. Moreover, the only stagnation points in that neighborhood are true solutions up to global phase [1402.0550].

In generalized alternating projections on manifolds, the Jacobian at the solution is exactly the linearized relaxed-projection operator on tangent spaces,
\[
J\,S(\bar x)=S_T,
\]
and the local contraction estimate takes the form
\[
\|S(x)-P_{\M_1\cap\M_2}(x)\|\le c\,\|x-P_{\M_1\cap\M_2}(x)\|,
\qquad c=\|S_T-S_T^\infty\|<1.
\]
This yields local convergence to a point in \(\M_1\cap\M_2\) [2101.07286].

A common misconception is that local manifold convergence results automatically imply a global theorem. The literature does not support that conclusion. The semi-algebraic theory requires boundedness and tail containment in compact active sets; the smooth-manifold theory is explicitly local; and the generalized setting relies on neighborhood arguments or finite identification of active strata [1802.03889] [2101.07286].

## 4. Rates of convergence

The semi-algebraic/KL framework gives a rate determined by the KL exponent \(\theta\in[0,1)\) at the limiting critical point. If \(\theta=0\), the method terminates in finitely many steps. If \(0<\theta\le \tfrac12\), then there exist \(0<\rho<1\), \(C_1>0\), and \(k_0\) such that
\[
\|z_k-z^*\|\le C_1\rho^k,
\]
whereas for \(\tfrac12<\theta<1\),
\[
\|z_k-z^*\|\le C_2\,k^{-\frac{1-\theta}{2\theta-1}}
\]
for large \(k\). Thus \(\theta<\tfrac12\) yields linear convergence and \(\theta>\tfrac12\) yields sub-linear convergence [1802.03889].

In local smooth-manifold theory, rates are governed by principal or Friedrichs angles. For phase retrieval, if \(T_{\mathcal M}\) and \(T_{\mathcal N}\) are the tangent spaces at a solution, then the linearization has spectral radius
\[
\rho\bigl(d(P_{\mathcal M}P_{\mathcal N})_{z^*}\bigr)=\cos\theta_{\max}<1,
\]
and for nearby starts,
\[
\|z^{(k)}-z^*\|\le (\cos\theta_{\max})^k\,C.
\]
The contraction factor is therefore set by the maximal principal angle appearing in the local two-space model [1402.0550].

For generalized alternating projections on two linear subspaces, the optimal parameters are
\[
\gamma^*=1,\qquad
\beta_1^*=\beta_2^*=\frac{2}{1+\sin\theta_F},
\]
and the optimal asymptotic rate is
\[
\rho^*(\theta_F)=\frac{1-\sin\theta_F}{1+\sin\theta_F}.
\]
The manifold extension shows that the same asymptotic rate is obtained locally on smooth manifolds under the stated regularity assumptions [1703.10547] [2101.07286].

For convex sets, the semi-algebraic framework recovers the classical POCS conclusion: if \(X,Y\) are closed convex and \(\operatorname{ri}X\cap\operatorname{ri}Y\neq\emptyset\), then the alternating projection converges globally at a linear rate,
\[
\|z_k-z^*\|\le C\,\rho^k,\qquad \rho=1-\gamma<1,
\]
with \(\gamma>0\) depending on the relative angle/intersection geometry [1802.03889].

| Framework | Geometric quantity | Rate statement |
|---|---|---|
| KL semi-algebraic AP | KL exponent \(\theta\) | finite / linear / sub-linear |
| Smooth manifold AP | principal angles | factor \(\cos\theta_{\max}\) |
| GAP on subspaces/manifolds | Friedrichs angle \(\theta_F\) | \(\frac{1-\sin\theta_F}{1+\sin\theta_F}\) |
| Convex AP | relative angle/intersection geometry | global linear rate |

The rate results show that “alternating projection” is not a single asymptotic regime. Depending on geometry, the same algorithmic template can exhibit finite termination, linear convergence, or sub-linear decay.

## 5. Tangent-space, inexact, and retraction-based variants

A major line of development replaces expensive manifold projections by tangent-space approximations. For two \(C^2\) manifolds \(\M_1,\M_2\subset \K\), tangent-space-based alternating projections use the affine tangent plane \(\widetilde T_{\M_i}(x)=x+T_{\M_i}(x)\). Given \(y\in\K\) and \(x\in\M_i\), the orthogonal projection onto the affine tangent plane is
\[
P_{T_{\M_i}(x)}(y)
=
x+P_{T_{\M_i}(x)-\{0\}}(y-x),
\]
followed by exact re-projection to \(\M_i\). Under nontangential intersection, the resulting TAP iteration converges linearly to a common limit \(X_\infty\in\M\) and satisfies
\[
\|X_\infty-\pi(A)\|\le \epsilon\,\|A-\pi(A)\|,
\qquad
\|X_k-X_\infty\|\le C\,c^k\,\|A-\pi(A)\|
\]
for \(c\in(\sigma(A_0),1)\) and sufficiently local initialization [2003.10324].

For nonnegative low-rank matrix approximation, the manifold is
\[
M_r=\{X\in\mathbb R^{m\times n}\mid \operatorname{rank}(X)=r\},
\]
and the convex set is
\[
N=\{X\in\mathbb R^{m\times n}\mid X_{ij}\ge 0\}.
\]
If \(Y=U\Sigma V^T\in M_r\), the tangent-space projection is
\[
P_{T_{M_r}(Y)}(X)=U U^T X + X V V^T - U U^T X V V^T.
\]
TAP alternates
\[
Y^k=P_N(X^k),\qquad
Z^k=P_{T_{M_r}(X^k)}(Y^k),\qquad
X^{k+1}=P_{M_r}(Z^k),
\]
using only a small \(2r\times 2r\) SVD for retraction. Under the non-tangential condition \(\sigma(P)<1\), the sequence converges linearly to a point in \(M_r\cap N\) [2009.03998].

Generalized alternating projections with relaxation parameters extend this idea further. On manifolds, the local behavior is asymptotically identical to the linear-subspace model once the active manifolds are identified. However, finite identification cannot be assumed universally: the paper gives a counterexample with
\[
C=\{(x,y)\mid y\ge |x|\},\qquad D=\{y=0\}
\]
in \(\mathbb R^2\), for which the iterates converge linearly to the origin while alternating between two slanted faces of \(C\) and never identifying a single smooth branch [2101.07286].

The retraction viewpoint unifies exact and inexact schemes. Under clean intersection, the limit
\[
\psi(x,\eta)=\lim_{k\to\infty}\phi^k(x+\eta)
\]
defines a local retraction on \(\mathcal M=\mathcal M_1\cap\mathcal M_2\). If \(\mathcal M_1,\mathcal M_2\) are \(C^{2,1}\), then \(\psi\) is a first-order retraction with
\[
\psi(x,0)=x,\qquad D_\eta\psi(x,0)=I.
\]
If they are \(C^{3,1}\), then \(\psi\) is second-order. Exact APM, inexact AP, NewtonSLRA, and APHL all fit this framework [2605.17384].

## 6. Applications, extensions, and limitations

One application class is **structured tight frames**. For tight frames with prescribed column norms,
\[
X=\{D:\;DD^T=aI\},\qquad
Y=\{S:\|S_{:,j}\|^2=c_j\},
\]
the set \(Y\) satisfies the three-point property via a sphere-type identity, while \(X\) satisfies local contraction after excluding zero columns. The alternating projection method then converges to a critical point of \(\|D-S\|_F^2\). For equiangular tight frames,
\[
G\in G_a,\qquad H\in H_\xi,
\]
the same KL-based argument gives full convergence and rate estimates, with \(H\) convex and \(G\) a fixed-rank manifold on an active subset with spectral gap bounded below [1802.03889].

In ptychographic imaging, the constraint pair is
\[
\mathcal M=\mathrm{Range}(Q),\qquad \mathcal N=\{z:|z|=a\},
\]
and phase synchronization is used to build an accurate initial guess. The graph-connection-Laplacian
\[
L=D^{-1}W
\]
produces a spectral initializer whose top eigenvector synchronizes phases across all frames in a single step (GCL-PS), while a masked spectral alternative yields truncation-PS. These are used to accelerate the subsequent alternating-projection phase [1402.0550].

For nonnegative low-rank approximation, TAP is applied to data clustering, pattern recognition, and hyperspectral data analysis, and for quaternion color-image approximation it is applied to low-rank color-image reconstruction. In these studies the tangent-space scheme retains the convergence guarantees of the local manifold theory while reducing per-iteration cost relative to full projections [2009.03998] [2003.10324].

Broader geometric extensions show that the same conceptual template survives outside the Euclidean smooth-manifold setting. In CAT(\(\kappa\)) spaces with \(\kappa>0\), alternating projections onto closed convex sets are Fejér-monotone relative to the intersection, asymptotically regular, and \(\Delta\)-convergent; under bounded regularity or bounded linear regularity, one obtains strong or linear convergence [1611.01605]. In uniformly convex and uniformly smooth Banach spaces, alternating metric projections onto closed linear subspaces can be analyzed via alternating Bregman projections onto annihilators, again yielding linear convergence under bounded linear Bregman regularity [1905.00605].

The limits of the theory are also explicit. The nonconvex semi-algebraic results assume boundedness and eventual confinement to compact active sets [1802.03889]. The manifold results are local and depend on transversality, nontangentiality, or clean intersection [1402.0550] [2003.10324] [2605.17384]. Finite identification may fail even when the sequence still converges linearly [2101.07286]. In fixed-rank projector-splitting iteration, if the small-error hypothesis on the initial point fails, the iteration may converge to a spurious fixed point rather than the true solution [1604.02111].

Taken together, these results define a coherent theory of convergent alternating projection iterations on manifold-type sets. The theory explains why a gradient-free, step-size-free method can nonetheless admit precise convergence statements, and why rates are controlled not by a single universal principle but by the local geometry: KL exponents for semi-algebraic objectives, principal or Friedrichs angles for smooth intersections, and regularity constants for convex or metric-space variants [1802.03889] [1703.10547].

Source: https://www.emergentmind.com/topics/convergent-manifold-alternating-projection-iterations