Repeated Projected Gradient Descent (RPGD)

• Repeated Projected Gradient Descent (RPGD) is an iterative method that applies a gradient step followed by a projection to enforce feasibility in constrained optimization problems.
• It is versatile across domains, adapting projection schemes for setups ranging from quantum state tomography to model-based sparsity and low-rank matrix recovery.
• Algorithmic enhancements like momentum, backtracking, and preconditioning improve convergence rates while balancing computational cost and projection accuracy.

Searching arXiv for recent and foundational papers on projected gradient descent and repeated projected gradient descent across quantum tomography, constrained optimization, low-rank recovery, and nonconvex stationarity. “Repeated Projected Gradient Descent” (“RPGD”, Editor’s term) denotes the repeated application of a projected-gradient update of the form $x_{k+1}=\mathcal P(x_k-\alpha_k\nabla f(x_k))$, or domain-specific variants with the same gradient-plus-projection structure, on constrained optimization problems whose feasible sets range from convex sets to nonconvex closed sets, structured-sparse models, low-rank varieties, and quantum-state spaces (Lapucci et al., 23 Jan 2026). In the surveyed literature, the term itself is often implicit rather than explicit, but the underlying mechanism is stable across applications: a gradient step is computed in an ambient Euclidean or matrix space, and the resulting iterate is projected back onto a set that encodes feasibility, structure, or physical admissibility (Olikier et al., 2024). In quantum state tomography, for example, the basic update is written as $$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$, with $\mathcal S$ enforcing positivity and unit trace (Bolduc et al., 2016).

1. Canonical iteration and problem classes

At its most classical, RPGD addresses constrained smooth optimization problems of the form

$\min_{x\in\mathcal S} f(x),$

where $\mathcal S\subset\mathbb R^n$ is a closed, convex set and $f$ is smooth, possibly nonconvex. The baseline projected-gradient iteration is

$$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$

or, in a direction-search formulation,

$$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$

This setting underlies the general convergence and complexity theory developed for projected gradient methods with momentum (Lapucci et al., 23 Jan 2026).

A broader formulation allows $\mathcal C$ to be merely nonempty and closed, possibly highly nonconvex, with set-valued metric projection

$P_C(x):=\operatorname*{argmin}_{y\in C}\|x-y\|.$

In that setting, one projected-gradient step is

$$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$0

typically combined with an Armijo-type backtracking line search. The resulting infinite sequence is the object studied in the modern stationarity theory for PGD on arbitrary closed sets (Olikier et al., 2024).

The same template appears in highly structured inverse problems. In model-based sparsity, the iteration is

$$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$1

with projection onto a model-sparse ball (Bahmani et al., 2012). In low-rank matrix estimation, the update is

$$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$2

where $$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$3 is truncated SVD onto rank-$$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$4 matrices (Zhang et al., 2024). In spectral compressed sensing, projected gradient descent is applied to a non-convex factorized objective after a one-step hard-thresholding initialization (Cai et al., 2017). In generalized low-rank tensor regression, projected gradient descent is repeatedly applied to a potentially non-convex tensor constraint set $$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$5 (Chen et al., 2016).

These formulations differ in geometry, but they share a common separation of roles: the gradient step optimizes the objective in ambient coordinates, whereas the projection step reimposes feasibility.

2. Projection geometry and feasible sets

The projection operator is the defining structural element of RPGD. In convex settings it is the Euclidean projection

$$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$6

which is single-valued and central to the projected-gradient stationarity measure

$$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$7

used in general nonconvex-complexity analysis (Lapucci et al., 23 Jan 2026). When the feasible set is only closed, the metric projection may be set-valued, and the algorithm may choose any element of $$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$8 (Olikier et al., 2024).

Across applications, the projection map is adapted to the target structure.

Feasible structure	Projection mechanism	Representative papers
Density matrices $$\rho_{k+1}=\mathcal S[\rho_k-\delta\nabla\mathcal C(\rho_k)]$$9	Eigen-decomposition plus simplex projection on eigenvalues using Michelot’s algorithm	(Bolduc et al., 2016)
Model-based sparsity $\mathcal S$0	Best support in $\mathcal S$1, then scaling to radius $\mathcal S$2	(Bahmani et al., 2012)
Rank-$\mathcal S$3 matrices	Best rank-$\mathcal S$4 approximation by truncated SVD	(Zhang et al., 2024)
Incoherence-constrained low-rank factors	Row-wise clipping onto $\mathcal S$5 or $\mathcal S$6	(Xu et al., 2024, Cai et al., 2017)
Box and balance constraints for graph partitioning	Projection onto $\mathcal S$7 and balance slabs via alternating projections, Dykstra, or exact projection for $\mathcal S$8	(Avdiukhin et al., 2019)

In quantum tomography, the projection $\mathcal S$9 is implemented by diagonalizing a Hermitian matrix $\min_{x\in\mathcal S} f(x),$0, projecting the eigenvalue vector onto the probability simplex, and reconstructing $\min_{x\in\mathcal S} f(x),$1 (Bolduc et al., 2016). In structured sparsity, the bounded projection is reduced to an unbounded model projection followed by a radial scaling step (Bahmani et al., 2012). In low-rank optimization, truncated SVD realizes projection onto $\min_{x\in\mathcal S} f(x),$2 and is therefore the analogue of Euclidean projection for the determinantal variety (Zhang et al., 2024).

For nonconvex sets, projection regularity becomes a central analytical issue. The local least-squares theory of PGD uses differentiability and Lipschitz-continuous differentiability of $\min_{x\in\mathcal S} f(x),$3 at the solution and at the post-gradient point $\min_{x\in\mathcal S} f(x),$4, yielding an exact asymptotic linear rate through the linearized map

$\min_{x\in\mathcal S} f(x),$5

(Vu et al., 2021). A plausible implication is that, in RPGD, projection design is not a secondary implementation detail; it is often the decisive object controlling both local geometry and attainable rates.

3. Algorithmic variants, acceleration, and related projected schemes

The simplest RPGD scheme uses a single fixed or line-searched projected gradient step per iteration, but the literature contains several systematic variants. In quantum tomography, three projected-gradient algorithms are studied: Projected Gradient Descent with Momentum (PGDM), FISTA, and Projected Gradient Descent with Backtracking (PGDB) (Bolduc et al., 2016). PGDM updates a velocity term,

$\min_{x\in\mathcal S} f(x),$6

whereas FISTA uses Nesterov-type extrapolation,

$\min_{x\in\mathcal S} f(x),$7

PGDB instead enforces monotonic decrease by backtracking along a projected direction (Bolduc et al., 2016).

A more abstract momentum construction is developed for convex-constrained smooth optimization in the Projected Gradient Method with Momentum (PGMM). Rather than inserting a raw inertial term inside the projection, PGMM builds two projected directions,

$\min_{x\in\mathcal S} f(x),$8

and combines them as

$\min_{x\in\mathcal S} f(x),$9

with $\mathcal S\subset\mathbb R^n$0 chosen by a two-dimensional quadratic program and an Armijo line search on $\mathcal S\subset\mathbb R^n$1 (Lapucci et al., 23 Jan 2026). The paper explicitly notes that the acronym RPGD is not used there, but conceptually the method is a repeated projected-gradient scheme with momentum (Lapucci et al., 23 Jan 2026).

Preconditioning and scaling form another variant class. In deterministic matrix completion, standard PGD is applied to a lifted balanced objective $\mathcal S\subset\mathbb R^n$2, while a scaled PGD variant applies gradient steps preconditioned by $\mathcal S\subset\mathbb R^n$3 and $\mathcal S\subset\mathbb R^n$4 on the unregularized factorized objective $\mathcal S\subset\mathbb R^n$5, followed by projection onto an incoherence set (Xu et al., 2024). In low-rank matrix estimation, a perturbed projected-gradient method interleaves ordinary projected steps with tangent-space perturbation phases when progress stalls, in order to escape saddle points and converge to an approximate solution or a second-order local minimizer (Zhang et al., 2024).

Projected-projected gradient descent (PPGD) is closely related but distinct. In low-rank optimization on the determinantal variety, the method first projects $\mathcal S\subset\mathbb R^n$6 onto the Bouligand tangent cone, then projects the trial point back to the rank-bounded set. A new “P with rank reduction” algorithm augments this with rank-reduced projections and selects the best candidate decrease at each iteration (Olikier et al., 2022). This suggests that, within the broader RPGD family, the gradient direction itself may be projected before the iterate is projected.

4. Convergence, stationarity, and complexity

The convergence theory of RPGD is fragmented across geometric regimes. For smooth, convex-constrained optimization, the modern baseline is a worst-case nonconvex complexity of $\mathcal S\subset\mathbb R^n$7 to reach $\mathcal S\subset\mathbb R^n$8, where

$\mathcal S\subset\mathbb R^n$9

This bound is proved for classical projected gradient directions under Armijo line search, and the same $f$0 order is retained by the momentum-enhanced PGMM framework (Lapucci et al., 23 Jan 2026).

For arbitrary nonempty closed sets $f$1, the stationarity picture is sharper but more geometric. Under continuous differentiability, every accumulation point of PGD is Bouligand stationary, and under locally Lipschitz continuous gradient on the ambient space, every accumulation point is proximally stationary (Olikier et al., 2024). The same work emphasizes that Bouligand and proximal stationarity are the strongest stationarity properties that can be expected in that setting, and contrasts PGD with another projected method, P$f$2GD, that can converge to Mordukhovich-stationary points which are not Bouligand stationary (Olikier et al., 2024).

On the determinantal variety $f$3, first-order low-rank methods face the specific pathology called an apocalypse: a convergent sequence whose limit is not Bouligand stationary. The rank-reduction-enhanced projected-projected scheme of (Olikier et al., 2022) is designed precisely to avoid that phenomenon. Its finite termination point is stationary, and every accumulation point of an infinite sequence is Bouligand stationary (Olikier et al., 2022).

Local linear theory is particularly explicit for constrained least squares. If $f$4 is a fixed point of

$f$5

and the projection is Lipschitz-continuously differentiable at $f$6 and at $f$7, then the asymptotic linear rate is the spectral radius $f$8 of the linearized map $f$9, and the paper gives an explicit region of convergence and an iteration bound to reach any relative accuracy $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$0 (Vu et al., 2021). In manifold-like cases, the rate simplifies to the familiar restricted-Hessian expression

$$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$1

where $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$2 are the largest and smallest eigenvalues of the Hessian restricted to the tangent space (Vu et al., 2021).

Several application-specific linear convergence results complement this general theory. In deterministic matrix completion with Ramanujan sampling, PGD on the lifted balanced objective converges linearly, with a rate that depends on the condition number $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$3 of the ground truth, while scaled PGD achieves a linear rate independent of $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$4 under its local curvature and smoothness conditions (Xu et al., 2024). In low-rank matrix estimation under rank-$$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$5 restricted $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$6-smoothness and $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$7-strong convexity, projected gradient descent has a local convergence rate independent of the effective condition number $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$8; when $$x_{k+1}=\mathcal P_{\mathcal S}[x_k-\alpha_k\nabla f(x_k)],$$9, it also enjoys global linear convergence for an explicit interval of step sizes, and the constrained landscape has no spurious local minimizers (Zhang et al., 2024).

In quantum tomography, convergence is studied empirically rather than through a general theorem. PGDB is monotonic in the likelihood cost, whereas PGDM and FISTA are non-monotonic but converge rapidly in practice; all three methods reach the same maximum-likelihood state in the reported experiments (Bolduc et al., 2016).

5. Representative application domains

RPGD appears in the literature as a reusable computational pattern rather than as a single domain-specific algorithm.

Domain	Representative formulation	Representative papers
Quantum state tomography	Maximum-likelihood estimation over density matrices $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$0	(Bolduc et al., 2016)
Model-based sparsity and GLMs	$$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$1 subject to $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$2	(Bahmani et al., 2012)
Generalized low-rank tensor regression	PGD on non-convex tensor constraint sets $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$3	(Chen et al., 2016)
Spectral compressed sensing	PGD on a factorized low-rank Hankel objective with incoherence projection	(Cai et al., 2017)
Low-rank matrix completion and estimation	Rank-constrained PGD, scaled PGD, and perturbed projected gradient methods	(Xu et al., 2024, Zhang et al., 2024, Olikier et al., 2022)
Multi-dimensional balanced graph partitioning	Randomized PGD on a non-convex continuous relaxation with box and balance constraints	(Avdiukhin et al., 2019)

In quantum tomography, RPGD solves a convex maximum-likelihood problem over density matrices from noisy count data. The Gaussian negative log-likelihood cost

$$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$4

is minimized subject to positivity and unit trace, with per-iteration gradient evaluation and Born-rule evaluation costing $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$5 (Bolduc et al., 2016). The paper reports that PGD techniques reach $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$6 significantly faster than the diluted iterative algorithm and SDPT3 in the vast majority of scenarios (Bolduc et al., 2016).

In structured sparsity, the relevant constraint is a non-convex union of subspaces induced by a family of allowed supports $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$7. Under a Stable Model-Restricted Hessian condition, the projected gradient algorithm contracts linearly toward a model-consistent reference point, with an explicit decomposition into optimization error and approximation error (Bahmani et al., 2012). The same paper treats canonical generalized linear models as a principal application (Bahmani et al., 2012).

In generalized low-rank tensor regression, projected gradient descent is analyzed on potentially non-convex low-rank tensor sets through localized Gaussian width. The resulting statistical error scales with

$$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$8

and the paper argues that the non-convex approach has a superior rate for a number of examples relative to convex regularization (Chen et al., 2016). Low Tucker-rank estimation is the clearest example: the non-convex PGD bound depends on a minimum over matricization dimensions, whereas the convex bound depends on a maximum (Chen et al., 2016).

In spectral compressed sensing, the low-rank structure is transferred to the Hankel matrix $$\hat d_k=\mathcal P_{\mathcal S}[x_k-\eta_k\nabla f(x_k)]-x_k,\qquad x_{k+1}=x_k+\alpha_k\hat d_k.$$9. After one-step hard thresholding and projection, PGD is applied to a non-convex factorized objective that enforces both Hankel structure and data fidelity. Under a sampling-with-replacement model, $\mathcal C$0 observed entries are sufficient for successful recovery in the noiseless theorem, and the iterates converge linearly to the solution manifold (Cai et al., 2017).

In graph partitioning, randomized projected gradient descent is used on the continuous relaxation of multi-dimensional balanced graph partitioning. The iterate update is

$\mathcal C$1

with $\mathcal C$2 the intersection of the box $\mathcal C$3 and multiple balance slabs. Exact projection for $\mathcal C$4 has running time $\mathcal C$5, and experiments on graphs containing up to hundreds of billions of edges show superior performance to the compared approaches (Avdiukhin et al., 2019).

6. Limitations, misconceptions, and recurring trade-offs

A common misconception is that all projected first-order methods have essentially the same limiting behavior. The recent stationarity literature does not support that claim. PGD on arbitrary closed sets accumulates at Bouligand stationary points, and even proximally stationary points under locally Lipschitz gradient, whereas other projected methods can accumulate at weaker Mordukhovich-stationary points (Olikier et al., 2024). Likewise, low-rank projected-projected methods can suffer apocalypses unless the algorithm is modified with rank reduction (Olikier et al., 2022).

Another misconception is that adding momentum automatically preserves the convergence profile of baseline PGD. In the tomography literature, PGDM and FISTA are explicitly non-monotonic, even though they are often faster in practice; only PGDB is monotonic in the cost (Bolduc et al., 2016). In the abstract momentum framework, additional safeguards, spectral bounds on the local model, and Armijo line search are required to retain the standard projected-gradient complexity guarantees (Lapucci et al., 23 Jan 2026).

Projection cost is the main practical bottleneck in many RPGD instantiations. Exact model projections for general combinatorial sparsity can be difficult, and the structured-sparsity paper points to the need for specialized algorithms or approximate projections (Bahmani et al., 2012). In graph partitioning, exact Euclidean projection is practical only for small numbers of balancing dimensions; for larger $\mathcal C$6, alternating projections and Dykstra-type procedures become the scalable alternative (Avdiukhin et al., 2019). In low-rank varieties, larger numerical-rank thresholds can force exploration of multiple rank levels per iteration, increasing cost (Olikier et al., 2022).

Finally, global guarantees are highly problem-dependent. Local linear convergence for constrained least squares depends on projection smoothness and initialization inside an explicit region of convergence (Vu et al., 2021). Global linear convergence in low-rank matrix estimation requires the restricted-smoothness/strong-convexity regime with $\mathcal C$7 (Zhang et al., 2024). In quantum tomography, the main evidence is empirical rather than theorem-level global analysis (Bolduc et al., 2016). This suggests that “RPGD” names a structural algorithmic pattern, not a single theorem: its behavior is determined jointly by objective regularity, feasible-set geometry, projection exactness, and step-selection strategy.