Scaled Gradient Descent (ScaledGD)

• ScaledGD is a family of optimization methods that preconditions raw gradients with adaptive scaling operators, such as Gram matrices, to improve descent trajectories.
• It neutralizes ill-conditioning in low-rank factorization by using inexpensive, structured preconditioners, achieving linear convergence independent of the condition number.
• Variants extend to tensor recovery, convex constraints, and overparameterized models, offering robust empirical performance with only minimal extra computational cost.

Searching arXiv for relevant Scaled Gradient Descent papers to ground the article in the provided research context. Scaled Gradient Descent (ScaledGD) denotes a family of first-order optimization methods in which the raw gradient is multiplied by an explicit scaling or preconditioning operator before the parameter update. Across the literature, the term is used in several related but non-identical senses: diagonal or projected scaled gradients in constrained convex optimization (Jin et al., 2015), Gram-preconditioned factor updates for low-rank matrix and tensor estimation (Tong et al., 2020), Tucker- and t-SVD-based tensor recovery algorithms (Dong et al., 2022, Wu, 3 Jan 2025), damped preconditioning for overparameterized matrix sensing (Xu et al., 2023), Hessian-aware scalar gradient scalings (Smee et al., 6 Feb 2025), and task-specific gradient rescaling schemes for classification fine-tuning (Bui et al., 30 Apr 2026). The unifying principle is to alter the local metric of descent so that the update better matches curvature, factorization symmetry, constraint geometry, or sample-dependent optimization structure. In the low-rank estimation line of work, ScaledGD is especially associated with condition-number-robust linear convergence obtained by preconditioning each factor gradient with inverses of small Gram matrices (Tong et al., 2020, Dong et al., 2022).

1. Core definition and algorithmic forms

The most common contemporary use of “ScaledGD” arises in nonconvex low-rank factorization. For a rank-$r$ matrix parameterized as $X=LR^\top$, Tong, Ma, and Chi define the factorized loss $L(L,R)=f(LR^\top)$ and update the factors by

$$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$

with

$$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$

(Tong et al., 2020). The scaling matrices are thus iteration-varying, low-dimensional, and adapted to the opposing factor.

This same structural idea appears in tensor factorization. In Tucker-form tensor RPCA, one observes

$X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$

and minimizes

$L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$

over $F=(U^{(1)},U^{(2)},U^{(3)},C)$ and $S$, with scaled updates for each mode and the core tensor (Dong et al., 2022). In t-SVD/t-product tensor estimation, the factorization $\mathcal X_\star=U_\star\ast V_\star^H$ leads to updates of the form

$X=LR^\top$0

where $X=LR^\top$1 is the Gram tensor (Wu, 3 Jan 2025). A related t-SVD robust tensor PCA formulation uses

$X=LR^\top$2

together with sparse-threshold updates (Feng et al., 8 Jan 2025).

Outside low-rank estimation, the term can refer to broader variable-metric methods. In projected scaled gradient methods for convex optimization, the iterate is

$X=LR^\top$3

with diagonal positive-definite $X=LR^\top$4 and Euclidean projection $X=LR^\top$5 onto a convex set $X=LR^\top$6 (Jin et al., 2015). In Hessian-aware “first-ish order” methods, the scaling is a scalar $X=LR^\top$7 chosen from curvature information so that $X=LR^\top$8 satisfies a second-order descent condition (Smee et al., 6 Feb 2025). This suggests that ScaledGD is best understood as a design pattern rather than a single algorithm.

2. Preconditioning, symmetry, and conditioning

In low-rank factorization, the main motivation for scaling is to neutralize the bad geometry induced by ill-conditioning and factor non-uniqueness. The matrix product $X=LR^\top$9 is invariant under the reparameterization $L(L,R)=f(LR^\top)$0 for any $L(L,R)=f(LR^\top)$1. Standard gradient descent does not naturally respect this geometry, whereas ScaledGD uses the factor Gram matrices $L(L,R)=f(LR^\top)$2 and $L(L,R)=f(LR^\top)$3 as inexpensive local preconditioners (Tong et al., 2020, Ma et al., 2023).

The 2020 analysis makes this precise through a distance that both aligns the factors by an invertible matrix $L(L,R)=f(LR^\top)$4 and weights errors by $L(L,R)=f(LR^\top)$5: $L(L,R)=f(LR^\top)$6 (Tong et al., 2020). Under restricted strong convexity and smoothness over rank-$L(L,R)=f(LR^\top)$7 matrices, this metric contracts linearly with a rate independent of the matrix condition number $L(L,R)=f(LR^\top)$8, provided the initialization lies in a suitable local basin.

The condition-number robustness is a defining claim of the low-rank ScaledGD literature. For matrix sensing, robust PCA, and completion, the 2020 results show linear convergence with iteration count $L(L,R)=f(LR^\top)$9, independent of $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$0, in contrast to vanilla GD whose convergence depends linearly on $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$1 (Tong et al., 2020). The chapter-length synthesis in 2023 emphasizes the same point and interprets the Gram preconditioners as promoting desirable invariance properties without sacrificing the low per-iteration cost of factor-space GD (Ma et al., 2023).

A similar mechanism appears in Tucker and t-SVD tensor settings. In the Tucker RPCA algorithm, the update for each factor uses the inverse of a “preconditioned factor” Gram matrix derived from the current core tensor matricization, while the core update is scaled by inverse factor Grams in each mode (Dong et al., 2022). In simultaneous blind super-resolution and demixing, the authors explicitly state that the factors $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$2 and $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$3 “precondition” the gradient so that the method becomes robust to ill-conditioning and no extra balancing regularizer is needed (Chen, 2024).

3. Spectral initialization and basin-entry mechanisms

A recurring feature of ScaledGD in nonconvex low-rank estimation is the need for a tailored initialization that places the iterate inside a contraction region. The canonical construction is spectral.

For matrix recovery under Gaussian measurements, one computes $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$4, extracts its top-$$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$5 SVD $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$6, and sets

$$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$7

(Li et al., 31 Mar 2026). This “warm start” ensures

$$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$8

under suitable sampling conditions (Li et al., 31 Mar 2026).

In Tucker tensor RPCA, the spectral step is preceded by trimming: $$L_{t+1}=L_t-\eta_t\nabla_LL(L_t,R_t)(R_t^\top R_t)^{-1},\qquad R_{t+1}=R_t-\eta_t\nabla_RL(L_t,R_t)(L_t^\top L_t)^{-1},$$9 (Dong et al., 2022). Under a corruption fraction

$$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$0

the initialization obeys

$$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$1

thereby entering the basin in which ScaledGD exhibits global linear convergence (Dong et al., 2022).

In t-SVD tensor estimation, the initialization is also spectral but specialized to the corruption model. For robust PCA, one thresholds the observation tensor, performs a rank-$$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$2 truncated t-SVD, and splits the singular tensor symmetrically into factor tensors. For completion, one forms the scaled zero-filled observation $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$3, takes its t-SVD, and then projects the factors onto a scaled $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$4-ball to enforce incoherence (Wu, 3 Jan 2025).

The importance of initialization extends beyond merely attaining a local basin. A 2024 study on matrix factorization shows that Nyström initialization can qualitatively change the convergence regime of ScaledGD. In the symmetric case, setting $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$5 with Gaussian sketch $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$6 places the iterate in the column space of $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$7, and ScaledGD then exhibits a two-phase behavior: a linear phase followed by a quadratic phase (Li et al., 2024). In the asymmetric exact-parametrized setting, the same work proves one-step convergence with $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$8: $$\nabla_LL(L,R)=\bigl(\nabla f(LR^\top)\bigr)R,\qquad \nabla_RL(L,R)=\bigl(\nabla f(LR^\top)\bigr)^\top L$$9 under Nyström initialization $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$0 (Li et al., 2024). This suggests that, in some factorizations, initialization is not merely a technical device but a determinant of the attainable local rate.

4. Theoretical guarantees across major problem classes

The best-developed theoretical guarantees for ScaledGD concern linear convergence independent of condition number in low-rank estimation.

For general low-rank matrix estimation with restricted strong convexity and smoothness, if

$X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$1

then

$X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$2

(Tong et al., 2020). Problem-specific instantiations include matrix sensing, robust PCA, and matrix completion, all with iteration count $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$3 independent of the target matrix condition number $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$4 (Tong et al., 2020).

For Tucker tensor RPCA, Theorem 1 establishes that under $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$5-incoherence, Tucker rank $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$6, and fractional sparsity

$X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$7

with $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$8 and geometrically decaying threshold schedule $X=X_\star+S_\star,\qquad X_\star=(U_\star^{(1)},U_\star^{(2)},U_\star^{(3)})\bcdot C_\star,$9, the iterates satisfy

$L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$0

with a rate $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$1 independent of $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$2 (Dong et al., 2022). The theorem also gives an $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$3-error bound and implies exact recovery in $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$4 steps (Dong et al., 2022).

For t-SVD robust tensor PCA, under $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$5-incoherence and slice-wise sparsity

$L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$6

Theorem 1 states that

$L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$7

is the convergence rate and is independent of $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$8, together with explicit Frobenius, $L(F,S)=\tfrac12\Big\|(U^{(1)},U^{(2)},U^{(3)})\bcdot C+S-X\Big\|_F^2$9, and sparse-support guarantees (Feng et al., 8 Jan 2025). The broader 2025 t-SVD tensor estimation paper proves analogous condition-number-free linear convergence for both tensor RPCA and tensor completion (Wu, 3 Jan 2025).

For blind super-resolution and demixing formulated as low-rank matrix demixing, the main theorem states that if

$F=(U^{(1)},U^{(2)},U^{(3)},C)$0

then with high probability

$F=(U^{(1)},U^{(2)},U^{(3)},C)$1

again with convergence rate independent of $F=(U^{(1)},U^{(2)},U^{(3)},C)$2 (Chen, 2024).

A more recent 2026 result sharpens the matrix recovery theory under Gaussian designs. It shows that ScaledGD for general low-rank matrix recovery achieves both sample complexity $F=(U^{(1)},U^{(2)},U^{(3)},C)$3 and iteration complexity $F=(U^{(1)},U^{(2)},U^{(3)},C)$4, whereas earlier ScaledGD analyses had $F=(U^{(1)},U^{(2)},U^{(3)},C)$5 sample complexity (Li et al., 31 Mar 2026). The global linear convergence theorem gives

$F=(U^{(1)},U^{(2)},U^{(3)},C)$6

for $F=(U^{(1)},U^{(2)},U^{(3)},C)$7 (Li et al., 31 Mar 2026). This closes a previously noted gap between fast convergence and optimal sampling efficiency.

5. Variants beyond the standard low-rank setting

Several ScaledGD variants modify either the preconditioner or the meaning of “scaling.”

An important extension addresses overparameterization. In PSD matrix sensing with unknown true rank, ScaledGD$F=(U^{(1)},U^{(2)},U^{(3)},C)$8 uses the damped preconditioner

$F=(U^{(1)},U^{(2)},U^{(3)},C)$9

inside the update

$S$0

(Xu et al., 2023). The damping resolves the singularity of $S$1 when the factor rank is overspecified. From a small Gaussian initialization $S$2, the method converges in

$S$3

iterations, improving dramatically over the polynomial $S$4-dependence known for vanilla GD in the same regime (Xu et al., 2023). The 2023 ScaledGD overview presents this as a global convergence mechanism from small random initialization when rank is over-specified (Ma et al., 2023).

A stochastic analogue appears in low-rank matrix completion. The 2016 Scaled stochastic gradient descent method constructs minibatch preconditioners

$S$5

where $S$6 are global Grams, $S$7 are batch Grams, $S$8, and $S$9 trades off global and local second-order information (Mishra et al., 2016). This scaling is explicitly designed to restore scale invariance of the factorization and to improve conditioning. The paper states that in the full-batch case the block-diagonal Hessian approximation reduces the condition number of the local quadratic model to one (Mishra et al., 2016).

Not all methods labeled ScaledGD rely on factor Grams. In separable logistic regression, “ScaledGD” refers to gradient descent with a risk-dependent stepsize

$\mathcal X_\star=U_\star\ast V_\star^H$0

equivalently standard GD on the transformed objective

$\mathcal X_\star=U_\star\ast V_\star^H$1

(Zhang et al., 5 Apr 2025). The key theorem states that after burn-in $\mathcal X_\star=U_\star\ast V_\star^H$2, the averaged iterate achieves risk bounded by $\mathcal X_\star=U_\star\ast V_\star^H$3 with $\mathcal X_\star=U_\star\ast V_\star^H$4, and the paper proves a matching minimax lower bound showing $\mathcal X_\star=U_\star\ast V_\star^H$5 is optimal among first-order batch methods (Zhang et al., 5 Apr 2025). Here the “scaling” is in the stepsize rather than a matrix preconditioner.

In convex constrained optimization, projected scaled gradient methods use diagonal positive-definite $\mathcal X_\star=U_\star\ast V_\star^H$6 with projection onto a feasible set, unifying gradient projection, projected Newton, projected Landweber, and EM-type algorithms (Jin et al., 2015). The central result is convergence under bounded perturbations, and under boundedness of $\mathcal X_\star=U_\star\ast V_\star^H$7 the operator is bounded perturbation resilient (Jin et al., 2015). This line is historically distinct from the low-rank factorization literature but belongs to the same broader family of scaled-gradient ideas.

6. Computational profile, empirical behavior, and practical significance

A central claim of ScaledGD is that it obtains quasi-Newton-like benefits with first-order cost. In matrix factorization, the extra work beyond one gradient evaluation consists only of forming and inverting two $\mathcal X_\star=U_\star\ast V_\star^H$8 Gram matrices, an $\mathcal X_\star=U_\star\ast V_\star^H$9 overhead that is negligible when $X=LR^\top$00 (Tong et al., 2020). The 2020 paper explicitly contrasts this with alternating minimization, which solves larger least-squares systems, and with projected methods that require more expensive inner operations (Tong et al., 2020).

In Tucker tensor RPCA, the per-iteration work is dominated by forming the preconditioned factors $X=LR^\top$01, costing $X=LR^\top$02 for third-order tensors, while the scaled-gradient multiplications cost $X=LR^\top$03 per mode; the overall cost is reported as $X=LR^\top$04 (Dong et al., 2022). The paper contrasts this with Riemannian GD, which requires an HOSVD each step with $X=LR^\top$05 cost (Dong et al., 2022). This computational comparison is central to the method’s positioning: faster than second-order or manifold methods while more robust than unpreconditioned GD.

Empirical results consistently reflect the theoretical condition-number robustness. In synthetic tensor RPCA phase transitions, ScaledGD shows a sharp transition boundary approximately linear in $X=LR^\top$06 and almost independent of $X=LR^\top$07 (Dong et al., 2022). In shot-noise experiments it matches RiemannianGD in accuracy but runs an order of magnitude faster (Dong et al., 2022). On MNIST digit-sequence denoising, it recovers clean digits and flags outliers more sharply than convex TNN at a fraction of the runtime; on VIRAT video background-foreground separation, freezing two modes yields a $X=LR^\top$08 speedup with negligible quality loss (Dong et al., 2022).

The matrix recovery literature reports the same qualitative pattern. In the 2026 Gaussian measurement experiments with $X=LR^\top$09 and $X=LR^\top$10, ScaledGD converges in far fewer iterations than vanilla GD and matches Riemannian GD, while GD runtime grows with $X=LR^\top$11 and ScaledGD remains flat (Li et al., 31 Mar 2026). In low-rank matrix estimation more broadly, ScaledGD is reported to tolerate ill-conditioning up to $X=LR^\top$12, whereas vanilla GD slows by a factor proportional to $X=LR^\top$13 (Tong et al., 2020).

This empirical regularity has shaped a common interpretation: the factor-Gram preconditioners “balance the scales” of the factors and effectively whiten the local geometry. That interpretation is explicit in several sources (Tong et al., 2020, Chen, 2024), though the precise proofs rely on specialized Lyapunov distances, RIP or incoherence arguments, perturbation lemmas, and basin-entry conditions rather than on a direct Hessian equivalence.

7. Conceptual distinctions, misconceptions, and evolving directions

A common misconception is that “ScaledGD” refers to a single canonical algorithm. The literature instead contains several distinct families sharing the idea of gradient scaling. The Gram-preconditioned low-rank methods of Tong, Ma, Chi, and collaborators (Tong et al., 2020, Dong et al., 2022, Ma et al., 2023) are not the same as projected scaled gradient methods in convex optimization (Jin et al., 2015), risk-adaptive stepsize scaling for logistic regression (Zhang et al., 5 Apr 2025), scaled sign methods for communication-efficient optimization (Li et al., 2021), or Hessian-aware scalar scaling rules (Smee et al., 6 Feb 2025). The shared name reflects a common variable-metric intuition rather than a uniform update formula.

A second misconception is that the scaling is equivalent to full second-order optimization. In the low-rank setting, the preconditioner is deliberately inexpensive and highly structured: only small Gram matrices are inverted (Tong et al., 2020). In the projected convex setting, the scaling matrices are diagonal (Jin et al., 2015). In Hessian-aware scalar ScaledGD, only curvature along the gradient direction is used, together with one Hessian-vector product (Smee et al., 6 Feb 2025). These methods occupy an intermediate space between plain GD and Newton-type methods.

The field is also evolving in two notable directions. One is sharper theory: ScaledGD has progressed from local condition-number-free rates (Tong et al., 2020) to overparameterized global guarantees with damping (Xu et al., 2023) and then to optimal-sampling matrix recovery results in the general asymmetric case (Li et al., 31 Mar 2026). Another is architectural hybridization. The learnable self-supervised deep unfolding model RTPCA-LSGD turns t-SVD ScaledGD for robust tensor PCA into a trainable recurrent architecture with learnable $X=LR^\top$14, while preserving the original algorithmic structure as the backbone (Feng et al., 8 Jan 2025). This suggests a broader trend in which ScaledGD serves both as an analyzable optimization primitive and as a module within learned iterative systems.

Taken together, the ScaledGD literature presents a coherent methodological theme: use a carefully chosen scaling operator—diagonal, projected, Gram-based, damped, stochastic, or Hessian-aware—to reshape the local descent geometry while preserving the simplicity of first-order updates. In low-rank matrix and tensor estimation, this theme has yielded a particularly strong body of theory and algorithms with linear convergence at rates independent of the condition number of the target object (Tong et al., 2020, Dong et al., 2022, Wu, 3 Jan 2025, Li et al., 31 Mar 2026).