Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration

Abstract: Non-expansive two-time-scale stochastic approximation is governed by a slow stochastic Krasnoselskii--Mann fixed-point iteration rather than by contraction to a unique equilibrium. We study this regime under a contractive fast map and a non-expansive reduced slow map. We first prove a finite-horizon lower bound showing that, for any prescribed slow stepsize schedule $(β<em>k)$, the classical KM residual scale $(\sum</em>{i&lt;N}β_i(1-β_i))<sup>{-1}$ is worst-case sharp for the corresponding unregularized KM update. Combined with the raw fast-tracking leakage scale, this explains the previously observed $k<sup>{-1/4+o(1)}$ last-iterate mean-square residual exponent. We then introduce a residual-preconditioned slow oracle that cancels the first-order dependence on the fast tracking error. In a nested Tikhonov-KM algorithm, the uncorrected oracle yields total-sample rate $T<sup>{-1/4+o(1)}$, while the corrected oracle yields $T<sup>{-1/3+o(1)}$. This improvement comes from changing the slow-oracle bias from first order to second order in the fast error after all inner-loop samples are counted. Finally, we show that the repeated inner-loop cost of the nested method can be avoided in a smooth derivative-oracle model. A single-loop algorithm that tracks both the fast equilibrium and the leakage preconditioner online achieves $T<sup>{-1/2+o(1)}$ with $O(1)$ primitive samples per iteration.

• The paper demonstrates that a k^(-1/4) residual rate barrier arises from finite-horizon KM limits and fast-manifold leakage.
• It introduces a bias-corrected, residual preconditioning strategy that converts first-order errors into second-order effects.
• Both nested and single-loop TTSA frameworks benefit from improved sample complexity in bilevel and RL optimization tasks.

Non-Expansive Two-Time-Scale Stochastic Approximation: Rate Barriers and Acceleration Mechanisms

Overview and Problem Formulation

This paper presents a detailed analysis of two-time-scale stochastic approximation (TTSA) in the specific regime where the fast component is contractive but the slow component map is merely non-expansive rather than contractive. TTSA frameworks are widely encountered in stochastic bilevel optimization, actor-critic RL, stochastic variational inequalities, and minimax learning problems, where the algorithm alternates updates between two coupled recursions: a “fast” variable tracking an equilibrium as the “slow” variable evolves.

The standard mean-field TTSA recursion takes the form:

$$\begin{align*} X_{k+1} &= X_k + \alpha_k (f(X_k, Y_k) - X_k + W^x_{k+1}) \ Y_{k+1} &= Y_k + \beta_k (g(X_k, Y_k) - Y_k + W^y_{k+1}), \end{align*}$$

with step size schedules $\alpha_k \gg \beta_k$ and noise sequences $W^x, W^y$. The fast variable is expected to track the unique fixed point $x^*(Y_k)$ of $f(\cdot, Y_k)$. When the reduced "slow" map $h(y) = g(x^*(y), y)$ is contractive, classical arguments yield rates for mean-square distance to equilibrium. In contrast, for non-expansive (but not contractive) $h$, the fixed-point set may be non-singleton, and the natural error metric becomes the residual $\|h(y) - y\|$.

This study systematically dissects all sources of finite-time error and their interactions, providing sharp lower bounds, highlighting bottlenecks, and developing structured correction techniques—culminating in improved sample complexity in both nested and single-loop TTSA variants.

Sharpness of the One-Quarter Rate Barrier in Non-Expansive TTSA

The core insight of the paper is the precise origin of the $k^{-1/4 + o(1)}$ mean-square residual rate, previously observed empirically and established for non-expansive TTSA [chandak_2026_nonexpansive_ttsa]. The analysis demonstrates that this exponent is not a loose upper bound but emerges from an intersection of two unavoidable effects:

• Finite-horizon lower bound for KM iterations: For any fixed slow step size sequence $(\beta_k)$, the mean-square residual cannot improve the KM scale $\alpha_k \gg \beta_k$0 in the worst case. This is demonstrated via a planar rotation construction, extending classical results of Baillon–Bruck and Cominetti–Soto–Vaisman. Thus, no improvement is possible unless the schedule or algorithmic structure itself is fundamentally altered.
• First-order fast-manifold leakage: When evaluating $\alpha_k \gg \beta_k$1 (instead of $\alpha_k \gg \beta_k$2), any lag in fast tracking induces a first-order bias in the slow update. The raw fast error is $\alpha_k \gg \beta_k$3, where $\alpha_k \gg \beta_k$4 is the stochastic approximation variance floor and $\alpha_k \gg \beta_k$5 arises from lag in tracking the time-varying equilibrium. Consequently, unless $\alpha_k \gg \beta_k$6 ("two-time-scale separation"), the lag term becomes the bottleneck and does not yield a superior exponent. Even relaxing the separation condition does not ameliorate the rate; the weighted perturbation scales and high probability construction together cement the $\alpha_k \gg \beta_k$7 exponent as tight in the black-box, uncorrected regime.

Bias Correction: Residual Preconditioning and Rate Acceleration

The paper then introduces a bias correction strategy that analytically transforms first-order leakage into a second-order effect, facilitating quantifiable acceleration. This is realized by constructing a residual-preconditioned slow oracle:

$\alpha_k \gg \beta_k$8

where $\alpha_k \gg \beta_k$9 with $W^x, W^y$0, and $W^x, W^y$1. The Taylor analysis shows that

$W^x, W^y$2

so the slow oracle bias becomes second-order in the fast error. Practically, this correction can be interpreted as implementing a single Newton-like update for the implicit solution path, but the form avoids explicit second-order computation or hypergradient construction.

The authors study two principal algorithmic paradigms with this correction:

• Nested Tikhonov-KM Algorithm: Each slow step first solves the fast equilibrium approximately (via inner-loop SA), then executes a Tikhonov-regularized fixed-point update on the slow variable with a bias-corrected oracle. Compared to the uncorrected nested scheme (which remains bottlenecked at $W^x, W^y$3 in total samples), bias correction reduces the slow oracle’s squared bias from $W^x, W^y$4 to $W^x, W^y$5 (where $W^x, W^y$6 is inner loop length per outer iteration). Balancing all terms, the sample complexity improves to $W^x, W^y$7 for the fixed-point residual.

  This is under moderate assumptions: differentiability in the fast variable, compactness, bounded moments, and structured access to $W^x, W^y$8. The result sharpens total primitive sample bounds for TTSA in broad classes of bilevel and compositional learning problems.

• Single-Loop Learned-Preconditioner Algorithm: The cost of performing a full inner fast solve at every outer iteration can be eliminated under a richer oracle model. By tracking both $W^x, W^y$9 and $x^*(Y_k)$0 (the leakage preconditioner) online—using stochastic derivative samples of $x^*(Y_k)$1 and $x^*(Y_k)$2—the algorithm achieves $x^*(Y_k)$3 complexity for the residual, matching the parametric convergence rates for single-level stochastic fixed-point problems. Critically, only $x^*(Y_k)$4 primitive oracle samples are required per iteration. This derivatively-augmented regime aligns with recent advances in implicit sensitivity and amortized bilevel optimization.

Practical and Theoretical Implications

This work provides a comprehensive theoretical framework clarifying the rate bottlenecks for broad classes of coupled stochastic approximation procedures where the slower component is non-expansive. Its lower bound analysis rigorously justifies why naive scheduling or simple acceleration cannot surpass the $x^*(Y_k)$5 exponent and highlights the necessity for algorithmic innovation.

By introducing a bias-corrected oracle structure, the authors demonstrate a route to provable acceleration, contingent on modest smoothness and sensitivity access. The results imply that any practical algorithm targeting improved rates in this regime must either break the fixed-schedule barrier (as in the corrected/nested routines), introduce richer regularization (e.g., Tikhonov terms), or leverage differentiability to online-track necessary sensitivity information.

The implications extend to a variety of learning settings—stochastic bilevel optimization, robust control, RL methods with non-expansive evaluation, operator splitting, and minimax optimization—where the fundamental coupling mimics TTSA.

Future Directions

The structure and analysis motivate several avenues for further research:

• Relaxed Oracle Models: Investigating whether more general sensitivity estimators or black-box surrogates can yield similar residual-rate acceleration without explicit Jacobian samples.
• Strongly Monotone/Polyak-Łojasiewicz Regimes: Extending the analysis to settings where the fixed-point residual is not the target but distance to a unique solution, possibly yielding further acceleration.
• Finite-Memory and Adaptive Schedules: Designing adaptive, data-driven schedule selection schemes that match or improve upon the presented exponents without oracle tuning.
• Extensions to Non-Euclidean and Infinite-Dimensional Spaces: Adapting the analysis to Hilbert/Banach spaces or saddle-point architectures relevant in GANs and inverse problems.

Conclusion

The paper establishes the one-quarter residual rate as a sharp, unbreakable barrier in non-expansive TTSA under fixed schedules and uncorrected algorithms. It further demonstrates that structured bias correction, either through nested or learned preconditioners, can substantially accelerate convergence—achieving up to the optimal $x^*(Y_k)$6 rate under smooth derivative-access assumptions. The analysis tightly integrates operator theory, stochastic approximation, and algorithmic design and yields direct guidance for the development of practical algorithms in bilevel and compositional settings.

Reference: "Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration" (2607.13414)