APS: Adaptive Proximal Optimization

• The paper introduces APS as a proximal framework that adaptively adjusts the proximal parameter, eliminating the need for the weak convexity constant.
• In the deterministic setting, APS achieves an O(ε⁻²) iteration complexity via a one-trial proximal update and an accept/reject descent test.
• In the stochastic model, APS delivers high-probability convergence guarantees under weak oracle assumptions by effectively regulating noise through adaptive regularization.

Searching arXiv for the APS paper and closely related prox-guided weakly convex optimization work. Adaptive Prox-Guided Scheme (APS) is a proximal optimization framework for minimizing weakly convex objectives when the weak-convexity parameter is unknown. Introduced in “Adaptive Proximal Methods for Weakly Convex Optimization with Unknown Parameter: Deterministic and Stochastic Guarantees” (Xie, 15 Jun 2026), APS targets problems of the form $\min_{x\in\mathbb R^d} f(x)$ with $f$ bounded below and $\rho$-weakly convex, without requiring $f$ to be globally Lipschitz continuous or smooth. Its defining feature is a one-trial proximal update rule that adapts the proximal parameter online, in both directions, through a descent test. In the deterministic setting it attains an $O(\varepsilon^{-2})$ iteration complexity for producing an $\varepsilon$-subgradient stationary point, and in the stochastic setting it attains a high-probability $O(\varepsilon^{-2})$ iteration bound for driving the Moreau-envelope gradient below $\varepsilon$ under deliberately weak oracle assumptions (Xie, 15 Jun 2026).

1. Problem class and analytical setting

APS is formulated for objectives

$\min_{x\in\mathbb R^d} f(x),$

where $f:\mathbb R^d\to\mathbb R$ is $f$0-weakly convex and bounded below. Weak convexity means that

$f$1

is convex. The scheme is designed for the regime in which $f$2 is unknown, which is precisely the parameter that conventional proximal-point and prox-guided methods use to set a safe proximal scale (Xie, 15 Jun 2026).

The paper works with the subdifferential induced by weak convexity. If

$f$3

then

$f$4

Equivalently,

$f$5

The associated proximal map and Moreau envelope are

$f$6

and

$f$7

When $f$8, the minimizer is unique and

$f$9

A central analytical threshold is the safe proximal parameter

$\rho$0

Whenever $\rho$1, the proximal subproblem is $\rho$2-strongly convex with modulus at least $\rho$3. APS is built around the fact that this threshold exists but is hidden from the algorithm (Xie, 15 Jun 2026).

Two stationarity notions are used. In the deterministic setting, the target is an $\rho$4-subgradient stationary point: $\rho$5 In the stochastic setting, the target is

$\rho$6

that is, small Moreau-envelope gradient at the fixed safe parameter $\rho$7 (Xie, 15 Jun 2026).

2. Core mechanism and adaptive update rule

APS is a one-trial proximal algorithm. At outer iteration $\rho$8, with current iterate $\rho$9 and proximal parameter $f$0, it requests one candidate proximal point $f$1, forms the Moreau-gradient proxy

$f$2

and obtains a difference estimate

$f$3

It then applies the accept/reject test

$f$4

If the test passes, the iteration is successful: $f$5 with $f$6. Otherwise it is unsuccessful: $f$7 with $f$8. APS therefore increases $f$9 after success and decreases it after failure. This bidirectional adaptation is the defining algorithmic idea: APS does not merely shrink the proximal parameter until a condition passes, but also enlarges it when local behavior appears favorable (Xie, 15 Jun 2026).

The descent test depends only on observable quantities: a candidate point $O(\varepsilon^{-2})$0, a displacement proxy $O(\varepsilon^{-2})$1, and an estimated function difference $O(\varepsilon^{-2})$2. APS never needs to know whether $O(\varepsilon^{-2})$3 is below or above the hidden safe threshold $O(\varepsilon^{-2})$4. The analysis instead partitions iterations into safe and unsafe ones by the indicator

$O(\varepsilon^{-2})$5

To quantify how far $O(\varepsilon^{-2})$6 lies from $O(\varepsilon^{-2})$7, the paper introduces a log-scale geometry. One increase step is measured in units of decrease steps through

$O(\varepsilon^{-2})$8

It also defines

$O(\varepsilon^{-2})$9

$\varepsilon$0

These variables permit a counting argument in which adaptation contributes only a logarithmic overhead relative to the hidden safe scale (Xie, 15 Jun 2026).

A recurrent misconception is to assimilate APS to line search or trust-region methods. The paper explicitly distinguishes it from one-sided backtracking. APS does not repeatedly resolve the same proximal subproblem until acceptance; adaptation occurs across iterations. Relative to trust-region or line-search schemes, the analogous feature is accept/reject based on actual progress, but the adapted quantity is the proximal parameter controlling regularization and subproblem geometry, not a step along a fixed search direction (Xie, 15 Jun 2026).

3. Deterministic theory

In the deterministic specialization, the difference oracle is exact,

$\varepsilon$1

and the candidate oracle returns

$\varepsilon$2

For $\varepsilon$3, the oracle output may be arbitrary. The stopping time is

$\varepsilon$4

The deterministic analysis sets

$\varepsilon$5

A key descent scale is

$\varepsilon$6

The main theorem states that

$\varepsilon$7

Thus APS achieves the standard weakly-convex complexity order without knowing $\varepsilon$8, and the price of an inaccurate initial choice $\varepsilon$9 is only the additive term $O(\varepsilon^{-2})$0 (Xie, 15 Jun 2026).

The proof has two structural pillars. First, before stationarity is reached, all safe iterations are successful. If $O(\varepsilon^{-2})$1 and $O(\varepsilon^{-2})$2, then

$O(\varepsilon^{-2})$3

Moreover, for safe $O(\varepsilon^{-2})$4, exact proximal points satisfy sufficient descent: $O(\varepsilon^{-2})$5 Hence a safe exact step with $O(\varepsilon^{-2})$6 necessarily passes the acceptance test.

Second, any successful unsafe iteration yields a uniform objective decrease: $O(\varepsilon^{-2})$7 Since $O(\varepsilon^{-2})$8 is bounded below, the number of unsafe successful iterations is controlled by the initial objective gap $O(\varepsilon^{-2})$9. This converts hidden-safe-threshold uncertainty into a bounded amount of wasted progress rather than a breakdown of the algorithm (Xie, 15 Jun 2026).

The paper also proves an offline output theorem. If

$\varepsilon$0

and one selects among the indices $\varepsilon$1 satisfying $\varepsilon$2 an index

$\varepsilon$3

then

$\varepsilon$4

satisfies

$\varepsilon$5

This gives a deterministic certification route that uses the smallest safe-looking proximal scale among near-stationary detected iterates (Xie, 15 Jun 2026).

4. Stochastic oracle model and high-probability guarantees

The stochastic version of APS is deliberately permissive. It uses two noisy oracles. The Stochastic Difference Oracle returns $\varepsilon$6, with absolute error

$\varepsilon$7

The assumptions are

$\varepsilon$8

and

$\varepsilon$9

for some $\min_{x\in\mathbb R^d} f(x),$0. Difference estimates may therefore be biased and heavy-tailed (Xie, 15 Jun 2026).

The Stochastic Proximal Oracle, given $\min_{x\in\mathbb R^d} f(x),$1, returns $\min_{x\in\mathbb R^d} f(x),$2. In the safe regime $\min_{x\in\mathbb R^d} f(x),$3, it is required to satisfy

$\min_{x\in\mathbb R^d} f(x),$4

where

$\min_{x\in\mathbb R^d} f(x),$5

Outside the safe regime, no condition is imposed; the output may be arbitrary.

The inexact proximal analysis uses the consistency bound

$\min_{x\in\mathbb R^d} f(x),$6

It also uses a comparison inequality between safe Moreau gradients: $\min_{x\in\mathbb R^d} f(x),$7

The stochastic stopping time is

$\min_{x\in\mathbb R^d} f(x),$8

and the analysis uses

$\min_{x\in\mathbb R^d} f(x),$9

Defining

$f:\mathbb R^d\to\mathbb R$0

the paper marks iteration $f:\mathbb R^d\to\mathbb R$1 as true when the safe proximal call is accurate enough and the difference-oracle error is at most $f:\mathbb R^d\to\mathbb R$2. The key conditional-probability assumption is that there exists $f:\mathbb R^d\to\mathbb R$3 such that, on $f:\mathbb R^d\to\mathbb R$4,

$f:\mathbb R^d\to\mathbb R$5

A crucial lemma states that if $f:\mathbb R^d\to\mathbb R$6, $f:\mathbb R^d\to\mathbb R$7, and $f:\mathbb R^d\to\mathbb R$8, then the iteration must be successful. This is the stochastic analogue of the deterministic safe-step lemma and is the mechanism by which the analysis links hidden safety, noisy prox accuracy, and the observable acceptance rule (Xie, 15 Jun 2026).

The per-success decrease scale remains

$f:\mathbb R^d\to\mathbb R$9

and the paper proves a counting inequality controlling the number of true iterations in terms of the initial objective gap, cumulative difference-oracle noise, and the log-distance of $f$00 from $f$01. It then derives a high-probability bound. With

$f$02

and

$f$03

if $f$04, then for suitable $f$05 and

$f$06

the stopping time satisfies

$f$07

where

$f$08

Since $f$09, the required $f$10 scales as $f$11 (Xie, 15 Jun 2026).

For cumulative SDO noise, the paper gives two tail regimes. Under finite $f$12-th conditional moments,

$f$13

Under conditionally sub-exponential difference errors,

$f$14

The paper also states that $f$15 almost surely under these conditions (Xie, 15 Jun 2026).

5. Assumptions, robustness, and expected-risk instantiation

A central aspect of APS is the weakness of its assumptions. The paper explicitly does not require global Lipschitz continuity of $f$16, smoothness of $f$17, bounded subgradients for the outer objective, exact or unbiased function-difference estimates, light-tailed noise, highly reliable proximal-oracle success on every call, or any useful behavior of the proximal oracle when $f$18. This suggests that APS is intended as a robust proximal framework for weakly convex objectives in which neither the geometry nor the stochastic feedback is assumed benign (Xie, 15 Jun 2026).

The stochastic oracles are correspondingly broad. For the difference oracle, only the expected absolute error and a finite $f$19-th centered moment are controlled. For the proximal oracle, constant-probability accuracy is required only on safe calls. A plausible implication is that APS is designed for settings where robust progress certificates can be extracted from intermittent reliable information rather than from uniformly accurate model evaluations.

The paper also emphasizes local adaptivity. If larger $f$20 values still produce sufficient descent, APS increases $f$21; if local geometry is unfavorable or oracle behavior deteriorates, rejection decreases $f$22. In this sense APS is not merely parameter-free. It attempts to infer a locally suitable regularization scale online from observed descent behavior (Xie, 15 Jun 2026).

For expected-risk objectives

$f$23

the paper constructs an SDO by paired minibatch differences,

$f$24

using common random numbers, and an SPO by running a stochastic subgradient method on the proximal subproblem. The minibatch SDO yields $f$25, so to enforce $f$26 one needs

$f$27

per outer iteration, which leads to $f$28 total samples for the raw minibatch-difference construction. For the SPO, a stochastic subgradient inner solve with

$f$29

yields the required constant-probability accuracy, hence $f$30 total stochastic subgradient evaluations over $f$31 outer iterations (Xie, 15 Jun 2026).

The excerpted paper contains no experiments. The emphasis is theoretical: deterministic finite-time guarantees, stochastic high-probability guarantees, and robustness to weak oracle models.

6. Relation to adjacent proximal methods and acronym disambiguation

APS belongs to a broader proximal literature but occupies a distinct position within it. The paper contrasts APS with fixed-$f$32 prox-guided methods that require knowing $f$33, with monotone parameter schedules in parameter-free proximally guided methods, with prox-linear backtracking methods tied to composite structure, and with 4WD-Catalyst-type approaches that increase regularization to enforce convexity under stronger smoothness assumptions (Xie, 15 Jun 2026). Its distinguishing combination is unknown-$f$34 operation, bidirectional online adaptation of the proximal parameter, and high-probability stochastic analysis under biased heavy-tailed difference estimates and only constant-probability safe proximal accuracy.

A useful nearby reference is “On the Convergence of FedProx with Extrapolation and Inexact Prox” (Li et al., 2024). That paper analyzes prox-guided extrapolation with inexact local proximal solves in federated optimization, but its main theorems use fixed $f$35 and fixed $f$36, and it explicitly does not provide a full theory for adaptive APS-style rules with inexact prox. Its main relevance is as background on inexact proximal guidance and on the contrast between absolute and relative prox errors, whereas APS is explicitly about online adaptation to the unknown weak-convexity scale.

Another adjacent method is “Prox-NAG-GS: A Semi-Implicit Proximal Method for Composite Optimization” (Wiykiynyuy et al., 25 May 2026). That work is naturally interpretable as a prox-guided semi-implicit accelerated scheme with a tunable proximal curvature parameter, but its proved regime is fixed-parameter and deterministic, and the analysis centers on a two-sequence mismatch between the gradient-evaluation point and the proximal output. APS differs by targeting general weakly convex optimization with unknown $f$37 and by making the proximal parameter itself the adaptively controlled quantity.

The acronym APS is also ambiguous across arXiv. In fair division, APS denotes Any Price Share rather than any optimization method, as in “On MMS, APS and XOS” (Feige et al., 9 May 2026) and “Approximating APS under Submodular and XOS valuations with Binary Marginals” (Kulkarni et al., 2023). In uplink ISAC systems, APS denotes Adaptive Phase-Shifted in “Adaptive Phase-Shifted Pilot Design for Uplink Multiple Access in ISAC Systems” (Sümer et al., 4 Aug 2025). These usages are unrelated. In optimization, Adaptive Prox-Guided Scheme refers specifically to the weakly convex proximal method introduced in (Xie, 15 Jun 2026).

In that sense, APS is best understood as a proximal framework that replaces prior knowledge of the weak-convexity scale by an accept/reject mechanism operating on observable descent. Its technical contribution is not a new proximal subproblem, but a method for navigating the hidden boundary between safe and unsafe regularization regimes while preserving $f$38 complexity in both deterministic and stochastic settings (Xie, 15 Jun 2026).