--- title: 'APS: Adaptive Proximal Optimization' url: https://www.emergentmind.com/topics/adaptive-prox-guided-scheme-aps type: topic --- # APS: Adaptive Proximal Optimization Searching arXiv for the APS paper and closely related prox-guided weakly convex optimization work. Adaptive Prox-Guided Scheme (APS) is a proximal optimization framework for minimizing weakly convex objectives when the weak-convexity parameter is unknown. Introduced in “Adaptive Proximal Methods for Weakly Convex Optimization with Unknown Parameter: Deterministic and Stochastic Guarantees” [2606.17285], APS targets problems of the form \(\min_{x\in\mathbb R^d} f(x)\) with \(f\) bounded below and \(\rho\)-weakly convex, without requiring \(f\) to be globally Lipschitz continuous or smooth. Its defining feature is a one-trial proximal update rule that adapts the proximal parameter online, in both directions, through a descent test. In the deterministic setting it attains an \(O(\varepsilon^{-2})\) iteration complexity for producing an \(\varepsilon\)-subgradient stationary point, and in the stochastic setting it attains a high-probability \(O(\varepsilon^{-2})\) iteration bound for driving the Moreau-envelope gradient below \(\varepsilon\) under deliberately weak oracle assumptions [2606.17285]. ## 1. Problem class and analytical setting APS is formulated for objectives \[ \min_{x\in\mathbb R^d} f(x), \] where \(f:\mathbb R^d\to\mathbb R\) is \(\rho\)-weakly convex and bounded below. Weak convexity means that \[ x \mapsto f(x)+\frac{\rho}{2}\|x\|^2 \] is convex. The scheme is designed for the regime in which \(\rho\) is unknown, which is precisely the parameter that conventional proximal-point and prox-guided methods use to set a safe proximal scale [2606.17285]. The paper works with the subdifferential induced by weak convexity. If \[ h(x):=f(x)+\frac{\rho}{2}\|x\|^2, \] then \[ \partial f(x):=\{v\in\mathbb R^d:\ v+\rho x\in \partial h(x)\}. \] Equivalently, \[ v\in\partial f(x) \quad\Longleftrightarrow\quad f(y)\ge f(x)+\langle v,y-x\rangle-\frac{\rho}{2}\|y-x\|^2,\quad \forall y\in\mathbb R^d. \] The associated proximal map and Moreau envelope are \[ \operatorname{prox}_{\gamma f}(x)\in\arg\min_y\left\{f(y)+\frac{1}{2\gamma}\|y-x\|^2\right\}, \] and \[ f_\gamma(x):=\min_y\left\{f(y)+\frac{1}{2\gamma}\|y-x\|^2\right\}. \] When \(\gamma<1/\rho\), the minimizer is unique and \[ \nabla f_\gamma(x)=\frac{1}{\gamma}\bigl(x-\operatorname{prox}_{\gamma f}(x)\bigr). \] A central analytical threshold is the safe proximal parameter \[ \bar\gamma = \frac{1}{2\rho}. \] Whenever \(\gamma\le \bar\gamma\), the proximal subproblem is \((1/\gamma-\rho)\)-strongly convex with modulus at least \(\rho\). APS is built around the fact that this threshold exists but is hidden from the algorithm [2606.17285]. Two stationarity notions are used. In the deterministic setting, the target is an \(\varepsilon\)-subgradient stationary point: \[ \operatorname{dist}(0,\partial f(x))\le \varepsilon. \] In the stochastic setting, the target is \[ \|\nabla f_{\bar\gamma}(x)\|\le \varepsilon, \] that is, small Moreau-envelope gradient at the fixed safe parameter \(\bar\gamma\) [2606.17285]. ## 2. Core mechanism and adaptive update rule APS is a one-trial proximal algorithm. At outer iteration \(k\), with current iterate \(x_k\) and proximal parameter \(\gamma_k>0\), it requests one candidate proximal point \(y_k\), forms the Moreau-gradient proxy \[ d_k := \frac{x_k-y_k}{\gamma_k}, \] and obtains a difference estimate \[ \Delta_k \approx f(x_k)-f(y_k). \] It then applies the accept/reject test \[ \Delta_k \ge \frac{\sigma}{2}\gamma_k\|d_k\|^2-\epsilon_f \qquad\text{and}\qquad \|d_k\|\ge \varepsilon_{\mathrm{rej}}. \tag{AT} \] If the test passes, the iteration is successful: \[ x_{k+1}=y_k,\qquad \gamma_{k+1}=\beta_{\mathrm{inc}}\gamma_k, \] with \(\beta_{\mathrm{inc}}>1\). Otherwise it is unsuccessful: \[ x_{k+1}=x_k,\qquad \gamma_{k+1}=\beta_{\mathrm{dec}}\gamma_k, \] with \(\beta_{\mathrm{dec}}\in(0,1)\). APS therefore increases \(\gamma_k\) after success and decreases it after failure. This bidirectional adaptation is the defining algorithmic idea: APS does not merely shrink the proximal parameter until a condition passes, but also enlarges it when local behavior appears favorable [2606.17285]. The descent test depends only on observable quantities: a candidate point \(y_k\), a displacement proxy \(d_k\), and an estimated function difference \(\Delta_k\). APS never needs to know whether \(\gamma_k\) is below or above the hidden safe threshold \(\bar\gamma\). The analysis instead partitions iterations into safe and unsafe ones by the indicator \[ U_k := \mathbf 1\{\max\{\gamma_k,\gamma_{k+1}\}>\bar\gamma\}. \] To quantify how far \(\gamma_k\) lies from \(\bar\gamma\), the paper introduces a log-scale geometry. One increase step is measured in units of decrease steps through \[ m:=\frac{\ln \beta_{\mathrm{inc}}}{\ln(1/\beta_{\mathrm{dec}})}. \] It also defines \[ \ell_k^+ := \max\left\{\left\lceil \frac{\ln(\gamma_k/\bar\gamma)}{\ln(1/\beta_{\mathrm{dec}})}\right\rceil,0\right\}, \] \[ \ell_k^- := \max\left\{\left\lfloor \frac{\ln(\bar\gamma/\gamma_k)}{\ln(1/\beta_{\mathrm{dec}})}\right\rfloor,0\right\}, \qquad \ell_k:=\ell_k^++\ell_k^-. \] These variables permit a counting argument in which adaptation contributes only a logarithmic overhead relative to the hidden safe scale [2606.17285]. A recurrent misconception is to assimilate APS to line search or trust-region methods. The paper explicitly distinguishes it from one-sided backtracking. APS does not repeatedly resolve the same proximal subproblem until acceptance; adaptation occurs across iterations. Relative to trust-region or line-search schemes, the analogous feature is accept/reject based on actual progress, but the adapted quantity is the proximal parameter controlling regularization and subproblem geometry, not a step along a fixed search direction [2606.17285]. ## 3. Deterministic theory In the deterministic specialization, the difference oracle is exact, \[ \Delta_k = f(x_k)-f(y_k), \qquad \epsilon_f=0, \] and the candidate oracle returns \[ y_k=\operatorname{prox}_{\gamma_k f}(x_k) \quad\text{if } \gamma_k\le \bar\gamma. \] For \(\gamma_k>\bar\gamma\), the oracle output may be arbitrary. The stopping time is \[ T_\varepsilon := \inf\{k\ge 0:\ \operatorname{dist}(0,\partial f(y_k))\le \varepsilon\}. \] The deterministic analysis sets \[ \varepsilon_{\mathrm{rej}}=\varepsilon, \qquad \beta_{\mathrm{inc}}=1/\beta_{\mathrm{dec}}. \] A key descent scale is \[ h_{\mathrm{des}} := \frac{\sigma\bar\gamma\varepsilon_{\mathrm{rej}}^2}{2\beta_{\mathrm{inc}}} = \frac{\sigma\varepsilon^2}{4\rho\beta_{\mathrm{inc}}}. \] The main theorem states that \[ T_\varepsilon \le \ell_0 + \frac{2(f(x_0)-f_{\inf})}{h_{\mathrm{des}}} = \ell_0 + \frac{8\rho\beta_{\mathrm{inc}}(f(x_0)-f_{\inf})}{\sigma\varepsilon^2} = \mathcal O(\varepsilon^{-2}). \] Thus APS achieves the standard weakly-convex complexity order without knowing \(\rho\), and the price of an inaccurate initial choice \(\gamma_0\) is only the additive term \(\ell_0\) [2606.17285]. The proof has two structural pillars. First, before stationarity is reached, all safe iterations are successful. If \(y=\operatorname{prox}_{\gamma f}(x)\) and \(d=(x-y)/\gamma\), then \[ d\in\partial f(y), \qquad \operatorname{dist}(0,\partial f(y))\le \|d\|. \] Moreover, for safe \(\gamma\le \bar\gamma\), exact proximal points satisfy sufficient descent: \[ f(y)\le f(x)-\frac{2-\rho\gamma}{2}\gamma\|d\|^2 \le f(x)-\frac{\sigma}{2}\gamma\|d\|^2. \] Hence a safe exact step with \(\|d_k\|>\varepsilon\) necessarily passes the acceptance test. Second, any successful unsafe iteration yields a uniform objective decrease: \[ f(x_k)-f(x_{k+1}) \ge \frac{\sigma}{2}\gamma_k\varepsilon^2 \ge h_{\mathrm{des}}. \] Since \(f\) is bounded below, the number of unsafe successful iterations is controlled by the initial objective gap \(f(x_0)-f_{\inf}\). This converts hidden-safe-threshold uncertainty into a bounded amount of wasted progress rather than a breakdown of the algorithm [2606.17285]. The paper also proves an offline output theorem. If \[ N > \ell_0 + \frac{2(f(x_0)-f_{\inf})}{h_{\mathrm{des}}}, \] and one selects among the indices \(k