ProxSTORM: Stochastic Proximal TR Optimization

• The paper introduces ProxSTORM, a stochastic trust-region method that extends traditional proximal techniques to handle composite problems with nonsmooth convex terms.
• It leverages probabilistic stochastic models and adaptive trust-region updates to achieve first-order global convergence with an expected O(1/ε²) complexity.
• Numerical studies show ProxSTORM's superior robustness and accuracy over methods like Adam and deterministic sample average approximation in neural network training and topology optimization.

ProxSTORM is a stochastic trust-region algorithm for composite optimization problems of the form

$\min_{x \in \mathbb{R}^n} \; f(x) + g(x),$

where $f$ is a possibly nonconvex differentiable function with $L$-Lipschitz continuous gradient that is accessible only stochastically, and $g$ is a proper, closed, convex, and generally nonsmooth deterministic function for which evaluations and proximal mappings can be computed exactly. The method generalizes STORM, a stochastic trust-region method for smooth unconstrained optimization, and extends deterministic proximal trust-region ideas to stochastic composite settings. The ProxSTORM paper states first-order global convergence, expected complexity bounds, and numerical viability on $\ell^1$-regularized neural network training and topology optimization (Baraldi et al., 3 Oct 2025, Kapec et al., 2017).

1. Composite problem class and proximal structure

The target problem is the minimization of $f+g$, with stochastic smoothness in $f$ and deterministic convex nonsmoothness in $g$. The proximal mapping of $g$ is assumed to be computable exactly: $$\mathrm{prox}_{r g}(x) := \arg\min_{y\in \mathbb{R}^n} \left\{ g(y) + \tfrac{1}{2r}\|y-x\|^2 \right\},$$ for some $f$0 (Baraldi et al., 3 Oct 2025).

The assumptions described for this setting include that $f$1 is bounded below, that explicit domain regularity holds, and that both $f$2 and $f$3 are finite on their common domain. The paper identifies two representative application classes. One is $f$4-regularized neural network training, where $f$5. The other is stochastic programming with deterministic convex constraints, represented by taking $f$6 as an indicator function (Baraldi et al., 3 Oct 2025).

A central analytical object is the proximal gradient

$f$7

This serves as the first-order stationarity measure. A stationary point $f$8 satisfies $f$9, and when $L$0, the expression reduces to the ordinary gradient, $L$1 (Baraldi et al., 3 Oct 2025).

2. Algorithmic architecture

ProxSTORM is organized around iterates $L$2, trust-region radii $L$3, and stochastic models $L$4 of the smooth term $L$5. At iteration $L$6, the method approximately solves the proximal trust-region subproblem

$L$7

This inserts the nonsmooth term directly into the local subproblem rather than treating it through a smooth surrogate (Baraldi et al., 3 Oct 2025).

The model quality and step quality are assessed through predicted and computed reductions. The predicted reduction is

$L$8

while the computed reduction is a stochastic approximation to the objective decrease,

$L$9

Step acceptance uses both a reduction-ratio test and a stationarity-related threshold: $g$0 where $g$1 is the model proximal gradient at $g$2 and $g$3 are algorithmic parameters (Baraldi et al., 3 Oct 2025).

The trust-region update follows the standard accept/reject logic. If the step is accepted, then $g$4 and

$g$5

with $g$6. If the step is rejected, then $g$7 and

$g$8

The paper presents these operations in Algorithm 1 and characterizes the method as a stochastic trust-region generalization of proximal trust-region optimization (Baraldi et al., 3 Oct 2025).

3. Relation to STORM and deterministic proximal trust-region methods

The paper positions ProxSTORM as a direct generalization of STORM, originally developed for smooth stochastic unconstrained optimization by Chen et al. (Kapec et al., 2017). The extension consists of incorporating a deterministic convex nonsmooth term $g$9 through the proximal mapping, while retaining the stochastic trust-region structure and the martingale-based analysis framework (Baraldi et al., 3 Oct 2025).

The same source also states that ProxSTORM generalizes the deterministic proximal trust-region algorithm of Baraldi et al. (2022) by allowing stochastic model and reduction errors rather than deterministic vanishing errors. In this sense, the method sits at the intersection of three strands of theory: stochastic trust-region methods, proximal composite optimization, and nonsmooth constrained formulations (Baraldi et al., 3 Oct 2025).

Two limit cases are explicit. When $\ell^1$0, ProxSTORM reduces to STORM. When the models and reduction estimates are deterministic, it recovers the deterministic proximal trust-region method identified with Baraldi et al. (2022). The paper further states that some assumptions are generalized and, in some cases, simplified so that they reduce to a more succinct version of the assumptions used for STORM when the convex term is zero (Baraldi et al., 3 Oct 2025).

4. Probabilistic assumptions and convergence analysis

The analysis uses probabilistic model-accuracy and reduction-accuracy requirements. For the stochastic models, the paper assumes that, with probability at least $\ell^1$1,

$\ell^1$2

For the reduction estimates, it assumes that, with probability at least $\ell^1$3,

$\ell^1$4

A bound on the quadratic terms of the model is also required. These conditions are not imposed deterministically at every iteration; rather, they must hold sufficiently often through probabilistic events $\ell^1$5 and $\ell^1$6 (Baraldi et al., 3 Oct 2025).

The main global convergence claim is

$\ell^1$7

under the stated assumptions and with $\ell^1$8 sufficiently close to $\ell^1$9. The paper describes this as convergence to a stationary point of $f+g$0 in the proximal-gradient sense (Baraldi et al., 3 Oct 2025).

A Lyapunov function is used: $f+g$1 together with the conditional expected decrease estimate

$f+g$2

for some constant $f+g$3. The analysis then combines martingale arguments with trust-region radius dynamics to show that $f+g$4 and that the proximal gradient norm vanishes. The source explicitly states that martingale theory is used to navigate the inherent randomness and to establish both almost sure convergence and vanishing stationarity residuals (Baraldi et al., 3 Oct 2025).

The expected complexity result is stated in terms of the first hitting time

$f+g$5

The paper gives

$f+g$6

and more precisely

$f+g$7

where $f+g$8 is the Lyapunov gap and $f+g$9 depends on $f$0 and other parameters. The paper characterizes this as matching the $f$1 first-order complexity regime of smooth stochastic optimization while extending it to the composite nonsmooth stochastic case (Baraldi et al., 3 Oct 2025).

5. Numerical studies

The first numerical study concerns $f$2-regularized neural network training on the HIGGS dataset, described as having 11 million samples and 28 features. The model is a two-layer neural network classification task with logistic loss and regularization parameter $f$3, where $f$4 collects all network parameters. Each stochastic model and reduction estimate uses a minibatch of $f$5 samples, and steps are computed by a spectral proximal gradient method with at most 2 iterations per trust-region subproblem. The comparison baseline is Adam, described as standard but using a subgradient for the $f$6 term (Baraldi et al., 3 Oct 2025).

The reported protocol uses 100 independent trials and compares test loss across ProxSTORM and Adam under equal sample and iteration budgets, with Adam given more iterations to compensate for ProxSTORM’s Hessian-vector products. The observations reported are differentiated by the regularization regime: for small $f$7, Adam is slightly better; for non-negligible regularization, ProxSTORM is more accurate and reproducible, with a narrower histogram and lower test loss. The explanation given in the source is that ProxSTORM rigorously accommodates nonsmoothness through the prox mapping for $f$8, whereas Adam relies on subgradient heuristics that may not handle nonsmooth points properly (Baraldi et al., 3 Oct 2025).

The second numerical study addresses stochastic topology optimization. The objective is to design a material distribution minimizing expected elastic compliance under stochastic loading direction, subject to mass and box constraints on densities. After finite element discretization, the problem has $f$9 variables and takes the form

$g$0

where $g$1 involves solving a linear elasticity equation for each realization of $g$2. The implementation uses $g$3 samples of $g$4 per iteration for stochastic function, model, and reduction estimates. ProxSTORM is compared with a deterministic sample average approximation trust-region solver using the same sample set throughout (Baraldi et al., 3 Oct 2025).

For topology optimization, the reported evaluation uses 100 independent runs for each method and compares test error, defined as estimated compliance under unseen load samples. The paper states that ProxSTORM decisively outperforms the deterministic sample average approximation trust-region method in both accuracy and robustness, yielding lower and tighter distributions of test errors. The explanation supplied is that ProxSTORM’s dynamic sampling provides a more representative and statistically relevant search, whereas sample average approximation optimizes only for the small fixed sample set used during training (Baraldi et al., 3 Oct 2025).

6. Position within the optimization literature

The paper characterizes ProxSTORM as the first stochastic trust-region method for composite objective functions with stochastic smooth $g$5 and convex nonsmooth $g$6. It is presented as bridging nonsmooth regularized learning, stochastic variational problems, and convex-constrained engineering design (Baraldi et al., 3 Oct 2025).

A concise comparison with adjacent algorithmic families, using the categories stated in the source, is as follows.

Method	Structural scope	Typical use
TR	Smooth $g$7; deterministic; constraints	Convex or smooth nonconvex
Proximal TR	Smooth $g$8; deterministic; nonsmooth $g$9; constraints	LASSO, constraints
STORM	Smooth $g$0; stochastic; unconstrained	Smooth, unconstrained
ProxSTORM	Smooth $g$1; stochastic; nonsmooth $g$2; constraints	Regularized, sparse, constrained stochastic problems

The paper also emphasizes two points that clarify common misunderstandings. First, the probabilistic accuracy requirements are only enforced sufficiently often rather than at every iteration; this distinguishes the framework from deterministic error-control schemes. Second, the trust-region radius is not merely a step-size proxy but a core mechanism for robustness under stochastic error and for convergence analysis; the source notes that it necessarily decays to zero in the presence of stochasticity, unlike in deterministic algorithms (Baraldi et al., 3 Oct 2025).

Dynamic sampling strategies are discussed in the appendix as a practical route for implementing the probabilistic accuracy requirements. A plausible implication is that ProxSTORM is meant not only as a theoretical extension of STORM but also as a template for practical stochastic composite solvers in regimes where prox-compatible nonsmooth structure and noisy function information coexist (Baraldi et al., 3 Oct 2025).