CSSCA: Constrained Stochastic Optimization

• CSSCA is a surrogate-based framework that solves stochastic non-convex constrained problems by iteratively replacing them with convex surrogate subproblems.
• The methodology involves recursive surrogate updates, feasibility-restoration steps, and an averaging scheme to ensure convergence.
• CSSCA converges to stationary or KKT points under strong regularity conditions, diminishing step sizes, and, typically, feasible initialization.

Constrained Stochastic Successive Convex Approximation (CSSCA) is a framework for solving stochastic, non-convex, constrained optimization problems in which both the objective and the constraint functions are expectations over random states. In its canonical form, CSSCA replaces the original stochastic problem by a sequence of convex surrogate objective/feasibility problems, updates the decision variable by averaging toward the surrogate optimizer, and is proved to converge almost surely to stationary points under a feasible initial point, asymptotic surrogate consistency, and a Slater-type regularity condition (Liu et al., 2018).

1. Canonical problem class

The formulation that most directly defines CSSCA is

$$\begin{aligned} \min_{\boldsymbol{x}\in\mathcal{X}}\quad & f_0(\boldsymbol{x}) \triangleq \mathbb{E}\!\left[g_0(\boldsymbol{x},\xi)\right] \ \text{s.t.}\quad & f_i(\boldsymbol{x}) \triangleq \mathbb{E}\!\left[g_i(\boldsymbol{x},\xi)\right] \le 0,\quad i=1,\dots,m, \end{aligned}$$

where $\mathcal{X}$ is a compact, convex domain, $\xi$ is a random state, and each sample function $g_i(\boldsymbol{x},\xi)$ may be non-convex in $\boldsymbol{x}$. The defining feature is that stochasticity enters not only the objective but also the constraints, so feasibility itself is expectation-valued and cannot be enforced by a simple projection onto a deterministic convex set (Liu et al., 2018).

The baseline assumptions in the original framework are correspondingly strong but standard for stochastic approximation with non-convex constraints: $\mathcal{X}\subseteq\mathbb{R}^{n_x}$ is compact and convex; each $g_i(\cdot,\xi)$ is continuously differentiable; and $g_i$, $\nabla g_i$, and $\nabla^2 g_i$ are uniformly bounded. This places CSSCA between classical stochastic gradient schemes, which usually treat deterministic convex feasible sets, and constrained sample-average or robust formulations, which often require much heavier per-iteration computation (Liu et al., 2018).

Later work broadened the same basic template. Some papers considered convex but possibly nonsmooth regularizers $\mathcal{X}$0, convex feasible sets encoded through indicator functions, and decentralized decompositions over graphs (Idrees et al., 2024). Others considered convex smooth constraints $\mathcal{X}$1 together with smooth but possibly nonconvex functional constraints $\mathcal{X}$2, keeping the SCA principle while making the constraint class substantially richer (Idrees et al., 2024). A plausible implication is that “CSSCA” now denotes less a single algorithm than a surrogate-based design pattern for constrained stochastic non-convex optimization.

2. Surrogate construction and the core iteration

The canonical CSSCA iteration observes a fresh sample $\mathcal{X}$3, updates surrogate functions $\mathcal{X}$4 for all objective and constraint terms, solves a convex subproblem, and then takes a relaxed step: $\mathcal{X}$5 The step sizes satisfy

$\mathcal{X}$6

This averaging is central: the optimizer of the convex surrogate is not accepted as the new iterate outright, but only as a direction of motion (Liu et al., 2018).

CSSCA distinguishes two convex subproblems. If the current surrogate constraints are feasible, the method performs an objective update,

$\mathcal{X}$7

If that problem is infeasible, it instead solves a feasibility-restoration problem,

$\mathcal{X}$8

The alternation between objective improvement and feasibility recovery is one of the framework’s defining differences from unconstrained stochastic SCA (Liu et al., 2018).

A widely used recursive surrogate update is

$\mathcal{X}$9

where $\xi$0 is a convex approximation of the sample function around the current iterate. A canonical choice is the first-order-plus-quadratic model

$\xi$1

with $\xi$2. The original paper also emphasizes “structured surrogates,” in which a convex component of the sample function is preserved exactly and only the non-convex component is linearized. This gives CSSCA considerable modeling freedom and explains why the framework appears in problems as different as beamforming, CMDPs, and decentralized learning (Liu et al., 2018).

3. Regularity, stationarity, and the role of feasibility

The theoretical bridge from surrogate problems to the original stochastic program is asymptotic consistency. At the current iterate $\xi$3, the surrogate must satisfy

$\xi$4

In addition, the surrogates are assumed to be uniformly strongly convex, uniformly bounded together with their first and second derivatives, and to vary in a controlled Lipschitz-like manner across iterations (Liu et al., 2018).

The step-size coupling is also essential. Besides the iterate step $\xi$5, CSSCA uses a surrogate-averaging sequence $\xi$6 satisfying

$\xi$7

Typical polynomial choices are

$\xi$8

Under a feasible initial point and a Slater condition for the limiting surrogate problem, every limit point of the iterate sequence is a stationary point of the original stochastic constrained problem almost surely (Liu et al., 2018).

The original framework also makes the limitation of feasible initialization explicit. If the initial point is infeasible, the method may converge either to a stationary point of the original problem or to a stationary point of the constraint-minimization problem

$\xi$9

which the source describes as an undesirable attractor set. Later variants were designed precisely to reduce this dependence on feasible starts. One approach introduces slack variables and a penalized reformulation that is always feasible by construction (Ye et al., 2019). Another uses an exact-penalty model and develops an SCA variant that allows infeasible initialization while recovering approximate feasibility/KKT points under a strong Slater-type condition (Sharma et al., 28 Jan 2026).

4. Momentum, oracle complexity, and decentralized extensions

A major line of development augments CSSCA with recursive momentum. The method called CoSTA constructs convex surrogates for both the stochastic objective and the constraint functions, solves the resulting convex optimization problem at each iteration, and updates a gradient tracker through the STORM-like recursion

$g_i(\boldsymbol{x},\xi)$0

The analysis introduces a parameterized version of the standard Mangasarian-Fromowitz Constraints Qualification, proves a bound on the dual variables, and derives $g_i(\boldsymbol{x},\xi)$1 stochastic first-order complexity with adaptive step sizes and $g_i(\boldsymbol{x},\xi)$2 complexity with non-adaptive step sizes (Idrees et al., 2024).

The same acceleration theme appears in decentralized settings. D-MSSCA studies a fully decentralized stochastic nonconvex composite optimization problem over an undirected connected graph, where each agent solves a local strongly convex surrogate subproblem,

$g_i(\boldsymbol{x},\xi)$3

and then performs decentralized averaging, recursive momentum-gradient estimation, and gradient tracking. Its stated stochastic first-order complexity is $g_i(\boldsymbol{x},\xi)$4 for reaching an $g_i(\boldsymbol{x},\xi)$5-stationary point, with one sample per iteration (Idrees et al., 2024).

A related 2026 development, D-SCAMPL, reformulates nonlinear inequality-constrained decentralized stochastic optimization through an exact-penalty model and then applies a successive convex approximation step with linearized constraints and a strongly convex surrogate $g_i(\boldsymbol{x},\xi)$6. The method requires only local stochastic gradients and first-order constraint information, uses two communication rounds per iteration, permits infeasible initialization, and achieves oracle complexity $g_i(\boldsymbol{x},\xi)$7 under the stated smoothness and regularity assumptions (Sharma et al., 28 Jan 2026).

Taken together, these results indicate a clear shift in the CSSCA literature: from almost-sure stationarity under diminishing stepsizes toward non-asymptotic $g_i(\boldsymbol{x},\xi)$8-KKT complexity, variance reduction, and network-aware implementations. This suggests that the surrogate-based constrained framework is compatible with the same complexity targets that dominate modern unconstrained stochastic optimization.

5. CSSCA in constrained reinforcement learning

CSSCA has become a recurrent actor-update mechanism in constrained reinforcement learning, especially for average-cost CMDPs with deep or otherwise high-dimensional policies. In SCAOPO, the policy parameter $g_i(\boldsymbol{x},\xi)$9 is updated by solving convex surrogate objective or feasibility problems built from stochastic estimates

$\boldsymbol{x}$0

followed by the relaxation step

$\boldsymbol{x}$1

The paper uses off-policy reuse of the most recent $\boldsymbol{x}$2 transitions, proves asymptotic consistency of the surrogate value and gradient estimates, and shows that with a feasible initial point every limiting point satisfying the Slater condition for the limiting surrogate problem is a KKT point of the original CMDP almost surely (Tian et al., 2021).

The same template appears in single-loop deep actor-critic form. SLDAC explicitly states that the actor module uses CSSCA to replace the usual stochastic gradient descent update. The actor solves a convex quadratic program based on

$\boldsymbol{x}$3

while the critic DNNs are updated only once or a few finite times per iteration. Under feasible initialization, Slater condition, and $\boldsymbol{x}$4, every limiting point satisfies approximate KKT conditions whose error vanishes as the critic approximation error vanishes (Wang et al., 2023).

More specialized CRL systems embed the same mechanism in communications control. CAAC uses CSSCA to optimize a policy network that selects user priority weights and transmit power for QoS-aware WMMSE precoding, solving surrogate constrained problems or feasibility-restoration problems in the actor step (Wang et al., 19 Jun 2025). In non-stationary XR downlink power scheduling, a context inference module first augments the state with a latent traffic variable and reshapes sparse packet-drop feedback; CSSCA then solves the resulting CMDP through convex quadratic surrogate actor updates, with convergence to an approximate KKT point controlled by Q-network approximation error $\boldsymbol{x}$5 (Wang et al., 12 Mar 2025).

6. Applications, scope, and relation to adjacent SCA methods

The original CSSCA paper demonstrated the framework on several wireless communication problems: MIMO transmit signal design with imperfect CSI, robust beamforming with chance constraints, and massive MIMO hybrid beamforming. In those cases, CSSCA was reported to match or approach sample-average baselines while reducing CPU time, and to outperform online primal-dual baselines in power or sum-rate terms under the conditions tested (Liu et al., 2018). Closely related two-timescale hybrid precoding work, SSCA-THP, cast RF-precoder design, power allocation, and regularization selection into a unified stochastic constrained optimization problem and updated them by solving quadratic surrogate objective or feasibility problems once per frame (Liu et al., 2018).

Recent system-level formulations have preserved the same structure. In movable-antenna-enhanced MIMO downlink, the long-term transmit APV and covariance design problem is handled by CSSCA through recursive concave surrogates for average rate constraints and antenna-spacing constraints, with a separate feasibility-maximization problem when the surrogate problem is infeasible; every limit point of the generated sequence is stated to be a stationary point of the long-term problem almost surely (Chen et al., 4 Sep 2025).

A recurring misconception is that any stochastic SCA method over a convex feasible set is automatically CSSCA. The literature does not support that identification. S-NEXT, for example, is a distributed stochastic SCA method with dynamic consensus and recursive averaging, but its only explicit constraint is a common closed convex feasible set $\boldsymbol{x}$6; the source explicitly states that it “does not develop a special CSSCA-style mechanism for general coupled constraints or stochastic constraints” (Lorenzo et al., 2020). By contrast, CSSCA in the strict sense treats objective and constraint functions through surrogate construction, includes explicit feasibility management, and is motivated by settings in which stochastic feasibility cannot be reduced to projection onto a deterministic convex set (Liu et al., 2018).

In that narrower but technically important sense, CSSCA is best understood as the constrained branch of stochastic successive convex approximation: a family of algorithms that preserve feasibility structure through convexification, admit application-specific surrogate design, and have evolved from almost-sure stationary convergence results to momentum-accelerated, decentralized, and deep-RL variants with $\boldsymbol{x}$7-type guarantees in several modern formulations (Idrees et al., 2024).