Robust Safe Gradient Flow (RSGF)

• Robust Safe Gradient Flow (RSGF) is a continuous-time dynamics method that ensures constraint satisfaction by solving a strongly convex QCQP at each step.
• It incorporates a robustness term (β) to modify the descent direction, enhancing stability and convergence to KKT points while preserving feasibility.
• RSGF extends to applications like constrained reinforcement learning, bilevel optimization, and distributed control, offering anytime safe updates even with estimation errors.

Robust Safe Gradient Flow (RSGF) is a continuous-time dynamics for constrained optimization that is designed to be safe at every instant along the trajectory, not just at convergence. In its explicit formulation, RSGF selects a descent direction by solving a strongly convex quadratically constrained quadratic program (QCQP), so that the objective is decreased while constraint satisfaction is preserved through forward invariance of the feasible set. In the reinforcement-learning setting, this construction yields an anytime algorithmic backbone for constrained policy optimization, with high-probability safe-to-safe updates and recovery from unsafe policies. This suggests a narrow usage of the term for the QCQP-based robust extension of Safe Gradient Flow (SGF), and a broader usage for a family of safety-filtered gradient systems that includes bilevel optimization, receding-horizon inverter control, and reference-governed distributed optimization (Mestres et al., 1 Oct 2025, Allibhoy et al., 2022).

1. Foundational safe gradient flow

The immediate precursor of RSGF is Safe Gradient Flow for constrained nonlinear programming. The foundational problem is

$$\begin{aligned} &\underset{x \in \mathbb{R}^n}{\text{minimize}} && f(x) \ &\text{subject to} && g(x) \le 0 \ &&& h(x) = 0 , \end{aligned}$$

with feasible set

$$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$

The unconstrained gradient flow,

$\dot x = -\nabla f(x),$

does not enforce feasibility. SGF therefore augments the vector field with inputs along the gradients of the constraint functions,

$$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$

where $u\in\mathbb{R}^m_{\ge 0}$ and $v\in\mathbb{R}^k$ play a primal-dual role. The admissible set $K_\alpha(x)$ is defined by control-barrier-function conditions that imply

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$

and the implemented feedback is the minimum-norm correction in that admissible set. The resulting closed-loop system, termed safe gradient flow, can also be written as a continuous approximation of projected gradient flow:

$$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$

As $\alpha\to\infty$, $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$0 for $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$1. Under MFCQ, the feasible set is forward invariant and asymptotically stable, the vector field is locally Lipschitz on an open neighborhood of $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$2, and equilibria coincide with KKT points (Allibhoy et al., 2022).

This establishes the structural template that RSGF preserves: a nominal descent direction, a safety-enforcing correction derived from constraint geometry, and an anytime interpretation in which interruption of the dynamics still returns a feasible point.

2. Continuous-time RSGF dynamics

The explicit RSGF construction is formulated for constrained optimization problems of the form

$$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$3

In the constrained reinforcement-learning setting, $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$4 is the negative expected return and the $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$5 are expected cumulative constraint costs. RSGF is defined as the ODE

$$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$6

where $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$7 is the solution of the QCQP

$$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$8

subject to, for each constraint $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$9,

$\dot x = -\nabla f(x),$0

Here $\dot x = -\nabla f(x),$1 is a constraint-recovery/attractiveness gain, and $\dot x = -\nabla f(x),$2 is a robustness term; $\dot x = -\nabla f(x),$3 recovers the earlier SGF. On the feasible set, the paper gives the KKT-based expression

$\dot x = -\nabla f(x),$4

with $\dot x = -\nabla f(x),$5 the QCQP multipliers (Mestres et al., 1 Oct 2025).

This form makes the robust modification explicit. The SGF correction already reshapes the descent direction through the active constraints; the additional denominator induced by $\dot x = -\nabla f(x),$6 changes the effective step along that corrected direction. The paper states that the positive $\dot x = -\nabla f(x),$7 term increases robustness to modeling/estimation errors, and in the convergence proof it strengthens the dissipation term. A plausible implication is that RSGF should be understood not as a different geometry from SGF, but as an SGF regularization that inserts a quadratic safety margin directly into the admissible velocity conditions.

3. Safety, invariance, equilibria, and asymptotic behavior

The analytical guarantees for RSGF rely on regularity and constraint-qualification assumptions. The paper assumes that $\dot x = -\nabla f(x),$8 and $\dot x = -\nabla f(x),$9 are twice continuously differentiable; that the original constrained problem satisfies MFCQ on the feasible set $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$0; that the parametric QCQP satisfies a constant rank condition (CRC) at $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$1; and, outside the feasible set, that Slater’s condition for the QCQP and CRC hold. Under these assumptions, $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$2 is well-defined and locally Lipschitz, so the ODE has unique trajectories (Mestres et al., 1 Oct 2025).

A central structural fact is that equilibria coincide with KKT points. The paper states that if $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$3, then $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$4, and under MFCQ,

$$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$5

Safety is expressed as forward invariance of the feasible set. The QCQP constraints imply

$$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$6

On the boundary $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$7, this reduces to

$$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$8

and Nagumo’s theorem yields forward invariance of $$\dot{x} = -\nabla f(x) - {g(x)}_x^\top u - {h(x)}_x^\top v,$$9. This is the exact sense in which RSGF is anytime: trajectories initiated in the safe set remain safe throughout their evolution.

Inside the feasible set, the objective is nonincreasing along the flow:

$u\in\mathbb{R}^m_{\ge 0}$0

with equality only at KKT points. The derivation uses

$u\in\mathbb{R}^m_{\ge 0}$1

so the robust term appears as additional dissipation. For bounded trajectories that start in $u\in\mathbb{R}^m_{\ge 0}$2, LaSalle’s invariance principle implies convergence to the set of KKT points. For trajectories outside the feasible set, the paper introduces the modified LaSalle function

$u\in\mathbb{R}^m_{\ge 0}$3

and proves convergence of bounded trajectories to KKT points under additional assumptions (Mestres et al., 1 Oct 2025).

These results clarify a recurrent misconception. RSGF is not merely a projected-gradient heuristic with constraints checked post hoc. The constraint inequalities define the admissible velocity field itself, and the equilibrium characterization is optimization-theoretic rather than purely dynamical.

4. Discretization and constrained reinforcement learning

The RL realization of RSGF considers a finite-horizon constrained Markov decision process with parametric policy $u\in\mathbb{R}^m_{\ge 0}$4. The optimization problem is

$u\in\mathbb{R}^m_{\ge 0}$5

where

$u\in\mathbb{R}^m_{\ge 0}$6

The constraints act on the policy parameter $u\in\mathbb{R}^m_{\ge 0}$7, but the paper notes that they can encode state safety; with suitable costs $u\in\mathbb{R}^m_{\ge 0}$8, one obtains

$u\in\mathbb{R}^m_{\ge 0}$9

The continuous-time flow is discretized by forward Euler,

$v\in\mathbb{R}^k$0

Because the value functions and their gradients are not analytically available, the algorithm replaces them with episodic estimates and solves the estimated QCQP

$v\in\mathbb{R}^k$1

subject to

$v\in\mathbb{R}^k$2

together with an additional boundedness constraint for

$v\in\mathbb{R}^k$3

The resulting update is

$v\in\mathbb{R}^k$4

Off-policy value estimates are constructed by importance sampling, and the gradient estimator uses the policy-gradient theorem with a baseline (Mestres et al., 1 Oct 2025).

The algorithmic novelty is not only the discretization, but the preservation of safety under statistical uncertainty. The main theorem states that if the estimated QCQP is feasible, if

$v\in\mathbb{R}^k$5

and if the sample size is large enough so that the estimation errors of $v\in\mathbb{R}^k$6 and $v\in\mathbb{R}^k$7 are sufficiently small, then:

• if $v\in\mathbb{R}^k$8, then

$v\in\mathbb{R}^k$9

• if $K_\alpha(x)$0 but a computable margin $K_\alpha(x)$1, then the next iterate is still safe with the same probability;
• if the condition holds for all constraints, then

$K_\alpha(x)$2

• if $K_\alpha(x)$3, then $K_\alpha(x)$4.

The finite-horizon corollary extends this to

$K_\alpha(x)$5

This is the paper’s anytime guarantee. It covers both safe-to-safe updates and unsafe-to-safe recovery. Under vanishing estimation error,

$K_\alpha(x)$6

diminishing stepsizes $K_\alpha(x)$7 with $K_\alpha(x)$8, and boundedness enforced through $K_\alpha(x)$9, the iterates converge almost surely to the set of KKT points. In the reported experiments, RSGF-RL remains safe throughout training, can recover from an initially unsafe policy, outperforms CPO in both performance and safety metrics, and benefits from off-policy data and importance-sampling clipping (Mestres et al., 1 Oct 2025).

5. Generalizations and application-specific safe-flow architectures

Several later or parallel constructions retain the SGF/RSGF logic while changing the optimization variable, the safety representation, or the plant interface. The following summary is limited to formulations explicitly described in the cited works.

Variant	Problem class	Safety mechanism
SGF / relaxed SGF	Bilevel optimization	Velocity projection onto lower-level consistency constraints
Safe trajectory gradient flow	Grid-interfacing inverter control	QP-based correction on a rolling-horizon trajectory variable
Reference-governed distributed safe gradient flow	Safe optimal output agreement of nonlinear multi-agent systems	First-order CBFs on references plus dynamic safety margins

In bilevel optimization, Safe Gradient Flow is formulated as a single-loop continuous-time solver for

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$0

which is rewritten, under strong convexity of the lower-level problem, as

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$1

The safety condition is dynamic:

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$2

and the safe velocity is obtained by projecting the desired motion onto this constraint through a convex quadratic program. The relaxed compact variant replaces exact lower-level optimality by the sublevel set $$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$3, with

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$4

and yields the relaxed safe gradient flow

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$5

Because $$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$6 is strongly convex, feasibility in $$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$7 implies

$$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$8

so $$\dot g_i(x) \le -\alpha g_i(x), \qquad \dot h_j(x) = -\alpha h_j(x),$$9 directly controls the lower-level suboptimality neighborhood. The forward-invariance guarantee is preserved, and the asymptotic residual in the average hypergradient norm scales as $$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$0 (Sharifi et al., 27 Jan 2025).

In inverter control, the paper studies a three-phase voltage-source inverter connected to an infinite bus through an RL branch in a dq frame, with state $$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$1 and input $$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$2. The safety constraint is the current magnitude limit

$$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$3

so the safe set is a disk in the $$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$4-$$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$5 plane. The equilibrium nonlinear program is lifted to a finite-horizon trajectory optimization problem over the stacked variable $$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$6, and safe gradient flow is applied at trajectory level:

$$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$7

followed by the Euler update

$$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$8

At each control cycle, the method measures the current state, sets it as the first state in the horizon, predicts future states, performs a limited number $$G_\alpha(x)= \operatorname*{argmin}_{\xi\in\mathbb{R}^n} \frac12\|\xi+\nabla f(x)\|^2 \quad \text{s.t.}\quad {g(x)}_x\xi \le -\alpha g(x),\; {h(x)}_x\xi = -\alpha h(x).$$9 of optimization updates, applies only the first control action, and shifts the horizon forward. The paper states that the feasible set is intended to be forward invariant if predicted dynamics match the true dynamics and the step size is sufficiently small, and that the construction retains the anytime property: even if the optimization is interrupted early, the current iterate is still feasible. In the reported simulations, $\alpha\to\infty$0, the average single-step runtime of STGF is about 40 times faster than MPC, and the maximum STGF solve time is orders of magnitude smaller than MPC’s maximum, although the average STGF solve time is still longer than the 1 ms simulation step (Joswig-Jones et al., 15 Jan 2026).

In distributed multi-agent control, the proposed architecture moves the safety layer from plant inputs to reference dynamics. The static agreement problem is

$\alpha\to\infty$1

The upper-layer reference governor evolves as $\alpha\to\infty$2, with $\alpha\to\infty$3 determined by a distributed QP subject to consensus-flow equalities, first-order CBF constraints

$\alpha\to\infty$4

and dynamic safety margin (DSM) inequalities derived from a reference-dependent Lyapunov function. The paper proves forward invariance, optimal-solution preservation under DSM-compatibility conditions, and convergence by a Lyapunov small-gain argument. In the hexagonal workspace example, the method reaches

$\alpha\to\infty$5

with essentially zero average error, while preserving output safety throughout (Shangguan et al., 2 Jul 2026).

Taken together, these variants suggest that the core SGF/RSGF mechanism is portable across markedly different settings, but the location of the safety filter matters. In bilevel optimization it acts on velocity relative to a lower-level manifold; in the inverter it acts on trajectory variables in a receding horizon; in multi-agent systems it acts on references rather than plant inputs.

6. Interpretive scope, adjacent lines, and limitations

The term RSGF should be distinguished from nearby gradient-flow approaches that share some but not all of its defining elements. A particularly close neighbor is the PDE/sampler framework for distributionally robust optimization. That work studies

$\alpha\to\infty$6

rewrites the inner problem as a sampling problem for the worst-case distribution, and solves it through gradient flows on probability measures,

$\alpha\to\infty$7

The authors explicitly state that this is not an RSGF method per se, but a very close predecessor / cousin, and a direct predecessor / nearby ancestor to RSGF-style methods. The similarity lies in the gradient-flow viewpoint, the robust/worst-case objective, and the use of geometric dynamics to construct algorithms; the difference is that there is no explicit safe manifold, trust region, or safety filter beyond the DRO ambiguity regularization (Xu et al., 29 Oct 2025).

A second adjacent line is provably safe analytic-gradient RL with differentiable safeguard layers. There the safeguard is inserted into the differentiable pipeline

$\alpha\to\infty$8

so the effective gradient depends on the Jacobian $\alpha\to\infty$9:

$$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$00

The paper states that it is closely related to RSGF in spirit, but is not formulated as RSGF itself. Boundary projection is safe but rank-deficient on unsafe points, whereas ray mask is full-rank if the safe set is convex. The emphasis is therefore on differentiable safe mappings and Jacobian structure, not on a parameter-space flow defined by QCQP constraints (Walter et al., 2 Jun 2025).

The robustness notion itself is also context-dependent. In the explicit RSGF formulation, robustness refers to the positive $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$01 term, which increases robustness to modeling/estimation errors and strengthens dissipation in the convergence proof (Mestres et al., 1 Oct 2025). In safe trajectory gradient flow for inverter control, the paper states that robustness is “robust” in the safe-flow sense of preserving feasibility under partial optimization progress, not in the full robust optimization / worst-case disturbance sense; safety depends on predicted dynamics aligning with true dynamics, and disturbances and estimation errors are left for future work (Joswig-Jones et al., 15 Jan 2026). In RL-based voltage control with safe gradient flow, the method enforces reactive-power feasibility by a CBF-QP projection, but the paper explicitly states that it does not introduce explicit robustness to model uncertainty, uncertainty sets, disturbance attenuation guarantees, robust optimization layers, or adversarial or worst-case analysis of the safe gradient flow mechanism (Feng et al., 2023).

These distinctions matter because they separate three ideas that are often conflated: safety as forward invariance, robustness as resistance to modeling or sampling error, and robustness as worst-case optimization against adversarial uncertainty. RSGF, in its strict formulation, addresses the first directly and the second through the $$C = \{x \in \mathbb{R}^n \mid g(x)\le 0,\; h(x)=0\}.$$02 term; broader RSGF-style work may address only one of these senses, or may implement safety at the level of references, trajectories, or action mappings rather than directly on the optimization variables.