Model-Free and Gray-Box Feedback Optimization

• Model-free and gray-box feedback optimization are techniques that optimize dynamical systems using only output evaluations, avoiding explicit gradient computation.
• They employ periodic dithering and projection methods to estimate gradients and enforce hard constraints, ensuring robust feasibility in real-time.
• These methods have proven effective in applications like voltage control and robotics, offering convergence guarantees and safety against disturbances.

Model-free and gray-box feedback optimization techniques address the problem of steering dynamical systems to optimal operating points when explicit system models or sensitivities are unavailable or only partially known. These methods have developed rigorous frameworks for real-time optimization of nonlinear, possibly constrained systems via output feedback, encompassing both theoretical guarantees and practical implementation strategies.

1. Core Principles of Model-Free Feedback Optimization

Model-free feedback optimization methods aim to solve problems of the form

$\min_{x \in \mathcal{X}} f(x)$

where $f$ and constraint set $\mathcal{X}$ are known only via zeroth-order (output) evaluations, and $\mathcal{X}$ is closed and convex. These algorithms operate without requiring gradients or explicit system models, instead using real-time system interactions to infer optimality information.

A fundamental representative is the continuous-time projected zeroth-order (P-ZO) dynamics, defined by the ODE system: $$\begin{aligned} \dot x &= k_x\left(\mathcal{P}_{\mathcal{X}}(x-\alpha_x \xi) - x\right) \ \dot \xi &= \frac{1}{\varepsilon_\xi}\left( -\xi + \frac{2}{\varepsilon_a} f(\hat x) \hat\mu \right) \ \dot \mu &= \frac{1}{\varepsilon_\omega} \Lambda_\kappa \mu \end{aligned}$$ where $\hat x = x + \varepsilon_a \hat\mu$, and $\hat\mu$ comprises distinct high-frequency dither signals. This mechanism employs periodic dithering to generate persistently exciting probes, making it possible to estimate gradients by demodulation and subsequent low-pass filtering (Chen et al., 2023, Mehrnoosh et al., 15 Sep 2025).

P-ZO and related schemes generalize extremum-seeking approaches with projection operators, ensuring that hard constraints are satisfied at all times without recourse to barriers or penalties.

2. Gradient Learning and Projection Enforcement

Model-free feedback optimizers must estimate gradients from finite differences or periodic excitation. For P-ZO-type schemes, dithering in independent directions yields: $$\xi(t) \approx \frac{2}{\varepsilon_a T} \int_0^T f\left( x + \varepsilon_a \sin\omega t \right) \sin\omega t\,dt \approx \nabla f(x) + O(\varepsilon_a)$$ By continuously injecting dither and filtering outputs, high-confidence gradient estimates are obtained. This property is foundational for zeroth-order convergence analysis (Chen et al., 2023, Mehrnoosh et al., 15 Sep 2025).

Crucially, the algorithm enforces hard constraints by projecting updates onto $\mathcal{X}$: $$\mathcal{P}_{\mathcal{X}}(z) = \arg\min_{y \in \mathcal{X}} \|y - z\|$$ This ensures $f$0 for all $f$1, with practical ODEs designed to either update via projected flows or, in discontinuous variants, project vector fields onto the tangent cone of $f$2 (Chen et al., 2023, Chen et al., 2022).

3. Constrained Model-Free Primal-Dual Dynamics

To handle both hard and soft (asymptotic) constraints, projected primal-dual zeroth-order dynamics (P-PDZD) extend P-ZO by including Lagrange multipliers and corresponding estimator filters. For the constrained convex program

$f$3

the P-PDZD scheme simultaneously updates $f$4 and $f$5, relying on function probes and real-time output evaluations. The flow

$f$6

with auxiliary filters for gradient and constraint estimation, yields semi-global practical asymptotic stability (SGPAS) under standard regularity assumptions, robust to small bounded disturbances (Chen et al., 2022).

Decentralized extensions allow multi-agent implementation, with local agents communicating only on constraint coupling, and online stability/robustness guarantees extending identically to the centralized context.

4. Gray-Box Feedback Optimization: Adaptive Sensitivity Fusion

Gray-box feedback optimization systematically blends approximate model-based sensitivities with model-free updates. Suppose the gradient $f$7 is unavailable, but an approximate sensitivity $f$8 is accessible. The gray-box update law is given by

$f$9

with $\mathcal{X}$0 adaptively tuned. The controller selects $\mathcal{X}$1 based on the estimated bias in $\mathcal{X}$2:

• If $\mathcal{X}$3 with $\mathcal{X}$4, model-based updates dominate.
• For coarser or slowly improving $\mathcal{X}$5, gray-box or model-free dominates, mitigating bias while reducing variance.

Convergence and regret analyses show that the gray-box architecture achieves $\mathcal{X}$6 rates in the static unconstrained case, and favorable dynamic regret bounds in time-varying, nonconvex, and constrained problems (He et al., 2024, Mehrnoosh et al., 15 Sep 2025). When the sensitivity estimate converges sufficiently quickly, model-based is optimal; otherwise, the gray-box controller adaptively transitions to model-free behavior.

Notably, gray-box methods are extensible to time-varying settings, where the update becomes

$\mathcal{X}$7

with measurable ISS-type tracking guarantees and dynamic regret $\mathcal{X}$8 as a function of path length $\mathcal{X}$9 of the moving optimum.

5. Two-Point Random Gradient-Free Methods

Recent two-point random gradient-free methods extend static zeroth-order (ZOSGD) ideas to the feedback optimization setting. At each iteration, the algorithm uses two real-time function evaluations (plant probes) at $\mathcal{X}$0 and $\mathcal{X}$1 with $\mathcal{X}$2 to form the update: $\mathcal{X}$3 Such methods achieve $\mathcal{X}$4-stationary points for $\mathcal{X}$5-smooth, nonconvex costs with $\mathcal{X}$6 sample complexity. The stochasticity introduced by probing provides unbiased gradient estimates with controlled variance. When compared to gray-box and one-point estimators, the two-point methods display better scaling and approach idealized performance for $\mathcal{X}$7 complexity (Mehrnoosh et al., 15 Sep 2025, Hauswirth et al., 2021).

6. Robustness, Safety, and Practical Guarantees

Both model-free and gray-box feedback optimizers are furnished with explicit robustness and safety results:

• Forward invariance of the feasible domain is guaranteed by projection, provided initial feasibility.
• Under uniformly globally attractive projected flows, the P-ZO and P-PDZD systems converge to neighborhoods of the optimal set, with the radius controllable via dither/filter parameters (Chen et al., 2023, Chen et al., 2022).
• The system maintains robust safety with respect to bounded state and measurement disturbances, with convergence and constraint satisfaction preserved up to $\mathcal{X}$8 error (Chen et al., 2023, Chen et al., 2022).

Extension to time-varying and switching objectives leverages ISS-type analysis, ensuring graceful degradation and optimality tracking in nonstationary environments. Numerical experiments confirm sub-percent steady-state errors within a few time units and consistent hard constraint satisfaction across synthetic and practical case studies (e.g., voltage control in distribution grids).

7. Applications and Outlook

Model-free and gray-box feedback optimization has been applied to domains including autonomous grid control, robotic policy learning under sensing failures, and generic nonlinear output regulation. The performance trade-offs among model-based, gray-box, and model-free approaches are summarized as follows:

Approach	Model Knowledge Needed	Convergence Rate	Constraint Handling	Robustness to Model Error
Model-free	None	$\mathcal{X}$9	Projection, with limitations	High
Gray-box	Partial/approximate	$$\begin{aligned} \dot x &= k_x\left(\mathcal{P}_{\mathcal{X}}(x-\alpha_x \xi) - x\right) \ \dot \xi &= \frac{1}{\varepsilon_\xi}\left( -\xi + \frac{2}{\varepsilon_a} f(\hat x) \hat\mu \right) \ \dot \mu &= \frac{1}{\varepsilon_\omega} \Lambda_\kappa \mu \end{aligned}$$0	Projection + dual	Intermediate
Model-based	Exact $$\begin{aligned} \dot x &= k_x\left(\mathcal{P}_{\mathcal{X}}(x-\alpha_x \xi) - x\right) \ \dot \xi &= \frac{1}{\varepsilon_\xi}\left( -\xi + \frac{2}{\varepsilon_a} f(\hat x) \hat\mu \right) \ \dot \mu &= \frac{1}{\varepsilon_\omega} \Lambda_\kappa \mu \end{aligned}$$1	$$\begin{aligned} \dot x &= k_x\left(\mathcal{P}_{\mathcal{X}}(x-\alpha_x \xi) - x\right) \ \dot \xi &= \frac{1}{\varepsilon_\xi}\left( -\xi + \frac{2}{\varepsilon_a} f(\hat x) \hat\mu \right) \ \dot \mu &= \frac{1}{\varepsilon_\omega} \Lambda_\kappa \mu \end{aligned}$$2	Exact	Low if model mismatch

Gray-box schemes can exploit any available sensitivity knowledge, including online-estimated Jacobians, and naturally interpolate between model-based and model-free as knowledge quality varies (He et al., 2024, Hauswirth et al., 2021). Integration of acceleration, discrete-time and event-triggered versions, and hybrid learning-control strategies remain active research directions (Chen et al., 2022, Chen et al., 2023, Mehrnoosh et al., 15 Sep 2025).

In summary, model-free and gray-box feedback optimization synthesize extremum-seeking, projected gradient, and adaptive control concepts to produce algorithms with provable optimality, guaranteed safety, and robust performance, scalable from low-level physical systems to high-dimensional learning-driven domains.