Extremum Seeking Control (ESC)

• Extremum Seeking Control is a model-free adaptive feedback method that uses sinusoidal dithers and filtering to optimize steady-state performance in nonlinear systems.
• It extracts local gradient information through demodulation and averaging, enabling optimization without explicit knowledge of system derivatives.
• Recent advances integrate adaptive gains, oscillation reduction techniques, and hybrid strategies with reinforcement learning to enhance robustness and convergence.

Extremum Seeking Control (ESC) is a model-free adaptive feedback method for real-time optimization of a steady-state performance index using only measured outputs. In its classical perturbation-based form, ESC injects a sinusoidal dither, filters and demodulates the measured response, and integrates the resulting signal so that the closed loop approaches an extremum of an unknown steady-state map. Within the literature represented here, ESC appears both as a canonical steady-state optimizer for nonlinear plants and as a broader design paradigm that has been extended to adaptive normalization, vanishing perturbations, control-affine and non-affine systems, distributed-parameter actuation, and hybridizations with reinforcement learning and data-driven approximation (Trollberg et al., 2018, McNamee et al., 2024, Chang et al., 2024).

1. Canonical formulation and classical loop

For open-loop stable nonlinear SISO plants, ESC is commonly posed on systems of the form

$\dot x = f(x,u), \qquad y=h(x),$

with equilibria parameterized by the input through

$f(x,u)=0 \iff x=l(u),$

so that the steady-state input-output map is

$J(u)=h\circ l(u).$

In this setting, ESC seeks the extremum of $J(u)$ without using an explicit model of $J$ or its derivatives (Trollberg et al., 2018).

The classical perturbation-based ESC loop adds a sinusoidal dither $a\sin(\omega t)$ to a nominal input $\hat u$, measures the output, high-pass filters the measurement, demodulates by multiplying by $\sin(\omega t)$, low-pass filters the result to approximate gradient information, and integrates that signal to update $\hat u$. With first-order filters, one standard closed-loop realization is

$$\begin{aligned} \dot x &= f(x,\hat u + a\sin(\omega t)),\ y &= h(x),\ \dot \eta &= \omega_h(y - \eta),\ \dot \xi &= \omega_l\big((y-\eta)\sin(\omega t) - \xi\big),\ \dot{\hat u} &= k\xi. \end{aligned}$$

The paper on stationary non-uniqueness studies periodic stationary solutions of period

$f(x,u)=0 \iff x=l(u),$0

that is, period-one solutions synchronized with the dither (Trollberg et al., 2018).

A standard local result, attributed there to Krstić and Wang, is that under mild conditions and conservative tuning the ESC loop has a stable stationary periodic solution near an extremum of the steady-state map $f(x,u)=0 \iff x=l(u),$1. The same source emphasizes that this is only a local existence and stability result; it does not imply uniqueness or global convergence (Trollberg et al., 2018).

2. Demodulation, averaging, and gradient surrogates

The central ESC mechanism is the extraction of local slope information from an oscillatory probe. One formulation writes

$f(x,u)=0 \iff x=l(u),$2

$f(x,u)=0 \iff x=l(u),$3

where $f(x,u)=0 \iff x=l(u),$4 is a high-pass filter and $f(x,u)=0 \iff x=l(u),$5 is a low-pass filter. In the one-dimensional local analysis summarized for extremum-seeking action selection, the filtered and demodulated response yields

$f(x,u)=0 \iff x=l(u),$6

which shows exponential convergence of $f(x,u)=0 \iff x=l(u),$7 to a nearby local optimum $f(x,u)=0 \iff x=l(u),$8 provided the sign of $f(x,u)=0 \iff x=l(u),$9 matches the curvature. The same account stresses that the algorithm does not need the analytic derivative or curvature $J(u)=h\circ l(u).$0; the local optimization arises implicitly through filtering and perturbation (Chang et al., 2024).

A complementary abstraction distinguishes model-free ESC, its dither-averaged system, and the limiting model-based ESC. In one general form,

$J(u)=h\circ l(u).$1

with average system

$J(u)=h\circ l(u).$2

and model-based limit

$J(u)=h\circ l(u).$3

For gradient-based ESC, the model-based limit becomes

$J(u)=h\circ l(u).$4

while the model-free implementation uses dither signals of the form

$J(u)=h\circ l(u).$5

subject to nonresonance restrictions on the relative frequencies $J(u)=h\circ l(u).$6 (McNamee et al., 21 Jul 2025).

Recent adaptive variants retain the same averaging logic but modify the normalization of the gradient estimate. In RMSpESC, the parameter is decomposed as

$J(u)=h\circ l(u).$7

and the averaged gradient estimate satisfies

$J(u)=h\circ l(u).$8

as $J(u)=h\circ l(u).$9. The adaptation law then normalizes each component by an RMS-like denominator,

$J(u)$0

to reduce sensitivity of the local convergence rate to unknown curvature (McNamee et al., 2024).

3. Stability theory, phase effects, and non-uniqueness

A major refinement of the classical picture is that stationary ESC solutions need not be unique, even when the underlying optimization problem is convex. For low-amplitude period-$J(u)$1 stationary solutions, a necessary condition is

$J(u)$2

which, under the small-harmonics assumption $J(u)$3, simplifies to

$J(u)$4

Equivalently, when $J(u)$5,

$J(u)$6

For Hammerstein or Wiener type plants, this condition is tied directly to the steady-state gradient condition, but for more general dynamic plants it becomes a phase-lag condition. The consequence is that stationary solutions can exist at operating points unrelated to optimality (Trollberg et al., 2018).

That phase-based mechanism leads to bifurcation phenomena. Writing

$J(u)$7

stationary-solution branches can be traced in the $J(u)$8-plane, and cyclic fold bifurcations can create multiple stationary period-one solutions. In the chemical-reactor example summarized there, the ESC loop admits at least five stationary solutions simultaneously for realistic control parameters; three are stable and two are unstable. The near-optimal branch undergoes a fold bifurcation at

$J(u)$9

and one practical consequence is that one generally needs to initialize the loop close to the optimum to ensure convergence to the near-optimal solution (Trollberg et al., 2018).

More recent stability results partially decouple practical stability from the detailed stability of every dither-induced average system. One 2025 result shows that some selections of relative dither amplitudes and rates can make the average system unstable even when the corresponding model-based ESC is globally asymptotically stable. Its main theorem states that if the model-based ESC is globally asymptotically stable, then the average model-free ESC is semiglobally practically asymptotically stable and the original model-free ESC is semiglobally practically uniformly asymptotically stable. The design implication is that small dither amplitude and sufficiently large dither rate provide a practical stability route even when a particular average system exhibits a locally unstable equilibrium (McNamee et al., 21 Jul 2025).

Stability can also be strengthened by adapting the optimizer aggressiveness to gradient-estimation quality. In a discrete-time ESC scheme with batch least-squares gradient estimation, the adaptive step size

$J$0

is chosen from a worst-case error design, so that exploitation occurs only when the gradient estimate is sufficiently reliable. The resulting closed loop is proved input-to-state stable with respect to the dither signal (Danielson et al., 2021).

4. Adaptive, low-oscillation, and data-efficient variants

A large part of recent ESC research modifies the classical loop to improve rate uniformity, reduce steady-state oscillation, or increase robustness to uncertainty in the measured response. RMSpESC introduces a coordinatewise adaptive gain

$J$1

and proves that the full closed loop is semiglobally practically uniformly asymptotically stable to

$J$2

with respect to the small parameters $J$3. The proof uses a Lyapunov function assembled from the cost suboptimality and distances to contracting attractive sets associated with filter states, rather than a fixed quadratic construction (McNamee et al., 2024).

Several papers attack the classical ESC drawback that persistent dithering produces steady-state oscillations. Two perturbation-based SISO schemes use amplitude laws

$J$4

so that the excitation magnitude converges to zero as the gradient measure becomes small near the extremum; both schemes achieve practical asymptotic convergence to a neighborhood of the extremum with attenuated steady-state oscillations (Bhattacharjee et al., 2020). A related sampled-data construction, retrospective cost-based ESC, uses a vanishing perturbation $J$5 together with Kalman-filter gradient estimation and retrospective cost adaptive control, thereby avoiding perpetual dithering while retaining online adaptation to changing minimizers (Paredes et al., 2024). In control-affine ESC, a geometric extended Kalman filter estimates the Lie-bracket system, and the amplitude law

$J$6

drives $J$7 under the time-dependent condition

$J$8

yielding vanishing oscillations under a condition presented as easier to check than earlier generalized control-affine ESC conditions (Pokhrel et al., 2021).

Other variants address measurement economy and sensor noise. A kernel-based sampled-data ESC method constructs an online RKHS approximation of the unknown steady-state cost and uses certified descent and Armijo bounds to update parameters without new experiments whenever a decrease in the true cost can be guaranteed. In the reported nonlinear two-input simulation, it reached the same neighborhood of the optimum with 60 experiments instead of 100, a 40% reduction, and in 18 update steps instead of 25, a 28% reduction (Weekers et al., 28 Jan 2025). ESC/AISE replaces the conventional high-pass filter by adaptive input and state estimation used as a numerical differentiator. In the reported quadratic static-map example, the average RMSE changed from 0.476 for ESC to 0.240 for ESC/AISE; in the ABS wheel-slip example, the average RMSE changed from 2.0281 to 0.22208, the average $J$9 from 30.3806 s to 24.3548 s, and the fraction of trials stopping the wheel within 50 s from 64% to 100% (Verma et al., 8 Jan 2025).

5. Extensions beyond classical SISO steady-state optimization

ESC has also been generalized well beyond the standard control-affine, finite-dimensional, unconstrained SISO setting. For systems not affine in control,

$a\sin(\omega t)$0

a high-frequency dither of the form

$a\sin(\omega t)$1

produces an averaged stabilizing gradient-like drift through the highest odd power. If the averaged system is globally uniformly asymptotically stable, then the original oscillatory system is $a\sin(\omega t)$2-semi-globally practically uniformly asymptotically stable for sufficiently large $a\sin(\omega t)$3 (Scheinker et al., 2016).

For constrained nonlinear systems, numerical optimization-based ESC (NOESC) combines projected gradient descent,

$a\sin(\omega t)$4

with an inversion-based feedforward controller obtained from an input-output normal form and an internal-dynamics boundary value problem. The method is stated to i) explicitly deal with output constraints; ii) allow the performance function to consider a direct dependence on the states of the internal dynamics; and iii) not require the internal dynamics to be necessarily stable (Yuan et al., 2021).

Distributed-parameter generalizations are represented by ESC for wave-PDE actuation with distributed effects. There the optimization variable seen by the unknown static map is

$a\sin(\omega t)$5

with $a\sin(\omega t)$6 governed by a wave equation. The dither is redesigned through a trajectory-generation problem so that the distributed input seen by the map reproduces the intended sinusoid, after which backstepping boundary control ensures exponential stability of the averaged closed loop and infinite-dimensional averaging yields convergence to a neighborhood of the optimizer (Muchave et al., 5 Jan 2026).

Multivariable and output-feedback generalizations pursue simpler deployment under weaker sensing. A self-perturbing relay-based ESC for MISO systems uses stochastic relay gains to make the multivariable gradient identifiable and is described as requiring one configurable parameter per input channel for the static case and one additional parameter for the dynamic version (Salsbury et al., 19 May 2026). A different output-feedback scheme for uncertain nonlinear SISO systems combines a monitoring function with a norm state observer and extends from relative-degree one to arbitrary uncertain relative degree through time scaling; it is stated to achieve the extremum of an unknown nonlinear mapping across the entire domain of initial conditions and to allow arbitrarily small proximity to the desired optimal point through output feedback (Aminde et al., 2024).

6. Applications, hybrids, and recurring misconceptions

The application record in these papers is notably heterogeneous, which suggests that ESC is less a single algorithm than a family of perturbation-and-response optimization constructions. In reinforcement learning for continuous control, extremum-seeking action selection refines each sampled action before execution. Given a policy sample $a\sin(\omega t)$7, the method forms

$a\sin(\omega t)$8

queries the critic at $a\sin(\omega t)$9, updates

$\hat u$0

and executes $\hat u$1. The method is framed as local action refinement rather than mere exploration, and was evaluated as an add-on to PPO and SAC on inverted pendulum, hopper, walker, and a Gazebo/PX4 quadrotor task. The reported compute overhead is moderate, up to about 50% longer runtime per episode, while PPO+ESA and SAC+ESA learn faster and achieve better final returns than the corresponding baselines in the reported experiments (Chang et al., 2024).

In energy systems, continuous-time model-free ES schemes were used to maximize the mean extracted power of wave energy converters by tuning the reactive and resistive PTO coefficients. The comparison covered sliding mode, relay, least-squares gradient, self-driving, and perturbation-based methods; except for the self-driving ES algorithm, the other four schemes reliably converge for the two-parameter optimization problem, while self-driving is more suitable for optimizing a single parameter. For irregular sea states, the sliding mode and perturbation-based schemes were reported to have better convergence to the optimum than the other schemes considered (Parrinello et al., 2020).

In aeroacoustics, ESC and slope seeking were applied in high-fidelity compressible Navier–Stokes simulations for NACA0012 trailing-edge noise suppression. The controller optimized either actuator position or jet intensity using a pressure-fluctuation cost. Reported attenuation reached approximately $\hat u$2 dB in the best moving-actuation low-Reynolds-number cases and about $\hat u$3 dB in the best higher-Reynolds-number case. The same study argues that slope seeking is useful when the cost function has a plateau rather than a sharp minimum, because standard ESC may otherwise continue increasing control effort in a flat region (Oliveira et al., 2021).

Mechanical and biomimetic variants show a different interpretation of ESC: oscillatory actuation itself can be the physical perturbation. A vibrational-stabilization ESC for second-order mechanical systems uses one perturbation signal and admits generalized forces that are quadratic in velocities; it was demonstrated on mass-spring and inverted-pendulum systems and used for 1D model-free source-seeking of a flapping system (Elgohary et al., 5 Apr 2025). Experimental flapping-robot work then reported 1D model-free hovering and source seeking using only onboard measurements and no explicit dynamic model, with hovering represented by

$\hat u$4

and light-source seeking driven by a photoresistive measurement whose minimum corresponds to the source location (Elgohary et al., 28 Aug 2025).

Two recurrent misconceptions are explicitly corrected in this literature. First, convexity of the underlying steady-state optimization problem does not imply uniqueness of the ESC stationary solution, because dynamic phase-lag conditions can generate stable non-optimal stationary orbits (Trollberg et al., 2018). Second, in hybrid ESC-RL formulations, ESC is “not just an exploration strategy,” but a local sample-quality improvement mechanism whose role is distinct from the global policy improvement performed by the base reinforcement-learning algorithm (Chang et al., 2024).