Nested Extremum Seeking (nES)

• nES is a multi-loop optimization scheme that uses fast inner loops for motion generation and slow outer loops to optimize performance based solely on measured outputs.
• It leverages Lie-bracket averaging and game-theoretic formulations to create effective slow dynamics, enabling equilibrium and stability in complex systems.
• The architecture is applied to nonholonomic systems and multi-agent scenarios, where proper time-scale separation guarantees convergence and robust control.

Searching arXiv for recent and foundational papers on nested extremum seeking and Lie-bracket ES. Search query: nested extremum seeking Lie bracket averaging Stackelberg equilibrium extremum seeking nonholonomic systems Nested extremum seeking (nES) is a multi-loop extremum-seeking architecture in which multiple ES mechanisms are stacked across separated time scales, so that a fast inner loop realizes a lower-level optimization or motion-generation task and a slower outer loop optimizes a higher-level objective using only measured performance values. In the recent game-theoretic formulation, nES is presented as multiple ES loops running simultaneously on interacting variables, with equilibrium selection determined by the hierarchy of gains and dither frequencies rather than by changing the closed-loop structure itself (Ratto et al., 25 Mar 2026). In the nonholonomic control setting, the same nested idea appears as an inner Lie-bracket-based oscillatory tracking layer wrapped by an outer model-free ES layer, yielding a rigorous multiple-time-scale extremum-seeking scheme for driftless control-affine systems (Grushkovskaya et al., 2020).

1. Definition and architectural forms

The defining feature of nES is the nesting of ES loops in series. In the two-level case emphasized in current work, a leader loop and a follower loop evolve according to

$$\begin{aligned} \dot{x}_1 &= \sqrt{\alpha_1\omega_1}\cos\big(\omega_1 t + k_1 J_{\rm L}(x_1, x_2)\big),\ \dot{x}_2 &= \sqrt{\alpha_2\omega_2}\cos\big(\omega_2 t + k_2 J_{\rm F}(x_1, x_2)\big), \end{aligned}$$

with distinct dither frequencies and positive gains (Ratto et al., 25 Mar 2026). This representation is model-free in the usual ES sense: the costs are measured in real time, but analytic gradients are not required.

A second, structurally different but conceptually equivalent realization appears for nonholonomic systems. There, the plant is a driftless control-affine system

$\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$

with $m<n$, and the nested design introduces an auxiliary reference state $\xi\in\mathbb{R}^n$ through

$$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$

The inner layer uses model information and Lie-bracket approximations to make $x(t)$ track $\xi(t)$; the outer layer is a model-free ES law driven only by the scalar output $y=J(x)$ (Grushkovskaya et al., 2020).

Two architectural motifs recur across these formulations. First, the inner loop is faster than the outer loop. Second, the fast oscillations are not merely probing signals but are designed so that their averaged effect induces a useful slow vector field. In the nonholonomic setting, that slow vector field is a stabilizing or tracking dynamics in state space; in the game-theoretic setting, it is a gradient or best-response-like dynamics. This suggests that nES is best understood not as a single algorithmic template but as a hierarchy of oscillatory feedback layers whose averaged interactions realize a prescribed optimization geometry.

2. Lie-bracket and averaging foundations

The principal analytical foundation of nES is the Lie-bracket interpretation of extremum seeking. For an input-affine system with fast periodic perturbations,

$$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$

the associated Lie-bracket system is

$$\dot{z} = b_0(t,z) + \sum_{i<j} [b_i,b_j](t,z)\,\nu_{ji}(t),$$

where $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$0 depends only on the dithers (Dürr et al., 2011). The central conclusion is that, for sufficiently large dither frequency, trajectories of the true ES system are approximated on finite horizons by trajectories of this bracket system, and uniform asymptotic stability of the bracket system implies practical uniform asymptotic stability of the original ES dynamics (Dürr et al., 2011).

This viewpoint is especially important for nES because each loop can be designed so that its effective slow dynamics is itself the target of a higher-level loop. In the simplest scalar example,

$\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$1

the Lie-bracket reduction yields

$\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$2

making the optimizing behavior explicit (Dürr et al., 2011). In nested designs, the same mechanism is applied iteratively: fast dithers generate an averaged drift, and slower loops act on the state produced by that averaged drift.

Recent two-level Stackelberg analysis makes this iterative structure explicit through a three-step reduction: fast two-dimensional Lie-bracket averaging for the follower loop, singular perturbation to exploit fast follower adaptation, and slow one-dimensional Lie-bracket averaging for the leader loop (Ratto et al., 25 Mar 2026). In the nonholonomic setting, the interaction between layers is analyzed with averaging and Chen–Fliess expansions, and the remainder terms are controlled through constructive separation conditions on the time-scale parameters $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$3 and $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$4 (Grushkovskaya et al., 2020). Taken together, these results place nES within the broader theory of oscillatory control, averaged systems, and singular perturbations.

3. Nonholonomic nested extremum seeking

A prototypical nES construction for control systems is developed for nonlinear driftless control-affine systems satisfying a one-step bracket generating condition,

$\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$5

with an invertible matrix $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$6 built from selected vector fields and first-order Lie brackets (Grushkovskaya et al., 2020). The cost $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$7 is assumed twice continuously differentiable and strongly convex on $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$8, with unique minimizer $\dot x = \sum_{i=1}^m u_i f_i(x), \qquad y = J(x),$9.

The nested closed-loop system contains two explicit time-scale parameters. The inner stabilizing layer uses oscillatory controls $m<n$0 with fast scale $m<n$1, while the outer ES layer evolves the virtual reference $m<n$2 on the slower scale $m<n$3, under the design condition

$m<n$4

The inner control contains a constant component that generates motion along directly actuated vector fields and a high-frequency sinusoidal component, with amplitudes scaling like $m<n$5 and frequencies proportional to $m<n$6, that generates averaged motion along Lie brackets $m<n$7 (Grushkovskaya et al., 2020). Its coefficients are chosen through

$m<n$8

so that the averaged inner dynamics approximates $m<n$9.

The outer loop has the form

$\xi\in\mathbb{R}^n$0

with generating functions satisfying

$\xi\in\mathbb{R}^n$1

and dither amplitudes proportional to $\xi\in\mathbb{R}^n$2 with frequencies proportional to $\xi\in\mathbb{R}^n$3 (Grushkovskaya et al., 2020). After averaging, the $\xi\in\mathbb{R}^n$4-dynamics becomes

$\xi\in\mathbb{R}^n$5

so the outer loop behaves like gradient descent in the virtual state.

The analysis yields exponential convergence of $\xi\in\mathbb{R}^n$6 to an arbitrarily small neighborhood of $\xi\in\mathbb{R}^n$7 under suitable choices of $\xi\in\mathbb{R}^n$8, $\xi\in\mathbb{R}^n$9, and $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$0. More precisely, for initial conditions in a sufficiently small ball around $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$1,

$$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$2

for some $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$3, with residual $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$4 tunable through the time-scale parameters (Grushkovskaya et al., 2020).

The Brockett integrator,

$$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$5

serves as the numerical example. Its vector fields satisfy

$$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$6

and the chosen objective is $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$7 (Grushkovskaya et al., 2020). Two generating-function choices are illustrated: the classical $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$8, and a pair that vanishes at $$\dot\xi = g(y,t), \qquad u_i = \varphi_i^\varepsilon(x,\xi,t).$$9, which reduces steady-state oscillations while preserving the nested architecture.

4. Nash and Stackelberg regimes

In game-theoretic nES, the same two-loop dynamics can realize either Nash or Stackelberg behavior. A Nash equilibrium $x(t)$0 satisfies

$x(t)$1

whereas a Stackelberg equilibrium is defined through the follower best-response map

$x(t)$2

and the reduced leader cost

$x(t)$3

The Stackelberg point $x(t)$4 then satisfies

$x(t)$5

The crucial result is that equilibrium selection depends on parameter hierarchy, not on changing the loop structure. Stackelberg behavior is obtained by enforcing the four-time-scale ordering

$x(t)$6

so that follower dithering is fastest, follower adaptation is next, leader dithering is slower, and leader adaptation is slowest (Ratto et al., 25 Mar 2026). Under assumptions $x(t)$7, $x(t)$8, uniqueness and $x(t)$9-regularity of $\xi(t)$0, and strong convexity of both $\xi(t)$1 in $\xi(t)$2 and $\xi(t)$3, the original nES system is $\xi(t)$4-SPUAS to the unique Stackelberg equilibrium for sufficiently large $\xi(t)$5, sufficiently small $\xi(t)$6, and sufficiently large $\xi(t)$7 (Ratto et al., 25 Mar 2026).

The quadratic example

$\xi(t)$8

makes the distinction explicit. The follower best response is $\xi(t)$9; the Nash equilibrium is

$y=J(x)$0

whereas the Stackelberg equilibrium is

$y=J(x)$1

(Ratto et al., 25 Mar 2026). Simulations reported there show convergence to a neighborhood of the Nash point under comparable time scales and to a neighborhood of the Stackelberg point under hierarchical scaling.

The Fish War game provides a second illustration. With

$y=J(x)$2

and parameters

$y=J(x)$3

the reported equilibria are $y=J(x)$4 and $y=J(x)$5 (Ratto et al., 25 Mar 2026). Although the assumptions of the theorem do not hold globally for this example, the same scaling mechanism is reported to work locally.

5. Learning dynamics and effective objectives

A recurrent misconception is that ES, and therefore nES, directly optimizes the original unknown objective. A more precise statement is that perturbation-based ES recovers an averaged gradient, and the induced learning dynamics can be interpreted as gradient descent on an effective objective $y=J(x)$6, not necessarily on the original $y=J(x)$7 (Wildhagen et al., 2018).

For the scalar ES system

$y=J(x)$8

with $y=J(x)$9-periodic dithers and Lie-bracket relation

$$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$0

needle-variation analysis yields a recursion at period sampling times $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$1,

$$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$2

where $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$3 is expressed as a double integral involving the nominal trajectory, the dither signals, the state-transition matrix of the variational equation, and the derivative of $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$4 (Wildhagen et al., 2018). The recovered gradient is therefore nonlocal and averaged over a period.

The multidimensional extension uses sequential dithers so that each coordinate receives a component-wise averaged partial derivative with only higher-order cross-coupling (Wildhagen et al., 2018). For nested architectures, the direct implication is that each loop optimizes a smoothed version of the function it sees. This suggests that, under sufficient time-scale separation, an outer loop acts on a cost landscape already modified by the inner loop’s effective objective. A plausible implication is that the nested steady state is generally the minimizer of a composed effective cost rather than of the original multilevel objective.

This effective-objective viewpoint also clarifies why ES can sometimes traverse local extrema of the original function. Because the recovered gradient is averaged over a finite time window, local features can be evened out in the learning dynamics (Wildhagen et al., 2018). In nES, the same smoothing may appear at multiple layers.

6. Assumptions, limitations, and research directions

Existing rigorous nES analyses rely on strong regularity and separation assumptions. The Lie-bracket approximation framework assumes $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$5 vector fields, bounded derivatives on compact sets, and periodic zero-mean perturbations (Dürr et al., 2011). The nonholonomic construction assumes a one-step bracket generating condition, strong convexity and smoothness bounds for $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$6, bounded inverse $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$7, and a noise-free setting with access to the full state $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$8 for the inner control law (Grushkovskaya et al., 2020). The Stackelberg result is currently proved for two players with scalar decision variables under global strong convexity and smoothness assumptions on follower and reduced leader costs (Ratto et al., 25 Mar 2026).

Several limitations are explicit. Higher-order Lie brackets are not treated in the nonholonomic scheme, although their extension is identified as future work (Grushkovskaya et al., 2020). The Lie-bracket ES theory provides only practical, not exact, asymptotic convergence for the perturbed system (Dürr et al., 2011). The learning-dynamics characterization does not furnish a closed-form formula for $$\dot{x} = b_0(t,x) + \sum_{i=1}^{m} b_i(t,x)\sqrt{\omega}\, u_i(t,\omega t),$$9 in general, and it does not address measurement noise or saturation effects (Wildhagen et al., 2018). The Stackelberg proof does not yet cover $$\dot{z} = b_0(t,z) + \sum_{i<j} [b_i,b_j](t,z)\,\nu_{ji}(t),$$0-nested hierarchies or higher-dimensional decision variables (Ratto et al., 25 Mar 2026).

At the same time, the present literature points to several coherent directions. One is deeper hierarchy: multiple nested small parameters, each supporting a separate averaging or singular-perturbation reduction. Another is structural generalization: nonholonomic systems requiring higher-order brackets, broader classes of cost functions beyond the quadratic-like regime, and multi-agent settings in which distinct rationally related frequencies suppress unwanted mixed brackets [(Grushkovskaya et al., 2020); (Dürr et al., 2011)]. Application domains already identified include power grids, networked dynamical systems, and tuning of particle accelerators, all of which naturally exhibit hierarchical or multi-time-scale organization (Ratto et al., 25 Mar 2026).

In this sense, nES occupies an intersection of model-free optimization, geometric control, and hierarchical dynamical systems. Its technical core is not merely the use of dithers, but the deliberate construction of layered averaged dynamics whose slow limit reproduces a desired optimization principle.