Extremum Seeking of Static Maps in the Presence of Unknown Large Time-Varying Delays

Abstract: In this paper, we present the discrete-time unbiased extremum seeking (ES) algorithm for n-dimensional (nD) static quadratic maps in the presence of unknown time-varying measurement delays bounded by known constants which can be large. The existing ES results in the presence of large delays are usually confined to known constant or slowly-varying delays, which is restrictive. We provide the first ES algorithm, which is robust with respect to unknown large time-varying delays. Moreover, we achieve the unbiased exponential convergence. We manage with such delays by choosing dithers with frequencies of the order of \sqrtε, where the small parameter ε > 0 appears in the dynamics of the real-time estimator. As expected, larger delays lead to a slower convergence. We provide qualitative and quantitative results based on the averaging analysis via delay-free transformation. For the quantitative bounds on the controller parameters that ensure the exponential unbiased convergence of the ES system, we assume that the Hessian of the map is uncertain and lies within a known range. Differently from its continuous-time counterpart, the small parameter in the discrete-time case defines the decay rate of the estimation error system, making a quantitative bound on this parameter particularly important. We present also constructive conditions for the practical stability of the classical ES system. Our results are semi-global for globally quadratic maps, while for locally quadratic static maps, we provide a bound on the region of convergence. Our analysis shows that appropriate ES parameters can be found for any large unknown time-varying bounded delay. A numerical example highlights the efficiency of the method.

• The paper introduces a discrete-time unbiased extremum seeking (ES) algorithm that achieves exponential convergence to the optimizer, even when perturbations have unknown, large, time-varying delays
• The main contribution of the paper is a methodology that relies on dither frequencies scaled as O(√ε) and a special delay-free transformation for robust stability under any delay size.
• The new ES algorithm significantly extends the applicability of extremum seeking to previously untreated regimes, including systems with fast-varying delays and extremely large perturbations.

Problem setting and contribution

The paper addresses extremum seeking (ES) of an unknown $n$-dimensional static quadratic map $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$ with $H > 0$, when the measurements $y(j)$ of the map are subject to a time-varying delay $D(j)$ that is unknown, bounded by a known constant $D_M$, and may be arbitrarily large. The delay is not assumed constant, slowly varying, or small: the only requirement is $0 \le D(j) \le D_M$. This is a strictly broader class than treated in prior work, where ES under large delays required known constant or slowly varying delays ($\dot D < 1$), and time-varying delay uncertainties were restricted to be small. The authors state that this is the first ES algorithm robust to unknown large time-varying delays.

Two algorithms are analyzed:

• Unbiased ES: a discrete-time version of the unbiased scheme with a high-pass filter state $\eta(j)$, exponentially decaying perturbation $\alpha(j)S(j)$ and growing demodulation signal $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$0, achieving unbiased exponential convergence to $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$1.
• Classical ES ($$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$2, no filter): only practical stability to an $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$3 neighborhood is obtained.

Key mechanism: slow dithers at frequency $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$4

Robustness to large unknown time-varying delays is achieved through the choice of dither frequencies

$$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$5

where $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$6 is the small parameter appearing in the estimator dynamics. With this scaling, $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$7 and $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$8, so the discrepancy between delayed and current values of the dithered input over the delay window is $$Q(\theta) = Q^* + \frac{1}{2}|\theta - \theta^*|_H^2$$9. This makes the delay-induced error term enter the closed loop as a perturbation of order $H > 0$0 relative to the nominal averaged dynamics, regardless of how large $H > 0$1 is — provided $H > 0$2 is chosen small enough.

The averaging analysis proceeds via the delay-free transformation introduced for discrete-time delayed systems: functions $H > 0$3, $H > 0$4, are constructed so that $H > 0$5, and the change of variables $H > 0$6 converts the error system into a perturbed nominal system $H > 0$7 with $H > 0$8. The averaged system $H > 0$9 is exponentially stable, and the residual terms are absorbed via explicit bounds.

A structural difference from the continuous-time case is emphasized: in discrete time, the decay rate of the averaged system is $y(j)$0, which depends on $y(j)$1. Hence quantitative bounds on $y(j)$2 are essential — they directly determine the achievable convergence rate, not merely feasibility.

Main results

For the unbiased ES, Theorem 1 establishes semi-global exponential stability: given any delay bound $y(j)$3, Hessian range $y(j)$4, initial uncertainty ball of radius $y(j)$5, and output uncertainty $y(j)$6, there exist tuning parameters such that

$y(j)$7

i.e., exponential convergence with rate $y(j)$8. The conditions require the learning rates to dominate the exploration rate: $y(j)$9 and $D(j)$0, plus the discrete-time-specific ordering $D(j)$1 and $D(j)$2, ensuring the averaged system and filter converge faster than the full closed loop. Notably, these design inequalities do not depend on $D(j)$3; the delay enters only through the admissible $D(j)$4, so larger $D(j)$5 forces smaller $D(j)$6 and hence slower convergence — but feasibility is preserved for any $D(j)$7. A corollary gives the qualitative statement that for completely unknown maps, sufficiently small $D(j)$8 and frequencies $D(j)$9 always yield exponential stability.

For classical ES, Theorem 2 provides constructive conditions for practical stability with ultimate bound $D_M$0 and decay rate $D_M$1. For locally quadratic $D_M$2 maps, both results extend regionally, with the convergence radius reduced by the dither amplitude.

Numerical illustration

A 3D example with eigenvalues of $D_M$3 spanning roughly $D_M$4 to $D_M$5 and delay bound up to $D_M$6 shows the expected trade-off: increasing $D_M$7 from 0 to 50 shrinks the certified $D_M$8 from $D_M$9 to $0 \le D(j) \le D_M$0 (unbiased ES), with correspondingly slower convergence. Simulations tolerate substantially larger $0 \le D(j) \le D_M$1 than the theory certifies (e.g., $0 \le D(j) \le D_M$2 vs. $0 \le D(j) \le D_M$3 for $0 \le D(j) \le D_M$4), indicating considerable conservatism in the analytical bounds. The authors propose a practical two-step tuning procedure: use the theorems to initialize parameters safely, then enlarge $0 \le D(j) \le D_M$5 via simulation before deployment.

Limitations and open questions

Several restrictions are acknowledged or evident. Quantitative results require the Hessian range $0 \le D(j) \le D_M$6 and bounds on $0 \le D(j) \le D_M$7 and $0 \le D(j) \le D_M$8 to be known; unlike the constant-delay case, reliable online Hessian estimation is "mathematically intractable" under unknown time-varying delays. The theoretical $0 \le D(j) \le D_M$9 bounds are highly conservative relative to simulations, and convergence can require on the order of $\dot D < 1$0 iterations even without delay. The results are semi-global for globally quadratic maps and regional otherwise; extension to dynamic systems with time-varying delays remains open, as does tightening the conservatism of the bounds.

Conclusion

The paper extends constructive averaging-based ES analysis to the previously untreated regime of unknown, arbitrarily large, fast-varying measurement delays, for both unbiased (exponentially convergent) and classical (practically stable) discrete-time algorithms. The central device — dither frequencies scaled as $\dot D < 1$1 combined with a delay-free transformation — yields explicit, always-feasible parameter conditions for any delay bound, at the cost of convergence rates that degrade with $\dot D < 1$2 and bounds that are conservative in practice.