--- title: Randomized Two-Point Gradient-Free Optimization url: https://www.emergentmind.com/topics/randomized-two-point-gradient-free-feedback-optimization type: topic --- # Randomized Two-Point Gradient-Free Optimization Randomized two-point gradient-free feedback optimization is a class of zeroth-order methods in which an algorithm perturbs the current decision variable in a random direction, evaluates the objective at two points, and uses the resulting finite difference as a gradient proxy. In convex optimization, the framework appears as bandit convex optimization with two-point feedback and zero-order stochastic convex optimization with two function evaluations per round; in broader settings it includes online distributed optimization, stochastic optimization with Markovian noise, and model-free feedback optimization for dynamical systems. Across these formulations, the main technical pattern is randomized smoothing plus a two-point estimator whose expectation matches the gradient of a smoothed objective, followed by mirror descent, dual averaging, projected gradient descent, or accelerated first-order updates [1507.08752], [2211.13566], [2509.11666]. ## 1. Problem formulations and feedback models In the bandit convex optimization formulation, at each round \(t=1,\dots,T\), an adversary chooses a convex loss \(f_t:\mathbb{R}^d\to\mathbb{R}\), the learner chooses \(w_t\in W\), suffers loss \(f_t(w_t)\), and may query two additional points \(w_t',w_t''\in W\) to observe \(f_t(w_t')\) and \(f_t(w_t'')\). The objective is to minimize expected average regret, \[ \frac{1}{T}\sum_{t=1}^T f_t(w_t)-\min_{w\in W}\frac{1}{T}\sum_{t=1}^T f_t(w). \] The stochastic analogue is zero-order stochastic convex optimization for \[ F(w)=\mathbb{E}_\xi[f(w;\xi)], \] where only sample functions \(f(\cdot;\xi_t)\) are accessible and each sample may be queried at two points. In that setting, the averaged iterate \(\bar w_T=\frac1T\sum_{t=1}^T w_t\) inherits the online regret guarantee as an optimization error bound [1507.08752]. The same two-point logic appears in a more general convex black-box setting, \[ \min_{x\in Q\subseteq \mathbb{R}^d} f(x), \] with access restricted to zeroth-order oracles returning noisy function values. The modern survey literature organizes the analysis around three criteria: oracle complexity, iteration complexity, and the maximum admissible noise level. It treats smooth convex objectives with Lipschitz gradients, nonsmooth convex objectives handled via randomized smoothing, and noisy or stochastic evaluations of the form \(f_\delta(x)=f(x)+\delta(x)\), \(|\delta(x)|\le \Delta\), or \(f(x,\xi)\) [2211.13566]. Online zero-order optimization with two-point feedback uses the same oracle model under adversarially chosen convex losses, but emphasizes regret and feasible-set geometry. In distributed online optimization, each agent observes only local function values after action execution and exchanges information with neighbors; gradient-free feedback is then local, decentralized, and time-varying. In model-free feedback optimization for dynamical systems, the two evaluations are taken through plant measurements rather than a static oracle: the controller perturbs the input, observes the resulting outputs, and estimates the gradient of a steady-state cost without knowing the plant model [2205.13910], [1903.07106], [2509.11666]. This range of formulations shows that “feedback” is not tied to a single oracle semantics. In some papers it means two evaluations of the same sample function, in others two noisy values under canceling or adversarial noise, and in control-oriented work it means two successive closed-loop measurements. A plausible implication is that the unifying object is not the oracle interface itself but the paired perturbation structure used to recover directional information. ## 2. Gradient estimators, symmetry, and randomized smoothing The central estimator in the convex two-point literature is the symmetric Euclidean finite difference \[ \tilde g_t=\frac{d}{2\delta_t}\bigl(f_t(w_t+\delta_t u_t)-f_t(w_t-\delta_t u_t)\bigr)u_t, \] where \(u_t\) is uniform on the Euclidean unit sphere. Its importance is explicit: the paper introducing the estimator describes it as a “small but surprisingly powerful modification” of earlier forward-difference constructions. In one dimension it is exactly the central difference \[ \frac{f(w+\delta)-f(w-\delta)}{2\delta}, \] and in higher dimensions its second moment is only linear in \(d\), whereas the forward-difference-style estimator can have second moment scaling like \(d^2\) for nonsmooth functions [1507.08752]. The associated smoothed function is defined by spherical averaging, \[ \hat f_t(w)=\mathbb{E}_{u_t}[f_t(w+\delta_t u_t)], \] and the estimator is unbiased for the gradient of that smoothed objective: \[ \mathbb{E}[\tilde g_t\mid w_t]=\nabla \hat f_t(w_t). \] When \(f_t\) is \(G_2\)-Lipschitz in \(\|\cdot\|_2\), the smoothing bias satisfies \[ \sup_{w\in W}|\hat f_t(w)-f_t(w)|\le \delta_t G_2. \] This establishes the standard zeroth-order mechanism: two-point feedback does not usually estimate \(\nabla f_t\) directly, but rather \(\nabla \hat f_t\) for a nearby smoothed objective [1507.08752]. The survey treatment makes this pattern explicit across smooth and nonsmooth settings. For smooth problems, it uses the randomized two-point symmetric estimator \[ \tilde{\nabla} f(x,e)=d\,\frac{f_\delta(x+he)-f_\delta(x-he)}{2h}\,e, \qquad e\sim S_2^d(1). \] For nonsmooth convex optimization, it replaces direct finite-difference analysis by randomized smoothing \[ f_\gamma(x)=\mathbb{E}_u[f(x+\gamma u)], \qquad u\sim RB_2^d(1), \] and the corresponding two-point stochastic gradient \[ \nabla f_\gamma(x,e)=d\,\frac{f(x+\gamma e)-f(x-\gamma e)}{2\gamma}\,e, \qquad e\sim RS_2^d(1), \] which satisfies \(\mathbb{E}_e[\nabla f_\gamma(x,e)]=\nabla f_\gamma(x)\) [2211.13566]. Other geometries yield different but structurally analogous estimators. For \(\ell_1\)-structured problems, the estimator \[ g_t=\frac{d}{2h_t}(y_t'-y_t'')\,\operatorname{sign}(\zeta_t), \qquad \zeta_t\sim \mathrm{Unif}(\mathbb{S}_1^d), \] uses perturbations on the \(\ell_1\)-sphere, and its expectation equals the gradient of a smoothing over the \(\ell_1\)-ball: \[ \mathbb{E}\left[\frac{d}{2h}\big(f(x+h\zeta)-f(x-h\zeta)\big)\operatorname{sign}(\zeta)\right]=\nabla \mathsf{f}_h(x). \] For nonsmooth stochastic optimization with additional small noise, a different two-point construction uses “double smoothing”: \[ g_k=\big(f(x_k+\tau e_1+\mu e_2,\xi_k)-f(x_k+\tau e_1,\xi_k)\big)e_2, \] with \(e_1\) uniform on the unit ball and \(e_2\) uniform on the unit sphere [2205.13910], [1701.03821]. This family of formulas suggests a robust structural principle: randomized two-point methods are best understood as estimators of gradients of smoothed surrogates, where the choice of perturbation law encodes the geometry of the problem. ## 3. Convex guarantees and optimality results For convex Lipschitz objectives, the 2015 two-point mirror-descent method gives expected average regret bounded by \[ c\, p_* G_2 R \sqrt{\frac{d}{T}}, \] equivalently cumulative regret \[ O\!\left(p_* G_2 R \sqrt{dT}\right). \] In Euclidean geometry, where \(p_*=1\), this becomes \[ O\!\left(G_2 R\sqrt{\frac{d}{T}}\right), \] and the paper states that this matches the known lower bound from \(\texttt{dujww13}\), so it is optimal up to constants. The explicit significance assigned there is that the gap between upper and lower bounds for general convex Lipschitz functions is closed, not only for smooth objectives but also for nonsmooth ones [1507.08752]. The broader survey perspective states that, for nonsmooth convex optimization under the \(l_2\)-smoothing scheme, a batched accelerated first-order method applied to the smoothed problem achieves iteration complexity \[ O\!\left(\frac{d^{1/4}\sqrt{M M_2}R}{\varepsilon}\right) \] and zeroth-order oracle complexity \[ O\!\left(\frac{\min\{q,\ln d\}\,d^{2/q} M_2^2 R^2}{\varepsilon^2}\right). \] In the Euclidean case \(p=q=2\), the oracle complexity becomes \[ O\!\left(\frac{d M_2^2 R^2}{\varepsilon^2}\right), \] which the survey describes as optimal in both iteration complexity and oracle complexity for the two-point smoothing approach [2211.13566]. For smooth stochastic convex optimization with noisy function values, the derivative-free literature analyzes two-point random-direction methods that nearly recover first-order performance. The accelerated method ARDFDS combines random finite-difference directional gradients, linear coupling, and mirror descent; its complexity bound is stated to be only by a factor of \(\sqrt{n}\) larger than the bound for accelerated gradient-based algorithms. The same work gives a non-accelerated derivative-free method with a complexity bound similar to the stochastic-gradient-based algorithm, without any dimension-dependent factor except logarithmic, and identifies advantages of an \(1\)-norm proximal setup when the difference between the starting point and the solution is sparse [1802.09022]. For nonsmooth stochastic convex programming with two noisy function values per iteration, the double-smoothing mirror-descent construction is described as optimal up to a multiplicative factor. After choosing \[ \tau=\frac{\varepsilon}{4M},\qquad \mu=\frac{\varepsilon}{4Mn}, \] and imposing a small-noise condition \(\delta\le \delta_0\), the paper concludes \[ N = \frac{384\,c_q\,M R^2}{\varepsilon^2} \quad\Longrightarrow\quad \mathrm{Regret}_f(\{x_k\})\le \varepsilon. \] In the strongly convex case it reports \[ \mathrm{Regret}_f(\{x_k\}) = O\!\left(\frac{nM\ln N}{\gamma N}\right), \] again described as optimal up to a logarithmic factor [1701.03821]. Taken together, these results place two-point feedback at the center of minimax-optimal zeroth-order convex optimization. The recurring theme is that the main statistical price of gradient-free optimization is dimensional, while symmetry and smoothing determine whether that price is optimal or suboptimal. ## 4. Geometry, noise, and adaptive variants Non-Euclidean geometry enters the theory through mirror descent and norm-dependent moment bounds. In the general framework of the 2015 optimal algorithm, the regularizer \(r\) must be \(1\)-strongly convex with respect to a norm \(\|\cdot\|\), and the regret bound depends on \[ \sqrt[4]{\mathbb{E}\|u\|_*^4}\le p_*. \] For the simplex with \(\ell_1\)-geometry, using the entropic regularizer \[ r(w)=\sum_{i=1}^d w_i\log(dw_i) \] gives \(\sup_W r(w)\le \log d\), while the spherical moment bound \[ \sqrt[4]{\mathbb{E}\|u\|_\infty^4}\le c\sqrt{\frac{\log d}{d}} \] yields \[ \mathbb{E}[\text{regret}] \le c\, G_1 \sqrt{\frac{d\log^2 d}{T}}. \] The paper characterizes this as optimal up to a \(\sqrt{\log d}\) factor [1507.08752]. A geometry-specific refinement appears in the \(\ell_1\)-randomized online method. There the smoothing bias is controlled by \[ \mathrm{b}_q(d)\triangleq \frac{1}{d+1}\times \begin{cases} q\,d^{1/q}, & q\in[1,\log d),\\ e\log d, & q\ge \log d, \end{cases} \] so that for \(L\)-Lipschitz \(f\), \[ | \mathsf{f}_h(x)-f(x)|\le \mathrm{b}_q(d)\,Lh. \] Under canceling noise, the paper gives the special-case bounds \[ \mathbb{E}[\mathrm{Regret}_T]\le 11.9\,L\sqrt{dT} \] for the Euclidean case and \[ \mathbb{E}[\mathrm{Regret}_T]\le 11.9\,L\sqrt{dT\log d} \] for the simplex/\(\ell_1\) case, stating that the latter improves over the previous \(O(L\sqrt{dT}\log d)\) bound by a \(\sqrt{\log d}\) factor. Under adversarial noise, the guarantee contains a slower \(T^{3/4}\) term. The same paper also gives a fully data-driven, anytime, parameter-free step size, \[ \eta_t=\frac{R}{\sqrt{2.75\sum_{k=1}^{t-1}\|g_k\|_{p^*}^2}}, \] with regret bounds of the same leading order up to constants [2205.13910]. Noise tolerance is a separate axis of analysis. In the survey treatment of randomized gradient-free convex optimization, the admissible noise for the nonsmooth smoothing regime is controlled by \[ \Delta \lesssim \frac{\varepsilon^2}{M_2R\sqrt d}, \] while in the smooth higher-order finite-difference regime the maximum admissible noise is roughly \[ \Delta \lesssim \varepsilon^{\frac{n+1}{n}} d^{-\frac{n+1}{2n}}. \] The survey emphasizes that oracle complexity, iteration complexity, and admissible noise cannot all be simultaneously optimized in every regime [2211.13566]. The stochastic double-smoothing paper makes a related point for additional deterministic noise in function values. It derives a threshold \(\delta_0\) such that, if the extra nonrandom noise satisfies \(\delta\le \delta_0\), the convergence rates are preserved up to absolute multiplicative constants; qualitatively, the admissible level scales inversely with dimension and problem Lipschitzness and quadratically in the target accuracy [1701.03821]. These results show that geometry and noise are not secondary implementation details. They determine the correct perturbation distribution, the appropriate prox structure, and the sharp form of the bias-variance tradeoff. ## 5. Distributed, dynamical, and Markovian extensions In distributed online optimization, randomized two-point gradient-free feedback is combined with consensus dynamics. The distributed projected gradient descent method uses Gaussian smoothing \[ f_{i,\mu^i}^t(x)=\frac{1}{\kappa}\int_{\mathbb{R}^p} f_i^t(x+\mu_t^i\xi_t^i)e^{-\frac12\|\xi_t^i\|^2}d\xi_t^i \] and the local oracle \[ g_{\mu^i}^t(x_t)=\frac{f_i^t(x_t+\mu_t^i\xi_t^i)-f_i^t(x_t)}{\mu_t^i}\,\xi_t^i. \] Under convexity, Lipschitz continuity, compactness of \(\Omega\), and strong connectivity of the communication digraph, the main theorem bounds the expected dynamic regret of each agent by terms depending on the smoothing scales \(\hat\mu_t\), the minimizer path length \(\omega_T\), the step sizes \(\gamma_t\), and network constants. With \[ \gamma_t=\frac{\gamma_0}{(T+1)^\alpha},\qquad \mu_t^i=\frac{\mu_0}{(T+1)^\beta}, \qquad \alpha,\beta\in(0,1), \] the paper obtains \[ \mathbf{E}[\mathbf{R}_i(T)] \le \mathcal{O}\Big(\max\{\omega_T T^\alpha,\; T^{\max\{1-\beta,\;1-\alpha+\beta\}\}\Big), \] and for the special choice \(\alpha=\tfrac23\), \(\beta=\tfrac13\), \[ \mathbf{E}[\mathbf{R}_i(T)]\le \mathcal{O}\left(\omega_T^{1/3}T^{2/3}\right). \] The stated conclusion is that the dynamic regret is sublinear whenever the minimizer path length grows sublinearly [1903.07106]. In model-free feedback optimization for dynamical systems, the objective is the steady-state cost \[ \tilde\Phi(u)=\Phi(u,h(u,d)), \] where \(h(u,d)\) is the steady-state output map of an unknown plant. The proposed two-timescale controller perturbs the input with a Gaussian direction \(v_t\sim\mathcal N(0,I_p)\) and forms \[ g_t^\delta=\frac{v_t}{\delta}\Big(\Phi(u_{t+1},y_{t+2})-\Phi(u_t,y_{t+1})\Big), \qquad u_{t+2}=u_t-\eta\, g_t^\delta. \] The analysis compares this implemented estimator with the ideal smoothed zeroth-order estimator for \[ \tilde\Phi_\delta(u)=\mathbb E_{v\sim\mathcal N(0,I_p)}[\tilde\Phi(u+\delta v)], \] proves the bound \[ \mathbb E[\|e_t\|^2]\le \frac{4M_\Phi^2\mu p}{\delta^2}, \] and states that, under smoothness and plant stability assumptions, the method computes an \(\epsilon\)-stationary point in \(\mathcal O(\epsilon^{-1})\) iterations. The paper explicitly contrasts this with a one-point residual-feedback method having a worse rate, around \(\mathcal O(\epsilon^{-3/2})\), in the smooth nonconvex setting [2509.11666]. For Markovian stochasticity, the zero-order oracle samples are no longer i.i.d. but generated by a uniformly geometrically ergodic Markov chain with mixing time \(\tau\). The two-point estimator takes the form \[ \hat{g}(x, Z, e)= d\frac{F(x+te,Z)-F(x-te,Z)}{2t}e. \] The key contribution is a randomized batching and multilevel Monte Carlo construction showing that the final dependence is \(d+\tau\) rather than \(d\tau\). In particular, the paper emphasizes that when \(\tau