Vector-Field-Driven Mirror Update

• The paper introduces a vector-field-driven generalization of mirror descent that replaces the gradient with an arbitrary vector field while preserving Bregman geometry.
• It details both data-driven learned mirror maps via input-convex neural networks and derivative-free finite-difference methods, emphasizing robustness and improved convergence.
• The study provides theoretical guarantees using relative smoothness and certificate-driven backtracking, offering practical insights into certified optimization algorithms.

A vector-field-driven mirror update is a generalization of the classical mirror descent algorithm in convex optimization, in which the usual gradient term is replaced by a potentially arbitrary vector field. This approach accommodates derivative-free optimization, learned vector fields, and domain-informed updates, while preserving the geometric structure induced by a mirror map and its associated Bregman divergence. The vector-field-driven mirror update framework subsumes both traditional mirror descent with explicit gradients and modern data-driven or zeroth-order variants, and provides interfaces for rigorous analysis, algorithmic certification, and practical implementation (Tan et al., 2022, Hayashi, 31 Jan 2026).

1. Mirror Descent and Its Vector-Field Generalization

In classical mirror descent, the iteration is defined for a convex set $\mathcal{X}\subseteq\mathbb{R}^n$ and a strictly convex, differentiable distance-generating function (mirror potential) $\psi:\mathcal{X}\to\mathbb{R}$, with the associated mirror map $\nabla\psi:\mathcal{X}\to(\mathbb{R}^n)^*$. The update for minimizing a convex differentiable objective $f:\mathcal{X}\to\mathbb{R}$ is

• (Dual update) $y_k = \nabla\psi(x_k) - t_k\nabla f(x_k)$,
• (Primal update) $x_{k+1} = (\nabla\psi)^{-1}(y_k)$.

The framework replaces the gradient $\nabla f(x)$ by a general vector field $\Omega(x)$, yielding the generalized update: $$\theta_k = \nabla\psi(x_k),\quad \theta_{k+1} = \theta_k - t_k\Omega(x_k),\quad x_{k+1} = (\nabla\psi)^{-1}(\theta_{k+1})$$ or equivalently,

$$x_{k+1} = \nabla\psi^*\big( \nabla\psi(x_k) - t_k\,\Omega(x_k) \big)$$

where $\psi^*$ denotes the Fenchel-conjugate of $\psi$. The Bregman divergence induced by $\psi$ is

$$B_\psi(x, y) = \psi(x) - \psi(y) - \langle \nabla\psi(y), x-y\rangle.$$

This formulation allows the design of updates driven by functional data, finite-difference oracles, or learned vector fields (Tan et al., 2022, Hayashi, 31 Jan 2026).

2. Instantiations: Data-Driven and Derivative-Free Vector Fields

Two prominent instantiations of the vector-field-driven framework have been established.

a) Data-driven learned mirror maps: The mirror potential $\psi(x)$ is parameterized by an input-convex neural network (ICNN), providing a convex and differentiable function with learnable parameters. The vector field $\nabla\psi(x;\theta)$ replaces the Euclidean identity in gradient descent, adapting the geometry to data. Since the inverse mirror map $(\nabla\psi)^{-1}$ is tractable only for simple $\psi$, a separate neural network $M^*(\cdot; \vartheta)$ is trained to approximate this inverse (Tan et al., 2022).

b) Deterministic zeroth-order oracles: The vector field $\Omega_{FD}(x)$ is constructed using deterministic central finite differences, e.g.

$$m_i(x) = \frac{f(x+\epsilon e_i) - f(x-\epsilon e_i)}{2\epsilon}$$

for $i=1,\dots,d$, with $2d+1$ function value evaluations per iteration. Uniform dominance over cones around the direction of the true gradient is ensured via robust conic scaling, yielding a certified vector field appropriate for Bregman geometry (Hayashi, 31 Jan 2026).

3. Relative Smoothness and Trajectory-Wise Certification

Global and a posteriori guarantees for vector-field-driven mirror updates rely on relative smoothness-type inequalities with respect to the pair $(f, \Omega)$. Define the mixed Bregman-style discrepancy: $$D_{f,\Omega}(x\|y) := \langle \Omega(y), y-x\rangle - f(y) + f(x)$$ The global relative smoothness condition requires

$$t D_{f, \Omega}(x \| y) \leq B_\psi(x, y) \qquad \forall\, x, y,$$

ensuring monotonic decrease of $f(x_k)$. In practice, the weaker trajectory-wise (a posteriori) property is certified along the realized iterates: $$t_k D_{f, \Omega}(x_{k+1} \| x_k) \leq B_\psi(x_{k+1}, x_k),\quad k=0,1,\ldots$$ If additionally, the objective $f$ is star-convex outside a punctured neighborhood $U$ of a minimizer $x_*$ (i.e., $$\langle \Omega(x), x - x_* \rangle \geq f(x) - f(x_*)$$ for $x\notin U$), the following last-iterate guarantee holds: $$f(x_t) - f(x_*) \leq \max\left\{\frac{B_\psi(x_*, x_1)}{\sum_{k=1}^{t-1} t_k}, \max_{x\in U}[f(x) - f(x_*)] \right\}$$ Thus, the framework provides explicit last-iterate certificates even with nonstandard vector fields (Hayashi, 31 Jan 2026).

4. Learning Vector Fields and Mirror Potentials

Parameterizing the mirror potential via an ICNN, as in

$$z_0 = x, \quad z_{i+1} = \sigma(W_i^{(z)}z_i + W_i^{(x)}x + b_i),\quad i=0...L-1;\quad \psi(x) = h^T z_L + c,$$

with convex non-decreasing activations and elementwise nonnegativity, results in convex and $\sigma$-strongly convex (with added $\mu\|x\|^2$) $\psi$. Learning proceeds by minimizing a finite-horizon objective aggregating the empirical task loss and a forward-backward consistency penalty for the approximate inverse map $M^*$. The effective update is: $$\tilde{x}_{k+1} = M^*\left(\nabla\psi(\tilde{x}_k;\theta) - t_k \nabla f(\tilde{x}_k);\,\vartheta\right)$$ Empirical performance matches or exceeds classical gradient-based algorithms across SVMs, multi-class classifiers, and image inverse problems, with accelerated convergence when consistency is enforced and step-sizes are learned (Tan et al., 2022).

5. Robust Conic Dominance and Derivative-Free Mirror Descent

The finite-difference instantiation constructs a vector field $\Omega_{FD}(x) = \alpha m(x)$, with $m(x)$ from central differences and explicit scaling $\alpha$ to achieve "robust conic dominance." For any cone $C(x, c)$ about the gradient direction,

$$\langle \alpha m(x), y\rangle \geq \langle x, y\rangle\quad\forall\, y\in C(x, c)$$

is enforced by solving a dominance problem over bounded-uncertainty sets $m \pm r$ (with $r$ from second differences). The α-scaling formula is provided in closed form. The resulting method ensures a generalized star-convexity interface except for a resolution-dependent exceptional set, outside which full certification is possible. Within a small neighborhood of the optimum, an $O(\epsilon\sqrt{d})$ error floor is explicitly characterized (Hayashi, 31 Jan 2026).

6. Theoretical Guarantees and Algorithmic Template

For the learned-mirror and finite-difference variants, the regret and last-iterate bounds are explicitly controlled by the geometry and the quality of vector field approximation:

• In the learned-mirror case, with backward map error $$||\nabla\psi(\tilde{x}_{k+1}) - \nabla\psi(x_{k+1})||_*$$ bounded, the regret is

$$\sum_{k=1}^K t_k [f(\tilde{x}_k) - f(x_*)] \leq B_\psi(x_*, x_1) + O\left(\sum_{k=1}^K \frac{\varepsilon_k^2}{t_k}\right)$$

with sublinear or linear convergence up to an additive error (Tan et al., 2022).

• For certified finite-difference mirror descent, if star-convexity and trajectory certificates hold,

$$f(x_t) - f(x_*) \leq \max\left\{ \frac{B_\psi(x_*, x_1)}{\sum t_k},\,\max_{x\in U}[f(x)-f(x_*)] \right\}.$$

Practical implementation employs certificate-driven backtracking for step sizes and explicit construction of the scaled vector field according to the dominance criterion (Hayashi, 31 Jan 2026).

7. Connections and Empirical Evaluations

Vector-field-driven mirror descent includes as special cases: classical mirror descent, Blahut-Arimoto-type information-geometric algorithms, data-driven learned optimizers, and deterministic zeroth-order (finite-difference) methods. In extensive empirical evaluation, the learned mirror descent framework exhibits superior convergence compared to gradient descent and Adam across support vector machine classification, multi-class linear classification, and total variation-based image restoration tasks, often requiring significantly fewer iterations to reach comparable accuracy and loss metrics (Tan et al., 2022). The finite-difference variant exposes a hidden geometric structure linking Bregman telescoping identities, a posteriori certificate-driven analysis, and robust conic geometry (Hayashi, 31 Jan 2026).

---

References:

• Data-Driven Mirror Descent with Input-Convex Neural Networks (Tan et al., 2022)
• Deterministic Zeroth-Order Mirror Descent via Vector Fields with A Posteriori Certification (Hayashi, 31 Jan 2026)