Last-Iterate Convergence of Optimistic Multiplicative Weight Update

Abstract: Optimistic Gradient Descent Ascent (OGDA) and Optimistic Multiplicative-Weights Update (OMWU) are two very popular algorithms to solve convex/concave saddle-point problems, where OMWU is the non-Euclidean, entropic version of OGDA. It is known since the '80s that the last iterate of OGDA asymptotically converges to a saddle point in smooth problems. On the other hand, it is unknown if OMWU has the same property. In this paper, I show that OMWU converges asymptotically for smooth convex-concave saddle-point problems, with a small enough constant learning rate. The result does not require uniqueness, strict complementarity, an error bound, or initialization near a solution. The main new ingredient is a boundary argument showing that every cluster point satisfies the inactive-coordinate KKT inequalities. The boundary argument was discovered with assistance from ChatGPT and is documented in the appendix.

• The paper proves that OMWU achieves last-iterate convergence for smooth convex-concave minimax problems, independent of uniqueness and proximity of initialization.
• It employs a novel boundary argument with KL divergence and mirror descent to manage non-differentiability on simplex boundaries.
• The study establishes OMWU's parity with OGDA and highlights LLM assistance in generating critical proof components.

Last-Iterate Convergence of Optimistic Multiplicative Weight Update

Introduction and Motivation

The Optimistic Multiplicative-Weights Update (OMWU) algorithm occupies a central role in solving convex-concave saddle-point problems with non-Euclidean geometry, particularly where the feasible regions are probability simplexes and the regularization is entropic. OMWU is viewed as the entropic or "mirror" analogue of the well-studied Optimistic Gradient Descent Ascent (OGDA) method. Classic results ensure last-iterate convergence for OGDA in smooth saddle-point problems, notably via Popov’s analysis, but the global, non-asymptotic last-iterate convergence for OMWU, particularly without restrictive assumptions on uniqueness, error bounds, or strict complementarity, has remained unresolved.

The work systematically demonstrates that OMWU with a sufficiently small constant learning rate $\eta < 1/(3L)$ achieves global, asymptotic last-iterate convergence to an equilibrium for any smooth convex-concave minimax objective over a product of simplexes, without requirements for initialization near a solution, strict complementarity, uniqueness, or problem bilinearity. This closes a longstanding gap in the theory and establishes OMWU as theoretically on par with OGDA in terms of asymptotic trajectory convergence for a broad class of monotone variational inequalities with entropic regularization.

Preliminaries and Setting

Consider the saddle-point problem: $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$ where $f$ is convex in $x$, concave in $y$, and $L$-smooth ($G(z)$ is $L$-Lipschitz, with $z = (x, y)$). The algorithmic family is general mirror descent; for OMWU, the distance generating function $h(z)$ is the sum of the negative entropies on $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$0 and $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$1.

The mirror update sequence for OMDA (entropic version being OMWU) evolves as: $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$2 with $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$3 being the Kullback-Leibler divergence induced by the negative entropy.

Technical Contributions

Last-Iterate Convergence Without Strong Assumptions

The primary technical contribution is removing all prior limitations—uniqueness of equilibrium, error bounds, initialization conditions, or strict complementary slackness—that were previously necessary for OMWU asymptotic analysis. The main theorem asserts:

If OMWU is run over $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$4 for smooth convex-concave $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$5 with $$\min_{x \in \Delta_n} \max_{y \in \Delta_m} f(x, y),$$6 and initialization in the relative interior, then the generated sequence converges to a single saddle point.

The proof is constructed via an intricate argument that avoids reliance on the strong differentiability or interiority required by mirror descent analyses.

Structural Argument: Boundary Behaviors

The analysis critically exploits the geometry of the entropy regularizer. Due to the non-differentiability of negative entropy on the simplex boundary, classical arguments to certify optimality based on KKT conditions in the interior generally fail. The core lemma establishes necessary boundary (inactive coordinate) KKT-like inequalities by tracking the “bad mass” associated with support shrinkage in the limit and tying this behavior to mirror descent's three-point inequality.

A key innovation is showing that either inactive coordinates remain empty, or their limit behavior contradicts the summability results of KL-divergence between iterates, which is controlled via a boundary expansion lemma (proven with LLM assistance). This leads to the conclusion that every cluster point obeys all first-order saddle-point conditions, despite the regularizer's lack of smoothness at the boundary.

Robustness to Lack of Uniqueness

No strict complementarity or uniqueness is required. The construction proceeds by showing any cluster point satisfies both active and inactive KKT inequalities, and Pinsker's inequality (quantifying the contraction of the KL) enforces that iterates converge to a single equilibrium.

Equivalence with One-Sequence Formulations

The analysis includes a formal equivalence between one-sequence and two-sequence OMWU, reconciling notational and initialization conventions seen across the literature and ensuring robustness of the convergence argument.

Numerical and Theoretical Implications

• Optimality Scope: The result applies globally to all smooth convex-concave minimax problems over product simplexes with entropic regularization, subsuming zero-sum matrix games as well as much more general potential functions.
• No Error Bound Requirement: The result holds even in the absence of error bounds or strict convexity.
• No Local Convergence Restriction: Prior works (e.g., [LeiNPW21]) only gave local convergence under strong optimality conditions; this work establishes global results.
• Arbitrarily Slow Rates: The convergence is asymptotic, and recent work ([CaiFGCKLLZ24]) has demonstrated that OMWU may converge arbitrarily slowly in finite time, which does not contradict the present result but pins down the qualitative endpoint.

Methodological Significance

An interesting aspect of the work is that the pivotal boundary argument completing the proof was developed with the iterative assistance of a LLM (ChatGPT). This demonstrates the practical relevance of LLMs as mathematical research assistants, not only for heuristic exploration but in formal proof generation. The author details the prompt engineering process leading to the final proof as an appendix.

Future Directions

• Finite-Time Rates: The asymptotic nature of the result leaves the (possibly slow) finite-time convergence rates as an open area, particularly for generic (non-bilinear, non-strongly-convex) settings.
• Extension Beyond Simplexes: Extending last-iterate analysis for OMWU/mirror descent with non-differentiable regularizers to polytopes or general convex sets remains of high theoretical importance.
• General Variational Inequalities: The methods here suggest avenues for OMD/OCO tools applied to monotone variational inequalities with boundary-intricate regularizers.
• LLM Proof Assistance: The meta-methodological lesson signals that coupling even strong optimization theoretic intuition with “co-pilot” LLMs may be optimal for future proof discovery in high-dimensional non-Euclidean optimization.

Conclusion

The author establishes the definitive global last-iterate convergence of OMWU for smooth convex-concave saddle-point problems over product simplexes, removing all prior structural and initialization assumptions. The boundary argument filling the gap for mirror methods with non-differentiable regularizers stands as a significant technical advance. This result places OMWU on firm footing as a method not only for online learning and game theory but also as a first-class solution algorithm for general entropic minimax problems. The proof techniques and LLM-augmented development process indicate both new frontiers for algorithmic analysis and mathematical proof search with machine assistance.

---

References:

• "Last-Iterate Convergence of Optimistic Multiplicative Weight Update" (2606.11773)
• [LeiNPW21], [DaskalakisP18], [WeiLZL21], CaiFGCKLLZ24