Online Convex-Concave Optimization

• Online convex-concave optimization is a framework for sequential two-player saddle-point games where the payoff is convex in the primal variable and concave in the dual variable.
• It harnesses no-regret techniques such as mirror descent, FTRL, and multiplicative weights to achieve provable convergence rates and duality gap reductions in dynamic settings.
• The framework unifies online convex optimization with constrained and Lagrangian methods, enabling efficient solutions in applications like portfolio optimization, signal processing, and maximum-entropy estimation.

Searching arXiv for recent and foundational papers on online convex-concave optimization to ground the article. Online convex-concave optimization (OCCO) studies sequential decision making in time-varying two-player zero-sum games whose payoff at round $t$ is convex in a primal variable and concave in a dual variable. In the formulations considered in the literature, decision makers choose $(x_t,y_t)\in X\times Y$ before observing the current payoff function $f_t$, after which the payoff $f_t(x_t,y_t)$ is incurred (Meng et al., 2023, Rivera et al., 2018). The framework extends online convex optimization from a single-player regret setting to a saddle-point setting in which both sides must be controlled simultaneously. It also connects to approximate constrained optimization through Lagrangian saddle-point formulations, where repeated online updates yield $\epsilon$-approximate feasible and near-optimal solutions without inner quadratic-programming subroutines [0610119].

1. Formal setting and saddle-point viewpoint

A standard OCCO model specifies convex, compact decision sets $X\subset\mathbb R^n$ and $Y\subset\mathbb R^m$, and a sequence of payoff functions $f_t:X\times Y\to\mathbb R$ such that $f_t(\cdot,y)$ is convex for each $y$ and $(x_t,y_t)\in X\times Y$0 is concave for each $(x_t,y_t)\in X\times Y$1 (Meng et al., 2023, Meng et al., 9 Sep 2025). At each round, the players choose $(x_t,y_t)\in X\times Y$2 and $(x_t,y_t)\in X\times Y$3 without knowing the current or future payoffs, and only then observe $(x_t,y_t)\in X\times Y$4 (Rivera et al., 2018).

A foundational antecedent is the Lagrangian reformulation of constrained convex optimization. For a problem

$(x_t,y_t)\in X\times Y$5

one defines

$(x_t,y_t)\in X\times Y$6

with $(x_t,y_t)\in X\times Y$7 and $(x_t,y_t)\in X\times Y$8, where $(x_t,y_t)\in X\times Y$9 may be $f_t$0 or the simplex after suitable restriction or smoothing. The constrained problem is then equivalent, by Lagrange duality, to the saddle-point problem

$f_t$1

and can be viewed as a repeated zero-sum game over $f_t$2 rounds [0610119]. In that repeated game interpretation, the primal player chooses $f_t$3, the dual player chooses $f_t$4, and the stage loss is $f_t$5 [0610119].

This saddle-point perspective is structurally important because it places approximation algorithms, no-regret learning, mirror descent, multiplicative weights, and primal-dual methods within a common template. A plausible implication is that OCCO is best understood not as a minor extension of online convex optimization, but as a unifying framework for sequential equilibrium computation and online constrained optimization.

2. Performance criteria: from regret to duality gaps

The central conceptual issue in OCCO is the choice of benchmark. In ordinary online convex optimization, performance is measured by regret against the best fixed decision in hindsight. In OCCO, the analogous object must compare a sequence of online plays to an offline saddle-point or to a comparator sequence.

One line of work measures performance through the generalized duality gap

$f_t$6

which reduces to standard regret when $f_t$7 is a singleton (Meng et al., 2023). This formulation makes explicit the parallel between OCCO with Dual-Gap and online convex optimization with regret (Meng et al., 2023).

A closely related formulation is the online saddle point problem, where the aggregate payoff is

$f_t$8

and the benchmark is the offline saddle-point $f_t$9 of $f_t(x_t,y_t)$0. The associated performance measure is

$f_t(x_t,y_t)$1

together with equivalent one-sided expressions involving the primal and dual deviations from equilibrium play (Rivera et al., 2018).

For non-stationary environments, recent work defines the dynamic duality gap

$f_t(x_t,y_t)$2

which benchmarks the online trajectory against arbitrary comparator sequences $f_t(x_t,y_t)$3 (Meng et al., 9 Sep 2025). This quantity generalizes both dynamic regret in online convex optimization and the static duality gap in zero-sum games (Meng et al., 9 Sep 2025).

The literature also records criticism of alternative metrics. In particular, dynamic Nash-equilibrium regret, defined via stage-wise Nash equilibria, is reported to have inherent defects: it can be negative even when performance is poor, it fails to compose over rounds, and it is not robust to small perturbations in $f_t(x_t,y_t)$4 (Meng et al., 2023). By contrast, Dual-Gap is always nonnegative and cumulative (Meng et al., 2023). This has made duality-gap-style metrics the dominant formalization of online convex-concave performance.

3. Algorithmic foundations: mirror descent, FTRL, and multiplicative weights

The basic OCCO algorithmic toolkit is inherited from online learning. In the repeated Lagrangian game formulation, the primal player may run Online Gradient Descent,

$f_t(x_t,y_t)$5

while the dual player runs Follow-The-Regularized-Leader (FTRL) with a strongly convex regularizer, including entropic regularization that yields multiplicative-weights updates on the simplex [0610119]. With entropy regularization $f_t(x_t,y_t)$6, the update takes the closed form

$f_t(x_t,y_t)$7

This is one of the earliest explicit reductions from approximate convex programming to online game playing [0610119].

In the online saddle point literature, strong convexity-concavity leads to SP-FTL, a follow-the-leader procedure that at round $f_t(x_t,y_t)$8 computes the unique saddle-point of the cumulative payoff $f_t(x_t,y_t)$9 (Rivera et al., 2018). Under the assumptions that each $\epsilon$0 is $\epsilon$1-strongly convex in $\epsilon$2, $\epsilon$3-strongly concave in $\epsilon$4, and $\epsilon$5-Lipschitz, SP-FTL achieves

$\epsilon$6

(Rivera et al., 2018). In the general convex-concave case, SP-RFTL adds regularization to induce strong convexity-concavity, yielding

$\epsilon$7

for Euclidean regularizers on domains of diameter $\epsilon$8 (Rivera et al., 2018).

Another important strand is implicit Online Mirror Descent-Ascent (OMDA), defined through the update

$\epsilon$9

where $X\subset\mathbb R^n$0 are Bregman divergences generated by strongly convex mirror maps (Meng et al., 2023). Under $X\subset\mathbb R^n$1-Lipschitz payoffs and $X\subset\mathbb R^n$2-strongly convex mirror maps, OMDA satisfies

$X\subset\mathbb R^n$3

which yields $X\subset\mathbb R^n$4 after optimizing the constant step size (Meng et al., 2023). An optimistic variant uses a one-step predictor $X\subset\mathbb R^n$5 and attains the same $X\subset\mathbb R^n$6 order with smaller leading factors on predictable sequences (Meng et al., 2023).

For simplex-constrained games, optimistic multiplicative methods provide a different behavior. Optimistic Multiplicative-Weights Update (OMWU), an entropy-regularized optimistic FTRL scheme, updates the $X\subset\mathbb R^n$7- and $X\subset\mathbb R^n$8-players by exponentiating a combination of current and previous gradients, with coefficients $X\subset\mathbb R^n$9 and $Y\subset\mathbb R^m$0 for the minimizing player and the sign-reversed version for the maximizing player (Lei et al., 2020). In the setting

$Y\subset\mathbb R^m$1

with $Y\subset\mathbb R^m$2 convex in $Y\subset\mathbb R^m$3, concave in $Y\subset\mathbb R^m$4, twice differentiable, and admitting a strictly complementary equilibrium, OMWU exhibits local last-iterate convergence to the saddle point for sufficiently small $Y\subset\mathbb R^m$5, and the convergence is locally exponential (Lei et al., 2020).

4. Rates, assumptions, and structural accelerations

The classical no-regret analysis of repeated saddle-point play produces $Y\subset\mathbb R^m$6-type guarantees. In the Lagrangian framework, if each $Y\subset\mathbb R^m$7 is $Y\subset\mathbb R^m$8-Lipschitz on $Y\subset\mathbb R^m$9 and $f_t:X\times Y\to\mathbb R$0 has diameter $f_t:X\times Y\to\mathbb R$1, then the primal and dual regrets satisfy

$f_t:X\times Y\to\mathbb R$2

with the standard choice $f_t:X\times Y\to\mathbb R$3, and the average iterates obey a saddle-point gap bounded by $f_t:X\times Y\to\mathbb R$4 [0610119]. Consequently, a gap at most $f_t:X\times Y\to\mathbb R$5 requires $f_t:X\times Y\to\mathbb R$6 in the general Lipschitz case [0610119].

A principal acceleration mechanism is smoothing or regularization that introduces strong convexity in one variable or strong concavity in the other. In the 2006 reduction from convex optimization to online game playing, adding an $f_t:X\times Y\to\mathbb R$7-strong convex smoothing term in the dual allows OGD on $f_t:X\times Y\to\mathbb R$8 and FTRL on $f_t:X\times Y\to\mathbb R$9 to produce an average point $f_t(\cdot,y)$0 with $f_t(\cdot,y)$1 for all constraints and $f_t(\cdot,y)$2 after

$f_t(\cdot,y)$3

rounds, using only gradients, projections onto $f_t(\cdot,y)$4, and simple entropic updates [0610119]. The paper emphasizes that no quadratic-programming subroutine is needed [0610119].

The same qualitative pattern appears in the online saddle point problem. Strongly convex-concave payoffs permit logarithmic SP-Regret via SP-FTL, whereas generic convex-concave payoffs require regularization to recover sublinear control (Rivera et al., 2018). This suggests that curvature is not merely a technical convenience, but a decisive property separating $f_t(\cdot,y)$5 and $f_t(\cdot,y)$6 regimes in online saddle-point learning.

More recent work on non-stationary OCCO identifies path-length adaptation as the relevant analogue of dynamic-regret adaptivity. Defining

$f_t(\cdot,y)$7

a modular algorithm can achieve a minimax optimal dynamic duality gap upper bound, up to a logarithmic factor,

$f_t(\cdot,y)$8

and more precisely

$f_t(\cdot,y)$9

where $y$0 is a data-dependent environmental variation bound (Meng et al., 9 Sep 2025). The same work also proves a lower bound

$y$1

showing that the achieved dependence on the path variation is minimax optimal up to logarithmic factors (Meng et al., 9 Sep 2025).

5. Variants: bilinear games, bandit feedback, alternating dynamics, and distributed settings

A particularly tractable subclass is the online matrix game, where $y$2, $y$3, and

$y$4

Using negative-entropy regularizers over restricted simplices, OMG-RFTL reduces the dimension dependence in SP-Regret from linear in the dimension to logarithmic: $y$5 (Rivera et al., 2018). Under bandit feedback, where each round only one entry $y$6 is observed after sampling $y$7, $y$8, an unbiased one-point estimator

$y$9

for the sampled pair leads to

$(x_t,y_t)\in X\times Y$00

against an adaptive adversary (Rivera et al., 2018).

Another direction replaces standard regret by alternating regret. For adversarial online convex optimization, Continuous Hedge achieves

$(x_t,y_t)\in X\times Y$01

alternating regret under $(x_t,y_t)\in X\times Y$02, and a 3rd-order-smooth FTRL construction yields $(x_t,y_t)\in X\times Y$03 in smooth and self-concordant settings, with $(x_t,y_t)\in X\times Y$04 when $(x_t,y_t)\in X\times Y$05, such as linear or quadratic losses (Hait et al., 18 Feb 2025). When both players in a convex-concave zero-sum game use alternating-regret algorithms, the average iterate $(x_t,y_t)\in X\times Y$06 has duality gap

$(x_t,y_t)\in X\times Y$07

so $(x_t,y_t)\in X\times Y$08 per player yields an $(x_t,y_t)\in X\times Y$09 rate for saddle-point approximation (Hait et al., 18 Feb 2025). This is distinct from the standard $(x_t,y_t)\in X\times Y$10 averaging behavior associated with ordinary no-regret bounds.

Distributed and stochastic versions introduce communication constraints and predictive dynamics. In a multiagent network with time-varying directed graph, the DOSMD-CCO algorithm combines stochastic mirror descent, Bregman projections, predictive mappings $(x_t,y_t)\in X\times Y$11, and consensus steps. Under assumptions including unbiased noisy gradients, strong convexity of distance-generating functions, and nonexpansive predictive mappings, the expected dynamic saddle-point regret of each agent satisfies

$(x_t,y_t)\in X\times Y$12

where $(x_t,y_t)\in X\times Y$13 is the path-variation of the time-varying saddle points (Zhang et al., 13 Aug 2025). With $(x_t,y_t)\in X\times Y$14, this becomes $(x_t,y_t)\in X\times Y$15 (Zhang et al., 13 Aug 2025). A multiple-consensus variant tightens the constant in the consensus term (Zhang et al., 13 Aug 2025).

6. Applications and adjacent problem classes

The Lagrangian game perspective immediately yields applications to approximate constrained optimization. The 2006 framework explicitly lists maximum-entropy estimation, portfolio optimization with convex risk constraints, and computational problems in signal processing as examples with strictly convex constraints [0610119]. In maximum-entropy estimation with moment matching,

$(x_t,y_t)\in X\times Y$16

the dual variables correspond to moment constraints and the same online convex optimization machinery recovers the Gibbs distribution in $(x_t,y_t)\in X\times Y$17 rounds [0610119].

Portfolio optimization with convex risk constraints is also cast in this form: choose $(x_t,y_t)\in X\times Y$18 to minimize $(x_t,y_t)\in X\times Y$19 subject to VaR or CVaR constraints $(x_t,y_t)\in X\times Y$20, update dual weights multiplicatively over scenarios, and update the primal via OGD [0610119]. Robust linear and semidefinite programs fit the same template when adversarial constraints are embedded in the dual strategy set [0610119].

The online saddle point formalism further connects to online convex optimization with knapsacks. With budget $(x_t,y_t)\in X\times Y$21, concave rewards $(x_t,y_t)\in X\times Y$22, and convex consumptions $(x_t,y_t)\in X\times Y$23, one defines

$(x_t,y_t)\in X\times Y$24

and obtains a regret decomposition in which one term is the dual player’s individual-regret and the other is SP-Regret (Rivera et al., 2018). A primal-dual RFTL method then achieves

$(x_t,y_t)\in X\times Y$25

regret in the stochastic i.i.d. setting (Rivera et al., 2018).

Learning-augmented online packing and covering provide a related but not identical adjacent class. For online concave packing, a switching algorithm mixes a black-box online packing method with an advice vector $(x_t,y_t)\in X\times Y$26, obtaining

$(x_t,y_t)\in X\times Y$27

with approximate feasibility $(x_t,y_t)\in X\times Y$28 (Grigorescu et al., 2024). For online convex covering, a primal-dual learning-augmented algorithm yields

$(x_t,y_t)\in X\times Y$29

(Grigorescu et al., 2024). The source describes these results as part of a unified theory of online convex-concave optimization with predictions (Grigorescu et al., 2024). This suggests a broader view in which advice, predictors, and side information act as regularizing signals for sequential saddle-point methods.

7. Current themes, misconceptions, and open directions

A recurrent misconception is that separate no-regret guarantees for the two players automatically solve the online saddle-point problem. The literature shows that the correct benchmark is not merely the sum of two independent regrets, but a saddle-point-compatible metric such as Dual-Gap, SP-Regret, or D-DGap (Rivera et al., 2018, Meng et al., 2023). This distinction is central because equilibrium quality involves both players jointly.

A second misconception is that averaging is the only meaningful convergence notion. Average-iterate guarantees are indeed standard: the no-regret-to-equilibrium lemma in the repeated Lagrangian framework bounds the saddle-point gap of $(x_t,y_t)\in X\times Y$30 by $(x_t,y_t)\in X\times Y$31 [0610119], and alternating-regret analyses also convert into average-iterate Nash guarantees (Hait et al., 18 Feb 2025). However, last-iterate convergence has become an important separate objective, particularly in constrained simplex domains. OMWU establishes local pointwise convergence of the actual iterates, not only their averages, under convex-concave smoothness and strict complementarity assumptions (Lei et al., 2020).

A third issue concerns predictability and non-stationarity. Existing algorithms had been reported to fail to deliver optimal performance in stationary or predictable environments, motivating a modular OCCO method with three components: an Adaptive Module, a Multi-Predictor Aggregator, and an Integration Module (Meng et al., 9 Sep 2025). The resulting procedure attains both a path-length-based minimax rate and a prediction-error-driven rate,

$(x_t,y_t)\in X\times Y$32

and with multiple predictors,

$(x_t,y_t)\in X\times Y$33

(Meng et al., 9 Sep 2025). In the perfectly predictable case, one predictor has zero error and the dynamic duality gap becomes $(x_t,y_t)\in X\times Y$34 (Meng et al., 9 Sep 2025).

Open directions are explicitly identified in the non-stationary OCCO literature: one-sided feedback, two-player partial observation or bandit saddle-point problems, and extensions to multi-player non-zero-sum games with dynamic equilibria (Meng et al., 9 Sep 2025). Distributed stochastic OCCO further points toward predictive multiagent saddle-point learning under communication constraints (Zhang et al., 13 Aug 2025). Taken together, these directions indicate that the modern field has expanded beyond static adversarial convex-concave games into a broader study of dynamic, predictive, partially observed, and networked equilibrium computation.