---
title: Online Convex-Concave Optimization
url: https://www.emergentmind.com/topics/online-convex-concave-optimization
type: topic
---

# Online Convex-Concave Optimization

Searching arXiv for recent and foundational papers on online convex-concave optimization to ground the article.
Online convex-concave optimization (OCCO) studies sequential decision making in time-varying two-player zero-sum games whose payoff at round \(t\) is convex in a primal variable and concave in a dual variable. In the formulations considered in the literature, decision makers choose \((x_t,y_t)\in X\times Y\) before observing the current payoff function \(f_t\), after which the payoff \(f_t(x_t,y_t)\) is incurred [2312.06957, 1806.08301]. The framework extends online convex optimization from a single-player regret setting to a saddle-point setting in which both sides must be controlled simultaneously. It also connects to approximate constrained optimization through Lagrangian saddle-point formulations, where repeated online updates yield \(\epsilon\)-approximate feasible and near-optimal solutions without inner quadratic-programming subroutines [0610119].

## 1. Formal setting and saddle-point viewpoint

A standard OCCO model specifies convex, compact decision sets \(X\subset\mathbb R^n\) and \(Y\subset\mathbb R^m\), and a sequence of payoff functions \(f_t:X\times Y\to\mathbb R\) such that \(f_t(\cdot,y)\) is convex for each \(y\) and \(f_t(x,\cdot)\) is concave for each \(x\) [2312.06957, 2509.07901]. At each round, the players choose \(x_t\in X\) and \(y_t\in Y\) without knowing the current or future payoffs, and only then observe \(f_t\) [1806.08301].

A foundational antecedent is the Lagrangian reformulation of constrained convex optimization. For a problem
\[
\min_{x\in X} f_0(x)\qquad \text{subject to } f_i(x)\le 0,\ i=1,\dots,m,
\]
one defines
\[
L(x,y)=f_0(x)+\sum_{i=1}^m y_i f_i(x),
\]
with \(x\in X\) and \(y\in Y\), where \(Y\) may be \(\mathbb R_+^m\) or the simplex after suitable restriction or smoothing. The constrained problem is then equivalent, by Lagrange duality, to the saddle-point problem
\[
\min_{x\in X}\max_{y\in\mathbb R_+^m} L(x,y),
\]
and can be viewed as a repeated zero-sum game over \(T\) rounds [0610119]. In that repeated game interpretation, the primal player chooses \(x_t\), the dual player chooses \(y_t\), and the stage loss is \(L(x_t,y_t)\) [0610119].

This saddle-point perspective is structurally important because it places approximation algorithms, no-regret learning, mirror descent, multiplicative weights, and primal-dual methods within a common template. A plausible implication is that OCCO is best understood not as a minor extension of online convex optimization, but as a unifying framework for sequential equilibrium computation and online constrained optimization.

## 2. Performance criteria: from regret to duality gaps

The central conceptual issue in OCCO is the choice of benchmark. In ordinary online convex optimization, performance is measured by regret against the best fixed decision in hindsight. In OCCO, the analogous object must compare a sequence of online plays to an offline saddle-point or to a comparator sequence.

One line of work measures performance through the generalized duality gap
\[
\mathrm{DualGap}_T
:= \max_{y\in Y}\sum_{t=1}^T f_t(x_t,y)
-\min_{x\in X}\sum_{t=1}^T f_t(x,y_t),
\]
which reduces to standard regret when \(Y\) is a singleton [2312.06957]. This formulation makes explicit the parallel between OCCO with Dual-Gap and online convex optimization with regret [2312.06957].

A closely related formulation is the online saddle point problem, where the aggregate payoff is
\[
F_T(x,y):=\sum_{t=1}^T f_t(x,y),
\]
and the benchmark is the offline saddle-point \((x^*,y^*)\) of \(F_T\). The associated performance measure is
\[
\mathrm{SP\mhyphen Regret}_T
:=\left|\sum_{t=1}^T f_t(x_t,y_t)-F_T(x^*,y^*)\right|,
\]
together with equivalent one-sided expressions involving the primal and dual deviations from equilibrium play [1806.08301].

For non-stationary environments, recent work defines the dynamic duality gap
\[
\mathrm{D\mhyphen DGap}_T
=\sup_{(u_t,v_t)\in X\times Y}
\sum_{t=1}^T \bigl[f_t(x_t,v_t)-f_t(u_t,y_t)\bigr],
\]
which benchmarks the online trajectory against arbitrary comparator sequences \(\{(u_t,v_t)\}_{t=1}^T\subset X\times Y\) [2509.07901]. This quantity generalizes both dynamic regret in online convex optimization and the static duality gap in zero-sum games [2509.07901].

The literature also records criticism of alternative metrics. In particular, dynamic Nash-equilibrium regret, defined via stage-wise Nash equilibria, is reported to have inherent defects: it can be negative even when performance is poor, it fails to compose over rounds, and it is not robust to small perturbations in \(f_t\) [2312.06957]. By contrast, Dual-Gap is always nonnegative and cumulative [2312.06957]. This has made duality-gap-style metrics the dominant formalization of online convex-concave performance.

## 3. Algorithmic foundations: mirror descent, FTRL, and multiplicative weights

The basic OCCO algorithmic toolkit is inherited from online learning. In the repeated Lagrangian game formulation, the primal player may run Online Gradient Descent,
\[
x_{t+1}=\Pi_X\!\Bigl(x_t-\eta_t\nabla_x L(x_t,y_t)\Bigr),
\]
while the dual player runs Follow-The-Regularized-Leader (FTRL) with a strongly convex regularizer, including entropic regularization that yields multiplicative-weights updates on the simplex [0610119]. With entropy regularization \(R(y)=\sum_i y_i\log y_i\), the update takes the closed form
\[
y_{t+1,i}
=
\frac{y_{t,i}\exp\bigl(+\eta f_i(x_t)\bigr)}
{\sum_{j=1}^m y_{t,j}\exp\bigl(+\eta f_j(x_t)\bigr)}.
\]
This is one of the earliest explicit reductions from approximate convex programming to online game playing [0610119].

In the online saddle point literature, strong convexity-concavity leads to SP-FTL, a follow-the-leader procedure that at round \(t+1\) computes the unique saddle-point of the cumulative payoff \(H_t(x,y)=\sum_{\tau=1}^t L_\tau(x,y)\) [1806.08301]. Under the assumptions that each \(L_t\) is \(H\)-strongly convex in \(x\), \(H\)-strongly concave in \(y\), and \(G\)-Lipschitz, SP-FTL achieves
\[
\mathrm{SP\mhyphen Regret}_T \le (8G^2/H)\cdot(1+\ln T)=O(\log T)
\]
[1806.08301]. In the general convex-concave case, SP-RFTL adds regularization to induce strong convexity-concavity, yielding
\[
\mathrm{SP\mhyphen Regret}=O(GD\sqrt{T\ln T})
\]
for Euclidean regularizers on domains of diameter \(D\) [1806.08301].

Another important strand is implicit Online Mirror Descent-Ascent (OMDA), defined through the update
\[
(x_{t+1},y_{t+1})
=
\underset{x\in X}{\arg\min}\;
\underset{y\in Y}{\arg\max}
\Bigl\{ \eta_t f_t(x,y)+B_1(x,x_t)-B_2(y,y_t)\Bigr\},
\]
where \(B_1,B_2\) are Bregman divergences generated by strongly convex mirror maps [2312.06957]. Under \(G\)-Lipschitz payoffs and \(\sigma\)-strongly convex mirror maps, OMDA satisfies
\[
\mathrm{DualGap}_T \le \frac{R^2}{\eta}+\eta G^2T,
\]
which yields \(O(RG\sqrt{T})\) after optimizing the constant step size [2312.06957]. An optimistic variant uses a one-step predictor \(\hat f_t\) and attains the same \(\sqrt{T}\) order with smaller leading factors on predictable sequences [2312.06957].

For simplex-constrained games, optimistic multiplicative methods provide a different behavior. Optimistic Multiplicative-Weights Update (OMWU), an entropy-regularized optimistic FTRL scheme, updates the \(x\)- and \(y\)-players by exponentiating a combination of current and previous gradients, with coefficients \(-2\eta\) and \(+\eta\) for the minimizing player and the sign-reversed version for the maximizing player [2002.06768]. In the setting
\[
\min_{x\in\Delta_n}\max_{y\in\Delta_m} f(x,y),
\]
with \(f\) convex in \(x\), concave in \(y\), twice differentiable, and admitting a strictly complementary equilibrium, OMWU exhibits local last-iterate convergence to the saddle point for sufficiently small \(\eta\), and the convergence is locally exponential [2002.06768].

## 4. Rates, assumptions, and structural accelerations

The classical no-regret analysis of repeated saddle-point play produces \(O(\sqrt{T})\)-type guarantees. In the Lagrangian framework, if each \(f_i\) is \(G\)-Lipschitz on \(X\) and \(X\) has diameter \(D\), then the primal and dual regrets satisfy
\[
R_x(T)\le DG\sqrt{T},\qquad
R_y(T)\le \frac{\log m}{\eta}+\eta G^2T,
\]
with the standard choice \(\eta\approx \sqrt{\log m}/(G\sqrt{T})\), and the average iterates obey a saddle-point gap bounded by \((R_x(T)+R_y(T))/T\) [0610119]. Consequently, a gap at most \(\epsilon\) requires \(T=O(1/\epsilon^2)\) in the general Lipschitz case [0610119].

A principal acceleration mechanism is smoothing or regularization that introduces strong convexity in one variable or strong concavity in the other. In the 2006 reduction from convex optimization to online game playing, adding an \(\eta\)-strong convex smoothing term in the dual allows OGD on \(x\) and FTRL on \(y\) to produce an average point \(\bar x\) with \(f_i(\bar x)\le \epsilon\) for all constraints and \(f_0(\bar x)\le f_0(x^*)+\epsilon\) after
\[
T=\left\lceil 2GD/\epsilon \right\rceil
\]
rounds, using only gradients, projections onto \(X\), and simple entropic updates [0610119]. The paper emphasizes that no quadratic-programming subroutine is needed [0610119].

The same qualitative pattern appears in the online saddle point problem. Strongly convex-concave payoffs permit logarithmic SP-Regret via SP-FTL, whereas generic convex-concave payoffs require regularization to recover sublinear control [1806.08301]. This suggests that curvature is not merely a technical convenience, but a decisive property separating \(O(\log T)\) and \(O(\sqrt{T\ln T})\) regimes in online saddle-point learning.

More recent work on non-stationary OCCO identifies path-length adaptation as the relevant analogue of dynamic-regret adaptivity. Defining
\[
P_T=\sum_{t=1}^T \|u_t-u_{t-1}\|+\|v_t-v_{t-1}\|,
\]
a modular algorithm can achieve a minimax optimal dynamic duality gap upper bound, up to a logarithmic factor,
\[
\widetilde O\bigl(\sqrt{(1+P_T)T}\bigr),
\]
and more precisely
\[
\widetilde O\bigl(\sqrt{(1+\min\{P_T,C_T\})T}\bigr),
\]
where \(C_T\) is a data-dependent environmental variation bound [2509.07901]. The same work also proves a lower bound
\[
\Omega\bigl(\sqrt{(1+P)T}\bigr),
\]
showing that the achieved dependence on the path variation is minimax optimal up to logarithmic factors [2509.07901].

## 5. Variants: bilinear games, bandit feedback, alternating dynamics, and distributed settings

A particularly tractable subclass is the online matrix game, where \(X=\Delta_{d_1}\), \(Y=\Delta_{d_2}\), and
\[
L_t(x,y)=x^\top A_t y,\qquad \|A_t\|_\infty\le 1.
\]
Using negative-entropy regularizers over restricted simplices, OMG-RFTL reduces the dimension dependence in SP-Regret from linear in the dimension to logarithmic:
\[
O(\sqrt{T}\ln T+\sqrt{T}\max\{\ln d_1,\ln d_2\})+o(1)\cdot \max\{d_1,d_2\}
\]
[1806.08301]. Under bandit feedback, where each round only one entry \(A_t(i,j)\) is observed after sampling \(i\sim x_t\), \(j\sim y_t\), an unbiased one-point estimator
\[
\hat A_{ij}=c/(x_i y_j)
\]
for the sampled pair leads to
\[
\mathbb E[\mathrm{SP\mhyphen Regret}]
=O((d_1+d_2)\ln T\,T^{5/6})
\]
against an adaptive adversary [1806.08301].

Another direction replaces standard regret by alternating regret. For adversarial online convex optimization, Continuous Hedge achieves
\[
O\!\bigl(d^{2/3}T^{1/3}\ln^{2/3}T\bigr)
\]
alternating regret under \(|f_t(x)|\le 1\), and a 3rd-order-smooth FTRL construction yields \(\tilde O(T^{2/5})\) in smooth and self-concordant settings, with \(O(T^{1/3})\) when \(C=0\), such as linear or quadratic losses [2502.12529]. When both players in a convex-concave zero-sum game use alternating-regret algorithms, the average iterate \((\bar x,\bar y)\) has duality gap
\[
O\!\Bigl(\frac{R^x+R^y}{T}\Bigr),
\]
so \(R=O(d^{2/3}T^{1/3})\) per player yields an \(O(d^{2/3}T^{-2/3})\) rate for saddle-point approximation [2502.12529]. This is distinct from the standard \(O(T^{-1/2})\) averaging behavior associated with ordinary no-regret bounds.

Distributed and stochastic versions introduce communication constraints and predictive dynamics. In a multiagent network with time-varying directed graph, the DOSMD-CCO algorithm combines stochastic mirror descent, Bregman projections, predictive mappings \(B_t,C_t\), and consensus steps. Under assumptions including unbiased noisy gradients, strong convexity of distance-generating functions, and nonexpansive predictive mappings, the expected dynamic saddle-point regret of each agent satisfies
\[
O\bigl(\max\{T^{\theta_1},T^{\theta_2}(1+V_T)\}\bigr),
\]
where \(V_T\) is the path-variation of the time-varying saddle points [2508.09411]. With \(\gamma_1=\gamma_2=\frac12\), this becomes \(O(\sqrt{T}(1+V_T))\) [2508.09411]. A multiple-consensus variant tightens the constant in the consensus term [2508.09411].

## 6. Applications and adjacent problem classes

The Lagrangian game perspective immediately yields applications to approximate constrained optimization. The 2006 framework explicitly lists maximum-entropy estimation, portfolio optimization with convex risk constraints, and computational problems in signal processing as examples with strictly convex constraints [0610119]. In maximum-entropy estimation with moment matching,
\[
\min_{q\in\Delta}\sum_s q(s)\log q(s)
\quad \text{s.t.}\quad
\mathbb E_q[\phi_i(s)]=\mu_i,\ i=1,\dots,m,
\]
the dual variables correspond to moment constraints and the same online convex optimization machinery recovers the Gibbs distribution in \(O(1/\epsilon)\) rounds [0610119].

Portfolio optimization with convex risk constraints is also cast in this form: choose \(x\in\Delta_n\) to minimize \(-\mu^\top x\) subject to VaR or CVaR constraints \(f_i(x)\le 0\), update dual weights multiplicatively over scenarios, and update the primal via OGD [0610119]. Robust linear and semidefinite programs fit the same template when adversarial constraints are embedded in the dual strategy set [0610119].

The online saddle point formalism further connects to online convex optimization with knapsacks. With budget \(b\in\mathbb R_+^m\), concave rewards \(r_t(x)\), and convex consumptions \(c_t(x)\), one defines
\[
L_t(x,y)=-r_t(x)-y^\top(b/T-c_t(x)),\qquad y\in Y=[0,y_{\max}]_m,
\]
and obtains a regret decomposition in which one term is the dual player’s individual-regret and the other is SP-Regret [1806.08301]. A primal-dual RFTL method then achieves
\[
O(\sqrt{T})
\]
regret in the stochastic i.i.d. setting [1806.08301].

Learning-augmented online packing and covering provide a related but not identical adjacent class. For online concave packing, a switching algorithm mixes a black-box online packing method with an advice vector \(y'\), obtaining
\[
g(y)\ge (1-\lambda)g(y') \quad\text{and}\quad
g(y)\ge (\lambda/\alpha)\cdot \mathrm{OPT},
\]
with approximate feasibility \(A^\top y\le (2-\lambda)\beta b\) [2411.08332]. For online convex covering, a primal-dual learning-augmented algorithm yields
\[
f(x)\le O(1/(1-\lambda))\cdot f(x')
\quad\text{and}\quad
f(x)\le O((p\log(d/\lambda))^p)\cdot \mathrm{OPT}
\]
[2411.08332]. The source describes these results as part of a unified theory of online convex-concave optimization with predictions [2411.08332]. This suggests a broader view in which advice, predictors, and side information act as regularizing signals for sequential saddle-point methods.

## 7. Current themes, misconceptions, and open directions

A recurrent misconception is that separate no-regret guarantees for the two players automatically solve the online saddle-point problem. The literature shows that the correct benchmark is not merely the sum of two independent regrets, but a saddle-point-compatible metric such as Dual-Gap, SP-Regret, or D-DGap [1806.08301, 2312.06957]. This distinction is central because equilibrium quality involves both players jointly.

A second misconception is that averaging is the only meaningful convergence notion. Average-iterate guarantees are indeed standard: the no-regret-to-equilibrium lemma in the repeated Lagrangian framework bounds the saddle-point gap of \((\bar x,\bar y)\) by \((R_x(T)+R_y(T))/T\) [0610119], and alternating-regret analyses also convert into average-iterate Nash guarantees [2502.12529]. However, last-iterate convergence has become an important separate objective, particularly in constrained simplex domains. OMWU establishes local pointwise convergence of the actual iterates, not only their averages, under convex-concave smoothness and strict complementarity assumptions [2002.06768].

A third issue concerns predictability and non-stationarity. Existing algorithms had been reported to fail to deliver optimal performance in stationary or predictable environments, motivating a modular OCCO method with three components: an Adaptive Module, a Multi-Predictor Aggregator, and an Integration Module [2509.07901]. The resulting procedure attains both a path-length-based minimax rate and a prediction-error-driven rate,
\[
\widetilde O\Bigl(\min\{\sum_t \rho_t(f,h),\sqrt{(1+\min\{P_T,C_T\})T}\}\Bigr),
\]
and with multiple predictors,
\[
\widetilde O\Bigl(\min\{\min_{k\in[d]}\sum_t\rho_t(f,h^k),\sqrt{(1+\min\{P_T,C_T\})T}\}\Bigr)
\]
[2509.07901]. In the perfectly predictable case, one predictor has zero error and the dynamic duality gap becomes \(O(\ln T)\) [2509.07901].

Open directions are explicitly identified in the non-stationary OCCO literature: one-sided feedback, two-player partial observation or bandit saddle-point problems, and extensions to multi-player non-zero-sum games with dynamic equilibria [2509.07901]. Distributed stochastic OCCO further points toward predictive multiagent saddle-point learning under communication constraints [2508.09411]. Taken together, these directions indicate that the modern field has expanded beyond static adversarial convex-concave games into a broader study of dynamic, predictive, partially observed, and networked equilibrium computation.

Source: https://www.emergentmind.com/topics/online-convex-concave-optimization