Matrix Multiplicative Weights Algorithm

• Matrix multiplicative weights (MMW) are a generalization of multiplicative-weights and exponentiated-gradient methods, replacing probability distributions with positive semidefinite trace-one matrices (density matrices) and scalar normalization with trace normalization.
• MMW is used in a variety of applications including semidefinite programming, zero-sum and quantum games, online eigenvector problems, and quadratic optimization, offering a convergence rate to Nash equilibria in zero-sum games.
• Standard MMW has an average regret of order O(√(T logn)), which can be achieved with choice of learning rate η ∝ √log(n)/T; efficient computational approximations are possible.

Matrix multiplicative weights (MMW) is a matrix-valued generalization of multiplicative-weights and exponentiated-gradient methods in which probability distributions are replaced by positive semidefinite trace-one matrices, payoff vectors by Hermitian matrices, and scalar normalization by trace normalization. The canonical decision domain is the spectrahedron

$$\mathcal A_n=\{X\in\mathbb S^n:X\succeq 0,\ \operatorname{tr}(X)=1\},$$

whose elements are density matrices. MMW is used for online optimization over semidefinite domains, zero-sum and quantum games, semidefinite programming, online eigenvector problems, and quadratic optimization. Its central update maps an accumulated symmetric gain matrix $Y$ to

$$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$

1. Mathematical formulation and canonical update

At round $t$, an MMW learner selects $X_t\in\mathcal A_n$, receives a symmetric gain matrix $G_t\in\mathbb S^n$, and obtains payoff

$$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$

The regret against the best fixed matrix in $\mathcal A_n$ is

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$

Because

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$

the comparator term equals

$Y$0

The standard matrix exponentiated-gradient update selects

$Y$1

where $Y$2 is the learning rate. If

$Y$3

then

$Y$4

The matrix exponential ensures positive semidefiniteness, while trace normalization ensures unit trace.

The associated potential is

$Y$5

with gradient

$Y$6

Its convex conjugate is the negative von Neumann entropy, represented on positive definite $Y$7 by

$Y$8

The induced Bregman divergence is quantum relative entropy,

$Y$9

When $$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$0, standard MMW gives average regret of order

$$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$1

and cumulative regret of order

$$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$2

A choice such as

$$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$3

produces this scale. The $$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$4 term arises from the entropy diameter of the spectrahedron.

MMW is distinct from a self-coupled Hedge map used in some analyses of symmetric games. In that setting,

$$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$5

where the same strategy $$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$6 supplies both the population distribution and the opponent against which payoffs are evaluated. This is a nonlinear single-population dynamical system, not the usual two-process row-player/column-player matrix MMW algorithm. Its fixed points and KL-divergence identities do not by themselves establish convergence of empirical play to Nash equilibrium (Avramopoulos, 2016).

2. Mirror descent, entropy, and regret analysis

The geometry of MMW is generated by the von Neumann entropy. For density matrices $$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$7 and $$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$8, the quantum relative entropy is

$$P_{\mathrm{mw}}(Y)=\frac{e^Y}{\operatorname{tr}(e^Y)}.$$9

In payoff-based analyses, the entropy satisfies the semidefinite Pinsker-type inequality

$t$0

This strong convexity supplies the stability term in the mirror-descent analysis.

For an MMW update driven by $t$1, the basic energy inequality is

$t$2

Summing over rounds yields a matrix regret estimate of the form

$t$3

For one player, the entropy diameter can be bounded by $t$4; in a two-player setting the corresponding quantity is

$t$5

In two-player zero-sum semidefinite games, the averaged iterates

$t$6

satisfy a duality-gap estimate

$t$7

where $t$8 is a Lipschitz or gradient bound. Thus $t$9 iterations suffice for an $X_t\in\mathcal A_n$0-Nash equilibrium under full matrix-gradient feedback.

Relative entropy also appears in continuous-time and quantum formulations. In a two-player zero-sum quantum game with a fully mixed Nash equilibrium $X_t\in\mathcal A_n$1, quantum replicator dynamics preserve

$X_t\in\mathcal A_n$2

where

$X_t\in\mathcal A_n$3

The conservation law is a continuous-time analogue of the conserved divergence in classical zero-sum replicator dynamics. It constrains the joint evolution of the two density matrices but does not imply that either state has a fixed spectrum or that the trajectory converges to equilibrium (Jain et al., 2022).

3. Computational approximations and low-rank sketches

The dominant computational obstacle in standard MMW is the matrix exponential. General-purpose eigendecomposition costs approximately $X_t\in\mathcal A_n$4 in practice, while full matrix exponentiation requires maintaining an $X_t\in\mathcal A_n$5 matrix.

A rank-one randomized sketch replaces the full matrix softmax by

$X_t\in\mathcal A_n$6

where $X_t\in\mathcal A_n$7 is sampled uniformly from the unit sphere. The resulting matrix is positive semidefinite, rank one, and has unit trace. The sketched algorithm plays

$X_t\in\mathcal A_n$8

This sketch is not unbiased for the standard MMW projection: $X_t\in\mathcal A_n$9 in general. Instead, define

$G_t\in\mathbb S^n$0

The averaged projection is the gradient of the convex potential

$G_t\in\mathbb S^n$1

Its convex conjugate supplies the regularizer for the averaged algorithm.

The relevant Bregman divergence satisfies the smoothness bound

$G_t\in\mathbb S^n$2

and the diameter bound

$G_t\in\mathbb S^n$3

These properties allow mirror-descent analysis with only constant-factor degradation relative to standard MMW. The resulting cumulative regret is bounded by

$G_t\in\mathbb S^n$4

For $G_t\in\mathbb S^n$5, this gives average regret

$G_t\in\mathbb S^n$6

The actual random sketch requires an adversary condition: conditional on the past, $G_t\in\mathbb S^n$7 must be independent of the fresh random vector $G_t\in\mathbb S^n$8. Under this condition,

$G_t\in\mathbb S^n$9

so expected regret is controlled by the deterministic averaged-projection analysis. High-probability bounds follow by treating

$$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$0

as a martingale difference.

The matrix-exponential–vector product $$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$1 can be approximated using $$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$2 Lanczos iterations. If

$$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$3

is the Lanczos tridiagonalization, then

$$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$4

approximates $$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$5. The resulting computational cost is

$$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$6

where $$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$7 is the matrix-vector multiplication cost and $$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$8 is the floating-point word size. The rank-one method is advantageous when matrix-vector multiplication is substantially cheaper than dense eigendecomposition, particularly for sparse or structured matrices (Carmon et al., 2019).

4. Applications to eigenvectors, semidefinite programming, and quadratic optimization

The spectrahedral domain contains rank-one matrices $$\langle G_t,X_t\rangle=\operatorname{tr}(G_t^\top X_t).$$9, so MMW directly induces algorithms for online eigenvector problems. The benchmark is

$\mathcal A_n$0

The rank-one method selects

$\mathcal A_n$1

With exact exponential-vector products, the expected regret is

$\mathcal A_n$2

With Lanczos approximation, the total matrix-vector-product complexity for high-probability $\mathcal A_n$3-average regret is

$\mathcal A_n$4

and the method improves the previous best online-eigenvector complexity by a factor of at least $\mathcal A_n$5.

MMW and Oja’s algorithm are related but not equivalent in general. If the input matrices $\mathcal A_n$6 share a common orthonormal eigenbasis $\mathcal A_n$7, with

$\mathcal A_n$8

then the squared eigen-coordinates of Oja’s normalized vector,

$\mathcal A_n$9

satisfy

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$0

After rewriting this expression, $$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$1 follows an ordinary multiplicative-weights recursion. Thus Oja’s vector dynamics reduce exactly to scalar MW over eigen-directions in the common-eigenbasis regime. For arbitrary noncommuting matrices, the squared-coordinate dynamics do not close, and this reduction fails (Garber, 2023).

MMW also applies to semidefinite programs in saddle-point form. Let

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$2

and

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$3

For symmetric matrices $$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$4, define

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$5

The matrix player uses the rank-one sketch, while the simplex player uses ordinary multiplicative weights. The duality gap is

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$6

If

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$7

then the averaged strategies satisfy

$$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$8

To achieve expected gap at most $$\max_{X\in\mathcal A_n}\sum_{t=1}^T\langle G_t,X\rangle -\sum_{t=1}^T\langle G_t,X_t\rangle.$$9, it suffices to take

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$0

5. Quantum games and payoff-based MMW

In quantum games, a mixed strategy is a density matrix

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$1

For a two-player game, Alice and Bob use density matrices $$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$2 and $$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$3. A payoff observable $$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$4 induces the bilinear payoff

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$5

The matrix multiplicative-weights updates are

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$6

and

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$7

The negative sign in Bob’s update reflects minimization of Alice’s payoff. When all matrices are diagonal in a common basis, MMWU reduces exactly to classical exponential MWU. Noncommutativity is the principal quantum distinction: accumulated payoff operators need not commute with their instantaneous derivatives, and the derivative of $$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$8 is generally

$$\max_{X\in\mathcal A_n}\langle Y,X\rangle=\lambda_{\max}(Y),$$9

The continuous limit defines quantum replicator dynamics through

$Y$00

and

$Y$01

For a fully mixed Nash equilibrium, the total quantum relative entropy is conserved: $Y$02 The same setting yields Poincaré recurrence for almost every interior initial condition. The proof uses canonical coordinates, volume preservation, boundedness from the relative-entropy invariant, and the Poincaré recurrence theorem. The result establishes recurrence rather than convergence: generic trajectories return arbitrarily close to their initial conditions infinitely often.

Full-information MMW assumes access to the complete payoff-gradient matrix. Payoff-based learning replaces it with an estimator. In minimal-information matrix multiplicative weights, or 3MW, the update is

$Y$03

The estimator decomposes as

$Y$04

where $Y$05 is smoothing bias and $Y$06 is a martingale-noise term.

For deterministic scalar payoff feedback, a two-point estimator achieves

$Y$07

with a factor linear in the effective matrix dimension $Y$08. For a single random payoff-observable realization, a one-point estimator has variance of order $Y$09, leading to

$Y$10

The rate difference arises from the bias–variance tradeoff: the smoothing bias is $Y$11, whereas the one-point estimator variance is $Y$12. In general non-zero-sum games, a regularized 3MW method has local, high-probability, last-iterate convergence to equilibria satisfying variational stability (Lotidis et al., 2023).

6. Evolutionary, game-theoretic, and dynamical interpretations

MMW also arises exactly in the marginal allele dynamics of sexual evolutionary models. For selection before recombination, let $Y$13 be the joint population distribution and let

$Y$14

be the allele marginal. If

$Y$15

is the conditional fitness of allele $Y$16, then the evolutionary update satisfies

$Y$17

which is exactly the parameter-free, correlation-sensitive polynomial-weights update. This identity holds for arbitrary nonnegative fitness matrices, recombination rates, initial distributions, and numbers of loci.

For recombination before selection, the marginal update uses a mixture of correlation-sensitive and independent payoffs: $Y$18 where

$Y$19

The resulting update is

$Y$20

Thus recombination before selection corresponds to an interpolated parameter-free PW process.

The correspondence is between allele marginals and MW strategies, not between the complete joint population distribution and independent mixed strategies. A product distribution satisfies

$Y$21

whereas linkage disequilibrium is

$Y$22

Product distributions are generally not preserved, even under weak selection. Consequently, the uncorrelated MWUA based only on marginal distributions need not describe actual evolutionary dynamics, and it can converge to a different equilibrium from the sexual population process. The precise correspondence therefore retains the evolving correlations through conditional payoffs (Meir et al., 2015).

In symmetric bimatrix games, the Hedge map

$Y$23

has fixed points characterized by equal payoffs on the support of $Y$24. Interior fixed points are interior symmetric equilibria, while boundary fixed points need not be equilibria of the full game. The map admits a relative-entropy identity

$Y$25

whose second derivative is an exponentially tilted payoff variance and is therefore nonnegative. These are valid local and variational properties of the Hedge map, but they do not imply convergence of the dynamics to equilibrium or establish a polynomial-time equilibrium algorithm. In particular, the claimed implication that every symmetric game without an equalizer has a weakly dominated pure strategy is not established, and the resulting claim $Y$26 is not a valid consequence of the argument (Avramopoulos, 2016).

Across these settings, the principal distinction is between exact matrix-valued multiplicative weights, low-rank or payoff-based approximations, and scalar multiplicative-weights reductions available under additional structure. Standard MMW accommodates arbitrary symmetric, including noncommuting, matrix sequences. Rank-one sketches preserve regret through a different averaged mirror map. Oja’s algorithm reduces to ordinary MW only under a common eigenbasis. Quantum MMW extends the geometry to density matrices and yields conservation and recurrence phenomena in zero-sum games. Evolutionary dynamics reproduce MMW exactly at the level of allele marginals while retaining correlations in the underlying joint population.