---
title: 'Rot Mover’s Distance: Regularized OT'
url: https://www.emergentmind.com/topics/rot-mover-s-distance-rmd
type: topic
---

# Rot Mover’s Distance: Regularized OT

The Rot Mover’s Distance (RMD) is a generalization of the classic Earth Mover’s Distance (EMD) within the framework of discrete optimal transport. RMD augments the standard transport problem by introducing a smooth convex regularization penalty on the joint transport plan, yielding a new class of metrics rooted in matrix nearness with respect to Bregman divergences. This construction enables the interpolation between classical EMD and minimal-regularized couplings, where the nature of regularization controls plan smoothness, sparsity, or other desired structure. RMD recovers established methods such as Sinkhorn–Knopp for entropic regularization and extends to a wide spectrum of regularizers and induced divergences, with efficient algorithms tailored to the structure of each case [1610.06447].

## 1. Mathematical Formulation: Primal and Dual RMD

Given probability vectors $a, b\in \Sigma_d$ in the $d$-simplex, a nonnegative cost matrix $C\in\mathbb{R}_+^{d\times d}$, and a convex, smooth regularizer $\Omega:\mathbb{R}^{d\times d}\to\mathbb{R}\cup\{+\infty\}$, RMD is formulated on the transport polytope
\[
U(a,b) = \{X\in\mathbb{R}_+^{d\times d}\mid X\mathbb{1}=a,\; X^T\mathbb{1}=b\}.
\]

- **Primal (Constrained) Formulation:**  
Given an "allowance" $\alpha\geq 0$ for the regularizer,
\[
d'_{C,\alpha,\Omega}(a,b) = \min_{X\in U(a,b),\, \Omega(X)\leq\Omega(X^{\star})+\alpha}\;\langle C,X\rangle,
\]
where $X^{\star}$ solves $\min_{X\in U(a,b)} \Omega(X)$.

- **Dual (Penalized) Formulation:**  
Introducing a Lagrange parameter $\lambda\geq 0$,
\[
d_{C,\lambda,\Omega}(a,b) = \min_{X\in U(a,b)}\;\langle C,X\rangle + \lambda\,\Omega(X).
\]
For $\alpha$ below a threshold, there exists a unique $\lambda$ with $d'_{C,\alpha,\Omega}(a,b)=d_{C,\lambda,\Omega}(a,b)$ so that the primal and dual minimizers coincide. Classical EMD is recovered as $\lambda\to 0$, while $\lambda\to\infty$ yields the minimal-$\Omega$ coupling [1610.06447].

## 2. Bregman-Projection Matrix-Nearness Equivalence

Let $\Omega$ be the Bregman-type information regularizer generated by a convex function $\varphi$, either separable ($\varphi(X) = \sum_{ij} \varphi_{ij}(X_{ij})$) or general. The Fenchel conjugate $\psi = \varphi^{\ast}$ induces the Bregman divergence:
\[
D_{\varphi}(X\|Y) = \varphi(X) - \varphi(Y) - \langle X-Y, \nabla\varphi(Y) \rangle.
\]
The dual RMD problem equivalently minimizes the Bregman divergence
\[
\min_{X\in U(a,b)} D_{\varphi}(X\|Y),
\]
where $Y = \nabla\psi(-C/\lambda)$ is obtained by unconstrained minimization of $\langle C, X\rangle + \lambda \varphi(X)$. For the entropic regularizer, $Y_{ij} = \exp(-C_{ij}/\lambda)$, yielding a Kullback–Leibler projection interpretation. This general Bregman-projection framework enables the use of projection algorithms for regularized OT [1610.06447].

## 3. Iterative Bregman Projection Algorithms: ASA and NASA

Efficient solution of the RMD projection is based on iterative Bregman projections, with two principal algorithmic frameworks determined by the domain of the regularizer $\varphi$:

- **Nonnegative Alternate Scaling Algorithm (NASA):**  
Used when $\varphi$ does not enforce $X\geq 0$. Dykstra's algorithm augments alternate Bregman projections with correction variables, cycling through nonnegativity, row-sum, and column-sum constraints. Newton–Raphson solves the one-dimensional per-row and per-column projection equations when $\varphi$ is separable. Each iteration maintains correction vectors to ensure convergence.

- **Alternate Scaling Algorithm (ASA):**  
Applied when $\varphi$'s domain is contained in $\mathbb{R}_+^{d\times d}$, so $X\geq 0$ is implicit. The method alternates between row-sum and column-sum Bregman projections with no correction variables. For separable $\varphi$, updates decouple into per-row and per-column monotone equations efficiently solved by Newton–Raphson.

Both schemes generalize the classical projection-on-convex-sets (POCS) framework and leverage the explicit structure of $\nabla\varphi$ and $\psi^{\prime}$ for efficient updates [1610.06447].

## 4. Regularizer Families and Induced Divergences

The RMD framework supports a broad gallery of convex regularizers $\varphi$ (see Table 1), each yielding a distinct Bregman divergence $D_{\varphi}$ and associated geometric and statistical properties.

| Regularizer Type             | $\varphi(\pi)$ Definition                        | Induced $D_{\varphi}$                     |
|------------------------------|--------------------------------------------------|--------------------------------------------|
| Entropic (KL)                | $\pi\log\pi-\pi+1$                               | $\pi\log(\pi/\xi)-\pi+\xi$                |
| Burg (Itakura–Saito)         | $\pi-\log\pi-1$                                  | $\pi/\xi-\log(\pi/\xi)-1$                 |
| Fermi–Dirac                  | $\pi\log\pi+(1-\pi)\log(1-\pi)$                  | $\pi\log(\pi/\xi)+(1-\pi)\log\left(\frac{1-\pi}{1-\xi}\right)$ |
| $\ell_p$ (quasi-norms, $p<1$)| $-\pi^p$                                         | —                                         |
| $\ell_p$ (norms, $p>1$)      | $|\pi|^p$                                        | —                                         |
| Euclidean ($p=2$)            | $\frac{1}{2}\pi^2$                               | $\frac{1}{2}\|\pi-\xi\|^2$                |
| Hellinger-type               | $-\sqrt{1-\pi^2}$                                | —                                         |
| Mahalanobis (quadratic form) | $\frac{1}{2}\mathrm{vec}(X)^T P\,\mathrm{vec}(X)$| —                                         |

Notably, the framework recovers Sinkhorn–Knopp scaling for KL entropic regularization (where $\psi^{\prime}(\theta)=\exp\theta$ and Newton projections become matrix rescalings), but allows for fundamentally different plan structures, smoothing, and sparsification depending on the regularizer choice [1610.06447].

## 5. Empirical Properties and Algorithmic Considerations

RMD exhibits several algorithmic and empirical characteristics:

- **Interplay of $\lambda$ and $\varphi$:**  
Varying the regularizer parameter $\lambda$ yields a continuous interpolation between sharply optimal (EMD-like) and highly regularized transport plans. The geometry, such as anisotropy or smoothness, is strongly modulated by the nature of $\varphi$.
- **Computational Complexity:**  
For moderate $d$, Newton subproblems within ASA/NASA scale as $O(d)$ or $O(\text{nnz})$ per projection, with overall quadratic complexity per outer iteration. Sinkhorn–Knopp (KL) admits the fastest implementation; ASA is empirically faster than NASA due to the absence of correction variables.
- **Sparsity via Pruning:**  
Transport forbidden by $C_{ij}=+\infty$ is handled by sparse extensions, simply excluding these indices from updates without affecting result correctness under broad conditions [1610.06447].

## 6. Applications and Empirical Results

Synthetic experiments on two-mode densities demonstrate that tuning $\lambda$ and choosing different $\varphi$ yields qualitatively distinct mass redistributions, such as $\ell_1$- versus $\ell_2$-like smoothing and various anisotropic effects. In audio-scene classification benchmarks (specifically, DCASE16), RMD-induced kernels—where each segment is encoded as a GMM over MFCCs, with OT ground cost given by pairwise Jeffrey-divergence—achieve superior or competitive accuracy to classical EMD-based SVM kernels. The Hellinger-type and certain $\ell_p$ penalties ($p\approx0.1$–$0.5$) outperform classic Euclidean ($p=2$) or Burg-IS regularizers in discriminative capacity. This suggests that fine-grained adjustment of $\lambda$ and $\varphi$ can significantly enhance OT-based kernel methods for pattern recognition and statistical tasks [1610.06447].

## 7. Connections and Generalizations

RMD provides a principled interpolation and generalization over standard optimal transport, embedding the classic EMD, entropic regularization (Sinkhorn), and other divergences in a single algorithmic and theoretical scaffold. The Bregman-projection viewpoint enables leverage of convex duality and optimization-theoretic tools, including Newton–Raphson projection for separable $\varphi$, Dykstra’s algorithm for general convex settings, and efficient sparse extensions. The framework is compatible with a variety of regularizer classes encountered in machine learning and information geometry, supporting both spread-promoting and sparsity-inducing couplings according to analytic or empirical desiderata [1610.06447].

Source: https://www.emergentmind.com/topics/rot-mover-s-distance-rmd