---
title: Sinkhorn–Knopp Matrix Scaling Algorithm
url: https://www.emergentmind.com/topics/sinkhorn-knopp-matrix-scaling-algorithm
type: topic
---

# Sinkhorn–Knopp Matrix Scaling Algorithm

The Sinkhorn–Knopp matrix scaling algorithm, also known as the RAS algorithm, is a fundamental iterative procedure for scaling a nonnegative matrix so that it achieves prescribed row and column sums. Its wide impact spans combinatorial optimization, numerical linear algebra, statistics, and optimal transport. The method alternates normalizing the rows and columns of a matrix, producing a sequence converging—under suitable conditions—to a matrix which satisfies the specified marginals exactly. Beyond its original use case for doubly stochastic scaling, the Sinkhorn–Knopp algorithm underlies fast solvers for entropic regularized optimal transport and is deeply connected to geometric programming, Bregman projections, quantum algorithms, and structural properties in combinatorial matrix theory.

## 1. Problem Statement and Classical Algorithm

Given a nonnegative matrix $A \in \mathbb{R}^{n \times m}_{\ge 0}$, and target row and column sum vectors $r \in \mathbb{R}^n_{+}$, $c \in \mathbb{R}^m_{+}$ such that $\sum_i r_i = \sum_j c_j = h$, the matrix scaling problem is to find diagonal matrices $R \in \mathbb{R}^{n \times n}_+$ and $S \in \mathbb{R}^{m \times m}_+$ so that $X := R A S$ has row sums $r$ and column sums $c$:
\[
\sum_j X_{ij} = r_i \quad \forall i, \qquad \sum_i X_{ij} = c_j \quad \forall j
\]
A particularly important special case is doubly stochastic scaling: $n = m$, $r_i = c_j = 1$ for all $i, j$.

The canonical Sinkhorn–Knopp algorithm alternates between row-normalization and column-normalization:

- Initialize $x^{(0)} = 1$, $y^{(0)} = 1$
- For $t = 0, 1, 2, \ldots$:
  \[
  \begin{cases}
  x^{(t+1)}_i = \frac{r_i}{\sum_j A_{ij} y^{(t)}_j} \\
  y^{(t+1)}_j = \frac{c_j}{\sum_i A_{ij} x^{(t+1)}_i}
  \end{cases}
  \]
Or, equivalently, alternately scale the matrix by the relevant diagonal matrices so that in each step either the rows or the columns match the target sums [1801.02790], [1609.06349], [1306.0895], [1905.09426].

The iteration is stopped when both the maximum deviation in row sums and column sums falls below a prescribed tolerance $\varepsilon$ [1905.09426].

## 2. Theoretical Guarantees: Existence, Uniqueness, and Convergence

**Existence & Uniqueness:**  
The algorithm produces positive diagonal matrices $R$, $S$ such that $RAS$ has prescribed marginals if and only if $A$ has a "doubly stochastic pattern"—there exists a matrix with the same support as $A$ that is already scalable to the desired marginals [1609.06349]. For strictly positive $A$, the diagonal scalings are unique up to a common scalar factor.

**Convergence:**  
- Each scaling step is a contraction in Hilbert’s projective metric.
- Alternation of two such contractions yields linear (geometric) convergence to the unique fixed point in the positive orthant [1905.09426], [1609.06349].
- For matrices with total support (no row/column of all zeros) and connected support graph, convergence to the unique solution (or Sinkhorn limit $S(A)$) is always achieved.

**Finiteness Results:**  
In the generic case, the algorithm converges asymptotically. However, for certain block-structured matrices which are already row- or column-stochastic, the process may terminate in one or two iterations [1903.06778], [1912.00095]. Cohen–Nathanson proved that no instance terminates exactly in more than two steps [1912.00095].

## 3. Quantitative Error Analysis and Iteration Bounds

The convergence rate of the Sinkhorn–Knopp algorithm is governed by the structure of $A$ and the target tolerance $\varepsilon$. Key results [1801.02790]:

- **KL-divergence Tracking:**  
  The decrease in the KL-divergence between the current row sum distribution and the target is closely tracked at each iteration. Pinsker’s inequality and a new KL vs $\ell_1/\ell_2$ bound connect KL divergence to $\ell_1$ and $\ell_2$ errors.
- **Iteration Bounds:**  
  To achieve $\ell_1$ error $\le \varepsilon$,
  \[
  O\left( \frac{h^2}{\varepsilon^2} \log\frac{\Delta \rho}{\nu} \right)
  \]
  steps suffice, where $\nu =$ (min nonzero entry)/(max entry), $\rho = \max_i r_i, \max_j c_j$, and $\Delta = \max_j |\{i : A_{ij}>0\}|$.

  For $\ell_2$ error $\le \varepsilon$,
  \[
  O\left( \rho h \log\frac{\Delta \rho}{\nu} \left( \frac{1}{\varepsilon} + \frac{1}{\varepsilon^2} \right)\right)
  \]
  suffices.

- **Comparisons to Prior Bounds:**  
  Earlier analyses had $h^2$ dependence for both errors; improved results replace this with $h$ for $\ell_2$ and log-scale dependence on the matrix parameters due to the strengthened KL-related inequalities.

- **Exponential and Dimension-Free Convergence:**  
  For matrices whose normalized version satisfies an explicit density condition (bulk mass), iteration complexity is $O(\log n - \log \varepsilon)$, independent of $\nu$ and the maximum regularized cost [2604.03787], [2507.09711]. Pre-scaling by marginals enables a dimension-free $O(\log (1/\varepsilon))$ iteration complexity [2604.03787].

- **Phase Transition in Complexity:**  
  There exists a sharp phase transition at matrix density $\gamma = 1/2$: above this, convergence is logarithmic in $1/\varepsilon$; below, dependence becomes polynomial in $1/\varepsilon$ and $n$ [2507.09711], [2604.03787].

## 4. Interpretations and Connections to Optimization

**Variational Views:**

- **Bregman Projection / KL Minimization:**  
  Alternating row and column scaling is an instance of Bregman projection (specifically, KL-projection) onto affine subspaces of fixed row and column sums. The algorithm minimizes the KL-divergence $D(P||K)$ under these constraints [2404.00003], [1801.02790], [1306.0895].

- **Geometric Programming (Capacity Maximization):**  
  The process alternates minimization of a geometric log-capacity objective, equivalent to maximizing the permanent over certain normalized classes under fixed marginals [2204.07425]. The decrease in capacity is directly linked to progress toward feasibility via an exact formula involving KL divergence.

- **Duality in Entropic Regularized Optimal Transport:**  
  In entropic OT, diagonal scaling finds the potentials ($u, v$) in $P = \mathrm{diag}(u)K\mathrm{diag}(v)$ solving $\min_{P \in U(r,c)} \langle C, P \rangle - \varepsilon H(P)$, with $K = \exp(-C/\varepsilon)$.

## 5. Algorithmic Enhancements, Quantum Algorithms, and Practical Extensions

**Algorithmic Accelerations:**
- Regularization scheduling (doubling the entropic penalty periodically) and scaling reductions (cost, capacity) yield exponential convergence and optimal $O(\log 1/\varepsilon)$ complexity for entropic OT with integer marginals [2207.00736].
- Hierarchical low-rank and Kronecker-structured representations allow $O(n \log^3 n)$ complexity for large-scale regularized OT when the cost matrix decomposes favorably [2004.12511].

**Quantum Analogues:**
- Quantum implementations of Sinkhorn (using amplitude estimation) achieve $\widetilde{O}(\sqrt{mn}/\varepsilon^4)$ scaling, a polynomial speedup in $n$ at a cost in $\varepsilon$-dependence, and are provably optimal for constant-error scaling [2011.12823].

**Extensions:**
- The SK scheme generalizes to operator scaling (positive linear maps), where iteration on positive semidefinite matrices yields trace-preserving, unital normal forms with connections to quantum information theory [1609.06349].
- Constrained scaling with prescribed zeros ("forbidden entries") is handled by zeroing out respective entries in $K$ and restricting scaling support, with convergence proven under mild support assumptions [2404.00003].

## 6. Combinatorial and Structural Implications

- **Perfect Matching and Blockers:**  
  For the case $A$ is a $0$-$1$ matrix, the existence of a doubly stochastic scaling is equivalent to the support graph admitting a perfect matching (via Hall's condition) [2204.07425]. When not scalable, the limiting matrix encodes all Hall blockers (deficiency subsets), accessible from the plateau structure of the limiting marginals.
- **Block Decompositions:**  
  The Sinkhorn–Knopp limit in the non-scalable case is block upper-triangular, precisely reflecting the principal partition, polymatroid base-polytope structure, and Dulmage–Mendelsohn decomposition in combinatorial optimization.
- **Explicit Formulae and Arithmetic:**  
  For symmetric $3 \times 3$ matrices with two distinct entries, closed-form doubly stochastic scalings ("Sinkhorn limits") can be calculated, sometimes involving roots of low-degree equations. Finite-term scaling is structurally characterized and rare [1905.09426], [1902.04544].

## 7. Applications and Practical Considerations

**Applications:**
- **Preconditioning Linear Systems:**  
  Matrix balancing as a preprocessing step increases numerical stability in large-scale linear algebra [2011.12823].
- **Optimal Transport and Learning:**  
  The SK algorithm is the workhorse for entropic OT solvers in machine learning, enabling scalable computation of Wasserstein distances for images, signals, and distributions [1306.0895], [2004.12511].
- **Combinatorial Optimization:**  
  Used for perfect matching, permanent approximation, and biproportional allocation in graphs and networks [2204.07425], [2507.09711].

**Practical Issues and Recommendations:**
- Scaling matrices with extreme values (small $\nu$ or massive entries in $C$) can produce poor theoretical bounds, but practical performance is governed by the well-boundedness (bulk density) of the cost/mass, robust to outliers [2604.03787].
- Pre-scaling by the target marginals is recommended to improve both numerical stability and convergence rates.
- Stopping criteria based on threshold error in marginal sums or primal feasibility are standard; log-domain stabilization and exploitation of sparsity are often necessary in large-scale problems [1306.0895], [2404.00003].

---

For foundational results, optimal iteration bounds, structural implications, and advanced algorithmic enhancements, see [1801.02790], [2507.09711], [2604.03787], [1609.06349], [2004.12511], and [2011.12823].

Source: https://www.emergentmind.com/topics/sinkhorn-knopp-matrix-scaling-algorithm