---
title: Sinkhorn–Knopp Centering Process
url: https://www.emergentmind.com/topics/sinkhorn-knopp-centering-process
type: topic
---

# Sinkhorn–Knopp Centering Process

The Sinkhorn–Knopp centering process—also known as matrix balancing, matrix scaling, or row/column normalization—is an iterative algorithm to compute positive diagonal scalings of a nonnegative matrix so that the resulting matrix achieves prescribed row and column marginals. It is fundamental to entropic regularization for discrete optimal transport, matrix balancing, and doubly stochasticization problems. The process combines principles from convex optimization, geometric analysis, and dynamical systems, and exhibits rapid convergence under broad regularity conditions [2309.09089].

## 1. Problem Statement and Matrix-Scaling Formulation

Given a strictly positive matrix \(K \in \mathbb{R}_+^{n \times n}\) and two strictly positive vectors \(a, b \in \mathbb{R}^n_{++}\), the Sinkhorn–Knopp problem is to find diagonal scaling matrices \(\operatorname{diag}(u)\) and \(\operatorname{diag}(v)\) such that
\[
P = \operatorname{diag}(u) K \operatorname{diag}(v)
\]
satisfies
\[
P \mathbb{1} = a, \qquad P^\top \mathbb{1} = b
\]
that is, the row sums of \(P\) equal \(a\) and the column sums equal \(b\) [2309.09089, 1710.06635, 1306.0895]. In component form,
\[
u_i K_{ij} v_j
\]
must sum to \(a_i\) over \(j\) and to \(b_j\) over \(i\). This formulation includes the classic doubly stochastic scaling as the special case \(a = b = \mathbb{1}_n\).

The process also arises in entropic optimal transport, where the problem is to minimize the cost
\[
\langle C, P \rangle + \varepsilon \sum_{i,j} P_{ij} (\log P_{ij} - 1)
\]
subject to prescribed marginals, with \(K = e^{-C/\varepsilon}\) as the Gibbs kernel [1710.06635, 1306.0895].

## 2. Iterative Centering Algorithm

The Sinkhorn–Knopp iteration alternates between normalizing to enforce the row and column marginal constraints. In vector form, the updates are
\[
u^{(k+1)} = a \,/\, (K v^{(k)}), \qquad v^{(k+1)} = b \,/\, (K^\top u^{(k+1)})
\]
where division is entrywise [2309.09089, 1710.06635, 1801.02790]. The initial vectors \(u^{(0)}\) and \(v^{(0)}\) can be set to all-ones or any strictly positive values, which ensures positivity is preserved throughout the iterations.

Each iterate \(P^{(k)} = \operatorname{diag}(u^{(k)}) K \operatorname{diag}(v^{(k)})\) alternately matches the row or column marginals, and the iteration converges to a unique scaling (up to rescaling) when \(K > 0\) and \((a, b) > 0\).

Pseudocode of the classical Sinkhorn–Knopp process is as follows:

```python
# Inputs: K, a, b, tolerance tau
u = np.ones(n)
v = np.ones(n)
while not converged:
    u = a / (K @ v)
    v = b / (K.T @ u)
# Stopping: e.g., max(norm(P 1 - a, inf), norm(P.T 1 - b, inf)) < tau
P = np.diag(u) @ K @ np.diag(v)
```
[2309.09089, 1710.06635]

## 3. Geometric and Dynamical Interpretation

The Sinkhorn–Knopp algorithm can be viewed geometrically as alternating Bregman projections (specifically, Kullback–Leibler projections) onto the affine sets of matrices with prescribed row and then column sums within the transport polytope \(\{P \ge 0 \mid P 1 = a,\, P^T 1 = b\}\) [2309.09089, 1711.01851].

From a continuous perspective, the Sinkhorn iteration discretizes a coupled nonlinear gradient flow on densities—specifically, a time-splitting of the integral equations
\[
\partial_s a = -a \log\left( a\, e^{\varepsilon \Delta} b / \rho_0 \right), \qquad \partial_s b = -b \log\left( b\, e^{\varepsilon \Delta} a / \rho_1 \right)
\]
with \(e^{\varepsilon\Delta}\) representing heat kernel convolution and \(\rho_0, \rho_1\) the prescribed marginals in continuous space [2309.09089].

Under the Benamou–Brenier entropic regularization, a dynamical (continuous-time ODE) formulation yields
\[
\dot a = \varepsilon \Delta a, \qquad \dot b = \varepsilon \Delta b
\]
and Lie–Trotter splitting (with explicit Euler discretization) matches the discrete SK updates [2309.09089].

The projection viewpoint connects the SK process to alternating minimization of KL divergence and underpins interpretability in terms of metric contraction and convex geometry [1801.02790, 1711.01851].

## 4. Convergence Analysis and Rate

Sinkhorn and Knopp originally proved that, for any positive \(K\) and positive marginals, the alternated diagonal scaling converges to a unique solution [2309.09089, 1902.04544]. The convergence is geometric: the error in the marginals (as measured by, e.g., \(\ell_\infty\), \(\ell_1\), or Kullback–Leibler divergence) contracts at a rate determined by the Hilbert projective metric contraction ratio for the scaling map [2309.09089, 1801.02790].

The precise iteration complexity depends on structure:
- For dense, well-bounded, or high-density matrices (\(\gamma > 1/2\)), the iteration count is \(O(\log n - \log \varepsilon)\) to reach error \(\varepsilon\) [2507.09711, 2604.03787].
- For positive matrices with lower density or poorly bounded entries, lower bounds of \(\Omega(\sqrt n/\varepsilon)\) or even \(\Omega(n/\varepsilon)\) can be attained [2507.09711].
- The contraction is monotonic for a suitable potential function—either a Lyapunov energy or the KL divergence of the current marginal [2309.09089, 1801.02790].

A summary of bounds appears below:

| Error Norm | Iteration Bound (dense)     | Reference                |
|------------|-----------------------------|--------------------------|
| $\ell_1$   | $O(n^2 \varepsilon^{-2} \log(\cdot))$ | [1801.02790]      |
| $\ell_2$   | $O(n (\varepsilon^{-1} + \varepsilon^{-2}) \log(\cdot))$ | [1801.02790] |
| General $\gamma > 1/2$ | $O(\log n - \log \varepsilon)$ | [2507.09711]      |
| Sparse/low-density | $\Omega(n/\varepsilon)$ | [2507.09711]              |

Over-relaxation techniques, including adaptive selection of the relaxation parameter, yield further iteration count reductions, particularly in the regime where the spectral gap of the associated operator is small [1711.01851].

## 5. Applications and Extensions

The Sinkhorn–Knopp centering process has broad applications:

- **Entropic Optimal Transport**: Directly computes regularized transport plans with prescribed marginals, minimizing a combined cost and entropy objective [1306.0895, 1710.06635].
- **Preconditioning and Matrix Balancing**: Used to transform matrices into forms suitable for fast solvers or numerical stability, e.g., producing a doubly stochastic matrix from a positive input [1801.02790, 1902.04544].
- **Machine Learning and Data Analysis**: Fundamental for efficient computation of Sinkhorn distances between histograms or feature vectors [1306.0895].
- **Image Processing and Denoising**: Underpins normalization in Gaussian mixture model-based smoothing filters; alternates EM (E-step = column normalization, M-step = row normalization) for statistical interpretation [1601.00088].
- **Quantum Computing**: The conjectured analog involves scaling unitary matrices via phase correction to achieve prescribed line sums; with implications for quantum gate decompositions [1401.7883].
- **Constrained Transport and Prior Zeros**: Extensions handle sparse plans with enforced zeros by setting forbidden entries in $K$ to zero, preserving Bregman projection structure and convergence [2404.00003].
- **Acceleration and High-Performance Computing**: Arnoldi-type and Newton accelerations exploit the underlying nonlinear eigenproblems for further speedup in challenging regimes [1810.09700, 1710.06635].

## 6. Measure-Theoretic and Infinite-Dimensional Generalizations

Beurling’s theorem gives a measure-theoretic generalization of the Sinkhorn–Knopp process, establishing uniqueness of scaling solutions for Radon measures on continuous spaces \(M \times M\) via positive integral kernels [2309.09089]. The scaling of measures (as opposed to finitely supported vectors) generalizes the algorithm to infinite dimensions with well-posedness and stability results.

Under appropriate kernel positivity, the map from arbitrary nonnegative measures to product measures is a homeomorphism, and the scaling process inherits uniqueness and continuity properties with respect to data perturbations.

## 7. Practical Considerations

Each Sinkhorn–Knopp iteration has computational cost \(O(n^2)\) for dense \(n \times n\) matrices, dominated by matrix-vector multiplication \(Kv\) and \(K^T u\). For kernel matrices arising from structured cost (e.g., Gaussian or Toeplitz), FFT-based or fast transform techniques can reduce per-iteration complexity [2309.09089].

Stability is a concern when the regularization parameter $\varepsilon$ is small, as the kernel $K_{ij} = \exp(-C_{ij}/\varepsilon)$ can underflow to numerically zero. Log-domain stabilizations (operating on $\log u$, $\log v$) and log-sum-exp formulas are standard practice [1710.06635].

Stopping criteria are typically based on violation of the marginal constraints in $\ell_\infty$ or relative terms, with tolerances of $10^{-6}$–$10^{-9}$ common in applications [2309.09089, 1306.0895]. When mass cannot be matched (e.g., in the presence of prior zeros), unbalanced variants add penalty terms with competitive convergence properties [2404.00003].

Overrelaxation and acceleration are crucial for ill-conditioned or nearly decomposable matrices, and recent advances establish adaptive step selection schemes with global Lyapunov guarantees [1711.01851]. Arnoldi and Newton-type eigen-acceleration realizes dramatic gains for hard instances with clustered spectrum [1810.09700, 1710.06635].

## References

- "On the geometry and dynamical formulation of the Sinkhorn algorithm for optimal transport" [2309.09089]
- "A Sinkhorn-Newton method for entropic optimal transport" [1710.06635]
- "Better and Simpler Error Analysis of the Sinkhorn-Knopp Algorithm for Matrix Scaling" [1801.02790]
- "Overrelaxed Sinkhorn-Knopp Algorithm for Regularized Optimal Transport" [1711.01851]
- "Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances" [1306.0895]
- "Understanding Symmetric Smoothing Filters: A Gaussian Mixture Model Perspective" [1601.00088]
- "On the Efficiency of Sinkhorn-Knopp for Entropically Regularized Optimal Transport" [2604.03787]
- "Accelerating the Sinkhorn-Knopp iteration by Arnoldi-type methods" [1810.09700]
- "Scaling a unitary matrix" [1401.7883]
- "Phase transition of the Sinkhorn-Knopp algorithm" [2507.09711]
- "Matrix scaling, explicit Sinkhorn limits, and arithmetic" [1902.04544]
- "Algorithms for constrained optimal transport" [2404.00003]

Source: https://www.emergentmind.com/topics/sinkhorn-knopp-centering-process