Sinkhorn Algorithm and Matrix Scaling

• The Sinkhorn algorithm is an iterative procedure for scaling nonnegative matrices to near-doubly stochastic form by alternately normalizing rows and columns.
• It underpins applications in optimal transport, image processing, and graph matching, capitalizing on rapid empirical convergence in dense matrices.
• The analysis reveals a sharp phase transition driven by the density parameter, with exponential convergence in high-density regimes and slower rates for sparse inputs.

The Sinkhorn algorithm is an iterative scaling procedure for transforming a nonnegative square matrix into a (nearly) doubly stochastic matrix—one where all row and column sums are 1—by alternately normalizing rows and columns. Originally introduced for the matrix scaling problem, it underpins modern computational optimal transport via its capacity to efficiently and robustly solve entropically regularized mass transportation problems. The recent theoretical analysis in "Phase transition of the Sinkhorn-Knopp algorithm" (He, 13 Jul 2025) provides tight iteration complexity bounds and uncovers a sharp phase transition in iteration behavior governed by a structural density parameter, which explains the robust practical performance observed in applied contexts.

1. Matrix Scaling and the Sinkhorn–Knopp Iterates

The Sinkhorn–Knopp algorithm operates on a nonnegative matrix $A \in \mathbb{R}^{n \times n}$ by alternately rescaling rows and columns to achieve prescribed sums (typically unity, for doubly stochasticity). At each iteration, row scaling is performed to set all row sums to 1, followed by column scaling to set all column sums to 1. Formally, one seeks diagonal matrices $X$, $Y$ such that $B = XAY$ is nearly doubly stochastic. The discrete process is continued until the maximum residual deviation from the doubly stochastic property is below a pre-set threshold $\varepsilon$.

This algorithm is foundational not only in classical matrix scaling (He, 13 Jul 2025), but also as the core iteration for solving the entropically regularized optimal transport problems prevalent in computational imaging, deep learning, and combinatorial optimization.

2. Practical Performance and Density-driven Regimes

Despite the pseudopolynomial theoretical iteration bound, the Sinkhorn–Knopp algorithm exhibits extremely fast empirical convergence in applications such as dense graph matching, statistical data normalization, image transport, and preconditioning. The key insight, rigorously established, is that these problem instances typically yield dense matrices—meaning that each row and column of the normalized input has a significant fraction $\gamma$ of its entries exceeding a uniform positive threshold $\rho$. This density structure ensures favorable convergence properties absent in worst-case sparse inputs.

Such empirical success arises because, under high density, scaling vectors remain well-conditioned and stabilization of row and column sums proceeds geometrically.

3. Sharp Iteration Complexity and the Density Phase Transition

The iteration bound for Sinkhorn–Knopp is precisely characterized in terms of the matrix's density parameter $\gamma$:

• High-density regime ($\gamma > 1/2$): For nonnegative $n \times n$ matrices whose normalized version has density $\gamma > 1/2$ (i.e., each row/column has at least $\lceil \gamma n \rceil$ entries $\geq \rho$), the number of iterations required to attain $\varepsilon$-approximate doubly stochasticity is $O(\log n - \log \varepsilon)$. Each iteration incurs $O(n^2)$ arithmetic operations, yieldings overall complexity $\widetilde{O}(n^2)$, which is optimal in this regime.
• Low-density regime ($\gamma < 1/2$): There exist matrices—constructed explicitly in the analysis—for which convergence is slow: the number of iterations is $\Omega(n/\varepsilon)$ in the $\ell_1$-norm or $\widetilde{\Omega}(n^{1/2}/\varepsilon)$ in the $\ell_2$-norm.

This establishes a phase transition at $\gamma = 1/2$; above this, the algorithm admits exponentially fast convergence in iteration count (logarithmic in both $n$ and $1/\varepsilon$), while below this density the convergence rate deteriorates dramatically.

Density $\gamma$	Iterations (Upper Bound)	Iterations (Lower Bound)
$\gamma > 1/2$	$O(\log n - \log \varepsilon)$
$\gamma < 1/2$		$\Omega(n/\varepsilon)$ ($\ell_1$), $\widetilde{\Omega}(n^{1/2}/\varepsilon)$ ($\ell_2$)

The derivation employs combinatorial analysis tracking the propagation of non-negligible entries throughout the scaling process and leverages inequalities for the evolution of scaling factors.

4. Theoretical Tightness and Extremal Constructions

The provided bounds are shown to be tight up to logarithmic factors. For $\gamma > 1/2$, convergence in $O(\log n - \log \varepsilon)$ is optimal: no faster algorithm can exist for producing a nearly stochastic scaling in all such matrices, as every entry must be "touched" in $\widetilde{O}(n^2)$ total operations. For $\gamma < 1/2$, explicit matrix constructions demonstrate that the lower bound is achieved, solidifying the sharp phase transition at $\gamma = 1/2$ as a theoretical barrier for geometric convergence.

5. Significance for Applications and Algorithm Engineering

The density-based classification provides a direct explanation for the frequent observation of rapid convergence—logarithmic in both input size and numerical error—in applications involving the Sinkhorn–Knopp algorithm. Dense cost structures are ubiquitous in entropic optimal transport, image and graph problems, and permanent approximation algorithms. For these, the scaling method yields robust, scalable results matching the theoretical optimum.

Conversely, in settings where the matrix density falls below the threshold (e.g., highly sparse matrices), the performance is provably degraded, guiding practitioners to consider alternative preprocessing or scaling methods.

6. Mathematical Formulation of the Density Parameter and Scaling Evolution

The density parameter $\gamma$ is formally defined: a normalized $n \times n$ matrix has density $\gamma$ if there exists $\rho > 0$ such that at least $\lceil \gamma n \rceil$ entries in each row and column are at least $\rho$ (He, 13 Jul 2025). The analysis traces the evolution of the scaling through inequalities such as $1 + x \leq \exp(x)$, and via tracking the minimal and maximal values of scaling vectors, ultimately leading to the precise iteration complexity estimates.

7. Implications and Extensions

The identification of the phase transition at $\gamma = 1/2$ not only affirms the algorithm's suitability for dense problems but also provides a foundation for designing new algorithms tailored for either dense or sparse regimes. Furthermore, these findings have direct consequences for approximation algorithms for the permanent of dense $0$–$1$ matrices and broadly inform matrix scaling subroutines in linear algebra and combinatorial optimization workflows.

In summary, the Sinkhorn–Knopp algorithm exhibits a phase transition in convergence rate, governed by the matrix density parameter $\gamma$, and is both theoretically and practically optimal in the high-density regime prevalent in real-world applications (He, 13 Jul 2025).