Matrix Scaling: Theory, Algorithms & Applications

• Matrix scaling is a process that adjusts a matrix’s rows and columns through diagonal multiplication to meet prescribed marginals, most notably achieving doubly stochastic matrices via Sinkhorn’s theorem.
• The Sinkhorn–Knopp algorithm iteratively normalizes rows and columns with geometric convergence for positive matrices, while alternative methods address challenges in sparse or ill-conditioned cases.
• Matrix scaling is applied in combinatorics, optimization, quantum information, and data normalization, serving as a practical tool for preconditioning and matching problems.

Matrix scaling is the process of pre- and post-multiplying a given matrix by diagonal matrices to enforce prescribed properties on its rows and columns, typically to achieve specified marginals such as all ones (the doubly stochastic case) or to equilibrate norms. The central mathematical paradigm is the Sinkhorn scaling: for a nonnegative matrix, finding two positive diagonal matrices such that the scaled matrix is doubly stochastic. Matrix scaling is foundational for network theory, optimization, data normalization, quantum information, and combinatorial enumeration, and encompasses a rich theory extending to noncommutative settings and broader norm constraints.

1. The Matrix Scaling Problem and Sinkhorn’s Theorem

Given a nonnegative matrix $A \in \mathbb{R}^{n \times n}$ and target marginals $r, c \in \mathbb{R}^n_{>0}$ with $\sum_i r_i = \sum_j c_j$, the equivalence scaling problem is to find positive diagonal matrices $D_1 = \operatorname{diag}(x)$ and $D_2 = \operatorname{diag}(y)$ so that the scaled matrix $B = D_1 A D_2$ satisfies $B \mathbf{1} = r$ and $B^T \mathbf{1} = c$. In the doubly stochastic case, $r = c = \mathbf{1}$.

Sinkhorn’s theorem establishes that:

• For $A > 0$, there exist unique (up to scalar) $D_1, D_2$ such that $D_1 A D_2$ is doubly stochastic.
• For general $A \geq 0$, existence and uniqueness of the scaling is characterized by the “pattern” (total support or full indecomposability); the solution is unique if and only if the pattern precludes a nontrivial block triangular form (Idel, 2016).

The system can be recognized as a coupled set of nonlinear equations or as an optimization problem with various convex and fixed-point formulations.

2. Algorithms: Sinkhorn–Knopp Iteration and Alternatives

The Sinkhorn–Knopp algorithm (also known as RAS or iterative proportional fitting) alternates row normalization and column normalization:

1. Given $B^{(0)} = A$, for $k = 0, 1, 2, \ldots$:
   • Row-scale: $$B^{(2k+1)}_{ij} = B^{(2k)}_{ij} \cdot (r_i / \sum_j B^{(2k)}_{ij})$$
   • Column-scale: $$B^{(2k+2)}_{ij} = B^{(2k+1)}_{ij} \cdot (c_j / \sum_i B^{(2k+1)}_{ij})$$

Convergence is geometric for positive matrices and is governed by contraction in Hilbert’s projective metric, with the rate determined by the spread of the matrix entries and the de facto spectral gap (Idel, 2016, Kwok et al., 2019). For sparse or ill-conditioned data, the rate can slow, and variants using Newton or interior point methods provide improved complexity (Apeldoorn et al., 2020, Gribling et al., 2021).

Algorithmic variants include:

• Projective decomposition: alternates scaling rows and columns to unit root-mean-square (RMS) norms to produce a canonical scale-invariant representative, ideal for ratio-scale data (Robinson, 2019).
• Approximate equilibration: stochastic or matrix-free updates based on randomized projections, especially for signed or large matrices (Bradley et al., 2011).
• Quantum algorithms: rely on amplitude estimation for marginal calculations, offering $\widetilde{O}(\sqrt{mn}/\varepsilon^4)$ complexity for $n \times n$ matrices with $m$ nonzeros, with lower bounds showing optimality for constant $\varepsilon$ (Apeldoorn et al., 2020, Gribling et al., 2021).

3. Theoretical Structure, Convergence, and Explicit Formulae

Matrix scaling admits multiple theoretical formulations:

• Fixed-point contraction: Iteration sequence contracts in Hilbert projective metric, leading to uniqueness and convergence (Idel, 2016, Kwok et al., 2019).
• Potential/entropy minimization: Each step reduces Kullback–Leibler (KL) divergence or a related convex potential (e.g., $$f(x,y) = \sum_{ij} A_{ij} e^{x_i+y_j} - r^T x - c^T y$$) (Apeldoorn et al., 2020).
• Geometric programming: Scaling as alternating minimization in a ratio-form functional, with duality linking to Hall’s Marriage Theorem and geometrically to network flows and polymatroids (Hayashi et al., 2022).
• Spectral analysis: Convergence is linear (exponential decay) under a spectral gap assumption on the matrix: if $\sigma_2(B) \leq (1-\delta) s(B)/\sqrt{mn}$, then the $l_2$-error decays as $$F(B^{(k)}) \leq F(B^{(0)})(1-\frac{1}{2}\delta s)^k$$ (Kwok et al., 2019).

For some $3 \times 3$ symmetric matrices, closed-form expressions for the Sinkhorn limit can be obtained via solving polynomial equations deriving from the scaling conditions (Nathanson, 2019, Nathanson, 2019). However, for $n \geq 4$, the degree becomes generically high, and one must resort to numerical iteration.

A notable aspect in certain parameter families is the possibility for the Sinkhorn algorithm to terminate in finitely many steps (e.g., two-parameter cases constructed with determinant zero); whether full-rank positive matrices can realize this for $n \geq 3$ is open (Nathanson, 2019).

4. Applications and Extensions

Matrix scaling has a wide range of applications across areas that require balanced representations or invariance to scaling:

• Combinatorics: Deciding the existence of a perfect matching in a bipartite graph via the zero–one bi-adjacency matrix; Sinkhorn iterations can efficiently certify perfect matchings or identify Hall blockers, connecting scaling to the combinatorics of the underlying graph (Hayashi et al., 2022).
• Optimization and Numerical Linear Algebra: Preconditioning linear systems, reducing condition number by equilibration of row and column norms, enabling efficient iterative solvers (Bradley et al., 2011).
• Quantum Information: Scaling positive maps or quantum channels to trace-preserving, unitality-constrained forms (non-commutative Sinkhorn scaling), featuring in quantum process tomography and tensor network simulation (Idel, 2016).
• Statistical Inference and Data Normalization: Normalizing contingency tables, adjustment for marginal totals in multiway frequency tables and network flow problems, as well as for ratio-preserving normalization of measurement data (Robinson, 2019).
• Permanent Estimation: Lower bounds by van der Waerden-type theorems, with the Sinkhorn scaling yielding structured input for approximation algorithms (Kwok et al., 2019).
• Principal Partition and Decomposition: Limiting behavior for nonscalable matrices relates to the Dulmage–Mendelsohn decomposition, polymatroids, and parametric network flow (Hayashi et al., 2022).
• Quantum Algorithms: Polynomial speedups for large but moderate-precision scaling using amplitude estimation, quantum Laplacian sparsification, and exploitation of block-diagonal structure (Apeldoorn et al., 2020, Gribling et al., 2021).

5. Generalizations and Open Problems

Matrix scaling extends in several directions:

• Pattern constraints: Zero–nonzero pattern (support) determines scalability conditions; for noncommutative settings, the analogous notion is rank non-decreasing maps.
• Norms and variants: Beyond $\ell_1$ (stochasticity), $\ell_2$-norm scaling and equilibration are studied, with stochastic approximation algorithms and structural tradeoffs (Bradley et al., 2011).
• Non-commutative scaling: Positive linear maps between matrix algebras, subject to trace and unitality conditions as in quantum information; scaling is possible if and only if capacity is positive and attained, uniqueness is characterized by indecomposability of the map (Idel, 2016).
• Unitary matrices: Heuristic and numerical evidence suggest a phase (unitary) variant of Sinkhorn scaling is possible, conjecturally extending unique scaling to unitaries with fixed line sums, relevant for quantum circuit decomposition (Vos et al., 2014).
• Terminating procedures: Classifying all positive matrices for which the Sinkhorn algorithm terminates in finitely many steps, particularly for $L > 1$, is unresolved (Nathanson, 2019).
• Complexity bounds and quantum limitations: Precise classical and quantum resource tradeoffs depend on accuracy regime; quantum speedups are provably optimal in most low-precision cases, but new methods may be required for faster high-precision scaling (Apeldoorn et al., 2020, Gribling et al., 2021).

6. Summary Table: Key Aspects of Matrix Scaling

Aspect / Method	Main Characterization	Reference
Existence/Uniqueness	Pattern (total support/indecomposability); unique in positive case	(Idel, 2016)
Algorithmic Paradigm	Sinkhorn–Knopp (RAS), Newton, IPM	(Idel, 2016, Gribling et al., 2021)
Rate/Complexity	Geometric convergence (positive case); polynomial/entropy bounds; spectral gap exponent decay	(Kwok et al., 2019, Apeldoorn et al., 2020)
Closed-form Solutions	$3\times3$ symmetric, two-value cases; general: numeric or Gröbner methods	(Nathanson, 2019, Nathanson, 2019)
Applications	Preconditioning, network flow, data normalization, quantum, matching	(Bradley et al., 2011, Hayashi et al., 2022)
Quantum Speedup	$\sqrt{mn}$ factor for moderate precision, lower bounds tight	(Apeldoorn et al., 2020, Gribling et al., 2021)
Extensions	Non-commutative, ratio-scale (projective), phase/unitary, tensor	(Idel, 2016, Robinson, 2019, Vos et al., 2014)

Matrix scaling continues to be a central problem at the interface of combinatorics, optimization, numerical linear algebra, and quantum theory, with ongoing research devoted to its computational complexity, structural generalizations, and domain-specific applications.