Alternate Scaling Algorithm Overview

• ASA is an iterative algorithm that alternates row and column normalization to convert a positive matrix into its unique doubly stochastic form, known as the Sinkhorn limit.
• It generalizes to operator and tensor scaling, providing effective methods for applications in combinatorics, optimization, and quantum information.
• Convergence is guaranteed by the Sinkhorn–Knopp theorem with geometric progress in the Hilbert metric and polynomial complexity for approximate computations.

The Alternate Scaling Algorithm (ASA) is a class of alternating minimization procedures that transform an initial array or tensor, typically a positive matrix, toward a canonical “doubly stochastic” form by iterative normalization of its marginals. In the classical matrix case, ASA alternately rescales the rows and columns so that their sums become uniform, producing a sequence converging to the unique doubly stochastic matrix associated to the original, known as its Sinkhorn limit. ASA generalizes naturally to operator tuples and tensors, playing a central role in matrix, operator, and tensor scaling. The approach provides both foundational theory—establishing existence, uniqueness, and convergence—and effective algorithms underpinning computational applications in combinatorics, optimization, complexity theory, quantum information, and beyond (Nathanson, 2019, Garg et al., 2018).

1. Iterative Procedure and Scaling Operators

Consider a positive matrix $A \in \mathbb{R}^{n \times n}$ with $a_{ij} > 0$. ASA involves two alternating “scaling” steps applied to a sequence $A^{(k)}$:

• The row-scaling matrix $X(M)$ for a matrix $M$ is defined as

$$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$

Left-multiplying $M$ by $X(M)$ normalizes its rows to sum to $1$.

• The column-scaling matrix $Y(M)$ is defined as

$a_{ij} > 0$0

Right-multiplying $a_{ij} > 0$1 by $a_{ij} > 0$2 normalizes its columns to sum to $a_{ij} > 0$3.

The iterative process proceeds as

$a_{ij} > 0$4

or, in recursion,

$a_{ij} > 0$5

This limiting process generalizes to operator scaling (acting on tuples of matrices by invertible transformations) and tensor scaling (acting along different modes), following a shared alternating-normalization template (Nathanson, 2019, Garg et al., 2018).

2. Convergence Guarantees and Theoretical Foundations

The convergence of ASA for strictly positive matrices is governed by the Sinkhorn–Knopp theorem:

• For $a_{ij} > 0$6, there exist unique positive diagonal matrices $a_{ij} > 0$7 such that $a_{ij} > 0$8 is doubly stochastic—i.e., all row sums and column sums are $a_{ij} > 0$9.
• Iterates $A^{(k)}$0 of ASA converge:

$A^{(k)}$1

and this limit depends only on $A^{(k)}$2.

The convergence mechanisms can be understood as:

• Alternating minimization of the Kullback–Leibler divergence, with constraints imposed (row or column sums fixed at each step).
• Contractivity in the Hilbert projective metric: each scaling step is strictly contractive on the cone of positive matrices.

For nonnegative matrices (with possible zero-entries), additional feasibility checks are required: for instance, existence of a perfect matching (permanent $A^{(k)}$3) is necessary for matrix scalability.

Generalizations for operator and tensor scaling exploit invariant-theoretic potential functions. The algorithm's progress can be quantified via a polynomial potential $A^{(k)}$4 associated to invariants under reductive group actions, ensuring that convergence occurs in a number of alternating steps polynomial in the problem size and data bitlength (Nathanson, 2019, Garg et al., 2018).

3. Exact Finitely-Terminating Cases and Families

In generic cases with $A^{(k)}$5, ASA does not exactly terminate in finitely many steps; only convergence in the limit is guaranteed. However, explicit families exist that reach a doubly stochastic matrix after a single scaling. The notable construction by Nathanson provides, for each $A^{(k)}$6, a two-parameter family of row-stochastic but not column-stochastic positive matrices $A^{(k)}$7 that become doubly stochastic after one column-scaling. Key features:

• The matrices have a block structure:
  • Rows $A^{(k)}$8 to $A^{(k)}$9: $X(M)$0
  • Rows $X(M)$1 to $X(M)$2: $X(M)$3
  • Rows $X(M)$4 to $X(M)$5: $X(M)$6
  • where $X(M)$7 and $X(M)$8; $X(M)$9 are determined accordingly.
• Every row sums to $M$0 but column sums are nonuniform. Application of $M$1—a diagonal matrix scaling columns—produces $M$2 whose every row and column sums to $M$3, yielding exact doubly stochasticity in a single step.
• All explicitly constructed matrices in this family have determinant zero.

In the $M$4 case, a full classification shows at most two scalings are ever required; all one-step exact cases are characterized (Nathanson, 2019).

4. Algorithmic Implications, Complexity, and Rate of Convergence

General ASA, except for rare structured cases, typically requires infinitely many alternating steps to reach the exact doubly stochastic limit. There is no uniform finite upper bound $M$5 such that every $M$6 positive matrix is transformed in at most $M$7 steps.

For approximate stopping, practitioners measure deviation of row and column sums from $M$8—using metrics such as the infinity-norm or Frobenius norm of the marginal errors,

$M$9

and halt when a prescribed threshold $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$0 is reached.

Convergence is geometric (linear) in the Hilbert metric, with rate determined by the spread of matrix entries (ratio of largest to smallest) and the support pattern. Matrices closer to being doubly stochastic or with entries of similar magnitude converge more rapidly.

In matrix scaling, each iteration is $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$1; total time for $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$2-approximation is $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$3 for bitlength $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$4 (Garg et al., 2018). Operator and tensor scaling generalizations have analogous polynomial complexity estimates, with details depending on tensor dimensions and marginal computation costs.

5. Extensions, Applications, and Open Problems

ASA underpins a spectrum of fundamental and applied problems:

• Permanent estimation: Deterministic $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$5-approximation via Sinkhorn scaling, leveraging van der Waerden bounds for doubly stochastic matrices.
• Perfect matching testing: Matrix scalability is equivalent to positive permanent; operator scalability yields deterministic polynomial-time solutions to the non-commutative Edmonds problem.
• Design-matrix lower bounds: Matrix and operator scaling yield new rank lower bounds critical to incidence geometry.
• Brascamp–Lieb polytope feasibility: Operator scaling provides efficient membership and separation oracles for these convex sets, generalizing results in communication complexity.
• Quantum information: Tensor scaling describes stochastic local operations on multipartite systems, relevant to entanglement distillation.

Numerical performance is robust: for moderate $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$6, convergence (to high precision) is achieved rapidly in practice both for matrices and higher-order generalizations (Garg et al., 2018).

Open directions include:

• Existence of row-stochastic, non-column-stochastic $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$7 matrices with $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$8 yielding doubly stochasticity after one scaling.
• Classification of positive $$X(M) = \operatorname{diag}\left( \frac{1}{\sum_{j=1}^n m_{1j}}, \dots, \frac{1}{\sum_{j=1}^n m_{nj}} \right).$$9 matrices requiring a prescribed finite number $M$0 of alternations to terminate exactly.
• Inverse problems and rational-entry versions: characterizing all $M$1 such that row-scaling $M$2 produces a given $M$3. These questions remain unresolved, especially in low dimensions beyond $M$4 (Nathanson, 2019).

6. Generalizations and Unified Framework

ASA is the prototypical example of an alternating normalization (minimization) algorithm, with extensions across mathematical structures:

• Operator scaling: Alternates left and right normalization of a tuple of matrices, targeting approximate operator-stochasticity (marginals equal to the identity).
• Tensor scaling: Sequentially normalizes along $M$5 modes, achieving approximate $M$6-stochasticity. A unified analysis, based on invariant-theoretic potential functions, demonstrates convergence in all settings for scalable inputs, and quantifies step-wise progress by potential improvements, establishing polynomial iteration bounds in the relevant parameters (Garg et al., 2018).

ASA thus serves as a unifying methodology in the analysis of scaling-type problems, blending aspects of optimization, algebraic invariants, and computational tractability.