---
title: Non-negative Alternate Scaling (NASA)
url: https://www.emergentmind.com/topics/non-negative-alternate-scaling-algorithm-nasa
type: topic
---

# Non-negative Alternate Scaling (NASA)

The Non-negative Alternate Scaling Algorithm (NASA), also known as the Sinkhorn algorithm or the Sinkhorn-Knopp iteration, is a foundational class of iterative methods for transforming a non-negative (or strictly positive) matrix into a doubly stochastic matrix via alternating normalizations. Generalizations of NASA play a central role in scaling problems, optimal transport, matrix analysis, and algorithmic applications, including polynomial-time permanent approximation, design matrix analysis, statistical estimation, and regularized optimal transport. NASA encompasses both classical matrix scaling and modern extensions to general Bregman divergences and non-automatic non-negativity constraints.

## 1. Problem Formulation and Definitions

Given a non-negative matrix $A \in \mathbb{R}^{n \times n}_{\ge0}$, doubly stochasticity requires that all row and column sums equal 1:
\[
\sum_{j=1}^n A_{ij} = 1 \quad \forall i,\qquad \sum_{i=1}^n A_{ij} = 1 \quad \forall j
\]
A key functional to measure deviation from a doubly stochastic matrix is
\[
\operatorname{ds}(A) = \sum_{i=1}^n (r_i - 1)^2 + \sum_{j=1}^n (c_j - 1)^2
\]
where $r_i$ and $c_j$ denote the $i$-th row sum and $j$-th column sum, respectively.

The $\varepsilon$-scaling problem asks: for given $\varepsilon > 0$, find diagonal $B, C \succ 0$ such that $\operatorname{ds}(BAC) \le \varepsilon$ (where possible). Scalability—existence of such scaling for every $\varepsilon > 0$—is equivalent to positivity of the matrix permanent and to the bipartite support graph having a perfect matching [1808.09669].

For regularized optimal transport (ROT), the NASA is applied to solve problems of the form
\[
\min_{\pi \in \Pi(p,q)} \langle \pi, C \rangle + \lambda \varphi(\pi)
\]
where $\Pi(p,q)$ is the set of couplings (joint distributions) with prescribed marginals $p,q$, $C$ is a cost matrix, and $\varphi$ is a smooth convex regularizer, possibly not restricting the domain to the non-negative orthant [1610.06447].

## 2. Iterative Algorithmic Structure

The classical NASA iterates between affine projections that alternately normalize rows and columns:
- **Row-scaling:** For a matrix $B$, define
  \[
  (R(B))_{ij} = \frac{B_{ij}}{\sum_{j'} B_{ij'}}
  \]
- **Column-scaling:** Likewise,
  \[
  (C(B))_{ij} = \frac{B_{ij}}{\sum_{i'} B_{i'j}}
  \]
The update sequence is then:
\[
A^{(0)} = A; \quad A^{(2k+1)} = R(A^{(2k)}); \quad A^{(2k+2)} = C(A^{(2k+1)})
\]
This sequence converges to a unique doubly stochastic scaling $S(A)=D_rAD_c$, where $D_r,D_c$ are positive diagonals [1903.06778, 1902.04544].

For convex regularized transport, NASA generalizes to compute Bregman projections in dual coordinates. When the regularizer’s domain does not guarantee non-negativity, explicit projections onto the non-negative orthant are required. Corrections (Dykstra’s algorithm) are maintained to ensure convergence when projecting cyclically onto the constraints:
- $C_0 = \{\pi \ge 0\}$ (non-negativity)
- $C_1 = \{\pi 1 = p\}$ (row marginals)
- $C_2 = \{\pi^\top 1 = q\}$ (column marginals)
NASA cycles as $C_0 \to C_1 \to C_0 \to C_2 \to C_0$, with Newton-type inner steps for affine projections in the case of separable $\varphi$ [1610.06447].

## 3. Convergence Theory and Complexity

The classical Sinkhorn–Knopp theorem asserts that for every strictly positive $A \in \mathbb{R}^{n\times n}$, the NASA produces positive diagonal matrices $D_r,D_c$ such that $D_rAD_c$ is uniquely doubly stochastic. The iterative map is non-expansive in the Hilbert metric on the positive cone, and the process exhibits geometric (linear) convergence [1903.06778, 1902.04544].

In the entropy-regularized setting, convergence is seen as monotonic minimization of the Kullback–Leibler divergence $D(XAY \Vert A)$ over diagonal $(X,Y)$ scalings, and each iteration strictly decreases a potential. The number of NASA iterations needed to achieve $\operatorname{ds}(BAC)\le\varepsilon$ is $O(n (b + \log n)/\varepsilon)$, assuming the matrix entries have bit complexity $b$. Each iteration incurs $O(n^2)$ arithmetic operations [1808.09669].

For general Bregman divergences, under mild technical conditions (Legendre type, cofinite, full domain for conjugate, etc.), Dykstra’s algorithm ensures unique Bregman projection convergence at a linear rate (Theorem 2.1 of [1610.06447]). For separable $\varphi$, each outer iteration has $O(d^2)$ complexity, dominated by vectorized Newton updates. For non-separable regularizers (e.g., Mahalanobis), complexity grows to $O(d^4)$. Empirical studies confirm practical quadratic scaling for moderate problem sizes [1610.06447].

## 4. Algorithmic Pseudocode and Procedures

The concrete NASA scheme for matrix scaling (Sinkhorn’s algorithm) is as follows [1808.09669, 1903.06778]:
1. If any row or column of $A$ is all-zero, terminate and report “not scalable.”
2. Initialize $B\gets I_n$, $C\gets I_n$.
3. For $t=0,1,\dots,T-1$:
   - Compute $M=BAC$ and $\operatorname{ds}(M)$.
   - If $\operatorname{ds}(M)\le\varepsilon$, return $(B,C)$.
   - If $\sum_i (r_i(M)-1)^2>\varepsilon/2$, set $B \gets \operatorname{diag}(1/r_1(M),...,1/r_n(M)) \cdot B$; else $C \gets C \cdot \operatorname{diag}(1/c_1(M),...,1/c_n(M))$.
4. Output “not scalable” if no feasible scaling is found.

For regularized optimal transport:
- NASA performs in the dual parameterization $\theta=\nabla\varphi(\pi)$, maintaining correction variables for each constraint.
- Each cycle consists of projections (generally using Newton–Raphson for affine marginals), interleaved with non-negativity projections applied elementwise [1610.06447].

Pseudocode details for both cases are explicitly provided in [1808.09669] and [1610.06447].

## 5. Applications and Illustrative Use Cases

The NASA has a spectrum of applications across mathematics, statistics, and machine learning:
- **Permanent Approximation:** Any $A$ with $\operatorname{perm}(A)>0$ can be nearly doubly-stochastically scaled. Since $\operatorname{perm}(D)$ for a doubly stochastic $D$ satisfies $\frac{n!}{n^n} \leq \operatorname{perm}(D) \leq 1$, NASA delivers a deterministic $e^n$-approximation to $\operatorname{perm}(A)$ in polynomial time [1808.09669].
- **Rank Bounds in Design Matrices:** Scaling a design matrix to (near) doubly stochasticity provides lower bounds on matrix rank, crucial for quantitative Sylvester–Gallai theorems and improved bounds in combinatorial geometry [1808.09669].
- **Maximum-Entropy Estimation:** The scaling problem is equivalent to entropy-regularized convex optimization for contingency tables and joint distribution estimation, with NASA iterations realized as block coordinate minimizations [1808.09669, 1610.06447].
- **Regularized Optimal Transport (ROT):** In problems where regularization is not entropic, NASA enables efficient solution via explicit alternated projections, supporting sparsity, additional constraints, and richer regularization [1610.06447]. Experiments include synthetic 1D distributions (Gaussian and bimodal), runtime comparisons for various regularizers, and real-world audio scene classification with improved classification rates using the NASA-optimized kernel [1610.06447].

## 6. Special Cases, Finite-Termination Phenomena, and Limitations

Classically, NASA achieves convergence only asymptotically. Recent work constructs explicit families of $n\times n$ positive matrices that terminate after finitely many steps: for each $n\ge 3$, the family of non-invertible, row-stochastic but not column-stochastic matrices constructed by Nathanson become doubly stochastic after a single column-scaling (yet no earlier) [1903.06778]. All such examples have determinant zero, and it is open whether a strictly positive, invertible $A$ can have finite NASA termination.

Further, explicit Sinkhorn limits have been calculated for all symmetric $3\times 3$ matrices with two values, providing links to Diophantine approximation and sequences of rational numbers converging to algebraic irrationals via NASA iterates [1902.04544].

## 7. Practical Implementations and Advanced Considerations

NASA is well-suited for high-dimensional, structured, or sparse data due to its operation via vectorized row/column-wise updates, which are GPU/parallel-friendly [1610.06447]. For problems with infinite cost entries, NASA naturally restricts computation to active submatrices. Choice of regularization controls solution smoothness and sparsity, with larger $\lambda$ promoting faster convergence and increased smoothing. Termination criteria are application-dependent, involving either direct error in margin deviation or iterations on dual variables.

In the separable case, practical performance matches the theoretical $O(d^2)$ scaling, outperforming classical, combinatorial Earth Mover’s Distance solvers (which are super-cubic). For non-separable $\varphi$, NASA remains applicable but computationally more intensive.

NASA provides a unifying approach to matrix scaling and regularized transport, extending the reach of classical iterative scaling to general convex frameworks and broad application domains [1808.09669, 1610.06447, 1903.06778, 1902.04544].

Source: https://www.emergentmind.com/topics/non-negative-alternate-scaling-algorithm-nasa