Post-Hoc Matrix Scaling

• Post-Hoc Matrix Scaling is a family of methods that decomposes matrices and classifier outputs into diagonal scale factors and a normalized core to preserve ratio invariants.
• The techniques include diagonal equilibration, classwise multiplicative scaling, and full softmax recalibration to address numerical conditioning and calibration challenges.
• Practical implementations use iterative Sinkhorn-Knopp algorithms and stochastic gradient updates to efficiently reduce condition numbers and avoid overfitting.

Post-hoc matrix scaling denotes a family of procedures that rescale an already existing matrix, data table, or fixed classifier output after the primary measurements or model have been produced. In the supplied literature, the term covers at least three distinct but related constructions: diagonal row/column rescaling of a matrix to equalize norms or isolate a scale-invariant representative, classwise multiplicative scaling of a pretrained classifier under prior shift, and multiclass logistic recalibration with a learned weight matrix acting on logits or log-probabilities (Robinson, 2019, Bradley et al., 2011, Wei et al., 2023, Berta et al., 5 Nov 2025). The shared motif is that the underlying object is held fixed and only a post-hoc scaling transformation is learned or computed.

1. Conceptual scope and formal patterns

A recurring formal pattern is the factorization of an object into scale factors and a normalized core. For a real $m\times n$ data matrix $A$, projective decomposition seeks

$A = D_r\,S\,D_c,$

where $D_r=\mathrm{diag}(r_1,\dots,r_m)$ and $D_c=\mathrm{diag}(c_1,\dots,c_n)$ are diagonal matrices of positive row- and column-scale factors, and $S$ is an $m\times n$ scale-normalized matrix whose rows and columns each have unit root-mean-square (RMS) (Robinson, 2019). In numerical linear algebra, equilibration uses the same diagonal-scaling template,

$\hat A = D_r A D_c,$

but the normalization target is approximate equality of row- and column-$2$-norms rather than preservation of ratio structure (Bradley et al., 2011).

In post-hoc classification under prior shift, scaling is applied not to matrix entries directly but to a pretrained probability vector $\hat\eta(x)\in\Delta_m$. A nonnegative scaling vector $A$0 defines

$A$1

or equivalently a logit shift $A$2 (Wei et al., 2023). In multiclass calibration, a more expressive post-hoc recalibrator applies a full softmax layer to centered log-probabilities,

$A$3

with $A$4, $A$5, and $A$6 (Berta et al., 5 Nov 2025).

These formulations are not interchangeable. Diagonal matrix scaling, classwise vector scaling, and full matrix scaling act on different objects and preserve different invariants. A plausible implication is that “matrix scaling” is best understood as a structural family of post-hoc rescaling methods rather than as a single algorithm.

2. Projective decomposition and scale-invariant normalization

Projective decomposition treats a data matrix as defining a relationship between information labeling rows and information labeling columns, and normalizes both organizations simultaneously rather than standardizing only one axis (Robinson, 2019). The canonical form $A$7 is defined by unit RMS constraints on every row and every column: $A$8 with RMS operators

$A$9

The associated equivalence relation is “equivalent up to scale.” Two matrices $A = D_r\,S\,D_c,$0 and $A = D_r\,S\,D_c,$1 of the same size $A = D_r\,S\,D_c,$2 are equivalent up to scale if there exist strictly positive vectors $A = D_r\,S\,D_c,$3 and $A = D_r\,S\,D_c,$4 such that

$A = D_r\,S\,D_c,$5

equivalently $A = D_r\,S\,D_c,$6. The set of all decompositions $A = D_r\,S\,D_c,$7 with positive $A = D_r\,S\,D_c,$8 yields an equivalence class of matrices that differ only by row and column dilations, and $A = D_r\,S\,D_c,$9 is the canonical representative once the unit-RMS constraints are imposed (Robinson, 2019).

A central property is preservation of pairwise relative ratios. Writing $D_r=\mathrm{diag}(r_1,\dots,r_m)$0, one obtains, for any two distinct rows $D_r=\mathrm{diag}(r_1,\dots,r_m)$1 and two distinct columns $D_r=\mathrm{diag}(r_1,\dots,r_m)$2,

$D_r=\mathrm{diag}(r_1,\dots,r_m)$3

The source text states this in words as preservation of every within-row or within-column ratio in $D_r=\mathrm{diag}(r_1,\dots,r_m)$4, so that $D_r=\mathrm{diag}(r_1,\dots,r_m)$5 is “ratio-scale invariant” (Robinson, 2019). This distinguishes projective decomposition from z-transformation, which enforces zero mean and unit standard deviation down each column, does nothing to the rows, and alters ratio structure because it subtracts column means.

Computation proceeds by an iterative proportional-scaling algorithm of Sinkhorn-Knopp type adapted to $D_r=\mathrm{diag}(r_1,\dots,r_m)$6 norms. Starting from $D_r=\mathrm{diag}(r_1,\dots,r_m)$7, $D_r=\mathrm{diag}(r_1,\dots,r_m)$8, and $D_r=\mathrm{diag}(r_1,\dots,r_m)$9, one alternates row normalization and column normalization until the maximum deviations of row and column RMS values from $D_c=\mathrm{diag}(c_1,\dots,c_n)$0 fall below a tolerance. For sufficiently supported matrices, standard results adapted to $D_c=\mathrm{diag}(c_1,\dots,c_n)$1 guarantee existence of the limit and uniqueness up to the one-dimensional diagonal scale symmetry (Robinson, 2019).

3. Diagonal equilibration in numerical linear algebra

In numerical linear algebra, post-hoc matrix scaling usually means equilibration: finding diagonal matrices $D_c=\mathrm{diag}(c_1,\dots,c_n)$2 and $D_c=\mathrm{diag}(c_1,\dots,c_n)$3 so that the scaled matrix $D_c=\mathrm{diag}(c_1,\dots,c_n)$4 has row and column $D_c=\mathrm{diag}(c_1,\dots,c_n)$5-norms that are equal, or approximately equal (Bradley et al., 2011). For signed matrices, the problem is conveniently transferred to the nonnegative matrix

$D_c=\mathrm{diag}(c_1,\dots,c_n)$6

and one seeks positive vectors $D_c=\mathrm{diag}(c_1,\dots,c_n)$7, $D_c=\mathrm{diag}(c_1,\dots,c_n)$8 such that $D_c=\mathrm{diag}(c_1,\dots,c_n)$9 is doubly stochastic, where $S$0 and $S$1. Once these are found, one sets

$S$2

The classical exact formulation is Sinkhorn-Knopp iteration. If $S$3 has total support, the iteration

$S$4

converges to the unique scalings up to a scalar; the resulting doubly stochastic matrix is unique (Bradley et al., 2011). For symmetric $S$5, the iteration

$S$6

produces a symmetric scaling vector.

The matrix-free contribution of Bradley and Murray is to approximate the required actions of $S$7 and $S$8 using only matrix-vector products with $S$9 and $m\times n$0. If $m\times n$1 has iid zero-mean, unit-variance entries, then for fixed $m\times n$2,

$m\times n$3

entrywise, so $m\times n$4 can be estimated by squaring the output of a single matrix-vector product rather than forming $m\times n$5 explicitly (Bradley et al., 2011). The resulting nonsymmetric stochastic iteration uses two matrix-vector products per iteration, while the symmetric version uses one. Storage is $m\times n$6, and the method can be applied when the matrix is available only through a fast operator.

The stated motivation is conditioning. Empirically and theoretically, equilibration often drives down the $m\times n$7-norm condition number $m\times n$8 relative to $m\times n$9, thereby accelerating iterative solvers or improving numerical stability of factorizations. In the reported experiments on 700+ nonsymmetric and 400+ symmetric matrices, sizes up to $\hat A = D_r A D_c,$0, after only $\hat A = D_r A D_c,$1 iterations the ratio $\hat A = D_r A D_c,$2 falls to $\hat A = D_r A D_c,$3–$\hat A = D_r A D_c,$4, nearly indistinguishable from exact equilibration, and $\hat A = D_r A D_c,$5 is reduced almost as much as with exact Sinkhorn-Knopp or classic binormalization (Bradley et al., 2011).

4. Distributionally robust post-hoc class scaling under prior shift

For fixed pretrained classifiers, post-hoc scaling can be used to address changes in class-prior or group-prior distributions. The DROPS procedure introduces a nonnegative scaling vector $\hat A = D_r A D_c,$6, one factor per class, and defines the post-scaled scorer

$\hat A = D_r A D_c,$7

equivalently

$\hat A = D_r A D_c,$8

A common parameterization is $\hat A = D_r A D_c,$9, where $2$0 and $2$1 is the empirical class-prior on a held-out validation set (Wei et al., 2023).

The optimization target is a distributionally robust loss around a chosen target prior $2$2. Let $2$3 denote the average loss on class $2$4. Then

$2$5

where $2$6 is a convex divergence on the simplex and $2$7 determines the size of the neighborhood around $2$8 (Wei et al., 2023). The paper states that $2$9 recovers pure average-class risk, $\hat\eta(x)\in\Delta_m$0 recovers true worst-class risk, and intermediate $\hat\eta(x)\in\Delta_m$1 values trade off average versus worst performance.

Theorem 3.1 identifies the Bayes-optimal form of the scorer for proper convex losses: $\hat\eta(x)\in\Delta_m$2 This justifies post-hoc classwise scaling as the relevant solution class. On a validation set, the algorithm alternates a projected $\hat\eta(x)\in\Delta_m$3-update, an exponentiated-gradient ascent step for $\hat\eta(x)\in\Delta_m$4, and a closed-form scaling update for $\hat\eta(x)\in\Delta_m$5; in the common case $\hat\eta(x)\in\Delta_m$6, the $\hat\eta(x)\in\Delta_m$7-update has a closed form (Wei et al., 2023).

Theorem 4.1 gives a post-hoc convergence guarantee. Under Lipschitz and convexity conditions, if the validation-set size satisfies $\hat\eta(x)\in\Delta_m$8 and one runs $\hat\eta(x)\in\Delta_m$9 steps with suitable learning rates, then with probability at least $A$00 the returned averaged scorer $A$01 satisfies

$A$02

where $A$03 is the class of all scalings of $A$04 (Wei et al., 2023). The reported practical consequence is that once $A$05 is fixed, deployment requires only adding $A$06 to logits or multiplying probabilities by $A$07 and renormalizing; no retraining or expensive computation is required at inference time beyond one vector addition per example.

5. Multiclass calibration via full and structured matrix scaling

In multiclass calibration, “matrix scaling” usually means a multinomial logistic recalibrator acting on a classifier’s log-probabilities. Given a black-box classifier $A$08 producing $A$09, one defines centered log-probabilities

$A$10

and fits

$A$11

with $A$12 and $A$13 (Berta et al., 5 Nov 2025). The centering step is required because softmax is invariant to adding the same scalar to every entry of its input. The paper motivates this construction by an idealized generative model in which logistic regression on the pre-softmax logits is well-specified, and notes that a quadratic term may arise but is often omitted in practice.

The central difficulty is parameter growth: a full $A$14 matrix $A$15 has $A$16 parameters plus $A$17 intercepts, which can overfit when $A$18 is small. Structured Matrix Scaling (SMS) addresses this by decomposing $A$19 into disjoint parameter groups,

$A$20

where $A$21 is a global scalar, $A$22 is a diagonal deviation vector, and $A$23 supplies off-diagonal terms (Berta et al., 5 Nov 2025). The fitting objective is a single convex problem combining multiclass log-loss with penalties on $A$24, $A$25, and $A$26, scaled by group size and calibration-set size. In the special case $A$27, one recovers Structured Vector Scaling (SVS).

Optimization is performed with a composite stochastic gradient solver, SAGA, with proximal updates to handle $A$28 or group-LASSO penalties. The implementation uses robust preprocessing: a standard temperature scaling fit with Laplace smoothing is first applied to the raw probabilities, then $A$29 is clipped to $A$30 (Berta et al., 5 Nov 2025). The default practical choices reported are norm order $A$31, exponents $A$32 and $A$33, and group weights $A$34.

The empirical evaluation includes 68 multiclass UCI-style datasets from TabRepo, 7 base learners, and $A$35 train/test folds $A$36 $A$37-fold CV for calibration, giving approximately 1400 experiments. Metrics are log-loss and Brier score on held-out test folds. SMS and SVS are reported to never overfit even on high-parameter regimes; standard unregularized matrix scaling often increases test loss on approximately $A$38 of datasets; SMS yields strictly better median log-loss and Brier improvements than all baselines; and fitting time is approximately comparable to torchcal Vector/Matrix and $A$39 faster than Dirichlet because the latter requires expensive cross-validated grid search (Berta et al., 5 Nov 2025). On CIFAR-10, CIFAR-100, and ImageNet benchmarks, SMS gives the largest absolute log-loss gains in nearly all cases, while unregularized matrix scaling catastrophically overfits on CIFAR-100.

6. Comparative properties, limitations, and recurrent points of confusion

One recurrent source of confusion is that post-hoc matrix scaling does not denote a single invariant-preserving operation. In projective decomposition, the salient invariant is the family of pairwise relative ratios, and the method is presented as appropriate for ratio-scale data in which zero has absolute meaning and comparisons are by division (Robinson, 2019). In equilibration, the target is approximate equality of row and column $A$40-norms, with the primary benefit a reduction in $A$41 and improved numerical behavior of iterative or direct solvers (Bradley et al., 2011). In post-hoc classification, the target is either robustness to a $A$42-ball of prior shifts around a specified prior $A$43 or improved probability calibration on a held-out set (Wei et al., 2023, Berta et al., 5 Nov 2025).

A second point is symmetry. Projective decomposition is explicitly symmetric in rows and columns and is invariant under transposition; applying it to $A$44 or $A$45 yields transposes of the same canonical form and the same scaling vectors (Robinson, 2019). Columnwise z-transformation is not symmetric: it standardizes columns, does nothing to rows, and alters ratios through centering. In classifier calibration, by contrast, the symmetry issue is not between rows and columns of a matrix but between classes: vector scaling treats classes independently through one factor per class, whereas full matrix scaling allows cross-class couplings through off-diagonal terms in $A$46 (Wei et al., 2023, Berta et al., 5 Nov 2025).

A third point is feasibility and regularity. For $A$47-adapted Sinkhorn-type normalization in projective decomposition, convergence is guaranteed for non-too-sparse matrices; in practice, any matrix where each row and column has at least one nonzero is fine (Robinson, 2019). Exact Sinkhorn-Knopp equilibration requires total support for scalability, and with support but not total support one may still observe partial equilibration (Bradley et al., 2011). DROPS requires a held-out validation set and states that the validation set must be approximately balanced across classes or groups so that $A$48 and $A$49 are reliable (Wei et al., 2023). Structured matrix scaling emphasizes the calibration-set-size regime explicitly: if $A$50, SMS is expected to collapse toward temperature or vector scaling, and for very large $A$51, such as ImageNet with $A$52, the recommendation is to use SVS or TS unless thousands of calibration examples are available (Berta et al., 5 Nov 2025).

A plausible synthesis is that post-hoc matrix scaling is best viewed as a controlled manipulation of scale degrees of freedom after model fitting or data acquisition. What varies across subfields is the object being rescaled, the invariants one intends to preserve, and the statistical or numerical criterion used to choose the scaling.