Top Singular Value in Sum-Products of Random Matrices

Abstract: We study the top singular value for a sum of $m$ independent $n \times n$ random matrices, each of which is a product of $N$ i.i.d. $n\times n$ Gaussian matrices. Our main conceptual observation is that when $m,n,N\rightarrow \infty$, the top singular value coincides with the partition function in a random energy model at the inverse temperature $β=\sqrt{2(N-1)/(n\log m)}$, with energies depending on the ratio $N/n$. We provide several non-asymptotic results making this approximation precise.

• The paper shows that the top singular value is characterized by a random energy model mapping, delineating high and low-temperature regimes.
• It develops non-asymptotic bounds that precisely quantify the concentration of log singular values around a deterministic REM prediction.
• The study reveals a sharp phase transition at β = √2, offering new insights into extreme value statistics in high-dimensional random matrix products.

Top Singular Value in Sum-Products of Random Matrices: An Expert Review

Problem Overview and Model

The paper "Top Singular Value in Sum-Products of Random Matrices" (2607.04047) rigorously analyzes the top singular value (equivalently, the top Lyapunov exponent) of random matrices constructed as follows: Consider $X = \frac{1}{\sqrt{m}} \sum_{i=1}^m X_i$ where each $X_i$ is a product of $N$ i.i.d. $n \times n$ Ginibre matrices. All entries are i.i.d. Gaussians with mean zero and variance $1/n$. The focus is the asymptotic regime $n, N, m \to \infty$, with explicit formulae for the top singular value, and a characterization of phase transitions in the asymptotics.

Critically, when $m, n, N$ all diverge, the leading singular value is shown to be tightly governed by an instance of the Random Energy Model (REM) at inverse temperature

$\beta = \sqrt{2(N-1)/(n \log m)}.$

This mapping enables precise non-asymptotic high-probability estimates using the statistical mechanics of disordered systems applied to random matrix theory.

Analytical Approach and Main Results

The paper demonstrates that when $N, n, m \to \infty$, the top singular value $s_1(X)$ is asymptotically determined by the log-partition function $X_i$0 of an REM with energies constructed from the underlying random matrix products, and that

$X_i$1

holds with explicit, non-asymptotic error bounds, uniform over various scaling regimes. The analysis distinguishes two sharply different regimes, characterized by the value of the parameter $X_i$2:

• High-temperature regime ($X_i$3): The top singular value is determined by the collective effect of many matrix product terms, with a law of large numbers-style averaging dominating the statistics.
• Low-temperature regime ($X_i$4): The top singular value is dominated by rare, large contributions from a handful of product realizations with exceptionally large singular values, leading to REM-style freezing.

Formal Non-asymptotic Results

Two principal theorems (Theorems 1 and 2 in the paper) precisely quantify the concentration of $X_i$5 around the deterministic REM prediction $X_i$6 with probability exponentially close to one as $X_i$7 diverge, with explicit error rates depending on the scaling of $X_i$8 and the value of $X_i$9.

• For $N$0,

$N$1

with probability at least $N$2.

• For $N$3, $N$4, $N$5,

$N$6

with exponentially high probability.

The limiting value $N$7 in the REM model is explicitly calculated:

$N$8

representing a sharp phase transition at $N$9.

Characterization of the Transition

The paper proves that at $n \times n$0, a non-analyticity develops in the scaling of $n \times n$1, indicating a freezing transition. This transposes the classic REM phase transition into the behavior of extremal singular values of random matrix sum-products, a phenomenon previously ambiguous in the random matrix literature. The result is robust to approximation of non-Gaussian energies by Gaussian ones, with precise error quantification via a Cramér-type moderate deviation theorem.

Methods and Proof Techniques

The proofs are based on several sophisticated ingredients:

• Reduction to the REM: The analysis first reduces the computation of the top singular value to the evaluation of the log-partition function of an REM, whose energies encode the fluctuations in the norms of long products of random Ginibre matrices.
• Control of Supremum over Spheres: Showing that, under appropriate scaling, the supremum over all directions can be well-approximated by the value in a fixed direction, leveraging invariance and concentration results.
• Non-uniform Gaussian Approximation: Precise control of the distributions of the sum-product energies, employing moderate deviation results to facilitate the REM mapping despite the non-Gaussian landscape.
• Laplace, Markov Inequalities and Concentration: Adaptations of Laplace's method, Markov's inequality, and REM concentration results from statistical mechanics are synthesized to obtain strong non-asymptotic bounds.

Relation to Prior Work

Previous results on products of random matrices typically focused on the case $n \times n$2, i.e., a single product of Ginibre matrices, with regimes characterized by either large $n \times n$3 for fixed $n \times n$4 [geman1980limit] [yin1988limit] [akemann2013products], or large $n \times n$5 for fixed $n \times n$6 [furstenberg1960products] [oseledets1968multiplicative]. In the regime where $n \times n$7 with fixed $n \times n$8, the top Lyapunov exponents and related distributions have been studied, but explicit phase transitions of the kind revealed here were not previously established.

The analysis fundamentally departs from prior work by:

• Considering the case where both the length of the products $n \times n$9 and the number $1/n$0 of summed products diverge, and elucidating nontrivial interaction between the sum and product structure.
• Demonstrating that the top singular value, in the large $1/n$1 regime, is not governed simply by the scaling $1/n$2, but instead by the inverse temperature parameter $1/n$3 encoding a more intricate dependence.

Theoretical and Practical Implications

This work provides a precise, non-asymptotic understanding of extremal singular values in high-dimensional sums and products of random matrices.

Theoretical implications:

The mapping to the REM with explicit calculation of the phase transition opens pathways to analyze other models in random matrix theory where extremal statistics are relevant, such as non-commutative random polynomials or more general random matrix valued functions. The methodology provides a blueprint to handle complex non-Gaussian ensembles, and may be adapted to study universality and large deviations in broader settings.

Practical implications:

At a technical level, understanding the limiting behavior of large products and sums of random matrices is fundamentally connected to the spectral properties of weight matrices in deep neural networks, particularly in initialization and training dynamics where both width and depth are large. Moreover, in high-dimensional statistics and wireless communications, the largest singular value can govern stability and performance limits.

Potential extensions:

Future work may relax the assumptions on independence or Gaussianity, extend to rectangular matrices or more general variance structures, or analyze the asymptotic joint law of several largest singular values. Another direction is the extension to real-world structured matrix ensembles relevant for deep learning, e.g., block-structured, convolutional, or sparse matrices.

Conclusion

The paper delivers a rigorous and technically sophisticated analysis of the top singular value for sums of products of large random matrices, revealing that its asymptotics can be sharply described via a phase transition in the random energy model, with explicit error controls and sharp characterization of high- and low-temperature regimes. This result both clarifies previous ambiguities in the asymptotic random matrix literature and provides robust tools for future studies of extreme value statistics in high-dimensional random matrix models (2607.04047).