Expectation-level Decompression Law (EDFL)

• EDFL is a fundamental law that defines the relationship between information budgets and reliability in both spectral estimation of large matrices and transformer language models.
• It utilizes Stieltjes/R-transforms in the spectral domain and expected KL divergences in language modeling to establish sharp, information-theoretic bounds on extrapolative uncertainty.
• Empirical validations on synthetic ensembles and real datasets underscore EDFL's role as a cornerstone for principled spectrum recovery and calibrated transformer performance.

The Expectation-level Decompression Law (EDFL) is a fundamental result that quantifies the relationship between information budgets and reliability in two distinct but mathematically unified contexts: the estimation of spectral properties of large random matrices from small submatrices, and the prediction reliability of transformer-based LLMs under limited informational support. EDFL delineates sharp, information-theoretic bounds on extrapolative uncertainty, operationalized through Stieltjes and R-transforms in the spectral domain, and through expected Kullback–Leibler divergences for LLMs. Its theoretical formulation and empirical validations have established EDFL as a cornerstone for principled spectral inference in impalpable-matrix settings and for risk-managed generation in deployed transformer systems (Ameli et al., 13 Jun 2025, Chlon et al., 14 Sep 2025).

1. Formal Statement of the Expectation-level Decompression Law

In spectral estimation, let $A\in\mathbb{R}^{n\times n}$ be a large Hermitian matrix whose spectrum is inaccessible; $A_{n_s}$ denotes an $n_s\times n_s$ Haar-random principal submatrix, with empirical spectral density $\rho_s(\lambda)$ and Stieltjes transform $m_s(z)$. Defining the decompression ratio $\alpha=n_s/n<1$ and the rescaled “time” $t= -\log\alpha$, the evolution of the expected Stieltjes transform $m(t,z)$ for the full system follows the quasilinear PDE:

$$\frac{\partial m}{\partial t} = -m + m^{-1}\frac{\partial m}{\partial z}, \quad m(0,z) = m_s(z),$$

with the spectrum of $A$ at 'full size' recovered via inversion:

$$\rho_\text{full}(x) = \frac{1}{\pi}\lim_{\varepsilon\to 0^+} \Im\,m(t, x + i\varepsilon).$$

Equivalently, via the R-transform $R(z)$:

$$R_\text{full}(z) = R_s(\alpha z) = R_s\left(\frac{n_s}{n}z\right).$$

In transformer language modeling, for evidence $X$, permutations $\pi$, model predictive distributions $S_\pi$, ground-truth $P$, and a Bernoulli predicate $g$, EDFL asserts:

$$\bar\Delta = \mathbb{E}_\pi[ KL(P\|S_\pi) ] \ge KL(\mathrm{Ber}(p)\|\mathrm{Ber}(\bar q)),$$

where $p = P\{g(Y) = 1\}$ and $\bar q = \mathbb{E}_\pi [ S_\pi\{g(Y) = 1\} ]$. Equality holds when $P$ is the exponential-tilt (information projection) of $\bar S$ onto $Q(g=1)=p$.

2. Assumptions and Theoretical Foundations

Spectral Estimation:

• Asymptotic Freeness: Submatrix selection via Haar-random permutation ensures asymptotic freeness as $n_s, n\to\infty$ with $\alpha$ fixed, enabling free probability techniques [Nica, 1993].
• Moment Convergence: For all $k$, $(1/n_s)\operatorname{Tr}(A_{n_s}^k) \to$ finite.
• Hermiticity and Regularity: $A$ is Hermitian; limiting densities are compactly supported and their Stieltjes transforms are analytic off the real axis (Ameli et al., 13 Jun 2025).

Predictive Reliability (Transformers):

• Absolute Continuity: $P \ll S_\pi$; model assigns nonzero probabilities to relevant events.
• Permuted Evidence: Transformers minimize expected, not instancewise, conditional description length over permutations.
• Bernoulli Predicate Reduction: Adjudication must be cast as a binary predicate for tightness of EDFL; generalization to long-form is achieved via aggregation (Chlon et al., 14 Sep 2025).

3. Sketches of Derivation and Mathematical Structure

Spectral Domain:

• The Nica–Speicher result for free compression relates the R-transform of the $\alpha$-compressed matrix to that of the original.
• Free decompression inverts this scaling, giving $R_\mathrm{full}(z) = R_{n_s}\big((n/n_s)z\big)$.
• Mapping between Stieltjes and R-transforms, and differentiation with respect to $t=\log(n/n_s)$, yields the EDFL PDE.
• The method of characteristics provides a closed-form solution for $m(t, z)$:

$$m(t, z) = m_0(\phi(t, z))\, e^{-t},\quad z = \phi - (e^t-1)[m_0(\phi)]^{-1}.$$

Information-Theoretic Domain:

• Convexity (Jensen’s inequality) implies $\mathbb{E}_\pi[KL(P\|S_\pi)] \ge KL(P\|\bar S)$.
• The data-processing inequality for KL divergence reduces the bound to Bernoulli KL via the predicate $g$.
• The EDFL thus quantifies the minimal information budget required to meet a target reliability by the gap between prior and desired Bernoulli probabilities.

4. Empirical Demonstrations and Applications

Spectral Estimation

• Benchmark Ensembles: Validation includes Wigner, Marchenko–Pastur, Kesten–McKay, Wachter, and free–Meixner synthetic laws. For Marchenko–Pastur, EDFL extrapolates $R$-transforms to reconstruct the bulk limiting distribution at large $n$ from small submatrices (Ameli et al., 13 Jun 2025).
• Real Data: Applied to the SNAP Facebook Laplacian (submatrix $n_s\approx 4096$, full $n=22470$) and ResNet50 Neural Tangent Kernel for CIFAR10 ($n\approx 50\,000$). Free decompression accurately recovers the full spectrum, with empirical errors in moments and divergences versus ground truth quantitatively tabulated.

Transformer Reliability

• Dose-response: A randomized dose–response experiment shows hallucination rates drop quasi-linearly with increased expected information budget ($\sim 0.127$ per nat of $\bar\Delta$; $0.375$ nats per additional support chunk), in agreement with EDFL rare-event asymptotics (Chlon et al., 14 Sep 2025).
• Operational Gating: Metrics such as Bits-to-Trust (B2T), Risk-of-Hallucination (RoH), and Information Sufficiency Ratio (ISR) derive analytically from EDFL. Calibrated gating at ISR$=1$ in audit trials achieves near-zero hallucinations at 24.1% abstention.

5. Error Growth, Stability, and Limitations

• Error Propagation (Spectral): If $\delta m_{n_s}$ is an error in the initial submatrix Stieltjes transform estimate, then propagated error in the decompressed estimate grows at most polynomially in $n/n_s$: $$\|\delta m_n\|_{L^2}\,\le\,(n/n_s)^\nu\,\|\delta m_{n_s}\|_{L^2}$$ for some $\nu>0$.
• Analytic Continuation: Accurate PDE integration across spectral edges requires analytic continuation of the Stieltjes transform; practical holomorphic extension is achieved via rational corrections glued to the empirical transform.
• Sensitivity to Initial Estimates: Decompression magnifies estimation errors, requiring spectrum smoothing (e.g., Chebyshev/Jacobi expansions) and enforcement of positivity/mass constraints.
• Sampling Regime: Asymptotic freeness is generally not guaranteed for non-Haar submatrices; proper randomization in row and column selection is required.
• Matrix Impalpability: Existing spectral methods are inapplicable when only masked snapshots are available; EDFL/free decompression is the only established method for full-spectrum estimation under such constraints (Ameli et al., 13 Jun 2025).
• Predictive Domain: Absolute continuity requires that model support covers the event of interest; degenerate probability assignments must be regularized.

6. Operational Significance and Broader Impact

EDFL provides a quantitative, information-theoretic law linking the achievability of target reliability to the available informational budget, both for spectrum recovery from subsampled matrices and for LLM prediction under limited evidence. In spectral analysis, it enables full-spectrum inference of impalpable operators—supporting downstream tasks such as log-determinant computation, matrix function tracing, and kernel analysis—without matrix-vector multiplication. In transformer LLMs, EDFL reinterprets hallucination as a predictable, quantifiable consequence of insufficient code-length, enabling precise refusal strategies and calibrated risk-managed generation through direct application of B2T, RoH, and ISR metrics. The law is validated both on synthetic ensembles (with exact recovery), on real large-scale matrix data, and by empirical, randomized controlled experiments in language modeling contexts. This central principle draws together modern free probability, compression-based learning theory, and risk-aware machine learning deployments (Ameli et al., 13 Jun 2025, Chlon et al., 14 Sep 2025).