Empirical Spectral Densities

• Empirical spectral densities are measures describing the eigenvalue distribution of data-driven matrices and reveal intrinsic noise and structure in high-dimensional systems.
• They employ scaling relations and universality principles to link complex spectral behaviors to simpler white noise paradigms using Fourier transforms and incomplete Gamma functions.
• Advanced methods like the Kernel Polynomial Method and Lanczos quadrature enable efficient spectral density approximations in massive matrices, useful in time series, physics, and spatial statistics.

Empirical spectral densities describe the distribution of eigenvalues of data-driven matrices, such as sample auto-covariance or covariance matrices, under finite or infinite-sample regimes. Their paper is crucial for understanding the intrinsic structure and noise properties present in high-dimensional time series, spatial data, random matrices, and a variety of physical, engineering, and statistical systems.

1. Fundamental Concepts and Mathematical Definitions

The empirical spectral density (ESD) of an $N \times N$ Hermitian matrix $C$ is defined as

$$\rho_N(\lambda; C) = \frac{1}{N} \sum_{i=1}^N \delta(\lambda - \lambda_i),$$

where $\lambda_i$ are the eigenvalues of $C$. Equivalently, a smoothed version can be extracted via the Stieltjes transform: $$\rho_N(\lambda; C) = \frac{1}{\pi N} \operatorname{Im} \operatorname{Tr}\, [\lambda_\varepsilon I - C]^{-1}, \quad \lambda_\varepsilon = \lambda - i\varepsilon.$$ For stationary stochastic processes, especially in time series, the auto-covariance matrix is central: $$C_{k\ell} = \frac{1}{M} \sum_{m=0}^{M-1} x_{m+k} x_{m+\ell}, \quad 1 \leq k, \ell \leq N,$$ where $\{x_t\}$ is a realization of the process. The spectral density then describes the typical (statistical) outcome for the spectra of such sample matrices given process properties and sample size.

2. Scaling Relations and Universality for Auto-Covariance Matrices

When analyzing spectra of empirical auto-covariance matrices constructed from second order stationary stochastic processes, a remarkable scaling relation emerges in the large-$N$, large-$M$ regime with fixed $\alpha = N/M$: $$\rho(\lambda) = \int_0^\pi \frac{dq}{\pi} \frac{1}{\hat C(q)}\, \rho^{(0)}_\alpha\left( \frac{\lambda}{\hat C(q)} \right),$$ where:

• $\rho^{(0)}_\alpha(\cdot)$ is the ESD for the i.i.d. (white noise) case,
• $\hat C(q)$ is the Fourier transform of the process's auto-covariance function:

$$\hat C(q) = \sum_{n=-\infty}^{\infty} \bar C(n) e^{iqn}.$$

This expresses the ESD for general stationary processes as a superposition of rescaled i.i.d. spectral densities, weighted by the frequency-dependent power spectrum. The universality resides in the fact that, for i.i.d. inputs with finite variance, $\rho^{(0)}_\alpha$ is independent of higher moments or the precise distribution, depending only on $\alpha$.

A closed-form analytic approximation for $\rho^{(0)}_\alpha(\lambda)$ is provided: $$\rho^{(0)}_\alpha(\lambda) = - \lim_{\varepsilon\to 0} \frac{1}{\pi} \mathrm{Im} \frac{\partial}{\partial \lambda} \ln I_\alpha\left( \frac{2}{\alpha} \lambda_\varepsilon \right),$$ with

$$I_\alpha(x) = i\, (-x)^{-1+2/\alpha}\, e^{-x} \Gamma(1-2/\alpha, -x),$$

where $\Gamma$ denotes the incomplete Gamma function.

3. Empirical Spectral Distributions for Covariance Matrices with Long Memory

For sample covariance matrices generated from $N$ independent copies of a stationary process (possibly with long memory), the ESD possesses a universal limit determined by the process's spectral density $f$: $F_{B_N} \xrightarrow{N,p\to\infty} F,$ where the limiting distribution is characterized by the Stieltjes transform $S(z)$, satisfying: $$z = -\frac{1}{S} + \frac{c}{2\pi} \int_{-\pi}^\pi \frac{f(\lambda)\,d\lambda}{1 + f(\lambda) S},$$ with $c = \lim p/N$ as the aspect ratio.

Critically, this limit depends only on the spectral density $f$, not on the decay rate of covariances or on higher moments, indicating a strong universality property for sufficiently regular processes.

4. Methods for Approximation and Computation

4.1. Dense/Sparse Matrix Cases (Kernel Polynomial and Lanczos Methods)

Physical and engineering problems often involve very large real symmetric or Hermitian matrices, making full diagonalization infeasible. Several scalable numerical algorithms have been developed:

• Kernel Polynomial Method (KPM):
  • Approximates the spectral density via a Chebyshev expansion, utilizing stochastic trace estimation.
  • Capable of handling matrices of very large size with only matrix-vector products.
• Lanczos Quadrature Methods:
  • Use Krylov subspace projection (Lanczos algorithm) to approximate the DOS via Gaussian quadrature.
  • Particularly effective for spectra with sharp features due to rapid convergence.
• Delta-Chebyshev Expansion:
  • Directly expands delta distributions at target points in Chebyshev polynomials, efficiently using the Krylov process.
• Haydock (Continued Fraction) Method:
  • Relates to the trace of resolvents and works particularly well for tightly localized spectral features.

Accuracy is typically measured via $L_2$ distances between smoothed versions of the sampled and estimated densities. Higher degree expansions (or more Krylov steps) yield finer resolution, at the cost of increased computation.

4.2. Empirical Spectral Density in Spatial Statistics and Irregular Data

Empirical spectral densities also appear in spatial statistics, especially for processes sampled on irregular grids. Dedicated approaches such as the spatial frequency domain empirical likelihood (SFDEL) and semiparametric estimators with smoothing splines for low frequencies and parametric algebraic tails for high frequencies are constructed to handle bias, model flexibility, and covariance structure inference in such settings.

5. Applications in Theory and Practice

• Time Series Analysis: Distinguishes true signal structures from finite-sample noise, improves hypothesis testing, model selection, and data-driven structure learning.
• Physics and Materials Science: The density of states underpins predictions of thermodynamic, spectral, and transport properties in quantum, electronic, and condensed matter systems.
• Signal Processing and Data Science: Used in spectral analysis of large graphs, network Laplacians, or big data covariance analysis where full matrix diagonalization is impractical.
• Statistics and Random Matrix Theory: Informs understanding of limiting eigenvalue distributions, high-dimensional "spiked" models, and PCA in high-noise regimes.

6. Limitations, Assumptions, and Universality

Analytical results for ESDs often invoke assumptions such as stationarity, ergodicity, or finite variance. Many scaling or universality results rely on large sample limits ($N, M \to \infty$ with fixed ratio), regularity conditions on the underlying process or matrix entries, and structural properties such as independence or weak dependence.

Most methodologies treat only the bulk of the spectrum robustly; large outliers or finite-size corrections may not be captured, and caution is required when working with small matrices or processes violating regularity assumptions.

Summary Table: Key Spectral Density Results and Methods

Setting / Method	Core Formula / Approach	Comments
Auto-covariance matrix, stationary process	$$\rho(\lambda) = \int \frac{dq}{\pi} \frac{1}{\hat{C}(q)}\rho^{(0)}_\alpha(\lambda/\hat{C}(q))$$	Scaling relation via spectral power
i.i.d./white noise (universal scaling)	Closed form for $\rho^{(0)}_\alpha$ via incomplete Gamma function	Universal for finite variance input
Sample covariance, long memory	Limiting ESD via Stieltjes transform equation with process spectral density	Universality: depends only on $f$, not details
Kernel Polynomial / Lanczos	Chebyshev expansion or Krylov subspace projection	Scalable to huge matrices; exploits matvecs only
Empirical likelihood (spatial)	EL in frequency domain, spectral estimating equations	Corrects for periodogram bias, non-parametric CIs

Empirical spectral densities provide a unifying language for analyzing and understanding the average spectral (eigenvalue) structure arising from high-dimensional data matrices, physical Hamiltonians, or spatial/temporal processes. They underpin robust inference, computation, and modeling across a breadth of scientific, engineering, and statistical disciplines.