Spectral Density Divergence

• Spectral density divergence is a functional on positive spectral objects that enforces positivity, invariance, and entropy-based sufficiency.
• It unifies metrics like Kullback–Leibler, Itakura–Saito, Rényi, and Bregman divergences in applications ranging from multivariate spectral estimation to quantum state analysis.
• Practical applications include multivariate spectrum approximation, robust frequency-domain inference, and phase retrieval using bounded-degree rational solutions.

Searching arXiv for recent and relevant papers on spectral density divergence across information geometry, multivariate spectral estimation, and robust frequency-domain divergences. Searching arXiv for “spectral density divergence”, “multivariate spectral estimation divergence”, and “spectral Rényi divergence”. In the cited literature, the expression “spectral density divergence” denotes several closely related discrepancy constructions on objects that carry spectral structure: matrix-valued power spectral densities, normalized innovation spectra, nonnegative measurement spectra, and density operators on spectral convex sets. Taken together, these works suggest a common organizing idea: a spectral density divergence is a functional on positive spectral objects that is designed to respect positivity, spectral calculus, invariance under an appropriate symmetry group, and, in many cases, prediction-theoretic or information-theoretic consistency. Central examples include multivariate Beta, Alpha, Itakura–Saito, Kullback–Leibler, Rényi, and entropy-generated Bregman divergences, as well as geodesic distances induced by Fisher–Rao-type metrics (Zorzi, 2012, Harremoës, 2016).

1. Conceptual domain and basic definitions

For multivariate time-series analysis, the basic domain is the cone $S_+^m$ of $m\times m$ matrix-valued spectral densities $\Phi(e^{i\omega})$ that are bounded and coercive, meaning that there exist constants $\mu_1 \ge \mu_2 > 0$ such that $\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$ for all $\omega$. Matrix powers and logarithms are defined pointwise on the unit circle by functional calculus. This setting supports divergences between power spectra that are integrated over frequency with respect to the normalized Lebesgue measure (Zorzi, 2012).

In convex-state-space information geometry, the corresponding objects are states in a finite-dimensional compact convex set $C$, with pure states given by extreme points and mixed states by convex combinations. A state is spectral if all orthogonal decompositions of that state have the same spectrum, and $C$ is spectral if all states are spectral. Positive trace-$1$ elements in Euclidean Jordan algebras, including complex density matrices, are the main examples. This spectrality condition is not merely descriptive: it controls which divergences are compatible with sufficiency (Harremoës, 2016).

A useful distinction runs through the literature. In one line of work, a spectral density divergence is a discrepancy functional between two positive spectral objects. In another, “divergence” refers to blow-up or instability of a spectral density or of its Fourier representation. A long-memory construction with an integrable spectral density whose Fourier series diverges unboundedly almost everywhere is an example of the second usage (Vidril, 19 May 2026).

2. Information-theoretic and state-space foundations

On general convex state spaces, the central divergence class is the Bregman family generated by a convex function $F$. In the differential formulation,

$m\times m$0

The paper on maximum entropy and sufficiency develops this on general convex state spaces and introduces a sufficiency condition: if an affine map $m\times m$1 is sufficient for $m\times m$2, then a divergence satisfies

$m\times m$3

The key structural result is that only spectral sets can have a Bregman divergence that satisfies this sufficiency condition; moreover, if $m\times m$4 has at least three orthogonal states and $m\times m$5 is local, then the generator must have the form $m\times m$6, where $m\times m$7 and $m\times m$8 is affine. Thus, up to scaling and affine terms, entropy is forced by sufficiency (Harremoës, 2016).

This gives the classical and quantum cases as canonical instances. On the simplex of probability distributions,

$m\times m$9

and the associated Bregman divergence is the Kullback–Leibler divergence

$\Phi(e^{i\omega})$0

For complex density matrices,

$\Phi(e^{i\omega})$1

and the Bregman divergence becomes the Umegaki relative entropy

$\Phi(e^{i\omega})$2

In this framework, sufficiency is the equality case of data processing, and on complex Hilbert spaces it coincides with the existence of a recovery map on the pair $\Phi(e^{i\omega})$3 (Harremoës, 2016).

The same work links spectrality to Euclidean Jordan algebras. Density operators in real, complex, quaternionic, exceptional, and spin-type Jordan algebras form spectral convex sets, and entropy

$\Phi(e^{i\omega})$4

is strictly concave on the cone of positive elements. This suggests a strong constraint on admissible “spectral density divergences” for state spaces: once sufficiency is required, spectral decomposition, entropy generation, and symmetry become inseparable (Harremoës, 2016).

3. Divergences for power spectral densities

For matrix-valued power spectra, the multivariate Beta divergence is a standard unifying family. For $\Phi(e^{i\omega})$5 and $\Phi(e^{i\omega})$6,

$\Phi(e^{i\omega})$7

Its limiting cases are the multivariate Itakura–Saito distance and the multivariate Kullback–Leibler divergence:

$\Phi(e^{i\omega})$8

The map $\Phi(e^{i\omega})$9 is strictly convex for fixed $\mu_1 \ge \mu_2 > 0$0, and the family smoothly connects KL and IS in the multivariate setting (Zorzi, 2012).

A prediction-theoretic alternative is the divergence family $\mu_1 \ge \mu_2 > 0$1, built from the normalized innovation spectrum

$\mu_1 \ge \mu_2 > 0$2

where $\mu_1 \ge \mu_2 > 0$3 is a canonical left spectral factor of the prior $\mu_1 \ge \mu_2 > 0$4. For $\mu_1 \ge \mu_2 > 0$5,

$\mu_1 \ge \mu_2 > 0$6

It satisfies

$\mu_1 \ge \mu_2 > 0$7

so it is exactly the Alpha/Beta divergence between the normalized innovation spectrum and identity. Its limits are the multivariate Itakura–Saito distance and the multivariate Kullback–Leibler divergence between $\mu_1 \ge \mu_2 > 0$8 and $\mu_1 \ge \mu_2 > 0$9 (Zorzi, 2014).

A differential-geometric formulation studies distances between “infinitesimally close” PSDs. One representative divergence is

$\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$0

which is symmetric, congruence-invariant, and inverse-invariant. Its infinitesimal metric is

$\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$1

and the corresponding geodesic distance is

$\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$2

This geometry is explicitly connected to the Fisher–Rao metric (Jiang et al., 2011).

Family	Domain	Defining feature
Multivariate Beta	$\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$3	Smoothly connects KL and IS
$\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$4	Multivariate PSDs with prior $\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$5	Compares normalized innovation spectrum to $\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$6
$\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$7 / $\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$8	Positive spectral density matrices	Congruence-invariant Fisher–Rao-type geometry
Spectral $\mu_2 I \le \Phi(e^{i\omega}) \le \mu_1 I$9-Rényi	Scalar spectral densities	Includes IS as the $\omega$0 limit

4. Approximation, estimation, and computational structure

In THREE-like spectrum approximation, a prior $\omega$1 and a rational filter bank

$\omega$2

are combined with the generalized moment constraint

$\omega$3

With the parametrization $\omega$4, $\omega$5, the multivariate Beta-divergence problem

$\omega$6

has the explicit solution family

$\omega$7

more precisely in the form reported in the paper,

$\omega$8

Its McMillan degree satisfies

$\omega$9

and the unique dual minimizer can be computed by a globally convergent matricial Newton method with backtracking (Zorzi, 2012).

A closely related scalar framework uses the Alpha divergence. After whitening the covariance constraint to $C$0, the minimizer has the closed form

$C$1

with the reverse-KL case

$C$2

and the $C$3 limit

$C$4

For finite $C$5, the solution is rational; the exponential limit recovers the principle of minimum discrimination information in nonrational form (Zorzi, 2013).

The prediction-theoretic divergence $C$6 yields an analogous multichannel family,

$C$7

with degree bound

$C$8

This family is designed to penalize departures of the normalized innovation spectrum from white, and the parameter $C$9 controls the tradeoff between model complexity and fidelity (Zorzi, 2014).

5. Robust variants and application-specific uses

For stationary scalar processes, the spectral $C$0-Rényi divergence is defined for $C$1 by

$C$2

Its $C$3 limit is the spectral Itakura–Saito divergence,

$C$4

This family is connected asymptotically to the $C$5-divergence in robust statistics through $C$6, and it admits primal and dual variational representations in terms of convex combinations of IS divergences. The practical consequence emphasized in the paper is robustness to spectral outliers: for contaminated periodograms, the change in the $C$7-Rényi objective is asymptotically independent of the model parameter, while the IS objective remains strongly parameter-dependent (Takabatake et al., 2023).

A different application appears in phase retrieval. There the objects being compared are nonnegative intensity vectors in $C$8, interpreted as spectra in the positive orthant. Spectral initialization is reformulated as approximate Bregman divergence minimization over rank-$C$9 positive semidefinite matrices in the lifted forward model. For KL and IS, this yields explicit sample-processing functions:

$1$0

and

$1$1

The leading eigenvector of

$1$2

is then used as the initializer. In the Gaussian sampling model, the IS design reduces to the previously known optimal processing rule $1$3 (Yonel et al., 2020).

These applications show that “spectral density divergence” is not restricted to stationary power spectra. The same formal ideas recur whenever the data are positive and naturally compared through scale-aware, entropy-like, or multiplicative geometries.

6. Ambiguities, edge cases, and open questions

Several edge cases recur across the literature. In the convex-state-space setting, sufficiency beyond complex and real matrix algebras remains open, and the conjectured route from locality plus symmetry to Jordan algebraic structure is not fully proved. In two dimensions, the only spectral sets are simplices and balanced sets, and in nonspectral convex sets such as squares, entropy may fail to be globally concave and local Bregman divergences cannot exist (Harremoës, 2016).

In multivariate spectrum approximation, coercivity is essential. Singular spectra break the functional calculus for logarithms and matrix powers, and for $1$4 the dual minimum may lie on the boundary where positivity is lost. The regularized, bounded-degree families therefore depend essentially on positivity, feasibility of the moment constraint, and the admissible-domain structure of the dual problem (Zorzi, 2012).

In robust frequency-domain inference, the choice of $1$5 in spectral Rényi divergence remains a hyperparameter. The cited work recommends practical exploration over $1$6, but does not supply a general selection theorem. This makes robustness controllable but not canonical (Takabatake et al., 2023).

A separate terminological ambiguity concerns “divergence” as failure of convergence rather than a discrepancy functional. One constructed long-memory Gaussian process has an absolutely continuous, log-summable spectral density whose Fourier series diverges unboundedly almost everywhere. That result implies that, without additional regularity such as regularly varying structure plus suitable conditions on a slowly varying function, the spectral density of a long-range dependent process should be handled as a density of the spectral measure rather than identified naively with its Fourier series (Vidril, 19 May 2026). A related but distinct fractal-measure setting studies divergence of Mock Fourier series for doubling spectral measures (Pan et al., 2022).

Taken together, these results indicate that spectral density divergence is best understood as a family of structurally constrained discrepancy principles rather than a single formula. In the strongest information-theoretic formulations, sufficiency forces entropy and spectrality; in multivariate time-series analysis, prediction and invariance lead to Beta-, Alpha-, IS-, KL-, and Fisher–Rao-type geometries; and in robust settings, Rényi-type regularization tempers the influence of narrow spectral spikes. The main unresolved issues concern the full Jordan-algebraic scope of sufficiency, robustness–efficiency tradeoffs, and the boundary between well-behaved spectral densities and genuinely divergent spectral representations (Harremoës, 2016, Takabatake et al., 2023).