Memory-Capacity Spectrum

• Memory-capacity spectrum is a structured decomposition of finite memory into distinct profiles along interpretable axes rather than a single aggregate value.
• It spans multiple domains—from delay dynamics in echo-state networks and pattern geometry in associative memory to frequency-domain capacities in communication channels.
• Each formulation ties capacity distribution to system dynamics, topology, and resource trade-offs, providing actionable insights for optimizing memory usage.

Memory-capacity spectrum denotes a family of technical formalisms in which memory is not summarized by a single scalar but resolved along a structured axis: delays in reservoir computing, pattern geometry in associative memory, memory resources in sensing, frequency and eigenmodes in channels with memory, or time-varying resource states in systems work. In the cited literature, the term therefore names a class of spectrum-like descriptions of where finite memory resides and how it is redistributed by dynamics, topology, correlations, thermodynamic constraints, or workload structure (Singh et al., 24 Jul 2025, Bielmeier et al., 2 Aug 2025, Hartich et al., 2015, Loyka et al., 2022).

1. Core meanings of the term

In the cited literature, “memory-capacity spectrum” is used in several non-equivalent but closely related senses. The common feature is that a finite memory budget is decomposed along an interpretable coordinate rather than reported as one aggregate number.

Domain	Capacity quantity	Spectrum axis
Echo-State Networks	$C(\tau)$, with $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$	delay $\tau$
Dense Associative Memory	$K_{\max}$	Hamming separation, feature correlations, polynomial degree $n$
Sensory systems	$C = l_y/\mathcal{T}_{x\rightarrow y}$	memory resources and efficiency
Channels with memory	$$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$	frequency $\omega$, eigenmode $i$

This multiplicity is substantive rather than terminological. In Echo-State Networks, the spectrum literally indexes how well a delayed input $u_{t-\tau}$ can be reconstructed from the present reservoir state. In Dense Associative Memory, it becomes a capacity surface over data geometry and model complexity. In stochastic thermodynamics, it measures how much of the information available in a full trajectory is already present in the instantaneous sensor state. In communication theory, it is a frequency-domain decomposition induced by channel memory and colored noise (Singh et al., 24 Jul 2025, Bielmeier et al., 2 Aug 2025, Hartich et al., 2015, Loyka et al., 2022).

A recurrent conceptual distinction follows. A scalar capacity such as $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$0, $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$1, or an integral channel capacity answers how much memory a system has. A memory-capacity spectrum answers where that capacity is concentrated, how quickly it decays, and what trade-offs redistribute it.

2. Delay-indexed spectra in reservoir computing

The most explicit formalization appears in the dynamical-systems treatment of Echo-State Networks. For an ESN with state update

$$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$2

the target at delay $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$3 is $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$4, and a linear readout is trained to reconstruct it. The linear memory capacity at delay $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$5 is

$$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$6

so $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$7 is the squared correlation between the true delayed input and its optimal linear reconstruction. The total linear memory capacity is

$$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$8

and the memory-capacity spectrum is the delay-indexed sequence $$\mathrm{MC}_{\mathrm{tot}}=\sum_{\tau=1}^\infty C(\tau)\le n$$9 itself (Singh et al., 24 Jul 2025).

This definition is tied directly to contractive dynamics. If the activation is globally Lipschitz with constant $\tau$0, then the sufficient algebraic condition

$\tau$1

guarantees the Echo State Property and, under the paper’s proposition, also yields a Fading-Memory Property. The induced filter is then geometrically forgetting: perturbations in the remote past are exponentially down-weighted. This constrains the tail of $\tau$2: strong contraction compresses the spectrum toward small delays, whereas weaker contraction extends the effective memory horizon but approaches instability and the edge of chaos. The same analysis links slow Lyapunov directions to larger long-delay capacities and fast-contracting directions to short-term memory (Singh et al., 24 Jul 2025).

The paper further shows that topology and leak rate redistribute delay-specific capacities without changing the Jaeger bound. Increasing spectral radius while remaining in the ESP regime broadens the spectrum toward larger $\tau$3; structured eigenvalue placements can produce quasi-delay-line behavior; and in leaky ESNs, small leak rate $\tau$4 shifts capacity toward longer delays while large $\tau$5 concentrates it at short delays. In this formulation, the spectrum is a dynamical fingerprint of contractivity, eigenstructure, and time-scale separation rather than merely a readout diagnostic (Singh et al., 24 Jul 2025).

A photonic realization appears in a silicon microring time-delay reservoir computer. There the total information processing capacity is computed as

$\tau$6

with capacities evaluated up to order $\tau$7, $\tau$8 virtual nodes, and a delay line $\tau$9. The parameter space in average input power and frequency detuning separates into three regimes: an approximately linear region, an optimal region with adequate nonlinearity, and a self-pulsing region in which strong time-domain discontinuities destroy consistency and sharply reduce memory. Shorter free-carrier lifetime $K_{\max}$0 increases the maximum memory capacity and extends the optimal region, while excessive FCD/TO nonlinearity collapses useful memory despite increasing dynamical complexity (Castro et al., 2024).

3. Associative-memory spectra over data geometry and online loading

In Dense Associative Memory, memory capacity is the largest number of stored patterns $K_{\max}$1 that can be retrieved with zero errors. The cited empirical framework turns this scalar into a spectrum by varying three axes simultaneously: pattern separation, feature correlations, and the polynomial degree $K_{\max}$2 in the rectified polynomial energy. Pattern separation is measured by average pairwise Hamming distance, and the main empirical law is that capacity grows exponentially with separation for both synthetic and feature-correlated MNIST data. Feature correlations do not change that qualitative exponential form, but at fixed separation they shift the capacity curve downward, and the gap between synthetic and MNIST data widens as $K_{\max}$3 increases. The resulting “memory-capacity spectrum” is therefore a capacity surface over data geometry and interaction order rather than a single storage number (Bielmeier et al., 2 Aug 2025).

This usage is important because it separates two often conflated effects. Pattern separation is the primary axis: larger average Hamming distance supports larger $K_{\max}$4. Feature correlations are a secondary but systematic deformation: they reduce capacity slightly at constant separation, and that reduction is amplified at higher polynomial degree, indicating that higher-order interactions are more limited in depicting higher-order interactions between features than patterns. A plausible implication is that any realistic account of associative-memory capacity must treat data distribution and model order jointly (Bielmeier et al., 2 Aug 2025).

A different but complementary spectrum appears in dynamic capacity estimation for Hopfield networks. There capacity is defined as the maximum number of stored patterns that can be perfectly recalled when probed with the desired pattern. Instead of using a static formula, the model tracks destructive crosstalk at each neuron and each stored pattern via

$K_{\max}$5

The operational threshold is local and sharp: the network is at or beyond capacity when $K_{\max}$6 for any neuron-pattern pair. This yields a state-dependent capacity spectrum over network size $K_{\max}$7, pattern bias, temporal correlation, and storage history. In simulation, the dynamic estimator achieves capacity estimates between $K_{\max}$8 and $K_{\max}$9 accurate and doubles the memory efficiency of Hopfield networks in comparison to the static and worst-case capacity estimate (Sarup et al., 2017).

Taken together, these results shift associative memory away from a single “patterns per neuron” number. Capacity becomes a geometry-sensitive and trajectory-sensitive object: it depends on how patterns are arranged in input space, on their correlations, and on the local crosstalk state induced by previous storage operations.

4. Thermodynamic, representational, and oscillatory phase spaces

Continuous Dense Associative Memory on the $n$0-sphere introduces a thermodynamic memory-capacity spectrum in the $n$1-plane, where $n$2 is the exponential load and $n$3 is temperature. The central object is the phase boundary $n$4 separating retrieval from spin-glass or paramagnetic behavior. In the sharp-kernel regime, both Gaussian (LSE) and Epanechnikov (LSR) kernels attain the same zero-temperature endpoint,

$n$5

but differ qualitatively at finite temperature. For LSE, the retrieval region extends to arbitrarily high temperatures as $n$6, yet spurious patterns always contribute to the noise floor. For LSR, finite support creates a threshold

$n$7

below which no spurious patterns contribute, producing a distinct sub-threshold retrieval regime. Here the memory-capacity spectrum is literally a phase diagram rather than a delay curve (Petrova et al., 8 Apr 2026).

A representational version of the spectrum appears in a biologically plausible two-layer dense associative memory. The older winner-takes-all hidden layer yielded capacity only linear in the number of hidden units, $n$8, because each hidden unit encoded one memory. Replacing that nonlinearity with a threshold rule allows distributed binary hidden codes, and in the regime $n$9 almost all hidden binary patterns become stable, giving

$C = l_y/\mathcal{T}_{x\rightarrow y}$0

This reframes memory-capacity spectrum as a continuum from localist to distributed codes: one-memory-per-hidden-unit at one end, combinatorial reuse at the other (Kafraj et al., 2 Jan 2026).

An even more explicit oscillatory construction is the Kuramoto-based associative memory on a 1D honeycomb graph. With $C = l_y/\mathcal{T}_{x\rightarrow y}$1 cycles of size $C = l_y/\mathcal{T}_{x\rightarrow y}$2, the number of stable phase-locked configurations is

$C = l_y/\mathcal{T}_{x\rightarrow y}$3

while the number of oscillators is $C = l_y/\mathcal{T}_{x\rightarrow y}$4, so the capacity

$C = l_y/\mathcal{T}_{x\rightarrow y}$5

grows exponentially in network size. The model also proves that there are no stable phase-locked configurations besides those identified in the constructive count, hence no spurious memories. In this setting the spectrum is architectural: it is induced by graph topology and independent per-cycle winding choices rather than by delay or thermodynamic variables (Guo et al., 4 Apr 2025).

These formulations all replace the usual storage-load threshold by a structured state space. Phase boundaries, representational regimes, and topological winding sectors all function as memory-capacity spectra because they resolve which configurations remain retrievable under changing load, temperature, or graph geometry.

5. Information-theoretic and communication formulations

In stochastic thermodynamics, the relevant quantity is sensory capacity,

$C = l_y/\mathcal{T}_{x\rightarrow y}$6

where $C = l_y/\mathcal{T}_{x\rightarrow y}$7 is the learning rate of the instantaneous sensor state and $C = l_y/\mathcal{T}_{x\rightarrow y}$8 is the transfer entropy rate carried by the full sensor trajectory. Adding a downstream memory $C = l_y/\mathcal{T}_{x\rightarrow y}$9 to a bare sensor $$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$0 increases $$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$1 but leaves $$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$2 unchanged, so capacity can only increase. This yields a genuine memory–capacity curve parameterized by memory noise, timescale, or internal resources. The same paper proves a universal thermodynamic trade-off: if $$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$3, then the efficiency $$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$4 must satisfy $$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$5. Memory therefore improves capacity at an energetic cost (Hartich et al., 2015).

For Gaussian MIMO channels with memory, channel memory induces a frequency-domain capacity spectrum. With noise-whitened channel

$$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$6

the capacity is

$$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$7

and the optimal power spectral density has matrix water-filling form

$$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$8

Here memory-capacity spectrum means the capacity density distributed across frequency and spatial eigenmodes created by intersymbol interference and colored noise (Loyka et al., 2022).

A related but non-stationary formulation appears for channels with sampled additive cyclostationary Gaussian noise of finite memory. Synchronous sampling yields a wide-sense cyclostationary, information-stable channel; asynchronous sampling yields a wide-sense almost-cyclostationary, generally non-information-stable channel. When a finite transmission delay is allowed, the asynchronous capacity is characterized as

$$c(\omega)=\sum_i \log\!\left(1+\lambda_i(\omega)d_i(\omega)\right)$$9

the limit inferior of capacities of approximating finite-memory WSCS channels. This exposes a spectrum over memory length, sampling synchronization, and initial phase rather than a single closed-form Shannon number (Dabora et al., 2023).

Gaussian thermal memory channels express the same idea through a spectral transmissivity

$\omega$0

with capacity obtained by integrating local Gaussian-channel contributions over $\omega$1 and allocating energy $\omega$2 by a water-filling-type rule. In this literature the memory-capacity spectrum is the continuum of effective Gaussian subchannels produced by a bosonic memory mode and thermal environment, together with the cutoff behavior that appears above a critical temperature (Palma et al., 2014).

6. Resource- and workload-indexed spectra in computing systems

A systems interpretation appears in paging with dynamic memory capacity. The cache size may grow and shrink over time, and classic static $\omega$3-competitiveness no longer suffices. The exact optimal dynamic competitive ratio is

$\omega$4

This creates a capacity spectrum indexed by instantaneous memory size $\omega$5. It also separates robust and non-robust algorithms: LRU, FIFO, CLOCK, FWF, MARK, and RAND achieve the optimal dynamic ratio, whereas LFRU, despite having an optimal static $\omega$6-competitive ratio, has no finite dynamic $\omega$7-competitive ratio for any $\omega$8 and arbitrarily large $\omega$9 (Peserico, 2013).

An analogous workload-indexed usage appears in Spark memory configuration. WSMC defines the Data Expansion Ratio

$i$0

classifies workloads as Expanding, Medium, or Shrinking, and then refines Expanding into Expanding.Rapid and Expanding.Medium by the Increasing Rate of Data Expansion. The resulting workload categories are assigned category-specific shuffle factors $i$1, which then drive a memory requirement prediction model based on input size, shuffle size, parallelism, and block size. In evaluation, WSMC can save over $i$2 memory capacity with the workload performance slight degradation (only $i$3), and compared to the proper configuration found out manually, the configuration with the guide of WSMC leads to only $i$4 increase in the memory waste with the workload's performance slight improvement (about $i$5) (Liang et al., 2017).

In these systems papers, “memory-capacity spectrum” does not mean pattern recall or delay decoding. It means that usable capacity is indexed by fluctuating resource state or workload class. A plausible implication is that the same conceptual move recurs: fixed-capacity summaries are replaced by profiles over operating regimes.

7. Broader neural-network continuum and recurring themes

A broader neural-network formulation appears in the capacity scaling law for perceptron networks. There the relevant critical points are the lossless memory dimension and the MacKay dimension. For a network with $i$6 weights,

$i$7

This yields a three-region capacity spectrum: full memorization for $i$8, partial memorization for $i$9, and under-capacity beyond $u_{t-\tau}$0. The formulation is explicitly information-theoretic: the network is embedded into Shannon’s communication model, and LM and MK are interpreted as capacities measured in bits (Friedland et al., 2017).

The survey of neural memory models places another continuum underneath these results. Unbounded Hebbian synapses provide unrealistically large capacities; simple bounded synapses enforce severe plasticity–stability trade-offs; sparse representations and correlated-memory compression alter the effective spectrum; the cascade model yields memory signal $u_{t-\tau}$1 with lifetime $u_{t-\tau}$2; and the bidirectional cascade model yields memory signal $u_{t-\tau}$3 with lifetime $u_{t-\tau}$4. In this usage, the spectrum is a continuum across synaptic complexity, sparseness, and retrieval assumptions rather than a single analytic curve (Fusi, 2021).

Across these literatures, several recurring principles emerge. First, total capacity is rarely the decisive object; the distribution of capacity across delays, orders, phases, or modes usually matters more. Second, increasing nonlinearity does not uniformly improve useful memory: ESNs near loss of ESP, microrings in self-pulsing regimes, high-degree DAMs on correlated data, and sensors driven toward $u_{t-\tau}$5 all expose compensating losses in stability, robustness, or efficiency (Singh et al., 24 Jul 2025, Castro et al., 2024, Bielmeier et al., 2 Aug 2025, Hartich et al., 2015). Third, topology and representation are often capacity-generating mechanisms in their own right: cycle structure in Kuramoto memories, hidden-code distributedness in two-layer DAMs, and eigenvalue placement in reservoirs all redistribute or enlarge the spectrum without changing the basic readout architecture (Guo et al., 4 Apr 2025, Kafraj et al., 2 Jan 2026, Singh et al., 24 Jul 2025).

The term therefore names not one theory but a unifying methodological move. A memory-capacity spectrum is any technically precise decomposition that shows how a finite memory resource is allocated, shifted, or degraded across the relevant coordinates of a system.