Reservoir Computing Metrics

• Reservoir Computing Metrics are a set of quantitative measures that assess the memory, computational power, nonlinearity, and reliability of dynamical systems used in time-series processing.
• These metrics include task-specific measures like short-term memory and parity check capacities, as well as task-independent ones such as kernel rank, generalisation rank, and spatially resolved statistics.
• They guide both hardware-software co-design and parameter tuning in diverse platforms—from skyrmion systems to magneto-ionic heterostructures—ensuring optimized trade-offs for temporal signal processing.

Reservoir computing metrics quantify the computational power, memory, nonlinearity, and reliability of dynamical systems that serve as reservoirs for time-series processing or classification tasks. These metrics encompass both hardware-agnostic analytical tools and specialized measures reflecting the physical or algorithmic substrate. Contemporary reservoir research uses a diverse array of task-specific, task-independent, spatial, and statistical metrics to systematically assess and optimize reservoir quality for temporal information processing.

1. Memory Capacity and Nonlinear Capacity

The foundational metrics in reservoir computing are short-term memory (STM) capacity and parity check (PC) capacity. STM captures the linear recall of previous inputs, while PC quantifies the reservoir's ability to reconstruct nonlinear functions such as parity over delayed inputs.

Formally, for a binary input stream $u(n)$ and reservoir state vector $h(n)$ sampled at $N$ virtual nodes, the target for STM at delay $d$ is $u(n-d)$, and for PC it is $$p_d(n) = [u(n) \oplus u(n-1) \oplus \cdots \oplus u(n-d)] \bmod 2$$. Training proceeds via ordinary least squares to fit $w$ such that $s_\mathrm{ro}(n) = w \cdot h(n)$ best reconstructs the target over held-out test data. The squared Pearson correlation at each delay gives $\mathrm{Cor}_d$, yielding:

$$C_\mathrm{STM} = \sum_{d=1}^{D_\mathrm{max}} (\mathrm{Cor}_d)^2, \qquad C_\mathrm{PC} = \sum_{d=1}^{D_\mathrm{max}} (\mathrm{Cor}_d^\mathrm{parity})^2$$

These metrics are widely used to benchmark both physical and emulated reservoirs, notably in magneto-ionic heterostructures (Rajib et al., 9 Dec 2024) and skyrmion-based systems (Rajib et al., 2021). For instance, MI reservoirs achieve STM = 1.44, PC = 2 (over 24 virtual nodes), while skyrmion islands may reach up to STM ≈ 4.4, PC ≈ 4.6.

2. Task-Independent Metrics: Kernel Rank, Generalisation Rank, Linear Memory Capacity

A distinct class of task-independent metrics quantifies the structural properties of the reservoir mapping itself (Vidamour et al., 2021):

• Kernel Rank (KR): Measures the rank of the output matrix $\mathbf{O}$ formed by the responses to $N$ distinct input streams; reflects nonlinear separability.
• Generalisation Rank (GR): The rank of the output matrix $\mathbf{O}'$ when driven by input streams sharing some ending sequence; low GR implies the reservoir collapses similar sequences as desired.
• Linear Memory Capacity (MC): The sum over delays of squared correlation between linear readout output and delayed input, matching the classical Jaeger MC.

$$\mathrm{KR} = \mathrm{rank}(\mathbf{O}), \quad \mathrm{GR} = \mathrm{rank}(\mathbf{O}')$$

$$\mathrm{MC} = \sum_{k=1}^{K} \frac{\mathrm{cov}^2(u_{i-k}, y_k)}{\mathrm{var}(u_{i-k})\,\mathrm{var}(y_k)}$$

Simultaneous optimization over KR, GR, and MC predicts high downstream accuracy for complex temporal tasks such as spoken digit recognition and sequential MNIST (Vidamour et al., 2021).

3. Spatially-Resolved and Local Metrics

Physical reservoirs with spatially distributed states, notably skyrmion and magnetic texture systems, motivate spatially resolved, task-agnostic metrics (Love et al., 2021):

• Local Memory Capacity ($\mathrm{MC}_n$): At readout node $n$, sum over delays of coefficient-of-determination between reconstructed and actual past input, leveraging local neighborhoods.
• Local Nonlinearity ($\mathrm{NL}_n$): Degree to which node $n$'s output cannot be predicted linearly from inputs; $\mathrm{NL}_n = 1 - R^2[y_n(t), \widehat{y}_n(t)]$.

Global averages, $\overline{\mathrm{MC}}$ and $\overline{\mathrm{NL}}$, determine how tuning spatial parameters shifts reservoir behavior, empirically yielding a trade-off: high nonlinearity generally diminishes fading memory, and optimal device operation follows a negative correlation line in $\overline{\mathrm{MC}}, \overline{\mathrm{NL}}$-space.

4. Consistency Spectrum, Capacity Hierarchy, and Multivariate Analysis

Multivariate correlation analysis yields the consistency spectrum $\{\gamma_k^2\}$ and total consistency capacity $\Theta$ (Jüngling et al., 2021):

$$C_c = \langle \tilde{x}(t) \tilde{x}'(t)^T \rangle, \quad C_c = Q_c G^2 Q_c^T, \quad \mathcal{C}_k = \gamma_k^2, \quad \Theta = \operatorname{Tr} C_c = \sum_{k=1}^N \gamma_k^2$$

Partial replica tests define a hierarchy of capacities $\Theta_{i_1...i_\omega}$ for channel or temporal subsets, with inclusion-exclusion yielding strictly nonlinear (multiplicative) contributions. The time-resolved capacity (fading memory profile) $\Phi(\tau)$ resolves recall at varying delays. This framework generalizes to all dynamical reservoir systems and reveals how parameter choices alter the number of "informational directions" preserved in output space.

5. Prediction Error and Probabilistic Metrics

For benchmarking predictive performance and reliability, normalized error measures and probabilistic scores are widely used (Goudarzi et al., 2014, Guerra et al., 2023, Kumar et al., 2021):

Error-based metrics:

• RNMSE (root normalized mean squared error):

$$\operatorname{RNMSE} = \sqrt{\frac{\langle (y(t)-\hat{y}(t))^2 \rangle}{\sigma_{\hat{y}}^2}}$$

• NRMSE (normalized RMSE by range):

$$\operatorname{NRMSE} = \frac{\sqrt{\langle (y(t)-\hat{y}(t))^2 \rangle}}{\max_t \hat{y}(t) - \min_t \hat{y}(t)}$$

• SAMP (Symmetric Absolute Mean Percent Error):

$$\operatorname{SAMP} = 100 \times \left\langle \frac{|y(t)-\hat{y}(t)|}{|y(t)| + |\hat{y}(t)|} \right\rangle$$

• NMSE, MAE, RMSE, and Word Error Rate (WER) are standard for classification and forecasting.

Probabilistic and interval metrics:

• Continuous Ranked Probability Score (CRPS), Negative Log-Likelihood (NLL), Prediction Interval Coverage Probability (PICP), and calibration curve error—all critically assess uncertainty quantification.

6. Synchronization-Based Metrics for Oscillator Reservoirs

Oscillator-based reservoirs, including VO$_2$ and spintronic systems, use specialized synchronization measures (Velichko et al., 2020):

• Fractional High-Order Synchronization Value (SHR$_{ij}$): Ratio $M_j / M_i$ for dominant subharmonic locking order between oscillators $i$ and $j$, revealing nonlinear dynamical coordination.
• Synchronization Efficiency ($u_{ij}$): Percentage of total runtime spent in the dominant synchronized state:

$u_{ij} = \frac{t_{M_i, M_j}}{T} \times 100\%$

These metrics collectively characterize the richness and persistence of synchronization plateaus, which in turn provide high-dimensional embeddings for reservoir computation tasks.

7. Dimension Estimation Metrics

Assessment of the attractor dimension of the reservoir’s states reveals manifold capacity and correlations to test error (Carroll, 2019):

• False Nearest Neighbor (D$_\mathrm{fnn}$): Minimum embedding dimension so that adding more coordinates does not alter neighborhood relationships.
• Covariance Dimension (D$_c$): Minimum dimension for which local clusters exhibit anisotropic covariance exceeding Gaussian noise bounds.
• Kaplan-Yorke Dimension (D$_\mathrm{KY}$): Computed from the Lyapunov spectrum as $$D_\mathrm{KY} = j + \frac{\sum_{i=1}^j \lambda_i}{|\lambda_{j+1}|}$$, where $j = \max \{\sum_{i=1}^j \lambda_i \geq 0\}$.

Matching reservoir fractal dimension to that of the training signal minimizes test error; excessive dimension (e.g., via increased spectral radius) yields over-fractalization and degraded performance.

8. Comparative Benchmarks and Substrate-Specific Analysis

Modern literature provides comparative tables of STM and PC capacities across hardware implementations, substantiating how physical interactions (e.g., ion migration, spin-wave versus dipole coupling) set capacity limits in magneto-ionic (Rajib et al., 9 Dec 2024), skyrmion (Rajib et al., 2021), and nano-oscillator (Vidamour et al., 2021) substrates. Quantum reservoir computing further distinguishes substrate statistics (bosonic, fermionic, spin) by linear/nonlinear memory profiles, showing fermions maximize linear recall under single-excitation, while bosonic multi-level injection can harness higher Hilbert-space dimensions (Llodrà et al., 2023).

Reservoir Platform	STM Capacity	PC Capacity	Energy per pulse
Magneto-ionic heterostructure (Rajib et al., 9 Dec 2024)	1.44	2	n/a
Vortex-core magnetic (Vidamour et al., 2021)	~1.5	~1.5	n/a
Spin-torque nano-oscillator (Vidamour et al., 2021)	0.5–1.0	0.5–1.0	n/a
Optical fiber loop (Vidamour et al., 2021)	10–50	3–10	n/a
Memristive RC (Vidamour et al., 2021)	5–10	2–4	n/a
Skyrmion discontinuous (Rajib et al., 2021)	4.4	4.6	~50 fJ

These benchmark results illustrate the range of capacity values and show how underlying physics and device engineering affect algorithmic capability.

9. Methodological Significance and Application

Reservoir computing metrics serve dual purposes: (i) guiding hardware-software co-design to achieve desired temporal recall and nonlinear transformation properties for target tasks; (ii) rapidly screening parameter regimes for optimal trade-offs between fading memory, nonlinearity, and generalization, leveraging task-agnostic or spatially parallel implementations whenever possible.

Systematic metric-based assessment, spanning STM, PC, MC, KR/GR, synchronization values, error rates, and consistency spectra, underpins the comparative evaluation of architectures from analog photonic delay lines to solid-state magnetic-ion reservoirs and quantum nets. Optimization protocols rely on these metrics to match device dynamical characteristics to computational workload requirements and energy constraints.

In summary, reservoir computing metrics deliver a rigorous, substrate-neutral framework for quantifying and tuning dynamical system suitability for temporal signal processing, with reproducible procedures and universally interpretable benchmarks now standard across state-of-the-art platforms.