Power law scaling for classification accuracy in physical neural networks

Abstract: Physical neural networks (PNNs) harness the intrinsic complexity of physical systems to perform neural computation, potentially at speeds and energy efficiencies inaccessible to conventional digital hardware. Yet, a principled framework for quantifying and predicting their computing accuracy across diverse substrates has remained elusive. Here we introduce the Hotelling Trace Criterion (HTC), a task-conditioned measure of PNN- state separability that can be evaluated without training. We demonstrate that it predicts PNN classification performance with high fidelity across highly nonlinear optical fibres, vertical-cavity surface-emitting lasers, and coupled nonlinear oscillator networks, for benchmark tasks of different difficulty. Classification loss follows a power law in HTC, with Pearson correlation coefficients exceeding 0.99 for MNIST and $\approx$0.97 for Fashion-MNIST, noteworthy experimental and simulated data from physically distinct systems collapse onto a single scaling curve determined by the task rather than the substrate. Applying HTC layer-by-layer during training further reveals that gradient-based optimisation distributes representational capacity unevenly across PNN layers, providing a quantitative diagnostic of training and architecture efficiency invisible to standard loss monitoring. Crucially, once the scaling exponent is established from a small number of trained calibration systems, all further performance predictions require no training since performance can be derived from the much more efficient HTC measurement. These results establish HTC as a substrate-agnostic figure of merit for comparing and scaling PNNs, advancing the field further towards a complete theory connecting fundamental hardware parameters to task performance through universal scaling laws.

• The paper introduces a substrate-agnostic Hotelling Trace Criterion (HTC) for training-free performance prediction in physical neural networks.
• Key empirical results reveal universal power law scaling between HTC and classification error across diverse PNN platforms and benchmark datasets.
• Layer-wise HTC analysis provides insights into dynamic class separability, offering practical guidance for optimizing PNN architectures.

Power Law Scaling for Classification Accuracy in Physical Neural Networks

Introduction

Physical Neural Networks (PNNs) leverage the intrinsic nonlinear dynamics and high-dimensional state-space of physical substrates to perform neural computations, potentially surpassing the performance and energy efficiency of digital hardware. This paper, "Power law scaling for classification accuracy in physical neural networks" (2606.31588), introduces a substrate-agnostic methodology—rooted in the Hotelling Trace Criterion (HTC)—for predicting and comparing the classification performance of such systems. By demonstrating universal power law scaling of classification error with HTC across disparate PNN platforms, the authors lay theoretical and practical foundations for task-informed design, benchmarking, and scaling of PNN architectures.

Figure 1: Schematic depiction of three PNN implementations—a highly-nonlinear optical fiber (HNLF), large-area VCSEL, and coupled nonlinear oscillator network (CNON)—illustrating diverse realizations of physical computation analogous to multilayer ANNs.

Methodology: The Hotelling Trace Criterion and Its Application to PNNs

Current metrics for analyzing PNN computing capacity—including PCA-based dimensionality, information processing capacity [dambre2012information], and eigentask analysis [tureci2023]—are inherently task-agnostic and often fail to correlate with practical classification accuracy. The HTC, a classical statistical measure of cluster separability ($\mathrm{tr}(\mathbf{S}_W^{-1}\mathbf{S}_B)$), directly quantifies the ratio of between-class variance to within-class variance for hidden-layer representations, encapsulating the geometric separability required for optimal classification.

In all investigated platforms—experimental HNLF, LA-VCSEL, and numerical CNON—the HTC can be computed from the hidden layer response matrix without any system training, making it a highly efficient performance predictor. The criterion is not only theoretically grounded (equivalent to the sum of non-zero eigenvalues representing class separability along principal directions) but also empirically validated in this study.

Figure 2: Illustration of HTC projections in CNON for low- and high-performance regimes, highlighting the separability/overlap of class clusters in hidden-layer space.

Empirical Results: Universal Power Law Scaling

Across PNN systems and benchmark datasets (MNIST, fashion MNIST), a robust negative correlation is observed between classification loss (MSE) and HTC, spanning both physical experiments and numerical simulations. This relationship strictly follows a task-dependent power law: $\mathrm{MSE} \propto \mathrm{HTC}^{\zeta}$, with $\zeta_{\mathrm{MNIST}} = -0.47$ and $\zeta_{\mathrm{fMNIST}} = -0.29$, and Pearson coefficients of $>0.99$ (MNIST) and $\approx 0.96$ (fashion MNIST). Crucially, data from physically distinct PNN substrates collapse onto a single scaling curve for a given task, confirming the substrate-independence of the scaling exponent.

Figure 3: Power law scaling of classification performance with HTC across HNLF, LA-VCSEL, and CNON PNNs for MNIST and fashion MNIST; experimental and simulation results coalesce onto universal task-defined scaling curves.

This universality enables training-free performance prediction: once the power law exponent is established via a small number of calibration runs, any new PNN implementation (given hidden-layer responses) can be benchmarked for task-specific accuracy using only HTC measurements.

Layer-wise Analysis and Training Diagnostics

Applying HTC analysis to multi-layer CNON networks, the authors demonstrate its utility as a dynamic, layer-wise diagnostic tool. During gradient-based training, HTC reveals (with high granularity) how representational capacity and class separability are distributed and evolved across layers. In the investigated three-layer CNON, only the latter layers align with the global power law, indicating early layer underutilization and marginal accuracy improvement with additional depth. This exposes architectural inefficiency not discernible via global loss monitoring and suggests potential optimization directions in both PNN and ANN contexts.

Figure 4: Evolution of HTC across layers and training epochs in CNON networks, revealing layer-wise separation dynamics and alignment (or divergence) with universal scaling laws.

Practical and Theoretical Implications

The establishment of task-conditioned, substrate-independent scaling laws offers several implications:

• Benchmarking and Design: HTC provides a training-free, predictive metric for substrate selection, hardware development, and system comparison in PNNs, reframing the optimization problem as maximizing class-conditioned separability.
• Training Efficiency: Layer-resolved HTC trajectories inform adaptive training strategies, highlight underutilized regions, and potentially reduce the cost of architectural exploration.
• Theory Development: The HTC framework bridges the gap between fundamental physical parameters (dimensionality, nonlinear response) and task-specific performance, laying groundwork for universal theory in physical computing and scaling laws analogous to Moore’s Law and AI compute-performance scaling [kaplan2020scaling, hoffmann2022training].

Future Directions

The authors note that while HTC is highly effective for classification, extensions to regression (or additional metrics for regression-specific tasks) remain open challenges. Further consolidation of universal scaling laws is required, encompassing broader PNN substrates, more diverse datasets, and deeper dynamic ranges. Integrating task-agnostic upper bounds (via PCA or eigentask analysis) with HTC-informed predictions could complete the theoretical loop—from substrate properties to accurate performance forecasting.

Conclusion

This work demonstrates that physical neural networks exhibit universal, task-dependent power law scaling of classification error with class separability, as quantified by the Hotelling Trace Criterion. The ability to predict PNN performance across disparate substrates without system training advances both practical benchmarking and theoretical understanding. HTC emerges as a potent figure of merit, fundamentally connecting the geometry of physical system responses to neural computation, and sets the stage for future developments in substrate-agnostic, scalable, and efficient physical computing architectures.