An analog-electronic implementation of a harmonic oscillator recurrent neural network

Abstract: Oscillatory recurrent networks, such as the Harmonic Oscillator Recurrent Network (HORN) model, offer advantages in parameter efficiency, learning speed, and robustness relative to traditional non-oscillating architectures. Yet, while many implementations of physical neural networks exploiting attractor dynamics have been studied, implementations of oscillatory models in analog-electronic hardware that utilize the networks' transient dynamics so far are lacking. This study explores the feasibility of implementing HORNs in analog-electronic hardware while maintaining the computational performance of the digital counterpart. Using a digital twin approach, we trained a four-node HORN in silico for sequential MNIST classification and transferred the trained parameters to an analog electronic implementation. A set of custom error metrics indicated that the analog system is able to successfully replicate the dynamics of the digital model in most test cases. However, despite the overall well-matching dynamics, when using the readout layer of the digital model on the data generated by the analog system, we only observed $28.39\%$ agreement with the predictions of the digital model. An analysis shows that this mismatch is due to a precision difference between the analog hardware and the floating-point representation exploited by the digital model to perform classification tasks. When the analog system was utilized as a reservoir with a re-trained linear readout, its classification performance could be recovered to that of the digital twin, indicating preserved information content within the analog dynamics. This proof-of-concept establishes that analog electronic circuits can effectively implement oscillatory neural networks for computation, providing a demonstration of energy-efficient analog systems that exploit brain-inspired transient dynamics for computation.