Supervised learning of spatial features with STDP and homeostasis using Spiking Neural Networks on SpiNNaker (2312.02659v2)

Abstract: Artificial Neural Networks (ANN) have gained significant popularity thanks to their ability to learn using the well-known backpropagation algorithm. Conversely, Spiking Neural Networks (SNNs), despite having broader capabilities than ANNs, have always posed challenges in the training phase. This paper shows a new method to perform supervised learning on SNNs, using Spike Timing Dependent Plasticity (STDP) and homeostasis, aiming at training the network to identify spatial patterns. Spatial patterns refer to spike patterns without a time component, where all spike events occur simultaneously. The method is tested using the SpiNNaker digital architecture. A SNN is trained to recognise one or multiple patterns and performance metrics are extracted to measure the performance of the network. Some considerations are drawn from the results showing that, in the case of a single trained pattern, the network behaves as the ideal detector, with 100% accuracy in detecting the trained pattern. However, as the number of trained patterns on a single network increases, the accuracy of identification is linked to the similarities between these patterns. This method of training an SNN to detect spatial patterns may be applied to pattern recognition in static images or traffic analysis in computer networks, where each network packet represents a spatial pattern. It will be stipulated that the homeostatic factor may enable the network to detect patterns with some degree of similarity, rather than only perfectly matching patterns.The principles outlined in this article serve as the fundamental building blocks for more complex systems that utilise both spatial and temporal patterns by converting specific features of input signals into spikes.One example of such a system is a computer network packet classifier, tasked with real-time identification of packet streams based on features within the packet content

• The paper shows that supervised learning with STDP and homeostasis effectively trains SNNs to recognize spatial patterns by adjusting synaptic weights.
• The study uses the SpiNNaker platform to simulate large-scale neural networks, achieving 100% accuracy for single pattern detection and detailed metrics for multiple patterns.
• Results demonstrate that performance varies with pattern complexity and Hamming distances, highlighting potential applications in traffic analysis and image recognition.

Understanding Supervised Learning in Spiking Neural Networks on SpiNNaker

In the field of artificial intelligence and machine learning, Spiking Neural Networks (SNNs) represent a third generation of neural network models that incorporate a key dimension in their architecture: time. These biologically inspired networks have demonstrated promise in various applications, but one significant hurdle has been training them effectively. Unlike traditional Artificial Neural Networks (ANNs) that rely on the famed backpropagation algorithm, SNNs pose unique challenges given their dynamic and temporally sensitive nature.

However, recent advances have shown that supervised learning, specifically through Spike Timing Dependent Plasticity (STDP) and homeostasis, can successfully train SNNs. A research team detailed their approach to teaching an SNN to recognize spatial patterns—patterns consisting of concurrent spikes that represent a form of data with no temporal component, akin to a static image or the cross-sectional profile of network traffic data.

The core of this supervised learning framework involves two main processes, STDP and homeostasis, both of which are deeply rooted in neuroscience. STDP is a learning mechanism where the connection strength (synaptic weight) between two neurons is increased or decreased based on the precise timing of their firing. Homeostasis is a balancing act maintaining the network’s stability, ensuring that neurons don't become too active or inactive. Together, these mechanisms were used to manage synaptic weights to recognize specific input patterns.

Using the SpiNNaker platform, a cutting-edge, parallel computing system designed to simulate large sets of neurons, the researchers trained SNNs to recognize single or multiple spatial patterns. The researchers calculated specific performance metrics such as accuracy, precision, and specificity to evaluate their model’s effectiveness.

The findings revealed that when the network was trained with a single pattern, it functioned as an "ideal detector" with a perfect accuracy score. This means the network was 100% correct in identifying the pattern it was taught to recognize. The researchers then added complexity by training the network on two and three patterns, observing that the performance metrics responded to the similarity between the multiple patterns trained.

When two patterns were used, the network's ability to differentiate between the patterns hinged on their Hamming distance—a measure of difference between patterns. As the distance increased, recognition performance declined due to the increasing numbers of "don’t care" synapses which have little to no influence on the output.

Adding a third pattern training demonstrated even more layered complexity. At times, one or two of the patterns trained couldn’t be recognized owing to the vast dissimilarity to other patterns, thus leading to discrepancies in the model's performance.

The breakthrough in this research is not limited to theoretical constructs but extends to practical applications. The demonstrated techniques can be especially valuable in domains like traffic analysis in computer networks, where each packet can be treated as a spatial pattern, or in image recognition tasks, converting images into spike sequences for an SNN to process.

The research on SpiNNaker opens doors beyond what traditional neural networks have offered to this point. By leveraging the intrinsic properties of time and biological learning rules through STDP and homeostasis, SNNs could pave the way toward more nuanced and efficient forms of computation that mirror the dynamics of the human brain. This development marks significant progress in the quest for more sophisticated artificial intelligence, inching closer to the capabilities of biological neural systems.