GHz spiking neuromorphic photonic chip with in-situ training (2506.14272v1)

Abstract: Neuromorphic photonic computing represents a paradigm shift for next-generation machine intelligence, yet critical gaps persist in emulating the brain's event-driven, asynchronous dynamics,a fundamental barrier to unlocking its full potential. Here, we report a milestone advancement of a photonic spiking neural network (PSNN) chip, the first to achieve full-stack brain-inspired computing on a complementary metal oxide semiconductor-compatible silicon platform. The PSNN features transformative innovations of gigahertz-scale nonlinear spiking dynamics,in situ learning capacity with supervised synaptic plasticity, and informative event representations with retina-inspired spike encoding, resolving the long-standing challenges in spatiotemporal data integration and energy-efficient dynamic processing. By leveraging its frame-free, event-driven working manner,the neuromorphic optoelectronic system achieves 80% accuracy on the KTH video recognition dataset while operating at ~100x faster processing speeds than conventional frame-based approaches. This work represents a leap for neuromorphic computing in a scalable photonic platform with low latency and high throughput, paving the way for advanced applications in real-time dynamic vision processing and adaptive decision-making, such as autonomous vehicles and robotic navigation.

• The paper presents a photonic chip that achieves a 4 GHz spiking rate with in-situ supervised learning using a ReSuMe-based STDP rule.
• It utilizes retina-inspired spike encoding to process human action video with 80% accuracy on the KTH benchmark, enhancing energy efficiency and reaction speed.
• The chip boasts a computing density over 114.3 GMAC/s/mm² and offers a two-order magnitude speed improvement, paving the way for advanced edge computing.

GHz Spiking Neuromorphic Photonic Chip with In-Situ Training

Introduction

The paper presents a novel GHz spiking neuromorphic photonic chip designed for bioinspired computing architectures that mimic human brain dynamics. This advancement addresses the existing challenges in neuromorphic computing by integrating spiking neural networks (SNN) on a CMOS-compatible silicon platform. It leverages high-speed nonlinear spiking dynamics, in-situ supervised learning, and retina-inspired spike encoding, thus catering to dynamic real-time vision processing applications with improved energy efficiency and processing speeds.

Silicon Neuromorphic Photonic Chip

The proposed photonic chip is structured to emulate neuron-like behaviors using an optoelectronic microring spiking neuron design that achieves a firing rate of 4 GHz. This setup circumvents previous limitations of materials like III-V, while maintaining high scalability on silicon. The communication control loops employed support a bio-plausible in-situ training mechanism utilizing synaptic plasticity. The transformative information encoding draws from biological retina mechanisms, enabling sparse spike trains that mitigate data redundancy, aligning seamlessly with the event-driven architecture of SNNs.

Bio-Plausible In-Situ Training

To address the gap between simulation and real-world application, the paper introduces a supervised in-situ training mechanism for the photonic SNN chip. The training paradigm integrates multiple hardware and software components, creating a feedback loop for continuous learning and adaptation. The key innovation here is the use of a ReSuMe-based STDP learning rule that adjusts synaptic weights based on precise spike timing information, paving the way for real-time adaptability and improved learning efficiency directly on the chip without external influence.

Human Action Video Recognition

The photonic chip demonstrates considerable potential in real-time human action recognition by exploiting a frame-free processing approach. Utilizing retina-inspired spike encoding, it processes video streams by isolating temporal changes, significantly optimizing information bandwidth. The chip achieves an 80% accuracy rate on the KTH benchmark dataset, a noteworthy achievement given its operational speed and latency advantages. The inherent sparsity and dynamic response of the encoding mechanism result in reduced data processing overhead and power consumption relative to traditional frame-based sensors and digital processors.

Performance Metrics and Comparison

The neuromorphic photonic system is characterized by a computing density exceeding 114.3 GMAC/s/mm² and energy efficiency above 2.75 GSOP/W. Frame processing translates to effective handling at speeds reaching 15.93 MHz. Notably, compared to existing platforms, this photonic processing paradigm offers a substantial two-order magnitude improvement in speed owing to its event-driven processing strategy, setting new benchmarks for efficiency in neuromorphic computation.

Conclusion

The research delineated in this paper marks a significant step forward in bridging neuroscience-inspired processes and silicon-based photonic technology. By achieving ultrafast, scalable, and integrated spiking neuron processing, the chip addresses long-standing limitations of neuromorphic computing. This development opens immense possibilities for low-power, high-speed applications tailored for edge computing in autonomous systems and real-time sensory networks. The work not only advances photonic SNNs but also redefines the potential for seamless integration of neuromorphic vision sensors in diverse applications, demonstrating the viability of complex bio-plausible systems within practical contexts.