Deep Learning with Coherent Nanophotonic Circuits (1610.02365v1)

Abstract: Artificial Neural Networks are computational network models inspired by signal processing in the brain. These models have dramatically improved the performance of many learning tasks, including speech and object recognition. However, today's computing hardware is inefficient at implementing neural networks, in large part because much of it was designed for von Neumann computing schemes. Significant effort has been made to develop electronic architectures tuned to implement artificial neural networks that improve upon both computational speed and energy efficiency. Here, we propose a new architecture for a fully-optical neural network that, using unique advantages of optics, promises a computational speed enhancement of at least two orders of magnitude over the state-of-the-art and three orders of magnitude in power efficiency for conventional learning tasks. We experimentally demonstrate essential parts of our architecture using a programmable nanophotonic processor.

• The paper presents a fully optical neural network that leverages coherent nanophotonic circuits to achieve significant improvements in speed and energy efficiency.
• The authors implement matrix multiplications via optical interference units and singular value decomposition on a nanophotonic processor for vowel recognition tasks.
• The ONN architecture attains up to 100x speed enhancement and 1000x better power efficiency compared to electronic neural networks, paving the way for scalable AI.

Deep Learning with Coherent Nanophotonic Circuits

The paper presents the development and demonstration of a fully optical neural network (ONN) architecture that leverages the inherent advantages of photonics to achieve a significant increase in computational speed and energy efficiency over conventional electronic implementations. Utilizing a nanophotonic processor, the authors address the limitations imposed by traditional electronic computing architectures, particularly with regards to artificial neural networks (ANNs).

Architectural Overview

The proposed ONN design capitalizes on photonic technologies to implement neural network computations with high speed and energy efficiency. This architecture exploits the capabilities of integrated photonic circuits to perform linear transformations, such as matrix multiplications, and certain nonlinear functions at the speed of light, achieving minimal energy consumption. Specifically, the paper describes the implementation of matrix operations using optical interference units (OIUs) through singular value decomposition (SVD) methods. The design reduces power consumption by implementing matrix multiplications through interference patterns, inherently conserving the optical energy within the system.

Experimental Implementation

The experimental validation involved constructing a two-layer, fully-connected neural network using a programmable nanophotonic processor with an array of Mach-Zehnder interferometers for vowel recognition tasks. Impressively, the ONN demonstrated a level of accuracy that rivals conventional digital computers, vindicating the feasibility and practicality of the optical approach. The work primarily attributes the remarkable power efficiency to the minimal consumption involved in matrix multiplications, an essential aspect of ANN computations.

Performance Metrics and Results

The results illustrate that the ONN can achieve a speed enhancement of at least two orders of magnitude and a three orders of magnitude improvement in power efficiency compared to typical electronic neural networks. The work includes comprehensive modeling of noise effects, including phase encoding and photodetection errors, to predict and understand the system's limitations and potential accuracy degradation.

Theoretical and Practical Implications

This research underscores the potential for ONNs to transform neural network execution, particularly in performing forward propagation during inference tasks where compactness, speed, and energy considerations are paramount. The optical approach not only outperforms in terms of speed and efficiency but also offers a pathway to seamless scaling in more extensive networks through coherent photonic devices.

Future Developments

The paper signals possible advancements in training methodologies by leveraging the optical framework's unique advantages. It suggests the prospect of direct gradient estimation, which could sidestep traditional, computationally expensive training paradigms like backpropagation.

Future explorations are poised to integrate additional optical nonlinear components to achieve all-optical implementations. Integrating phase-change materials for non-volatile computing and extending the photonic architectures to accommodate larger neural network models and more complex algorithms, such as convolutional and recurrent neural networks, remain vital stepping stones toward realizing the full potential of this paradigm.

In summary, this paper provides a detailed account of the conceptualization, implementation, and evaluation of ONNs and suggests an innovative path forward for highly efficient, scalable, and sustainable AI computing architectures.