Nonlinear unitary circuits for photonic neural networks

Abstract: Photonics has unlocked the potential for energy-efficient acceleration of deep learning. Most approaches toward photonic deep learning have diligently reproduced traditional deep learning architectures using photonic platforms, separately implementing linear-optical matrix calculations and nonlinear activations via electro-optical conversion, optical nonlinearities, and signal-encoded materials. Here we propose a concept of nonlinear unitary photonic circuits to achieve the integration of linear and nonlinear expressivity essential for deep neural networks. We devise a building block for two-dimensional nonlinear unitary operations, featuring norm-preserving mappings with nonconservative inner products, which enables the construction of high-dimensional nonlinear unitary circuits. Using deep nonlinear unitary circuits, we demonstrate exponential growth in trajectory length and near-complete coverage of the output space, both of which are essential for deep learning. Along with neuroevolutionary learning examples for the regression of a nonconvex function, our results pave the way to photonic neural networks with highly expressive inference and stable training.