Scalable Photonic Neural Networks via Surrogate Scattering-Matrix Inverse Design

Abstract: Inverse-designed nanophotonic media are a promising platform for compact optical neural networks, but training them end to end is expensive because each adjoint iteration couples the full-wave solver to the dataset minibatch, so the number of electromagnetic simulations scales with both the network depth and the batch size. We introduce a two-stage surrogate workflow that decouples task learning from electromagnetic realization. In the first stage, the trainable optical block is represented as a passive complex matrix with bounded singular values and the classification task is solved directly in matrix space at negligible cost. In the second stage, the selected target operator is transferred to a fabrication-aware freeform device through an adjoint problem driven by a Frobenius-norm transmission residual and a reflection penalty, which removes the minibatch dependence from the full-wave loop and yields a smoother loss landscape than intensity-domain cross-entropy. We further introduce a banded-router architecture composed with a fixed evanescent-coupling region, which exploits the bandwidth-additive property of matrix products to realize dense effective operators within a design region roughly half as long as a fully local router would require. The framework is validated on three tasks. On MedMNIST, the realized all-optical classifier reproduces the surrogate accuracy within $0.6$ percentage points after only 20 adjoint epochs. On RSSCN7, the banded router plus evanescent stage improves test accuracy by more than 15 percentage points over a linear readout baseline. A Yin-Yang task confirms that the same framework supports nonlinear decision boundaries. These results indicate that surrogate-guided inverse design is a practical route to training compact photonic processors with simulation budgets orders of magnitude smaller than direct geometry-to-task pipelines.

• The paper proposes a surrogate-guided inverse design that decouples data-driven learning from computationally expensive electromagnetic simulations.
• It demonstrates a two-stage methodology by first optimizing a surrogate scattering matrix and then applying fabrication-aware inverse design to drastically reduce simulation epochs.
• Experimental validation on MedMNIST, RSSCN7, and Yin-Yang tasks confirms high classification accuracy and the effective scalability of a compact photonic architecture.

Scalable Photonic Neural Networks via Surrogate Scattering-Matrix Inverse Design

Introduction

The paper "Scalable Photonic Neural Networks via Surrogate Scattering-Matrix Inverse Design" (2604.21301) introduces a rigorous methodology for training and realizing nanophotonic neural networks by decoupling data-driven learning from physics-based electromagnetic (EM) device optimization. The work addresses a fundamental scalability bottleneck in inverse-designed optical neural networks (ONNs): the high computational cost of geometry-to-task (end-to-end) optimization, which arises due to the necessity to run full-wave EM simulations for every data sample in each training minibatch. By establishing a two-stage workflow centered on surrogate matrix optimization followed by fabrication-aware EM inverse design, the approach achieves orders-of-magnitude reduction in simulation requirements and enables the scalable synthesis of compact photonic processors.

Methodology

Surrogate Operator Model

The core innovation resides in formulating the learning problem in a passive complex matrix space. In the surrogate stage, the trainable optical core is modeled as a bounded-singular-value matrix $M = U \Sigma(s) V^\dagger$, optimizing directly for classification accuracy (e.g., using cross-entropy) without recourse to EM simulations. Unitary parametrization ensures physical realizability and loss constraints. Feature data is optically encoded through amplitude or phase mappings and linearly combined by $M$, with detection yielding decision logits via output intensity. This surrogate optimization entirely sidesteps the cost of electromagnetic field simulation, allowing efficient exploration of network configurations and regularizers.

Fabrication-Aware Inverse Design

Once the optimal surrogate operator $T^\star$ is established, it serves as the transmission target for inverse design of a passive nanophotonic device. The physical device is parameterized as a continuous permittivity distribution, constrained by fabrication-aware projections (enforcing feature size and binarization). The realization stage employs an adjoint-based gradient descent, not on the task loss, but on the Frobenius norm between the simulated transmission matrix and $T^\star$, combined with a reflection suppression penalty. This batch-free objective dramatically reduces the number of required EM simulations—one forward and adjoint solve per epoch—independent of dataset size, and yields a much smoother optimization landscape compared to intensity-domain cross-entropy minimization.

Architectural Innovations

The paper further introduces a banded-router photonic architecture combined with a fixed evanescent-coupling region. This exploits the mathematics of sparse matrix factorization: even with locality-enforced banded matrices for the trainable stage, dense global mixing is achieved in the composite system due to the bandwidth-additive property of matrix multiplication. Consequently, hardware footprints are minimized and design regions are halved in size compared to fully dense mesh architectures, further enhancing scalability.

Experimental Validation and Numerical Results

The surrogate-guided design framework was validated on MedMNIST, RSSCN7, and a nonlinear “Yin-Yang” synthetic classification task:

• MedMNIST: A $6 \times 16$ all-optical passive classifier was trained (surrogate accuracy $98.75\%$). The physical realization reached $98.16\%$ test accuracy after just 20 adjoint epochs, with transmission loss closely matching the target (7.08 dB vs. 6.80 dB). This is an order-of-magnitude reduction in simulation compared to prior end-to-end EM inverse design pipelines (typically $\sim$150 epochs).
• RSSCN7: For a 16-class remote sensing scene task, the banded-router plus evanescent-stage system boosted test accuracy by more than 15 percentage points over a linear readout baseline (53.04% vs. 36.79%) after only 25 realization epochs, again matching the surrogate within 1% accuracy. The architecture demonstrated the practical effectiveness of mixing via a structured banded/dense composite.
• Yin-Yang Nonlinear Boundary: The framework enabled the implementation of a neuromorphic photonic front-end with classically nonlinear decision boundaries (surrogate and realization accuracy $93.17\%$ and $93.67\%$, respectively, in 25 epochs), confirming versatility beyond linear classification.

Theoretical and Practical Implications

The proposed workflow fundamentally alters the cost structure for photonic neural network design. By decoupling label fitting from physical realization, the computational burden scales as $M$0 per epoch (where $M$1 and $M$2 are forward and adjoint field solves) independent of dataset size. This is a reduction of $M$3-fold (minibatch size, often $M$4) in EM simulation demand. The use of the Frobenius loss for operator matching delivers smooth convergence and monotonic decay in physically relevant metrics (e.g., reflection, transmission error), circumventing nonconvexities associated with intensity-domain nonlinearity in the traditional approach.

Moreover, banded/local routers, when combined with fixed downstream mixing (evanescent coupling), enable compact, modular photonic architectures achieving global mixing in much shorter device footprints—a result underpinned by the algebra of sparse matrix products. This architectural strategy supports the realization of deeper photonic networks within feasible fabrication constraints.

Limitations and Future Directions

The work relies on a 2D effective-index model, which, although experimentally validated for slab waveguides, does not account for out-of-plane losses or chromatic dispersion. Further improvements in insertion loss are needed, potentially by combining surrogate training with direct transmission constraints or tighter singular-value bounds. While simulation-based, the approach is amenable to a range of photonic platforms, including silicon nitride and thin-film lithium niobate, and is compatible with extensions to 3D FDTD solvers.

The methodology is poised for experimental validation; hardware-in-the-loop co-design is a promising extension, where the surrogate guides device architecture and only the realization step is performed with high-fidelity simulation or direct measurement.

Conclusion

The surrogate-guided inverse-design strategy constitutes a scalable paradigm for synthesizing compact, high-performance photonic neural networks. By decoupling task learning from electromagnetic realization, the approach delivers substantial speedups in simulation and enables the design of photonic processors matching surrogate performance with minimal degradation and resource overhead. The architectural innovations in sparse-to-dense compositional routing further enhance scalability and practical deployment. Experimental implementation and expansion to more complex networks and tasks remain natural and impactful next steps for the field.