Physics-informed reinforcement learning for sample-efficient optimization of freeform nanophotonic devices

Abstract: In the field of optics, precise control of light with arbitrary spatial resolution has long been a sought-after goal. Freeform nanophotonic devices are critical building blocks for achieving this goal, as they provide access to a design potential that could hardly be achieved by conventional fixed-shape devices. However, finding an optimal device structure in the vast combinatorial design space that scales exponentially with the number of freeform design parameters has been an enormous challenge. In this study, we propose physics-informed reinforcement learning (PIRL) as an optimization method for freeform nanophotonic devices, which combines the adjoint-based method with reinforcement learning to enhance the sample efficiency of the optimization algorithm and overcome the issue of local minima. To illustrate these advantages of PIRL over other conventional optimization algorithms, we design a family of one-dimensional metasurface beam deflectors using PIRL, obtaining more performant devices. We also explore the transfer learning capability of PIRL that further improves sample efficiency and demonstrate how the minimum feature size of the design can be enforced in PIRL through reward engineering. With its high sample efficiency, robustness, and ability to seamlessly incorporate practical device design constraints, our method offers a promising approach to highly combinatorial freeform device optimization in various physical domains.