Applicability of Metalenses for Generalizable Computer Vision

Abstract: Optical neural networks (ONNs) are gaining increasing attention to accelerate machine learning tasks. In particular, static meta-optical encoders designed for task-specific pre-processing demonstrated orders of magnitude smaller energy consumption over purely digital counterpart, albeit at the cost of slight degradation in classification accuracy. However, a lack of generalizability poses serious challenges for wide deployment of static meta-optical front-ends. Here, we investigate the utility of a metalens for generalized computer vision. Specifically, we show that a metalens optimized for full-color imaging can achieve image classification accuracy comparable to high-end, sensor-limited optics and consistently outperforms a hyperboloid metalens across a wide range of sensor pixel sizes. We further design an end-to-end single aperture metasurface for ImageNet classification and find that the optimized metasurface tends to balance the modulation transfer function (MTF) for each wavelength. Together, these findings highlight that the preservation of spatial frequency-domain information is an essential interpretable factor underlying ONN performance. Our work provides both an interpretable understanding of task-driven optical optimization and practical guidance for designing high-performance ONNs and meta-optical encoders for generalizable computer vision.