Hyperpixels: Pixel Filter Arrays of Multivariate Optical Elements for Optimized Spectral Imaging (2403.16901v1)
Abstract: We introduce the concept of `hyperpixels' in which each element of a pixel filter array (suitable for CMOS image sensor integration) has a spectral transmission tailored to a target spectral component expected in application-specific scenes. These are analogous to arrays of multivariate optical elements that could be used for sensing specific analytes. Spectral tailoring is achieved by engineering the heights of multiple sub-pixel Fabry-Perot resonators that cover each pixel area. We first present a design approach for hyperpixels, based on a matched filter concept and, as an exemplar, design a set of 4 hyperpixels tailored to optimally discriminate between 4 spectral reflectance targets. Next, we fabricate repeating 2x2 pixel filter arrays of these designs, alongside repeating 2x2 arrays of an optimal bandpass filters, perform both spectral and imaging characterization. Experimentally measured hyperpixel transmission spectra show a 2.4x reduction in unmixing matrix condition number (p=0.031) compared to the optimal band-pass set. Imaging experiments using the filter arrays with a monochrome sensor achieve a 3.47x reduction in unmixing matrix condition number (p=0.020) compared to the optimal band-pass set. This demonstrates the utility of the hyperpixel approach and shows its superiority even over the optimal bandpass case. We expect that with further improvements in design and fabrication processes increased performance may be obtained. Because the hyperpixels are straightforward to customize, fabricate and can be placed atop monochrome sensors, this approach is highly versatile and could be adapted to a wide range of real-time imaging applications which are limited by low SNR including micro-endoscopy, capsule endoscopy, industrial inspection and machine vision.
- Q. Li, X. He, Y. Wang, H. Liu, D. Xu, and F. Guo, “Review of spectral imaging technology in biomedical engineering: Achievements and challenges,” 18, 100901.
- G. Lu and B. Fei, “Medical hyperspectral imaging: A review,” 19, 010901.
- J. Spigulis, D. Jakovels, and L. Elste, “Towards single snapshot multispectral skin assessment,” p. 82160L.
- J. R. Bauer, A. A. Bruins, J. Y. Hardeberg, and R. M. Verdaasdonk, “A Spectral Filter Array Camera for Clinical Monitoring and Diagnosis: Proof of Concept for Skin Oxygenation Imaging,” 5, 66.
- D. J. Waterhouse, C. R. Fitzpatrick, B. W. Pogue, J. P. O’Connor, and S. E. Bohndiek, “A roadmap for the clinical implementation of optical-imaging biomarkers,” 3, 339–353.
- N. T. Clancy, G. Jones, L. Maier-Hein, D. S. Elson, and D. Stoyanov, “Surgical spectral imaging,” 63, 101699.
- A. Siri Luthman, D. Waterhouse, L. Ansel-Bollepalli, J. Yoon, G. Gordon, J. Joseph, M. Di Pietro, W. Januszewicz, and S. Bohndiek, “Bimodal reflectance and fluorescence multispectral endoscopy based on spectrally resolving detector arrays,” 24.
- Y. Liu, H. Pu, and D.-W. Sun, “Hyperspectral imaging technique for evaluating food quality and safety during various processes: A review of recent applications,” 69, 25–35.
- K. Clarke, A. Hennessy, A. McGrath, R. Daly, S. Gaylard, A. Turner, J. Cameron, M. Lewis, and M. B. Fernandes, “Using hyperspectral imagery to investigate large-scale seagrass cover and genus distribution in a temperate coast,” 11, 4182.
- T. Adão, J. Hruška, L. Pádua, J. Bessa, E. Peres, R. Morais, and J. Sousa, “Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry,” 9, 1110.
- C. Williams, G. Gordon, S. Gruber, T. Wilkinson, and S. Bohndiek, “Grayscale-to-color: Single-step fabrication of bespoke multispectral filter arrays,” .
- T. W. Sawyer, M. Taylor-Williams, R. Tao, R. Xia, C. Williams, and S. E. Bohndiek, “Opti-MSFA: A toolbox for generalized design and optimization of multispectral filter arrays,” 30, 7591.
- R. Wu, Y. Li, X. Xie, and Z. Lin, “Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction,” 19, 2905.
- G. Turin, “An introduction to matched filters,” 6, 311–329.
- M. P. Nelson, J. F. Aust, J. A. Dobrowolski, P. G. Verly, and M. L. Myrick, “Multivariate Optical Computation for Predictive Spectroscopy,” 70, 73–82.
- S. Kozhukh, S. Zhukovsky, I. Yurkevich, S. Stepkova, S. Gaponenko, A. Lunevich, and Y. Glukhov, “Design of high-transmission multiband multilayer filters for Raman spectroscopy,” 6, 061704.
- J. Xiong, X. Cai, K. Cui, Y. Huang, J. Yang, H. Zhu, W. Li, B. Hong, S. Rao, Z. Zheng, S. Xu, Y. He, F. Liu, X. Feng, and W. Zhang, “Dynamic brain spectrum acquired by a real-time ultraspectral imaging chip with reconfigurable metasurfaces,” 9, 461.
- A. Ahamed, C. Bartolo-perez, A. S. Mayet, S. Ghandiparsi, S.-y. Wang, and M. S. Islam, “Reconstruction-based spectroscopy using CMOS image sensors with random photon-trapping nanostructure per sensor,” .
- X. Zou, Y. Zhang, R. Lin, G. Gong, S. Wang, S. Zhu, and Z. Wang, “Pixel-level Bayer-type colour router based on metasurfaces,” 13, 1–7.
- K. Monakhova, K. Yanny, N. Aggarwal, and L. Waller, “Spectral DiffuserCam: Lensless snapshot hyperspectral imaging with a spectral filter array,” 7.
- A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” 47, B44.
- M. Yako, Y. Yamaoka, T. Kiyohara, C. Hosokawa, A. Noda, K. Tack, N. Spooren, T. Hirasawa, and A. Ishikawa, “Video-rate hyperspectral camera based on a CMOS-compatible random array of Fabry–Pérot filters,” 17, 218–223.
- T. Goossens, “Pixel-integrated thin-film filter simulation and scaling trade-offs,” .
- M. Taylor-Williams, R. B. Cousins, C. Williams, S. E. Bohndiek, C. J. Mellor, and G. S. D. Gordon, “Spectrally tailored ’hyperpixel’ filter arrays for imaging of chemical compositions,” in Optical Biopsy XX: Toward Real-Time Spectroscopic Imaging and Diagnosis, R. R. Alfano, S. G. Demos, and A. B. Seddon, eds. (SPIE), p. 23.
- L. Gao and R. T. Smith, “Optical hyperspectral imaging in microscopy and spectroscopy - a review of data acquisition,” 8, 441–456.
- A. Goldsmith and P. Varaiya, “Capacity of fading channels with channel side information,” 43, 1986–1992 (Nov./1997).
- D. Yi, L. Kong, J. Wang, and F. Zhao, “Fabrication of multispectral imaging technology driven MEMS-based micro-arrayed multichannel optical filter mosaic,” p. 792711.
- C. M. Hooper, S. E. Bohndiek, and C. Williams, “Optimizing achromaticity in metalenses, and development of a layered thin-film metalens,” in High Contrast Metastructures XI, J. A. Fan, C. J. Chang-Hasnain, and W. Zhou, eds. (SPIE), p. 31.
- S.-W. Wang, X. Chen, W. Lu, L. Wang, Y. Wu, and Z. Wang, “Integrated optical filter arrays fabricated by using the combinatorial etching technique,” 31, 332.
- O. Soyemi, D. Eastwood, L. Zhang, H. Li, J. Karunamuni, P. Gemperline, R. A. Synowicki, and M. L. Myrick, “Design and Testing of a Multivariate Optical Element: The First Demonstration of Multivariate Optical Computing for Predictive Spectroscopy,” 73, 1069–1079.
- https://doi.org/10.17639/nott.7397