OpEnCam: Lensless Optical Encryption Camera (2312.01077v1)
Abstract: Lensless cameras multiplex the incoming light before it is recorded by the sensor. This ability to multiplex the incoming light has led to the development of ultra-thin, high-speed, and single-shot 3D imagers. Recently, there have been various attempts at demonstrating another useful aspect of lensless cameras - their ability to preserve the privacy of a scene by capturing encrypted measurements. However, existing lensless camera designs suffer numerous inherent privacy vulnerabilities. To demonstrate this, we develop the first comprehensive attack model for encryption cameras, and propose OpEnCam -- a novel lensless OPtical ENcryption CAmera design that overcomes these vulnerabilities. OpEnCam encrypts the incoming light before capturing it using the modulating ability of optical masks. Recovery of the original scene from an OpEnCam measurement is possible only if one has access to the camera's encryption key, defined by the unique optical elements of each camera. Our OpEnCam design introduces two major improvements over existing lensless camera designs - (a) the use of two co-axially located optical masks, one stuck to the sensor and the other a few millimeters above the sensor and (b) the design of mask patterns, which are derived heuristically from signal processing ideas. We show, through experiments, that OpEnCam is robust against a range of attack types while still maintaining the imaging capabilities of existing lensless cameras. We validate the efficacy of OpEnCam using simulated and real data. Finally, we built and tested a prototype in the lab for proof-of-concept.
- M. S. Asif, A. Ayremlou, A. Sankaranarayanan, A. Veeraraghavan, and R. G. Baraniuk, “Flatcam: Thin, lensless cameras using coded aperture and computation,” IEEE Transactions on Computational Imaging, vol. 3, no. 3, pp. 384–397, 2016.
- V. Boominathan, J. K. Adams, J. T. Robinson, and A. Veeraraghavan, “Phlatcam: Designed phase-mask based thin lensless camera,” IEEE transactions on pattern analysis and machine intelligence, vol. 42, no. 7, pp. 1618–1629, 2020.
- N. Antipa, G. Kuo, R. Heckel, B. Mildenhall, E. Bostan, R. Ng, and L. Waller, “Diffusercam: lensless single-exposure 3d imaging,” Optica, vol. 5, no. 1, pp. 1–9, 2018.
- N. Antipa, P. Oare, E. Bostan, R. Ng, and L. Waller, “Video from stills: Lensless imaging with rolling shutter,” in 2019 IEEE International Conference on Computational Photography (ICCP). IEEE, 2019, pp. 1–8.
- K. Monakhova, K. Yanny, N. Aggarwal, and L. Waller, “Spectral diffusercam: lensless snapshot hyperspectral imaging with a spectral filter array,” Optica, vol. 7, no. 10, pp. 1298–1307, 2020.
- P. Refregier and B. Javidi, “Optical image encryption based on input plane and fourier plane random encoding,” Optics letters, vol. 20, no. 7, pp. 767–769, 1995.
- J. Zang, Z. Xie, and Y. Zhang, “Optical image encryption with spatially incoherent illumination,” Opt. Lett., vol. 38, no. 8, pp. 1289–1291, Apr 2013. [Online]. Available: https://opg.optica.org/ol/abstract.cfm?URI=ol-38-8-1289
- J. Byrne, B. Decann, and S. Bloom, “Key-nets: Optical transformation convolutional networks for privacy preserving vision sensors,” in British Machine Vision Conference (BMVC), 2020.
- T. Nguyen Canh and H. Nagahara, “Deep compressive sensing for visual privacy protection in flatcam imaging,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019, pp. 0–0.
- Y. Ishii, S. Sato, and T. Yamashita, “Privacy-aware face recognition with lensless multi-pinhole camera,” in European Conference on Computer Vision. Springer, 2020, pp. 476–493.
- Z. W. Wang, V. Vineet, F. Pittaluga, S. N. Sinha, O. Cossairt, and S. Bing Kang, “Privacy-preserving action recognition using coded aperture videos,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
- X. Peng, H. Wei, and P. Zhang, “Chosen-plaintext attack on lensless double-random phase encoding in the fresnel domain,” Optics letters, vol. 31, no. 22, pp. 3261–3263, 2006.
- M. Liao, D. Lu, W. He, and X. Peng, “Optical cryptanalysis method using wavefront shaping,” IEEE Photonics Journal, vol. 9, no. 1, pp. 1–13, 2017.
- X. Peng, P. Zhang, H. Wei, and B. Yu, “Known-plaintext attack on optical encryption based on double random phase keys,” optics letters, vol. 31, no. 8, pp. 1044–1046, 2006.
- U. Gopinathan, D. S. Monaghan, T. J. Naughton, and J. T. Sheridan, “A known-plaintext heuristic attack on the fourier plane encryption algorithm,” Optics Express, vol. 14, no. 8, pp. 3181–3186, 2006.
- X. Peng, H.-Q. Tang, and J.-D. Tian, “Ciphertext-only attack on double random phase encoding optical encryption system,” 2007.
- C. Zhang, M. Liao, W. He, and X. Peng, “Ciphertext-only attack on a joint transform correlator encryption system,” Optics express, vol. 21, no. 23, pp. 28 523–28 530, 2013.
- X. Liu, J. Wu, W. He, M. Liao, C. Zhang, and X. Peng, “Vulnerability to ciphertext-only attack of optical encryption scheme based on double random phase encoding,” Optics express, vol. 23, no. 15, pp. 18 955–18 968, 2015.
- M. Liao, S. Zheng, S. Pan, D. Lu, W. He, G. Situ, X. Peng et al., “Deep-learning-based ciphertext-only attack on optical double random phase encryption,” Opto-Electronic Advances, vol. 4, no. 5, p. 05200016, 2021.
- M. Liao, W. He, D. Lu, and X. Peng, “Ciphertext-only attack on optical cryptosystem with spatially incoherent illumination: from the view of imaging through scattering medium,” Scientific Reports, vol. 7, no. 1, pp. 1–9, 2017.
- Y. Wang, J. Gawedzinski, M. E. Pawlowski, and T. S. Tkaczyk, “3d printed fiber optic faceplates by custom controlled fused deposition modeling,” Optics Express, vol. 26, no. 12, pp. 15 362–15 376, 2018.
- M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE signal processing magazine, vol. 25, no. 2, pp. 83–91, 2008.
- S. S. Khan, V. Sundar, V. Boominathan, A. Veeraraghavan, and K. Mitra, “Flatnet: Towards photorealistic scene reconstruction from lensless measurements,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
- D. Bagadthey, S. Prabhu, S. S. Khan, D. T. Fredrick, V. Boominathan, A. Veeraraghavan, and K. Mitra, “Flatnet3d: intensity and absolute depth from single-shot lensless capture,” JOSA A, vol. 39, no. 10, pp. 1903–1912, 2022.
- X. Pan, X. Chen, S. Takeyama, and M. Yamaguchi, “Image reconstruction with transformer for mask-based lensless imaging,” Optics Letters, vol. 47, no. 7, pp. 1843–1846, 2022.
- J. D. Rego, K. Kulkarni, and S. Jayasuriya, “Robust lensless image reconstruction via psf estimation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 403–412.
- G. Wetzstein, A. Ozcan, S. Gigan, S. Fan, D. Englund, M. Soljačić, C. Denz, D. A. Miller, and D. Psaltis, “Inference in artificial intelligence with deep optics and photonics,” Nature, vol. 588, no. 7836, pp. 39–47, 2020.
- R. Ranftl, A. Bochkovskiy, and V. Koltun, “Vision transformers for dense prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 12 179–12 188.
- C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang et al., “Photo-realistic single image super-resolution using a generative adversarial network,” in CVPR, 2017, pp. 4681–4690.
- A. Dosovitskiy and T. Brox, “Generating images with perceptual similarity metrics based on deep networks,” in Advances in Neural Information Processing Systems, 2016, pp. 658–666.
- K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in ICLR, 2015.
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in CVPR, 2009, pp. 248–255.
- K. Perlin, “Improving noise,” in Proceedings of the 29th annual conference on Computer graphics and interactive techniques, 2002, pp. 681–682.
- S. S. Khan, V. Adarsh, V. Boominathan, J. Tan, A. Veeraraghavan, and K. Mitra, “Towards photorealistic reconstruction of highly multiplexed lensless images,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7860–7869.
- K. Monakhova, J. Yurtsever, G. Kuo, N. Antipa, K. Yanny, and L. Waller, “Learned reconstructions for practical mask-based lensless imaging,” Optics express, vol. 27, no. 20, pp. 28 075–28 090, 2019.
- B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
- O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp. 211–252, 2015.