Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ExtremeMETA: High-speed Lightweight Image Segmentation Model by Remodeling Multi-channel Metamaterial Imagers (2405.17568v1)

Published 27 May 2024 in cs.CV

Abstract: Deep neural networks (DNNs) have heavily relied on traditional computational units like CPUs and GPUs. However, this conventional approach brings significant computational burdens, latency issues, and high power consumption, limiting their effectiveness. This has sparked the need for lightweight networks like ExtremeC3Net. On the other hand, there have been notable advancements in optical computational units, particularly with metamaterials, offering the exciting prospect of energy-efficient neural networks operating at the speed of light. Yet, the digital design of metamaterial neural networks (MNNs) faces challenges such as precision, noise, and bandwidth, limiting their application to intuitive tasks and low-resolution images. In this paper, we propose a large kernel lightweight segmentation model, ExtremeMETA. Based on the ExtremeC3Net, the ExtremeMETA maximizes the ability of the first convolution layer by exploring a larger convolution kernel and multiple processing paths. With the proposed large kernel convolution model, we extend the optic neural network application boundary to the segmentation task. To further lighten the computation burden of the digital processing part, a set of model compression methods is applied to improve model efficiency in the inference stage. The experimental results on three publicly available datasets demonstrate that the optimized efficient design improved segmentation performance from 92.45 to 95.97 on mIoU while reducing computational FLOPs from 461.07 MMacs to 166.03 MMacs. The proposed the large kernel lightweight model ExtremeMETA showcases the hybrid design's ability on complex tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  2. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
  3. M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” arXiv preprint arXiv:1311.2901, 2014.
  4. C. Szegedy, S. Ioffe, and V. Vanhoucke, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
  5. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
  6. X. Zhang, X. Zhou, M. Lin, and J. Sun, “Efficient and accurate approximations of nonlinear convolutional networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1984–1992.
  7. M. Sun, Z. Liu, X. Wang, W. Qiao, and K. Lin, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International Conference on Machine Learning, 2018, pp. 6105–6114.
  8. M. Lin, Q. Chen, and S. Yan, “Network in network,” arXiv preprint arXiv:1312.4400, 2013.
  9. G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708, 2017.
  10. Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, and D. Englund, “Deep learning with coherent nanophotonic circuits,” Nature Photonics, vol. 11, no. 7, pp. 441–446, 2017.
  11. X. Lin, Y. Rivenson, D. Teng, L. Wei, H. Günaydın, Y. Zhang, and A. Ozcan, “All-optical machine learning using diffractive deep neural networks,” Science, vol. 361, no. 6406, pp. 1004–1008, 2018.
  12. T. W. Hughes, M. Minkov, I. A. Williamson, Y. Shi, and S. Fan, “Training of photonic neural networks through in situ backpropagation and gradient measurement: supplementary material,” Optica, vol. Part F127-, 2018.
  13. A. N. Tait, M. A. Nahmias, B. J. Shastri, P. R. Prucnal, and J. S. Harris, “The physics of optical neural networks,” Applied Physics Reviews, vol. 4, no. 2, p. 021105, 2017.
  14. ——, “Optical implementation of deep networks,” Applied Optics, vol. 55, no. 36, pp. A71–A82, 2016.
  15. M. Miscuglio, J. Dambre, and P. Bienstman, “All-optical nonlinear activation function for photonic neural networks [invited],” Optical Materials Express, vol. 8, no. 12, pp. 3851–3863, 2018.
  16. L. Larger, M. C. Soriano, D. Brunner, L. Appeltant, J. M. Gutiérrez, I. Fischer, and C. R. Mirasso, “Photonic information processing beyond turing: an optoelectronic implementation of reservoir computing,” Optics Express, vol. 20, no. 3, pp. 3241–3249, 2012.
  17. S. Jutamulia and F. T. S. Yu, “Overview of hybrid optical neural networks,” Optics & Laser Technology, vol. 28, no. 2, pp. 85–97, 1996.
  18. K. M. Boehm, P. Khosravi, R. Vanguri, J. Gao, and S. P. Shah, “Harnessing multimodal data integration to advance precision oncology,” Nature Reviews Cancer, vol. 22, no. 2, pp. 71–88, 2022.
  19. M. Zhuge et al., “Kaleido-bert: Vision-language pre-training on fashion domain,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021.
  20. Y. B. Ovchinnikov et al., “Diffraction of a released bose-einstein condensate by a pulsed standing light wave,” Physical Review Letters, vol. 83, no. 2, p. 284, 1999.
  21. S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Advances in Neural Information Processing Systems, 2015, pp. 1135–1143.
  22. P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” in International Conference on Learning Representations, 2016.
  23. I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Quantized neural networks: Training neural networks with low precision weights and activations,” in arXiv preprint arXiv:1609.07061, 2017.
  24. E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in Advances in Neural Information Processing Systems, 2014, pp. 1269–1277.
  25. G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in arXiv preprint arXiv:1503.02531, 2015.
  26. W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen, “Compressing neural networks with the hashing trick,” in International Conference on Machine Learning, 2015, pp. 2285–2294.
  27. H. Park, L. L. Sjösund, Y. Yoo, J. Bang, and N. Kwak, “Extremec3net: Extreme lightweight portrait segmentation networks using advanced c3-modules,” arXiv preprint arXiv:1908.03093, 2019.
  28. X. Shen, A. Hertzmann, J. Jia, S. Paris, B. Price, E. Shechtman, and I. Sachs, “Automatic portrait segmentation for image stylization,” in Computer Graphics Forum, vol. 35, no. 2.   Wiley Online Library, 2016, pp. 93–102.
  29. J. Krause, J. Deng, M. Stark, and L. Fei-Fei, “Collecting a large-scale dataset of fine-grained cars,” 2013.
  30. A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
  31. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo et al., “Segment anything,” arXiv preprint arXiv:2304.02643, 2023.
  32. Q. Liu, H. Zheng, B. T. Swartz, Z. Asad, I. Kravchenko, J. G. Valentine, Y. Huo et al., “Digital modeling on large kernel metamaterial neural network,” arXiv preprint arXiv:2307.11862, 2023.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets