Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
146 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

NeuroLGP-SM: Scalable Surrogate-Assisted Neuroevolution for Deep Neural Networks (2404.08786v4)

Published 12 Apr 2024 in cs.NE and cs.AI

Abstract: Evolutionary Algorithms (EAs) play a crucial role in the architectural configuration and training of Artificial Deep Neural Networks (DNNs), a process known as neuroevolution. However, neuroevolution is hindered by its inherent computational expense, requiring multiple generations, a large population, and numerous epochs. The most computationally intensive aspect lies in evaluating the fitness function of a single candidate solution. To address this challenge, we employ Surrogate-assisted EAs (SAEAs). While a few SAEAs approaches have been proposed in neuroevolution, none have been applied to truly large DNNs due to issues like intractable information usage. In this work, drawing inspiration from Genetic Programming semantics, we use phenotypic distance vectors, outputted from DNNs, alongside Kriging Partial Least Squares (KPLS), an approach that is effective in handling these large vectors, making them suitable for search. Our proposed approach, named Neuro-Linear Genetic Programming surrogate model (NeuroLGP-SM), efficiently and accurately estimates DNN fitness without the need for complete evaluations. NeuroLGP-SM demonstrates competitive or superior results compared to 12 other methods, including NeuroLGP without SM, convolutional neural networks, support vector machines, and autoencoders. Additionally, it is worth noting that NeuroLGP-SM is 25% more energy-efficient than its NeuroLGP counterpart. This efficiency advantage adds to the overall appeal of our proposed NeuroLGP-SM in optimising the configuration of large DNNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. M. Aswathy and M. Jagannath. An svm approach towards breast cancer classification from h&e-stained histopathology images based on integrated features. Medical & biological engineering & computing, 59(9):1773–1783, 2021.
  2. Medical image classification with hand-designed or machine-designed texture descriptors: a performance evaluation. In Bioinformatics and Biomedical Engineering: 6th International Work-Conference, IWBBIO 2018, Granada, Spain, April 25–27, 2018, Proceedings, Part II 6, pages 266–275. Springer, 2018.
  3. Breakhis based breast cancer automatic diagnosis using deep learning: Taxonomy, survey and insights. Neurocomputing, 375:9–24, 2020.
  4. Improving kriging surrogates of high-dimensional design models by partial least squares dimension reduction. Structural and Multidisciplinary Optimization, 53:935–952, 2016.
  5. M. Brameier and W. Banzhaf. Linear genetic programming, volume 1. Springer, 2007.
  6. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321–357, 2002.
  7. Multiple instance learning of deep convolutional neural networks for breast histopathology whole slide classification. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 578–581. IEEE, 2018.
  8. Evolutionary neural architecture search for facial expression recognition. IEEE Transactions on Emerging Topics in Computational Intelligence, 7(5):1405–1419, 2023.
  9. Introduction to Evolutionary Computing. Springer Verlag, 2003.
  10. Data-efficient neuroevolution with kernel-based surrogate models. In Proceedings of the genetic and evolutionary computation conference, pages 85–92, 2018.
  11. E. Galván and P. Mooney. Neuroevolution in deep neural networks: Current trends and future challenges. IEEE Transactions on Artificial Intelligence, 2:476–493, 2021.
  12. E. Galván and M. Schoenauer. Promoting semantic diversity in multi-objective genetic programming. In A. Auger and T. Stützle, editors, Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, July 13-17, 2019, pages 1021–1029. ACM, 2019.
  13. E. Galván and F. Stapleton. Semantic-based distance approaches in multi-objective genetic programming. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pages 149–156. IEEE, 2020.
  14. E. Galván and F. Stapleton. Evolutionary multi-objective optimisation in neurotrajectory prediction. Applied Soft Computing, 146:110693, 2023.
  15. Semantics in multi-objective genetic programming. Applied Soft Computing, 115:108143, 2022.
  16. Semantic learning machine: a feedforward neural network construction algorithm inspired by geometric semantic genetic programming. In Progress in Artificial Intelligence: 17th Portuguese Conference on Artificial Intelligence, EPIA 2015, Coimbra, Portugal, September 8-11, 2015. Proceedings 17, pages 280–285. Springer, 2015.
  17. B. Greenwood and T. McDonnell. Surrogate-assisted neuroevolution. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1048–1056, 2022.
  18. V. Gupta and A. Bhavsar. An integrated multi-scale model for breast cancer histopathological image classification with joint colour-texture features. In Computer Analysis of Images and Patterns: 17th International Conference, CAIP 2017, Ystad, Sweden, August 22-24, 2017, Proceedings, Part II 17, pages 354–366. Springer, 2017.
  19. Prediction of neural network performance by phenotypic modeling. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 1576–1582, 2019.
  20. Y. Jin. Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation, 1(2):61–70, 2011.
  21. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13:455–492, 1998.
  22. R. Karthiga and K. Narasimhan. Automated diagnosis of breast cancer using wavelet based entropy features. In 2018 Second international conference on electronics, communication and aerospace technology (ICECA), pages 274–279. IEEE, 2018.
  23. K. Kumar and A. C. S. Rao. Breast cancer classification of image using convolutional neural network. In 2018 4th International Conference on Recent Advances in Information Technology (RAIT), pages 1–6. IEEE, 2018.
  24. Green algorithms: quantifying the carbon footprint of computation. Advanced science, 8(12):2100707, 2021.
  25. Deep learning. Nature, 521(7553):436–444, 2015.
  26. G. Menghani. Efficient deep learning: A survey on making deep learning models smaller, faster, and better. ACM Computing Surveys, 55(12):1–37, 2023.
  27. A.-A. Nahid and Y. Kong. Histopathological breast-image classification using local and frequency domains by convolutional neural network. Information, 9(1):19, 2018.
  28. Histopathological breast cancer image classification by deep neural network techniques guided by local clustering. BioMed research international, 2018, 2018.
  29. Histopathological breast-image classification with restricted boltzmann machine along with backpropagation. Biomedical Research, 29(10):2068–2077, 2018.
  30. S. Park and N. Kwak. Analysis on the dropout effect in convolutional neural networks. In Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13, pages 189–204. Springer, 2017.
  31. Semantic backpropagation for designing search operators in genetic programming. IEEE Transactions on Evolutionary Computation, 19(3):326–340, June 2015.
  32. S. Pratiher and S. Chattoraj. Diving deep onto discriminative ensemble of histological hashing & class-specific manifold learning for multi-class breast carcinoma taxonomy. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1025–1029. IEEE, 2019.
  33. W. Rawat and Z. Wang. Deep convolutional neural networks for image classification: A comprehensive review. Neural computation, 29(9):2352–2449, 2017.
  34. Neuroevolution with box mutation: An adaptive and modular framework for evolving deep neural networks. Applied Soft Computing, page 110767, 2023.
  35. Classification of breast tumors as benign and malignant using textural feature descriptor. In 2017 Ieee international conference on bioinformatics and biomedicine (bibm), pages 1110–1113. IEEE, 2017.
  36. Breast cancer histopathological image classification using convolutional neural networks. In 2016 international joint conference on neural networks (IJCNN), pages 2560–2567. IEEE, 2016.
  37. Neurolgp-sm: A surrogate-assisted neuroevolution approach using linear genetic programming. In International Conference on Optimization and Learning (OLA), 2024.
  38. F. Stapleton and E. Galván. Semantic neighborhood ordering in multi-objective genetic programming based on decomposition. In 2021 IEEE Congress on Evolutionary Computation (CEC), pages 580–587. IEEE, 2021.
  39. F. Stapleton and E. Galvan. Initial steps towards tackling high-dimensional surrogate modeling for neuroevolution using kriging partial least squares. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, GECCO ’23 Companion, page 83–84, New York, NY, USA, 2023. Association for Computing Machinery.
  40. Improving neuroevolution efficiency by surrogate model-based optimization with phenotypic distance kernels. In International Conference on the Applications of Evolutionary Computation (Part of EvoStar), pages 504–519. Springer, 2019.
  41. Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor. IEEE Transactions on Evolutionary Computation, 24(2):350–364, 2020.
  42. Evolving deep convolutional neural networks for image classification. IEEE Transactions on Evolutionary Computation, 24(2):394–407, 2019.
  43. Freeze-thaw bayesian optimization. arXiv preprint arXiv:1406.3896, 2014.
  44. B. Zoph and Q. V. Le. Neural architecture search with reinforcement learning. CoRR, abs/1611.01578, 2016.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets