Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multiple Population Alternate Evolution Neural Architecture Search (2403.07035v1)

Published 11 Mar 2024 in cs.NE and cs.LG

Abstract: The effectiveness of Evolutionary Neural Architecture Search (ENAS) is influenced by the design of the search space. Nevertheless, common methods including the global search space, scalable search space and hierarchical search space have certain limitations. Specifically, the global search space requires a significant amount of computational resources and time, the scalable search space sacrifices the diversity of network structures and the hierarchical search space increases the search cost in exchange for network diversity. To address above limitation, we propose a novel paradigm of searching neural network architectures and design the Multiple Population Alternate Evolution Neural Architecture Search (MPAE), which can achieve module diversity with a smaller search cost. MPAE converts the search space into L interconnected units and sequentially searches the units, then the above search of the entire network be cycled several times to reduce the impact of previous units on subsequent units. To accelerate the population evolution process, we also propose the the population migration mechanism establishes an excellent migration archive and transfers the excellent knowledge and experience in the migration archive to new populations. The proposed method requires only 0.3 GPU days to search a neural network on the CIFAR dataset and achieves the state-of-the-art results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Understanding and simplifying one-shot architecture search. In International conference on machine learning, pages 550–559. PMLR, 2018.
  2. Overcoming multi-model forgetting. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 594–603. PMLR, 09–15 Jun 2019.
  3. ProxylessNAS: Direct neural architecture search on target task and hardware. In International Conference on Learning Representations, 2019.
  4. Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1294–1303, 2019.
  5. Renas: Reinforced evolutionary neural architecture search. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4787–4796, 2019.
  6. Xuanyi Dong and Yi Yang. One-shot neural architecture search via self-evaluated template network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3681–3690, 2019.
  7. Unsupervised clustering based understanding of cnn. In CVPR Workshops, pages 9–11, 2019.
  8. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  9. Particle swarm optimization for compact neural architecture search for image classification. IEEE Transactions on Evolutionary Computation, 27(5):1298–1312, 2023.
  10. Nas-bench-nlp: neural architecture search benchmark for natural language processing. IEEE Access, 10:45736–45747, 2022.
  11. Opanas: One-shot path aggregation network architecture search for object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10195–10203, 2021.
  12. Progressive neural architecture search. In Proceedings of the European conference on computer vision (ECCV), pages 19–34, 2018.
  13. Darts: Differentiable architecture search. In International Conference on Learning Representations, 2018.
  14. Multiobjective evolutionary design of deep convolutional neural networks for image classification. IEEE Transactions on Evolutionary Computation, 25(2):277–291, 2020.
  15. Efficient neural architecture search via parameters sharing. In International conference on machine learning, pages 4095–4104. PMLR, 2018.
  16. Large-scale evolution of image classifiers. In International Conference on Machine Learning, pages 2902–2911. PMLR, 2017.
  17. Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, volume 33, pages 4780–4789, 2019.
  18. Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor. IEEE Transactions on Evolutionary Computation, 24(2):350–364, 2019.
  19. Completely automated cnn architecture design based on blocks. IEEE transactions on neural networks and learning systems, 31(4):1242–1254, 2019.
  20. Evolving deep convolutional neural networks for image classification. IEEE Transactions on Evolutionary Computation, 24(2):394–407, 2019.
  21. Automatically designing cnn architectures using the genetic algorithm for image classification. IEEE transactions on cybernetics, 50(9):3840–3854, 2020.
  22. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
  23. Mixconv: Mixed depthwise convolutional kernels. CoRR, abs/1907.09595, 2019.
  24. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2820–2828, 2019.
  25. A multipopulation evolutionary algorithm for solving large-scale multimodal multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 25(3):405–418, 2020.
  26. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10734–10742, 2019.
  27. Genetic cnn. In Proceedings of the IEEE international conference on computer vision, pages 1379–1388, 2017.
  28. SNAS: stochastic neural architecture search. In International Conference on Learning Representations, 2019.
  29. Partially-connected neural architecture search for reduced computational redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9):2953–2970, 2021.
  30. Cars: Continuous evolution for efficient neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1829–1838, 2020.
  31. Evaluating the search phase of neural architecture search. In International Conference on Learning Representations, 2019.
  32. Visualizing and understanding convolutional networks. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pages 818–833. Springer, 2014.
  33. You only search once: Single shot neural architecture search via direct sparse optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9):2891–2904, 2020.
  34. Efficient evolutionary search of attention convolutional networks via sampled training and node inheritance. IEEE Transactions on Evolutionary Computation, 25(2):371–385, 2021.
  35. One-shot neural architecture search: Maximising diversity to overcome catastrophic forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9):2921–2935, 2021.
  36. Evolutionary search for complete neural network architectures with partial weight sharing. IEEE transactions on evolutionary computation, 26(5):1072–1086, 2022.
  37. Multinomial distribution learning for effective neural architecture search. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1304–1313, 2019.
  38. Blockqnn: Efficient block-wise neural network architecture generation. IEEE transactions on pattern analysis and machine intelligence, 43(7):2314–2328, 2020.
  39. Bayesnas: A bayesian approach for neural architecture search. In International conference on machine learning, pages 7603–7613. PMLR, 2019.
  40. Neural architecture search with reinforcement learning. In International Conference on Learning Representations, 2016.
  41. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8697–8710, 2018.
  42. Ts-enas: Two-stage evolution for cell-based network architecture search. arXiv preprint arXiv:2310.09525, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets