Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MatchNAS: Optimizing Edge AI in Sparse-Label Data Contexts via Automating Deep Neural Network Porting for Mobile Deployment (2402.13525v1)

Published 21 Feb 2024 in cs.LG and cs.DC

Abstract: Recent years have seen the explosion of edge intelligence with powerful Deep Neural Networks (DNNs). One popular scheme is training DNNs on powerful cloud servers and subsequently porting them to mobile devices after being lightweight. Conventional approaches manually specialized DNNs for various edge platforms and retrain them with real-world data. However, as the number of platforms increases, these approaches become labour-intensive and computationally prohibitive. Additionally, real-world data tends to be sparse-label, further increasing the difficulty of lightweight models. In this paper, we propose MatchNAS, a novel scheme for porting DNNs to mobile devices. Specifically, we simultaneously optimise a large network family using both labelled and unlabelled data and then automatically search for tailored networks for different hardware platforms. MatchNAS acts as an intermediary that bridges the gap between cloud-based DNNs and edge-based DNNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Learning with Pseudo-Ensembles. In NIPS. https://api.semanticscholar.org/CorpusID:8307266
  2. Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016).
  3. ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring. In International Conference on Learning Representations. https://api.semanticscholar.org/CorpusID:213757781
  4. MixMatch: A Holistic Approach to Semi-Supervised Learning. ArXiv abs/1905.02249 (2019). https://api.semanticscholar.org/CorpusID:146808485
  5. Once for All: Train One Network and Specialize it for Efficient Deployment. ArXiv abs/1908.09791 (2019). https://api.semanticscholar.org/CorpusID:201666112
  6. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. ArXiv abs/1812.00332 (2018). https://api.semanticscholar.org/CorpusID:54438210
  7. Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective. ArXiv abs/2102.11535 (2021). https://api.semanticscholar.org/CorpusID:232013680
  8. ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 11390–11399. https://api.semanticscholar.org/CorpusID:56657862
  9. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009), 248–255. https://api.semanticscholar.org/CorpusID:57246310
  10. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv abs/1810.04805 (2019). https://api.semanticscholar.org/CorpusID:52967399
  11. EdgeMove: Pipelining Device-Edge Model Training for Mobile Intelligence. Proceedings of the ACM Web Conference 2023 (2023). https://api.semanticscholar.org/CorpusID:258333779
  12. The Pascal Visual Object Classes Challenge: A Retrospective. International Journal of Computer Vision 111, 1 (Jan. 2015), 98–136.
  13. MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2017), 1586–1595. https://api.semanticscholar.org/CorpusID:206596875
  14. Single Path One-Shot Neural Architecture Search with Uniform Sampling. In European Conference on Computer Vision. https://api.semanticscholar.org/CorpusID:90262841
  15. Edge Computing in 5G for Drone Navigation: What to Offload? IEEE Robotics and Automation Letters 6, 2 (2021), 2571–2578. https://doi.org/10.1109/LRA.2021.3062319
  16. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 770–778. https://api.semanticscholar.org/CorpusID:206594692
  17. Distilling the Knowledge in a Neural Network. ArXiv abs/1503.02531 (2015).
  18. Searching for MobileNetV3. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 1314–1324. https://api.semanticscholar.org/CorpusID:146808333
  19. 3D Object Representations for Fine-Grained Categorization. 2013 IEEE International Conference on Computer Vision Workshops (2013), 554–561. https://api.semanticscholar.org/CorpusID:14342571
  20. Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. https://api.semanticscholar.org/CorpusID:18268744
  21. Dong-Hyun Lee. 2013. Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks. https://api.semanticscholar.org/CorpusID:18507866
  22. Zen-NAS: A Zero-Shot NAS for High-Performance Image Recognition. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 337–346. https://api.semanticscholar.org/CorpusID:245835451
  23. Ilya Loshchilov and Frank Hutter. 2017. Decoupled Weight Decay Regularization. In International Conference on Learning Representations. https://api.semanticscholar.org/CorpusID:53592270
  24. Deep Learning at the Mobile Edge: Opportunities for 5G Networks. Applied Sciences (2020). https://api.semanticscholar.org/CorpusID:225525817
  25. Geoffrey J. McLachlan. 1975. Iterative Reclassification Procedure for Constructing An Asymptotically Optimal Rule of Allocation in Discriminant-Analysis. J. Amer. Statist. Assoc. 70 (1975), 365–369. https://api.semanticscholar.org/CorpusID:120764023
  26. Neural Architecture Search without Training. ArXiv abs/2006.04647 (2020). https://api.semanticscholar.org/CorpusID:219531078
  27. Maad M. Mijwil. 2022. Has the Future Started? The Current Growth of Artificial Intelligence, Machine Learning, and Deep Learning. Iraqi Journal for Computer Science and Mathematics (2022). https://api.semanticscholar.org/CorpusID:249688145
  28. Automatic differentiation in PyTorch. (2017).
  29. Large-Scale Evolution of Image Classifiers. ArXiv abs/1703.01041 (2017).
  30. CompOFA: Compound Once-For-All Networks for Faster Multi-Platform Deployment. ArXiv abs/2104.12642 (2021). https://api.semanticscholar.org/CorpusID:232286427
  31. Samsung. [n. d.]. Samsung Remote Test Lab. https://developer.samsung.com/remote-test-lab
  32. Evaluating the Search Phase of Neural Architecture Search. ArXiv abs/1902.08142 (2019).
  33. FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. ArXiv abs/2001.07685 (2020). https://api.semanticscholar.org/CorpusID:210839228
  34. MnasNet: Platform-Aware Neural Architecture Search for Mobile. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 2815–2823. https://api.semanticscholar.org/CorpusID:51891697
  35. Antti Tarvainen and Harri Valpola. 2017. Weight-averaged consistency targets improve semi-supervised deep learning results. ArXiv abs/1703.01780 (2017). https://api.semanticscholar.org/CorpusID:2759724
  36. . Technical Report CNS-TR-2011-001. California Institute of Technology.
  37. AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 6414–6423.
  38. Unsupervised Data Augmentation for Consistency Training. arXiv: Learning (2019). https://api.semanticscholar.org/CorpusID:195873898
  39. A First Look at Deep Learning Apps on Smartphones. The World Wide Web Conference (2018). https://api.semanticscholar.org/CorpusID:59158795
  40. GreedyNAS: Towards Fast One-Shot NAS With Greedy Supernet. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020), 1996–2005.
  41. BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models. In ECCV.
  42. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing (2019). https://api.semanticscholar.org/CorpusID:125608550
  43. Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016).
  44. Learning Transferable Architectures for Scalable Image Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 8697–8710.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets