2000 character limit reached
Evaluation Metrics for DNNs Compression (2305.10616v4)
Published 18 May 2023 in cs.LG and cs.CV
Abstract: There is a lot of ongoing research effort into developing different techniques for neural networks compression. However, the community lacks standardised evaluation metrics, which are key to identifying the most suitable compression technique for different applications. This paper reviews existing neural network compression evaluation metrics and implements them into a standardisation framework called NetZIP. We introduce two novel metrics to cover existing gaps of evaluation in the literature: 1) Compression and Hardware Agnostic Theoretical Speed (CHATS) and 2) Overall Compression Success (OCS). We demonstrate the use of NetZIP using two case studies on two different hardware platforms (a PC and a Raspberry Pi 4) focusing on object classification and object detection.
- “Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networks”, 2020 arXiv: http://arxiv.org/abs/2011.09398
- “BinaryDenseNet: Developing an architecture for binary neural networks” In Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019 IEEE, 2019, pp. 1951–1960 DOI: 10.1109/ICCVW.2019.00244
- Davis Blalock, Samuel Madden and John Guttag “Sprintz: Time Series Compression for the Internet of Things”, 2018 DOI: 10.1145/3264903
- “What is the State of Neural Network Pruning?”, 2020 arXiv: http://arxiv.org/abs/2003.03033
- “Language Models are Few-Shot Learners” In CoRR abs/2005.14165, 2020 arXiv: https://arxiv.org/abs/2005.14165
- “Quantized neural networks for modulation recognition” In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV 12113 SPIE, 2022, pp. 1211317 International Society for OpticsPhotonics DOI: 10.1117/12.2617678
- “On Determinism of Game Engines used for Simulation-based Autonomous Vehicle Verification”, 2021 arXiv: https://arxiv.org/abs/2104.06262
- “BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1” In CoRR abs/1602.02830, 2016 arXiv: http://arxiv.org/abs/1602.02830
- “Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1”, 2016 arXiv:1602.02830 [cs.LG]
- “ImageNet: A Large-Scale Hierarchical Image Database” In CVPR09, 2009
- “Learned Step Size Quantization”, 2020 arXiv:1902.08153 [cs.LG]
- “PySlowFast”, https://github.com/facebookresearch/slowfast, 2020
- “DepGraph: Towards Any Structural Pruning”, 2023 arXiv:2301.12900 [cs.AI]
- Hayden Field, Glen Anderson and Kerstin Eder “EACOF: A Framework for Providing Energy Transparency to Enable Energy-Aware Software Development” In Proceedings of the 29th Annual ACM Symposium on Applied Computing, SAC ’14 Gyeongju, Republic of Korea: Association for Computing Machinery, 2014, pp. 1194–1199 DOI: 10.1145/2554850.2554920
- “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”, 2019 arXiv:1803.03635 [cs.LG]
- “Larq: An Open-Source Library for Training Binarized Neural Networks” In Journal of Open Source Software 5.45 The Open Journal, 2020, pp. 1746 DOI: 10.21105/joss.01746
- “Operational Adaptation of DNN Classifiers using Elastic Weight Consolidation”, 2022, pp. 1–13 arXiv: http://arxiv.org/abs/2205.00147
- “Deep Residual Learning for Image Recognition”, 2015 arXiv:1512.03385 [cs.CV]
- “AMC: AutoML for model compression and acceleration on mobile devices” In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11211 LNCS, 2018, pp. 815–832 DOI: 10.1007/978-3-030-01234-2˙48
- Glenn Jocher “YOLOv5 by Ultralytics”, 2020 DOI: 10.5281/zenodo.3908559
- “Microsoft COCO: Common Objects in Context”, 2015 arXiv:1405.0312 [cs.CV]
- “Ternary Weight Networks” In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5 DOI: 10.1109/ICASSP49357.2023.10094626
- “Deep neural networks compression: A comparative survey and choice recommendations” In Neurocomputing 520, 2023, pp. 152–170 DOI: 10.1016/j.neucom.2022.11.072
- “Training binary neural networks with real-to-binary convolutions” In International Conference on Learning Representations, 2020 URL: https://openreview.net/forum?id=BJg4NgBKvH
- Gaurav Menghani “Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better” In ACM Computing Surveys 55.12, 2023, pp. 1–37 DOI: 10.1145/3578938
- Microsoft “Neural Network Intelligence”, 2021 URL: https://github.com/microsoft/nni
- “The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization”, 2022 arXiv:2112.00463 [cs.CV]
- James O’ Neill “An Overview of Neural Network Compression”, 2020, pp. 1–73 arXiv: http://arxiv.org/abs/2006.03669
- NVIDIA “NVIDIA TURING GPU ARCHITECTURE” NVIDIA, 2018
- NVIDIA “nvidia-smi - NVIDIA System Management Interface program” NVIDIA, 2016
- “Automatic differentiation in PyTorch”, 2017
- “Differentiable top-k classification learning” In International Conference on Machine Learning, 2022, pp. 17656–17668 PMLR
- PowerAPI “pyRAPL”, 2019 URL: https://github.com/powerapi-ng/pyRAPL
- “XNOR-net: Imagenet classification using binary convolutional neural networks” In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9908 LNCS, 2016, pp. 525–542 DOI: 10.1007/978-3-319-46493-0˙32
- Giampaolo Rodola, Jay Loden and Dave Daeschler “psutil”, 2009 URL: https://github.com/giampaolo/psutil
- “One-shot Network Pruning at Initialization with Discriminative Image Patches”, 2022 arXiv:2209.05683 [cs.CV]
- “Neural Network Distiller: A Python Package For DNN Compression Research” In CoRR abs/1910.12232, 2019 arXiv: http://arxiv.org/abs/1910.12232
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.