Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices

Abstract: Today, deep learning optimization is primarily driven by research focused on achieving high inference accuracy and reducing latency. However, the energy efficiency aspect is often overlooked, possibly due to a lack of sustainability mindset in the field and the absence of a holistic energy dataset. In this paper, we conduct a threefold study, including energy measurement, prediction, and efficiency scoring, with an objective to foster transparency in power and energy consumption within deep learning across various edge devices. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. This study results in the creation of three extensive energy datasets for edge devices, covering a wide range of kernels, state-of-the-art DNN models, and popular AI applications. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset. Evaluation results demonstrate the ability of our predictors to provide consistent and accurate energy estimations on unseen DNN models. Lastly, we introduce two scoring metrics, PCS and IECS, developed to convert complex power and energy consumption data of an edge device into an easily understandable manner for edge device end-users. We hope our work can help shift the mindset of both end-users and the research community towards sustainability in edge computing, a principle that drives our research. Find data, code, and more up-to-date information at https://amai-gsu.github.io/DeepEn2023.

• The paper presents a groundbreaking measurement study that built three extensive datasets to capture energy usage patterns across diverse DNN models and kernels on edge devices.
• It introduces the first kernel-level energy predictors that deliver accurate future estimations for unseen deep neural network models.
• The paper proposes two scoring metrics, PCS and IECS, to distill complex energy data into actionable insights for sustainable computing.

The paper "Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices" addresses the often neglected issue of energy efficiency in deep learning, particularly within the context of edge devices. The research comprises a comprehensive three-part study focusing on energy measurement, prediction, and efficiency scoring to enhance transparency in energy consumption of on-device deep learning.

Key Contributions:

1. Energy Measurement Study:
   • The authors conducted a pioneering measurement study to explore the energy consumption characteristics of deep learning on edge devices.
   • This effort resulted in the creation of three extensive energy datasets, which include a variety of kernels, contemporary deep neural network (DNN) models, and popular AI applications.
   • The datasets provide critical insights into how different models and applications impact the energy consumption across various edge devices.
2. Energy Prediction:
   • The paper introduces the first kernel-level energy predictors for edge devices, built upon the kernel-level energy dataset they constructed.
   • These predictors are designed to offer consistent and accurate energy estimations for unseen DNN models, enhancing the ability to forecast the energy needs of forthcoming applications and models effectively.
3. Efficiency Scoring Metrics:
   • Two scoring metrics, PCS (Power Consumption Score) and IECS (Integrated Energy Consumption Score), are proposed to simplify the complex data surrounding power and energy usage into an easily interpretable format for end-users.
   • These metrics aim to aid users in understanding and managing energy consumption, ultimately promoting a culture of sustainability in the use of edge computing resources.

The paper advocates for a shift in the deep learning community toward embracing sustainability, emphasizing that fostering transparency in energy use is crucial for environmentally conscious computing practices. The study not only provides tools and datasets but also serves as a call to action for researchers and practitioners to consider energy efficiency as a fundamental component of their work.