Learning-Based Computation Offloading for IoT Devices with Energy Harvesting (1712.08768v1)

Abstract: Internet of Things (IoT) devices can apply mobile-edge computing (MEC) and energy harvesting (EH) to provide the satisfactory quality of experiences for computation intensive applications and prolong the battery lifetime. In this article, we investigate the computation offloading for IoT devices with energy harvesting in wireless networks with multiple MEC devices such as base stations and access points, each with different computation resource and radio communication capability. We propose a reinforcement learning based computation offloading framework for an IoT device to choose the MEC device and determine the offloading rate according to the current battery level, the previous radio bandwidth to each MEC device and the predicted amount of the harvested energy. A "hotbooting" Q-learning based computation offloading scheme is proposed for an IoT device to achieve the optimal offloading performance without being aware of the MEC model, the energy consumption and computation latency model. We also propose a fast deep Q-network (DQN) based offloading scheme, which combines the deep learning and hotbooting techniques to accelerate the learning speed of Q-learning. We show that the proposed schemes can achieve the optimal offloading policy after sufficiently long learning time and provide their performance bounds under two typical MEC scenarios. Simulations are performed for IoT devices that use wireless power transfer to capture the ambient radio-frequency signals to charge the IoT batteries. Simulation results show that the fast DQN-based offloading scheme reduces the energy consumption, decreases the computation delay and the task drop ratio, and increases the utility of the IoT device in dynamic MEC, compared with the benchmark Q-learning based offloading.

• The paper introduces RL-based offloading strategies leveraging hotbooting Q-learning and fast DQN to accelerate convergence and boost performance.
• It demonstrates that advanced reinforcement learning minimizes energy consumption, task delays, and dropout rates in energy-harvesting IoT systems.
• The study highlights the potential for self-sustaining IoT networks via adaptive, real-time decision making in dynamic mobile edge computing environments.

An Analysis of Learning-Based Computation Offloading for IoT Devices with Energy Harvesting

The proliferation of Internet of Things (IoT) devices, such as sensors and wearable technologies, has necessitated advanced strategies to address their inherent constraints in power, computation, and memory. These restrictions are crucial, as they limit the capacity to support demanding applications like online gaming and facial recognition which require significant computational resources. The paper under review explores the intersection of computation offloading in mobile edge computing (MEC) with the sustainability of energy harvesting techniques for IoT devices, presenting innovative learning-based methodologies to enhance operational efficiency.

Computational Offload Framework

The authors propose a computation offloading framework that leverages reinforcement learning (RL) with a particular focus on Q-learning and deep Q-network (DQN) approaches facilitated by hotbooting techniques. This framework specifically targets IoT devices equipped with energy harvesting capabilities, enabling them to offload computational tasks to nearby MEC devices — without prior comprehensive knowledge of the MEC model, energy consumption, or latency profiles. This adaptability is framed as a Markov decision process (MDP), where the decision-making is driven by various state variables including radio bandwidth, predicted harvested energy, and the current battery level.

Notable Methodologies and Contributions

Two distinct RL-based offloading schemes are delineated:

1. Hotbooting Q-learning Based Offloading: This methodology enhances traditional Q-learning by utilizing transfer learning principles to expedite convergence rates. Through the incorporation of a hotbooting technique, the Q-values are initialized using experiences from analogous scenarios—resulting in reduced exploration time and improved learning efficiency.
2. Fast DQN-Based Offloading: This approach further elevates performance by integrating a CNN to approximate Q-values, thereby enabling efficient handling of large state spaces. This compression of state spaces through deep reinforcement learning (DRL) accelerates the learning process and improves quick adaptation to varying environmental conditions.

The results assert that the proposed schemes outperform traditional Q-learning-based strategies, particularly the DQN-based implementation, which achieves superior utility metrics, reduced energy consumption, and diminished task delay and dropout rates.

Implications and Future Direction

The robust numerical results of the proposed solutions highlight their potential in environments characterized by dynamic and diverse energy harvesting conditions. The implications extend to enhancing the sustainability and operational lifespan of IoT devices by optimizing offloading decisions without exhaustive pre-learned models. The research underscores a significant advancement toward self-sustaining IoT networks that leverage ambient energy efficiently while dynamically adapting to intricate computation-resource trade-offs.

The paper suggests that further exploration into real-world applications of these frameworks and their integration with power control and joint decision-making frameworks presents fertile ground for future investigation. This is crucial for accelerating the deployment of comprehensive solutions in energy-critical IoT scenarios, promoting ubiquitous deployment of resource-constrained devices across various industrial and consumer domains.

In conclusion, the findings corroborate the viability of using advanced reinforcement learning techniques to optimize computation offloading in IoT environments with energy harvesting, offering a concrete step toward enhancing the computational efficiency and power sustainability of future IoT networks.