Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multi-Objective Offloading Optimization in MEC and Vehicular-Fog Systems: A Distributed-TD3 Approach (2404.12584v1)

Published 19 Apr 2024 in cs.NI and eess.SP

Abstract: The emergence of 5G networks has enabled the deployment of a two-tier edge and vehicular-fog network. It comprises Multi-access Edge Computing (MEC) and Vehicular-Fogs (VFs), strategically positioned closer to Internet of Things (IoT) devices, reducing propagation latency compared to cloud-based solutions and ensuring satisfactory quality of service (QoS). However, during high-traffic events like concerts or athletic contests, MEC sites may face congestion and become overloaded. Utilizing offloading techniques, we can transfer computationally intensive tasks from resource-constrained devices to those with sufficient capacity, for accelerating tasks and extending device battery life. In this research, we consider offloading within a two-tier MEC and VF architecture, involving offloading from MEC to MEC and from MEC to VF. The primary objective is to minimize the average system cost, considering both latency and energy consumption. To achieve this goal, we formulate a multi-objective optimization problem aimed at minimizing latency and energy while considering given resource constraints. To facilitate decision-making for nearly optimal computational offloading, we design an equivalent reinforcement learning environment that accurately represents the network architecture and the formulated problem. To accomplish this, we propose a Distributed-TD3 (DTD3) approach, which builds on the TD3 algorithm. Extensive simulations, demonstrate that our strategy achieves faster convergence and higher efficiency compared to other benchmark solutions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. S. Ghazanfar, F. Hussain, A. U. Rehman, U. U. Fayyaz, F. Shahzad, and G. A. Shah, “Iot-flock: An open-source framework for iot traffic generation,” in 2020 International Conference on Emerging Trends in Smart Technologies (ICETST).   IEEE, 2020, pp. 1–6.
  2. H. Tran-Dang and D.-S. Kim, “Impact of task splitting on the delay performance of task offloading in the iot-enabled fog systems,” in 2021 International Conference on Information and Communication Technology Convergence (ICTC).   IEEE, 2021, pp. 661–663.
  3. L. Wang, J. Zhou, Y. Wang, and B. Lei, “Energy conserved computation offloading for o-ran based iot systems,” in ICC 2022-IEEE International Conference on Communications.   IEEE, 2022, pp. 4043–4048.
  4. B. Kar, W. Yahya, Y.-D. Lin, and A. Ali, “Offloading using traditional optimization and machine learning in federated cloud-edge-fog systems: A survey,” IEEE Communications Surveys & Tutorials, vol. 25, no. 2, 2023.
  5. P. Porambage, J. Okwuibe, M. Liyanage, M. Ylianttila, and T. Taleb, “Survey on multi-access edge computing for internet of things realization,” IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 2961–2991, 2018.
  6. B. Kar, Y.-D. Lin, and Y.-C. Lai, “Cost optimization of omnidirectional offloading in two-tier cloud–edge federated systems,” Journal of Network and Computer Applications, vol. 215, p. 103630, 2023.
  7. S. Kekki, W. Featherstone, Y. Fang, P. Kuure, A. Li, A. Ranjan, D. Purkayastha, F. Jiangping, D. Frydman, G. Verin et al., “Mec in 5g networks,” ETSI white paper, vol. 28, no. 2018, pp. 1–28, 2018.
  8. D. Wu, Y. Zhang, J. Luo, and R. Li, “Efficient data dissemination by crowdsensing in vehicular networks,” in 2014 IEEE 22nd International Symposium of Quality of Service (IWQoS).   IEEE, 2014, pp. 314–319.
  9. Y. Zhao, Y. Li, D. Wu, and N. Ge, “Overlapping coalition formation game for resource allocation in network coding aided d2d communications,” IEEE Transactions on Mobile Computing, vol. 16, no. 12, pp. 3459–3472, 2017.
  10. H. Li, H. Xu, C. Zhou, X. Lü, and Z. Han, “Joint optimization strategy of computation offloading and resource allocation in multi-access edge computing environment,” IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 10 214–10 226, 2020.
  11. W. Yahya, E. Oki, Y.-D. Lin, and Y.-C. Lai, “Scaling and offloading optimization in pre-cord and post-cord multi-access edge computing,” IEEE Transactions on Network and Service Management, vol. 18, no. 4, pp. 4503–4516, 2021.
  12. B.-S. Lin, B. Kar, T.-L. Chin, Y.-D. Lin, and C.-Y. Chen, “Cost optimization of cloud-edge-fog federated systems with bidirectional offloading: one-hop versus two-hop,” Telecommunication Systems, vol. 84, no. 4, pp. 487–505, 2023.
  13. Y.-D. Lin, J.-C. Hu, B. Kar, and L.-H. Yen, “Cost minimization with offloading to vehicles in two-tier federated edge and vehicular-fog systems,” in 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall).   IEEE, 2019, pp. 1–6.
  14. P. Agbaje, A. Anjum, A. Mitra, E. Oseghale, G. Bloom, and H. Olufowobi, “Survey of interoperability challenges in the internet of vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 22 838–22 861, 2022.
  15. B. Kar, K.-M. Shieh, Y.-C. Lai, Y.-D. Lin, and H.-W. Ferng, “Qos violation probability minimization in federating vehicular-fogs with cloud and edge systems,” IEEE Transactions on Vehicular Technology, vol. 70, no. 12, pp. 13 270–13 280, 2021.
  16. P. Dai, K. Hu, X. Wu, H. Xing, F. Teng, and Z. Yu, “A probabilistic approach for cooperative computation offloading in mec-assisted vehicular networks,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 2, pp. 899–911, 2020.
  17. M. Mukherjee, S. Kumar, C. X. Mavromoustakis, G. Mastorakis, R. Matam, V. Kumar, and Q. Zhang, “Latency-driven parallel task data offloading in fog computing networks for industrial applications,” IEEE Transactions on Industrial Informatics, vol. 16, no. 9, pp. 6050–6058, 2019.
  18. L.-H. Yen, J.-C. Hu, Y.-D. Lin, and B. Kar, “Decentralized configuration protocols for low-cost offloading from multiple edges to multiple vehicular fogs,” IEEE Transactions on Vehicular Technology, vol. 70, no. 1, pp. 872–885, 2020.
  19. X. He, H. Lu, M. Du, Y. Mao, and K. Wang, “Qoe-based task offloading with deep reinforcement learning in edge-enabled internet of vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 4, pp. 2252–2261, 2020.
  20. W. Zhan, C. Luo, J. Wang, C. Wang, G. Min, H. Duan, and Q. Zhu, “Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing,” IEEE Internet of Things Journal, vol. 7, no. 6, pp. 5449–5465, 2020.
  21. Y. Cui, L. Du, H. Wang, D. Wu, and R. Wang, “Reinforcement learning for joint optimization of communication and computation in vehicular networks,” IEEE Transactions on Vehicular Technology, vol. 70, no. 12, pp. 13 062–13 072, 2021.
  22. H. Zhou, K. Jiang, X. Liu, X. Li, and V. C. Leung, “Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing,” IEEE Internet of Things Journal, vol. 9, no. 2, pp. 1517–1530, 2021.
  23. N. Waqar, S. A. Hassan, A. Mahmood, K. Dev, D.-T. Do, and M. Gidlund, “Computation offloading and resource allocation in mec-enabled integrated aerial-terrestrial vehicular networks: A reinforcement learning approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 11, pp. 21 478–21 491, 2022.
  24. Y. Liu, H. Yu, S. Xie, and Y. Zhang, “Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks,” IEEE Transactions on Vehicular Technology, vol. 68, no. 11, pp. 11 158–11 168, 2019.
  25. Z. Ning, P. Dong, X. Wang, L. Guo, J. J. Rodrigues, X. Kong, J. Huang, and R. Y. Kwok, “Deep reinforcement learning for intelligent internet of vehicles: An energy-efficient computational offloading scheme,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 4, pp. 1060–1072, 2019.
  26. Z. Wu and D. Yan, “Deep reinforcement learning-based computation offloading for 5g vehicle-aware multi-access edge computing network,” China Communications, vol. 18, no. 11, pp. 26–41, 2021.
  27. Y. Xia, L. Wu, Z. Wang, X. Zheng, and J. Jin, “Cluster-enabled cooperative scheduling based on reinforcement learning for high-mobility vehicular networks,” IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 12 664–12 678, 2020.
  28. H. Zhou, Z. Zhang, Y. Wu, M. Dong, and V. C. Leung, “Energy efficient joint computation offloading and service caching for mobile edge computing: A deep reinforcement learning approach,” IEEE Transactions on Green Communications and Networking, 2022.
  29. I. Budhiraja, N. Kumar, H. Sharma, M. Elhoseny, Y. Lakys, and J. J. Rodrigues, “Latency-energy tradeoff in connected autonomous vehicles: a deep reinforcement learning scheme,” IEEE Transactions on Intelligent Transportation Systems, 2022.
  30. R. Sutton, “Barto: Reinforcement learning: An introduction,” IEEE Trans. Neural Netw, vol. 9, p. 1054, 1998.
  31. S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in International conference on machine learning.   PMLR, 2018, pp. 1587–1596.
  32. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
  33. E. Sergeant, “Epitools epidemiological calculators,” 2018.
  34. G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016.
  35. A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,” The Journal of Machine Learning Research, vol. 22, no. 1, pp. 12 348–12 355, 2021.
  36. S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi, “Optimization by simulated annealing,” science, vol. 220, no. 4598, pp. 671–680, 1983.
  37. S. M. A. Salehizadeh, P. Yadmellat, and M. B. Menhaj, “Local optima avoidable particle swarm optimization,” in 2009 IEEE Swarm Intelligence Symposium.   IEEE, 2009, pp. 16–21.
  38. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now