Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cooperative Multi-Agent Deep Reinforcement Learning Methods for UAV-aided Mobile Edge Computing Networks (2407.03280v1)

Published 3 Jul 2024 in cs.IT and math.IT

Abstract: This paper presents a cooperative multi-agent deep reinforcement learning (MADRL) approach for unmmaned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks. An UAV with computing capability can provide task offlaoding services to ground internet-of-things devices (IDs). With partial observation of the entire network state, the UAV and the IDs individually determine their MEC strategies, i.e., UAV trajectory, resource allocation, and task offloading policy. This requires joint optimization of decision-making process and coordination strategies among the UAV and the IDs. To address this difficulty, the proposed cooperative MADRL approach computes two types of action variables, namely message action and solution action, each of which is generated by dedicated actor neural networks (NNs). As a result, each agent can automatically encapsulate its coordination messages to enhance the MEC performance in the decentralized manner. The proposed actor structure is designed based on graph attention networks such that operations are possible regardless of the number of IDs. A scalable training algorithm is also proposed to train a group of NNs for arbitrary network configurations. Numerical results demonstrate the superiority of the proposed cooperative MADRL approach over conventional methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. N. Abbas, Y. Zhang, A. Taherkordi, and T. Skeie, “Mobile edge computing: A survey,” IEEE Internet Things J., vol. 5, no. 1, pp. 450–465, Feb. 2018.
  2. C. L. Chen, C. G. Brinton, and V. Aggarwal, “Latency minimization for mobile edge computing networks,” IEEE Trans. Mobile Comput., vol. 22, no. 4, pp. 2233–2247, Apr. 2023.
  3. L. Zhang, Y. Sun, Z. Chen, and S. Roy, “Communications-caching-computing resource allocation for bidirectional data computation in mobile edge networks,” IEEE Trans. Commun., vol. 69, no. 3, pp. 1496–1509, Nov. 2021.
  4. M. Masoudi and C. Cavdar, “Device vs edge computing for mobile services: Delay-aware decision making to minimize power consumption,” IEEE Trans. Mobile Comput., vol. 20, no. 12, pp. 3324–3337, Dec. 2021.
  5. K. Zhang, S. Leng, Y. He, S. Maharjan, and Y. Zhang, “Mobile edge computing and networking for green and low-latency internet of things,” IEEE Commun. Mag., vol. 56, no. 5, p. 39–45, May 2018.
  6. M. Kim, H. Lee, S. Hwang, M.Kim, M. Debbah, and I. Lee, “Decentralized learning framework for hierarchical wireless networks: A tree neural network approach,” IEEE Internet Things J., vol. 11, no. 10, pp. 17 780–17 796, May 2024.
  7. J. Park, S. Solanki, S. Baek, and I. Lee, “Latency minimization for wireless powered mobile edge computing networks with nonlinear rectifiers,” IEEE Trans. Veh. Technol., vol. 70, no. 8, pp. 8320–8324, Aug. 2021.
  8. M. Wu, W. Qi, J. Park, P. Lin, L. Guo, and I. Lee, “Residual energy maximization for wireless powered mobile edge computing systems with mixed-offloading,” IEEE Trans. Veh. Technol., vol. 71, no. 4, pp. 4523–4528, Apr. 2022.
  9. N. Kiran and C. Pan and S. Wang and C. Yin, “Joint resource allocation and computation offloading in mobile edge computing for SDN based wireless networks,” J. Commun. Netw., vol. 22, no. 1, pp. 1–11, Feb 2020.
  10. Y. Zeng, R. Zhang, and T. J. Lim, “Wireless communications with unmanned aerial vehicles: opportunities and challenges,” IEEE Commun. Mag., vol. 54, no. 5, pp. 36–42, May 2016.
  11. N. H. Motlagh, T. Taleb, and O. Arouk, “Low-altitude unmanned aerial vehicles-based internet of things services: Comprehensive survey and future perspectives,” IEEE Internet Things J., vol. 3, no. 6, pp. 899–922, Dec. 2016.
  12. B. Li, Z. Fei, and Y. Zhang, “UAV communications for 5G and beyond: Recent advances and future trends,” IEEE Internet Things J., vol. 6, no. 2, pp. 2241–2263, Apr. 2019.
  13. Z. Liu, J. Qi, Y. Shen, K. Ma, and X. Guan, “Maximizing energy efficiency in UAV-assisted NOMA–MEC networks,” IEEE Internet Things J., vol. 10, no. 24, pp. 22 208–22 222, Dec 2023.
  14. H. Zhou, Z. Wang, G. Min, and H. Zhang, “UAV-Aided computation offloading in mobile-edge computing networks: A stackelberg game approach,” IEEE Internet Things J., vol. 10, no. 8, pp. 6622–6633, Apr 2023.
  15. H. Xie, T. Zhang, X. Xu, D. Yang, and Y. Liu, “Joint sensing, communication and computation in UAV-assisted systems,” IEEE Internet Things J., Mar 2024, to be published.
  16. Y. Liu, S. Xie, and Y. Zhang, “Cooperative offloading and resource management for UAV-enabled mobile edge computing in power IoT system,” IEEE Trans. Veh. Technol., vol. 69, no. 10, pp. 12 229–12 239, Oct. 2020.
  17. Q. Liu, L. Shi, L. Sun, J. Li, M. Ding, and F. Shu, “Path planning for UAV-mounted mobile edge computing with deep reinforcement learning,” IEEE Trans. Veh. Technol., vol. 69, no. 5, pp. 5723–5728, May 2020.
  18. L. Wang, P. Huang, K. Wang, G. Zhang, L. Zhang, N. Aslam, and K. Yang, “Rl-based user association and resource allocation for multi-UAV enabled MEC,” in Proc. Int. Wireless. Commun. Mobile Comput. Conf., pp. 741–746, Jun. 2019.
  19. H. Wang, H. Ke, and W. Sun, “Unmanned-aerial-vehicle-assisted computation offloading for mobile edge computing based on deep reinforcement learning,” IEEE Access, vol. 8, pp. 180 784–180 798, Oct. 2020.
  20. Y. Peng, Y. Liu, and H. Zhang, “Deep reinforcement learning based path planning for UAV-assisted edge computing networks,” in Proc. IEEE Wireless Commun. Netw. Conf, pp. 1–6, Mar. 2021.
  21. L. Wang, K. Wang, C. Pan, W. Xu, N. Aslam, and A. Nallanathan, “Deep reinforcement learning based dynamic trajectory control for UAV-assisted mobile edge computing,” IEEE Trans. Mobile Comput., vol. 21, no. 10, pp. 3536–3550, Oct. 2022.
  22. S. Hwang, J. Park, H. Lee, M. Kim, and I. Lee, “Deep reinforcement learning approach for UAV-assisted mobile edge computing networks,” in Proc. IEEE Global Commun. Conf., Dec. 2022, pp. 3839–3844.
  23. Z. Chen and X. Wang, “Decentralized computation offloading for multi-user mobile edge computing: A deep reinforcement learning approach,” EURASIP J. Wireless. Commun. Netw., vol. 2020, no. 188, pp. 1–21, Sep. 2020.
  24. F. Song, H. Xing, X. Wang, S. Luo, P. Dai, Z. Xiao, and B. Zhao, “Evolutionary multi-objective reinforcement learning based trajectory control and task offloading in UAV-assisted mobile edge computing,,” IEEE Trans. Mobile Comput., vol. 22, no. 12, pp. 7387–7405, Dec. 2023.
  25. Y. M. Park, S. S. Hassan, Y. K. Tun, Z. Han, and C. S. Hong, “Joint trajectory and resource optimization of MEC-assisted UAVs in sub-THz networks: A resources-based multi-agent proximal policy optimization DRL with attention mechanism,” IEEE Trans. Veh. Technol., early access, doi: 10.1109/TVT.2023.3311537.
  26. B. Zhang, B. Tang, and F. Xiao, “Robust computation offloading and trajectory optimization for multi-UAV-assisted MEC: A multi-agent DRL approach,” IEEE Internet Things J., early access, doi: 10.1109/JIOT.2023.3300718.
  27. B. Li, W. Liu, W. Xie, N. Zhang, and Y. Zhang, “Adaptive digital twin for UAV-assisted integrated sensing, communication, and computation networks,” IEEE Trans. Green. Commun. Netw., vol. 7, no. 4, pp. 10 497–10 509, Dec. 2023.
  28. S. Hwang, H. Lee, J. Park, and I. Lee, “Decentralized computation offloading with cooperative UAVs: multi-agent deep reinforcement learning perspective,” IEEE Wireless Commun., vol. 29, no. 4, pp. 24–31, Aug. 2022.
  29. A. M. Seid, G. O. Boateng, B. Mareri, G. Sun, and W. Jiang, “Multi-agent DRL for task offloading and resource allocation in multi-UAV enabled IoT edge network,” IEEE Trans. Netw. Service Manage., vol. 18, no. 4, pp. 4534–4547, Dec. 2021.
  30. A. Gao, Q. Wang, W. Liang, and Z. Ding, “Game combined multi-agent reinforcement learning approach for UAV assisted offloading,” IEEE Trans. Veh. Technol., vol. 70, no. 12, pp. 12 888–12 901, Dec. 2021.
  31. H. Peng and X. Shen, “Multi-agent reinforcement learning based resource management in MEC- and UAV-assisted vehicular networks,” IEEE J. Sel. Areas Commun., vol. 39, no. 1, pp. 131–141, Jan. 2021.
  32. N. Zhao, Z. Ye, Y. Pei, Y. C. Liang, and D. Niyato, “Multi-agent deep reinforcement learning for task offloading in UAV-assisted mobile edge computing,” IEEE Trans. Wireless Commun., vol. 21, no. 9, pp. 6949–6960, Sept. 2022.
  33. Z. Ji, S. Wu, and C. Jiang, “Cooperative multi-agent deep reinforcement learning for computation offloading in digital twin satellite edge networks,” IEEE J. Sel. Areas Commun., early access, doi: 10.1109/JSAC.2023.3313595.
  34. R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multiagent actor-critic for mixed cooperative-competitive environments,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), pp. 6382–6393, Jun. 2017.
  35. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Deep reinforcement learning approach for UAV-assisted mobile edge computing networks,” in Proc. Int. Conf. Learn. Represent., Apr. 2018, pp. 1–13.
  36. S. Batabyal and P. Bhaumik, “Mobility models, traces and impact of mobility on opportunistic routing algorithms: A survey,” IEEE Commun. Surveys Tuts., vol. 17, no. 3, pp. 1679–1707, 3rd Quart.,2015.
  37. A. Al-Hourani, S. Kandeepan, and S. Lardner, “Optimal LAP altitude for maximum coverage,” IEEE Wireless Commun. Lett., vol. 3, no. 6, pp. 569–572, Dec. 2014.
  38. Y. Zeng, J. Xu, and R. Zhang, “Energy minimization for wireless communication with rotary-wing UAV,” IEEE Trans. Wireless Commun., vol. 18, no. 4, pp. 2329–2345, April 2019.
  39. Y. Wang, M. Sheng, X. Wang, L. Wang, and J. Li, “Mobile-edge computing: Partial computation offloading using dynamic voltage scaling,” IEEE Trans. Commun., vol. 64, no. 10, p. 4268, Oct. 2016.
  40. R. Goldberg, “Survey of virtual machine research,” IEEE Computer Mag., vol. 7, no. 6, pp. 34–45, Jun 1974.
  41. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Trans. Neural. Netw, vol. 20, no. 1, pp. 61–80, Jan. 2009.
  42. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, Jun 2014.
  43. S. Zhang, H. Tong, J. Xu, and R. Maciejewski, “Graph convolutional networks: A comprehensive review,” Comput. Social Netw., vol. 6, no. 1, pp. 1–23, Dec. 2019.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now