Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Traffic Learning and Proactive UAV Trajectory Planning for Data Uplink in Markovian IoT Models (2401.13827v1)

Published 24 Jan 2024 in cs.LG, cs.AI, and cs.NI

Abstract: The age of information (AoI) is used to measure the freshness of the data. In IoT networks, the traditional resource management schemes rely on a message exchange between the devices and the base station (BS) before communication which causes high AoI, high energy consumption, and low reliability. Unmanned aerial vehicles (UAVs) as flying BSs have many advantages in minimizing the AoI, energy-saving, and throughput improvement. In this paper, we present a novel learning-based framework that estimates the traffic arrival of IoT devices based on Markovian events. The learning proceeds to optimize the trajectory of multiple UAVs and their scheduling policy. First, the BS predicts the future traffic of the devices. We compare two traffic predictors: the forward algorithm (FA) and the long short-term memory (LSTM). Afterward, we propose a deep reinforcement learning (DRL) approach to optimize the optimal policy of each UAV. Finally, we manipulate the optimum reward function for the proposed DRL approach. Simulation results show that the proposed algorithm outperforms the random-walk (RW) baseline model regarding the AoI, scheduling accuracy, and transmission power.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. M. Latva-aho and K. Leppanen, “Key Drivers and Research Challenges for 6G Ubiquitous Wireless Intelligence,” 6G Flagship, University of Oulu, Finland, Sep 2019.
  2. N. H. Mahmood et al., “Six key features of machine type communication in 6G,” in 2020 2nd 6G Wireless Summit (6G SUMMIT), 2020, pp. 1–5.
  3. I. Jawhar, N. Mohamed, and J. Al-Jaroodi, “UAV-based data communication in wireless sensor networks: Models and strategies,” 2015 International Conference on Unmanned Aircraft Systems, ICUAS 2015, pp. 687–694, 07 2015.
  4. E. Eldeeb, M. Shehab, and H. Alves, “A learning-based fast uplink grant for massive IoT via support vector machines and long short-term memory,” IEEE Internet of Things Journal, vol. 9, no. 5, pp. 3889–3898, 2022.
  5. M. Laner, P. Svoboda, N. Nikaein, and M. Rupp, “Traffic models for machine type communications,” in ISWCS 2013; The Tenth International Symposium on Wireless Communication Systems, 2013, pp. 1–5.
  6. D. Minoli, K. Sohraby, and B. Occhiogrosso, “IoT considerations, requirements, and architectures for smart buildings-energy optimization and next-generation building management systems,” IEEE Internet of Things Journal, vol. 4, no. 1, pp. 269–283, 2017.
  7. A. Laya, L. Alonso, and J. Alonso-Zarate, “Is the random access channel of LTE and LTE-A suitable for M2M communications? a survey of alternatives,” IEEE Communications Surveys Tutorials, vol. 16, no. 1, pp. 4–16, 2014.
  8. E. Eldeeb et al., “Traffic prediction and fast uplink for hidden Markov IoT models,” IEEE Internet of Things Journal, vol. 9, no. 18, pp. 17 172–17 184, 2022.
  9. 3GPP, “TS 36.881 study on latency reduction techniques for LTE,” Tech. Spec., 2015.
  10. S. Ali, N. Rajatheva, and W. Saad, “Fast uplink grant for machine type communications: Challenges and opportunities,” IEEE Communications Magazine, vol. 57, no. 3, pp. 97–103, 2019.
  11. S. K. Sharma and X. Wang, “Toward massive machine type communications in ultra-dense cellular IoT networks: Current issues and machine learning-assisted solutions,” IEEE Communications Surveys Tutorials, vol. 22, no. 1, pp. 426–471, 2020.
  12. M. Khashei, M. Bijari, and G. A. Raissi Ardali, “Improvement of auto-regressive integrated moving average models using fuzzy logic and artificial neural networks (ANNs),” Neurocomputing, vol. 72, no. 4, pp. 956–967, 2009, brain Inspired Cognitive Systems (BICS 2006) / Interplay Between Natural and Artificial Computation (IWINAC 2007). [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231208002440
  13. R. Povinelli, M. Johnson, A. Lindgren, and J. Ye, “Time series classification using gaussian mixture models of reconstructed phase spaces,” IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 6, pp. 779–783, 2004.
  14. A. Vaswani et al., “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  15. S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” in 2012 Proceedings IEEE INFOCOM, 2012, pp. 2731–2735.
  16. A. Kosta, N. Pappas, and V. Angelakis, “Age of information: A new concept, metric, and tool,” 2017.
  17. M. Mozaffari et al., “A tutorial on UAVs for wireless networks: Applications, challenges, and open problems,” IEEE Communications Surveys Tutorials, vol. 21, no. 3, pp. 2334–2360, 2019.
  18. U. Challita, A. Ferdowsi, M. Chen, and W. Saad, “Machine learning for wireless connectivity and security of cellular-connected UAVs,” IEEE Wireless Communications, vol. 26, no. 1, pp. 28–35, 2019.
  19. Y. Gu et al., “Minimizing age of information in cognitive radio-based IoT systems: Underlay or overlay?” IEEE Internet of Things Journal, vol. 6, no. 6, pp. 10 273–10 288, 2019.
  20. N. Mahmood, N. Marchenko, M. Gidlund, and P. Popovski, “Wireless networks and industrial IoT applications, challenges and enablers: Applications, challenges and enablers,” 2021.
  21. N. Hossein Motlagh, T. Taleb, and O. Arouk, “Low-altitude unmanned aerial vehicles-based internet of things services: Comprehensive survey and future perspectives,” IEEE Internet of Things Journal, vol. 3, no. 6, pp. 899–922, 2016.
  22. M. Hatami, M. Leinonen, and M. Codreanu, “AoI minimization in status update control with energy harvesting sensors,” IEEE Transactions on Communications, vol. 69, no. 12, pp. 8335–8351, 2021.
  23. S. Sekander, H. Tabassum, and E. Hossain, “Multi-tier drone architecture for 5G/B5G cellular networks: Challenges, trends, and prospects,” IEEE Communications Magazine, vol. 56, no. 3, pp. 96–103, 2018.
  24. M. V. da Silva, R. D. Souza, H. Alves, and T. Abrão, “A NOMA-based Q-learning random access method for machine type communications,” IEEE Wireless Communications Letters, vol. 9, no. 10, pp. 1720–1724, 2020.
  25. S. Ali et al., “Sleeping multi-armed bandit learning for fast uplink grant allocation in machine type communications,” IEEE Transactions on Communications, vol. 68, no. 8, pp. 5072–5086, 2020.
  26. M. T. Islam, A.-e. M. Taha, and S. Akl, “A survey of access management techniques in machine type communications,” IEEE Communications Magazine, vol. 52, no. 4, pp. 74–81, 2014.
  27. Z. Zhou, R. Ratasuk, N. Mangalvedhe, and A. Ghosh, “Resource allocation for uplink grant-free ultra-reliable and low latency communications,” in 2018 IEEE 87th Vehicular Technology Conference (VTC Spring), 2018, pp. 1–5.
  28. O. Habachi, M.-A. Adjif, and J.-P. Cances, “Fast uplink grant for NOMA: A federated learning based approach,” in Ubiquitous Networking, O. Habachi, V. Meghdadi, E. Sabir, and J.-P. Cances, Eds.   Cham: Springer International Publishing, 2020, pp. 96–109.
  29. M. Mozaffari, W. Saad, M. Bennis, and M. Debbah, “Optimal transport theory for power-efficient deployment of unmanned aerial vehicles,” in 2016 IEEE International Conference on Communications (ICC), 2016, pp. 1–6.
  30. E. Kalantari, H. Yanikomeroglu, and A. Yongacoglu, “On the number and 3D placement of drone base stations in wireless cellular networks,” in 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall), 2016, pp. 1–6.
  31. J. Lyu, Y. Zeng, and R. Zhang, “UAV-aided offloading for cellular hotspot,” IEEE Transactions on Wireless Communications, vol. 17, no. 6, pp. 3988–4001, 2018.
  32. P. Tong et al., “Deep reinforcement learning for efficient data collection in UAV-aided internet of things,” in 2020 IEEE International Conference on Communications Workshops (ICC Workshops).   IEEE, 2020, pp. 1–6.
  33. C. Zhou et al., “Deep RL-based trajectory planning for AoI minimization in UAV-assisted IoT,” in 2019 11th International Conference on Wireless Communications and Signal Processing (WCSP).   IEEE, 2019, pp. 1–6.
  34. M. A. Abd-Elmagid, A. Ferdowsi, H. S. Dhillon, and W. Saad, “Deep reinforcement learning for minimizing age-of-information in UAV-assisted networks,” in 2019 IEEE Global Communications Conference (GLOBECOM), 2019, pp. 1–6.
  35. M. Yi et al., “Deep reinforcement learning for fresh data collection in UAV-assisted IoT networks,” in IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2020, pp. 716–721.
  36. Q. Zhang et al., “Predictive deployment of UAV base stations in wireless networks: Machine learning meets contract theory,” IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 637–652, 2021.
  37. Y. K. Tun et al., “Energy-efficient resource management in UAV-assisted mobile edge computing,” IEEE Communications Letters, vol. 25, no. 1, pp. 249–253, 2021.
  38. H. Peng and X. Shen, “Multi-agent reinforcement learning based resource management in MEC- and UAV-assisted vehicular networks,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 131–141, 2021.
  39. E. Eldeeb et al., “A learning-based trajectory planning of multiple UAVs for AoI minimization in IoT networks,” in 2022 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2022, pp. 172–177.
  40. A. M. Bedewy, Y. Sun, and N. B. Shroff, “Minimizing the age of information through queues,” IEEE Transactions on Information Theory, vol. 65, no. 8, pp. 5215–5232, 2019.
  41. L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, Feb 1989.
  42. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, pp. 1735–80, 12 1997.
  43. Y. Bengio, P. Simard, and P. Frasconi, “Learning long-term dependencies with gradient descent is difficult,” IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 157–166, 1994.
  44. V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–33, 02 2015.
  45. M. Hausknecht and P. Stone, “Deep recurrent Q-learning for partially observable mdps,” 2017.
  46. A. Hassan, M. R. Amin, A. K. A. Azad, and N. Mohammed, “Sentiment analysis on bangla and romanized bangla text using deep recurrent models,” in 2016 International Workshop on Computational Intelligence (IWCI), 2016, pp. 51–56.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Eslam Eldeeb (18 papers)
  2. Mohammad Shehab (29 papers)
  3. Hirley Alves (120 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube