Papers
Topics
Authors
Recent
Search
2000 character limit reached

Neural Network-Based Bandit: A Medium Access Control for the IIoT Alarm Scenario

Published 23 Jul 2024 in math.OC and cs.LG | (2407.16877v2)

Abstract: Efficient Random Access (RA) is critical for enabling reliable communication in Industrial Internet of Things (IIoT) networks. Herein, we propose a deep reinforcement learning based distributed RA scheme, entitled Neural Network-Based Bandit (NNBB), for the IIoT alarm scenario. In such a scenario, the devices may detect a common critical event, and the goal is to ensure the alarm information is delivered successfully from at least one device. The proposed NNBB scheme is implemented at each device, where it trains itself online and establishes implicit inter-device coordination to achieve the common goal. Devices can transmit simultaneously on multiple orthogonal channels and each possible transmission pattern constitutes a possible action for the NNBB, which uses a deep neural network to determine the action. Our simulation results show that as the number of devices in the network increases, so does the performance gain of the NNBB compared to the Multi-Armed Bandit (MAB) RA benchmark. For instance, NNBB experiences a 7% success rate drop when there are four channels and the number of devices increases from 10 to 60, while MAB faces a 25% drop.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. “The European market potential for (industrial) Internet of Things.” [Online]. Available: https://www.cbi.eu/market-information/outsourcing-itobpo/industrial-internet-things/market-potential
  2. E. Gelenbe, M. Nakıp, D. Marek, and T. Czachorski, “Mitigating the massive access problem in the Internet of Things,” in Proc. Int. Security Comput. Inf. Sci. (ISCIS), Jun. 2022, pp. 118–132.
  3. S. Su, J. Jiao, T. Yang, L. Xu, Y. Wang, and Q. Zhang, “Unequal timeliness protection massive access for mission critical communications in S-IoT,” IEEE Trans. Commun., vol. 72, no. 6, pp. 3211–3226, 2024.
  4. P. Raghuwanshi, O. L. A. López, P. Popovski, and M. Latva-Aho, “Channel scheduling for IoT access with spatial correlation,” IEEE Commun. Lett., vol. 28, no. 5, pp. 1014–1018, May 2024.
  5. O. L. A. López, N. H. Mahmood, M. Shehab, H. Alves, O. M. Rosabal, L. Marata, and M. Latva-Aho, “Statistical tools and methodologies for ultrareliable low-latency communication—a tutorial,” Proc. IEEE, vol. 111, no. 11, pp. 1502–1543, Nov. 2023.
  6. S. u. Haque, S. Chandak, F. Chiariotti, D. Günduz, and P. Popovski, “Learning to speak on behalf of a group: Medium access control for sending a shared message,” IEEE Commun. Lett., vol. 26, no. 8, pp. 1843–1847, Aug. 2022.
  7. M. A. Jadoon, A. Pastore, M. Navarro, and A. Valcarce, “Learning random access schemes for massive machine-type communication with MARL,” IEEE Trans. Mach. Learn. Commun. Netw., vol. 2, pp. 95–109, 2024.
  8. O. L. A. López, O. M. Rosabal, D. E. Ruiz-Guirola, P. Raghuwanshi, K. Mikhaylov, L. Lovén, and S. Iyer, “Energy-sustainable IoT connectivity: Vision, technological enablers, challenges, and future directions,” IEEE Open J. Commun. Soc., vol. 4, pp. 2609–2666, Oct. 2023.
  9. R. Kallimani, K. Pai, P. Raghuwanshi, S. Iyer, and O. L. López, “TinyML: Tools, applications, challenges, and future research directions,” Multimedia Tools Appl., vol. 83, pp. 29 015–29 045, Sept. 2023.
  10. L. Deng, D. Wu, J. Deng, P.-N. Chen, and Y. S. Han, “The story of 1/e1𝑒1/e1 / italic_e: Aloha-based and reinforcement-learning-based random access for delay-constrained communications,” arXiv preprint arXiv:2206.09779, 2022.
  11. L. Deng, D. Wu, Z. Liu, Y. Zhang, and Y. S. Han, “Reinforcement learning for improved random access in delay-constrained heterogeneous wireless networks,” arXiv preprint arXiv:2205.02057, 2022.
  12. D. Lee, Y. Zhao, and J. Lee, “Reinforcement learning for random access in multi-cell networks,” in Proc. Int. Conf. Artif. Intell. Inf. Commun. (ICAIIC), Apr. 2021, pp. 335–338.
  13. A. Rech and S. Tomasin, “Coordinated random access for industrial IoT with correlated traffic by reinforcement-learning,” in Proc. IEEE Globecom Workshops (GC Wkshps), Dec. 2021, pp. 1–6.
  14. Z. Jiang, S. Zhou, and Z. Niu, “Distributed policy learning based random access for diversified QoS requirements,” in Proc. IEEE Int. Conf. Commun. (ICC), May 2019, pp. 1–6.
  15. Y. Xu, J. Lou, T. Wang, J. Shi, T. Zhang, A. Paul, and Z. Wu, “Multiple access for heterogeneous wireless networks with imperfect channels based on deep reinforcement learning,” Electron., vol. 12, no. 23, 2023.
  16. Y. Yu, T. Wang, and S. C. Liew, “Deep-reinforcement learning multiple access for heterogeneous wireless networks,” IEEE J. Sel. Areas Commun., vol. 37, no. 6, pp. 1277–1290, Jun. 2019.
  17. M. A. Jadoon, A. Pastore, M. Navarro, and F. Perez-Cruz, “Deep reinforcement learning for random access in machine-type communication,” in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), Apr. 2022, pp. 2553–2558.
  18. Y. Z. Bekele and Y.-J. Choi, “Random access using deep reinforcement learning in dense mobile networks,” Sensors, vol. 21, no. 9, May 2021.
  19. O. Naparstek and K. Cohen, “Deep multi-user reinforcement learning for dynamic spectrum access in multichannel wireless networks,” in Proc. IEEE Global Commun. Conf. (GLOBECOM), Dec. 2017, pp. 1–7.
  20. O. L. A. López, G. Brante, R. D. Souza, M. Juntti, and M. Latva-Aho, “Coordinated pilot transmissions for detecting the signal sparsity level in massive IoT networks,” IEEE Trans. Commun., vol. 72, no. 3, pp. 1612–1624, Mar. 2024.
  21. I. J. Shapiro and K. S. Narendra, “Use of stochastic automata for parameter self-optimization with multimodal performance criteria,” IEEE Trans. Syst. Sci. Cybern., vol. 5, no. 4, pp. 352–360, Oct. 1969.
  22. I. Szita and A. Lörincz, “Learning tetris using the noisy cross-entropy method,” Neural Comput., vol. 18, no. 12, pp. 2936–2941, Dec. 2006.
  23. J. K. Gupta, M. Egorov, and M. Kochenderfer, “Cooperative multi-agent control using deep reinforcement learning,” in Proc. Int. Conf. Auton. Agents Multiagent Syst., Nov. 2017, pp. 66–83.
  24. M. A. Jadoon, A. Pastore, and M. Navarro, “Collision resolution with deep reinforcement learning for random access in machine-type communication,” in Proc. IEEE Veh. Technol. Conf. (VTC-Spring), Jun. 2022, pp. 1–6.
  25. Z. Li, J. Yang, C. Guo, J. Xiao, T. Tao, and C. Li, “A joint scheduling scheme for WiFi access TSN,” Sensors, vol. 24, no. 8, Apr. 2024.
  26. X. Jin, A. Saifullah, C. Lu, and P. Zeng, “Real-time scheduling for event-triggered and time-triggered flows in industrial wireless sensor-actuator networks,” in Proc. IEEE INFOCOM Conf. Comput. Commun., 2019, pp. 1684–1692.
  27. D. E. Ruíz-Guirola, O. L. López, and S. Montejo-Sánchez, “Configuring transmission thresholds in IIoT alarm scenarios for energy-efficient event reporting,” arXiv preprint arXiv:2407.03982, 2024.
  28. “Tensorflow.” [Online]. Available: https://www.tensorflow.org/api_docs/python/tf/clip_by_global_norm
  29. O. Nabati, T. Zahavy, and S. Mannor, “Online limited memory neural-linear bandits with likelihood matching,” in Proc. Int. Conf. Mach. Learn. (ICML), Jul. 2021, pp. 7905–7915.
  30. S. Petersen, “How to decide the number of hidden layers and nodes in a hidden layer?” 2013. [Online]. Available: https://www.researchgate.net/post/How-to-decide-the-number-of-hidden-layers-and-nodes-in-a-hidden-layer

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up