Papers
Topics
Authors
Recent
Search
2000 character limit reached

Collaborative Information Dissemination with Graph-based Multi-Agent Reinforcement Learning

Published 25 Aug 2023 in cs.LG, cs.AI, cs.MA, and cs.NI | (2308.16198v3)

Abstract: Efficient information dissemination is crucial for supporting critical operations across domains like disaster response, autonomous vehicles, and sensor networks. This paper introduces a Multi-Agent Reinforcement Learning (MARL) approach as a significant step forward in achieving more decentralized, efficient, and collaborative information dissemination. We propose a Partially Observable Stochastic Game (POSG) formulation for information dissemination empowering each agent to decide on message forwarding independently, based on the observation of their one-hop neighborhood. This constitutes a significant paradigm shift from heuristics currently employed in real-world broadcast protocols. Our novel approach harnesses Graph Convolutional Reinforcement Learning and Graph Attention Networks (GATs) with dynamic attention to capture essential network features. We propose two approaches, L-DyAN and HL-DyAN, which differ in terms of the information exchanged among agents. Our experimental results show that our trained policies outperform existing methods, including the state-of-the-art heuristic, in terms of network coverage as well as communication overhead on dynamic networks of varying density and behavior.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Deep reinforcement learning for multi-agent interaction. AI Commun., 35(4):357–368, 2022.
  2. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2023.
  3. How attentive are graph attention networks? In International Conference on Learning Representations (Poster), 2022.
  4. Multi-agent Reinforcement Learning: An Overview, pages 183–221. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.
  5. TarMAC: Targeted multi-agent communication. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1538–1546. PMLR, 09–15 Jun 2019.
  6. Optimized Link State Routing Protocol Version 2 (OLSRv2) and MANET Neighborhood Discovery Protocol (NHDP) Extension TLVs. RFC 7188, April 2014.
  7. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
  8. Learning to communicate with deep multi-agent reinforcement learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 2145–2153, Red Hook, NY, USA, 2016. Curran Associates Inc.
  9. Learning to sail dynamic networks: The marlin reinforcement learning framework for congestion control in tactical environments. In MILCOM 2023 - 2023 IEEE Military Communications Conference (MILCOM), pages 424–429, 2023.
  10. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, 1979.
  11. Information diffusion in online social networks: A survey. SIGMOD Rec., 42(2):17–28, jul 2013.
  12. Exploring network structure, dynamics, and function using networkx. 1 2008.
  13. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 1025–1035, Red Hook, NY, USA, 2017. Curran Associates Inc.
  14. Deep recurrent q-learning for partially observable mdps. In 2015 AAAI Fall Symposia, Arlington, Virginia, USA, November 12-14, 2015, pages 29–37. AAAI Press, 2015.
  15. A comprehensive survey on vanet broadcast protocols. In 2020 International Conference on Computation, Automation and Knowledge Management (ICCAKM), pages 298–302, 2020.
  16. Learning attentional communication for multi-agent cooperation. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 7265–7275, Red Hook, NY, USA, 2018. Curran Associates Inc.
  17. Graph convolutional reinforcement learning. In International Conference on Learning Representations, 2020.
  18. Deepcq+: Robust and scalable routing with multi-agent deep reinforcement learning for highly dynamic networks. In 2021 IEEE Military Communications Conference, MILCOM 2021, San Diego, CA, USA, November 29 - Dec. 2, 2021, pages 31–36. IEEE, 2021.
  19. Deepmpr: Enhancing opportunistic routing in wireless networks through multi-agent deep reinforcement learning, 2023.
  20. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
  21. Design and analysis of a robust broadcast scheme for vanet safety-related services. IEEE Transactions on Vehicular Technology, 61(1):46–61, 2012.
  22. J. Macker. Rfc 6621: Simplified multicast forwarding, 2012.
  23. Multiagent bidirectionally-coordinated nets: Emergence of human-level coordination in learning to play starcraft combat games, 2017.
  24. Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., USA, 1st edition, 1994.
  25. Multipoint relaying for flooding broadcast messages in mobile wireless networks. In Proceedings of the 35th Annual Hawaii International Conference on System Sciences, pages 3866–3875, 2002.
  26. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2009.
  27. Proximal policy optimization algorithms, 2017.
  28. Learning multiagent communication with backpropagation. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 2252–2260, Red Hook, NY, USA, 2016. Curran Associates Inc.
  29. Comparing performance of group communications protocols over scb versus routed manet networks. In MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM), pages 1011–1017, 2022.
  30. Adaptive information dissemination over tactical edge networks. In 2023 International Conference on Military Communications and Information Systems (ICMCIS), pages 1–7, 2023.
  31. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018.
  32. Pettingzoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34:15032–15043, 2021.
  33. Broadcasting in vanet. In 2007 Mobile Networking for Vehicular Environments, pages 7–12, 2007.
  34. Graph attention networks. In International Conference on Learning Representations, 2018.
  35. Dueling network architectures for deep reinforcement learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, page 1995–2003. JMLR.org, 2016.
  36. Tianshou: A highly modularized deep reinforcement learning library. Journal of Machine Learning Research, 23(267):1–6, 2022.
  37. Deepadmr: A deep learning based anomaly detection for MANET routing. In IEEE Military Communications Conference, MILCOM 2022, Rockville, MD, USA, November 28 - December 2, 2022, pages 412–417. IEEE, 2022.
  38. Performance evaluation indicators of space dynamic networks under broadcast mechanism. Space: Science & Technology, 2021, 2021.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up