Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning to generate Reliable Broadcast Algorithms (2208.00525v1)

Published 31 Jul 2022 in cs.DC, cs.DS, cs.LG, and cs.NI

Abstract: Modern distributed systems are supported by fault-tolerant algorithms, like Reliable Broadcast and Consensus, that assure the correct operation of the system even when some of the nodes of the system fail. However, the development of distributed algorithms is a manual and complex process, resulting in scientific papers that usually present a single algorithm or variations of existing ones. To automate the process of developing such algorithms, this work presents an intelligent agent that uses Reinforcement Learning to generate correct and efficient fault-tolerant distributed algorithms. We show that our approach is able to generate correct fault-tolerant Reliable Broadcast algorithms with the same performance of others available in the literature, in only 12,000 learning episodes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. A survey of machine learning for big code and naturalness. ACM Computing Surveys, 51(4):1–37, 2018.
  2. Tim Anema. Message efficient Byzantine reliable broadcast protocols on known topologies. Bachelor’s Thesis, TU Delft, 2021.
  3. A view of cloud computing. Communications of the ACM, 53(4):50–58, 2010.
  4. Automatic HTML code generation from mock-up images using machine learning techniques. In 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science, pages 1–4, 2019.
  5. Automatic discovery of mutual exclusion algorithms. In International Symposium on Distributed Computing, pages 136–150. Springer, 2003.
  6. Simplifying fault-tolerance: providing the abstraction of crash failures. Journal of the ACM, 48(3):499–554, 2001.
  7. Michael Ben-Or. Another advantage of free choice: Completely asynchronous agreement protocols. In Proceedings of the 2nd Annual ACM Symposium on Principles of Distributed Computing, pages 27–30, 1983.
  8. Practical Byzantine reliable broadcast on partially connected networks. arXiv preprint arXiv:2104.03673, 2021.
  9. Gabriel Bracha. An asynchronous [(n-1)/3]-resilient consensus protocol. In Proceedings of the 3rd Annual ACM Symposium on Principles of Distributed Computing, pages 154–162, 1984.
  10. Asynchronous consensus and broadcast protocols. Journal of the ACM, 32(4):824–840, 1985.
  11. Automatic code generation from design patterns. IBM Systems Journal, 35(2):151–171, 1996.
  12. Introduction to reliable and secure distributed programming. Springer Science & Business Media, 2011.
  13. Secure and efficient asynchronous broadcast protocols. In Annual International Cryptology Conference, pages 524–541. Springer, 2001.
  14. Practical Byzantine fault tolerance. In Proceedings of the 3rd USENIX Symposium on Operating Systems Design and Implementation, pages 173–186, 1999.
  15. Reliable broadcast protocols. ACM Transactions on Computer Systems, 2(3):251–273, 1984.
  16. Melanie Coggan. Exploration and exploitation in reinforcement learning. Research supervised by Prof. Doina Precup, CRA-W DMP Project at McGill University, 2004.
  17. Generating GPU compiler heuristics using reinforcement learning. arXiv preprint arXiv:2111.12055, 2021.
  18. From consensus to atomic broadcast: Time-free Byzantine-resistant protocols without signatures. The Computer Journal, 49(1):82–96, 2006.
  19. Asynchronous Byzantine consensus with 2f+1 processes. In Proceedings of the 2010 ACM Symposium on Applied Computing, pages 475–480, 2010.
  20. Model checking paxos in spin. Electronic Proceedings in Theoretical Computer Science, 161:131–146, aug 2014.
  21. Knowledge and common knowledge in a Byzantine environment: crash failures. Information and Computation, 88(2):156–186, 1990.
  22. Lightweight probabilistic broadcast. ACM Transactions on Computer Systems, 21(4):341–374, 2003.
  23. Tutorial on parameterized model checking of fault-tolerant distributed algorithms. In International School on Formal Methods for the Design of Computer, Communication and Software Systems, pages 122–171. Springer, 2014.
  24. On symmetry and quantification: A new approach to verify distributed protocols. In NASA Formal Methods Symposium, pages 131–150. Springer, 2021.
  25. Towards an automatic proof of Lamport’s Paxos. In Ruzica Piskac and Michael W Whalen, editors, Formal Methods in Computer-Aided Design, pages 112–122, New Haven, Connecticut, October 2021. doi:https://doi.org/10.34727/2021/isbn.978-3-85448-046-4_20.
  26. A modular approach to fault-tolerant broadcasts and related problems. Technical Report TR94-1425, Cornell University, Department of Computer Science, May 1994.
  27. Gerard J. Holzmann. The model checker SPIN. IEEE Transactions on Software Engineering, 23(5):279–295, 1997.
  28. How to train your robot with deep reinforcement learning: lessons we have learned. The International Journal of Robotics Research, 40(4-5):698–721, 2021.
  29. Simple and efficient reliable broadcast in the presence of Byzantine processes. arXiv preprint arXiv:1510.06882, 2015.
  30. Trading off t-resilience for efficiency in asynchronous Byzantine reliable broadcast. Parallel Processing Letters, 26(04):1650017, 2016.
  31. Towards modeling and model checking fault-tolerant distributed algorithms. In International SPIN Workshop on Model Checking of Software, pages 209–226. Springer, 2013.
  32. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–285, 1996.
  33. Generating test input with deep reinforcement learning. In 2018 IEEE/ACM 11th International Workshop on Search-Based Software Testing, pages 51–58, 2018.
  34. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11):1238–1274, 2013.
  35. ByMC: Byzantine model checker. In International Symposium on Leveraging Applications of Formal Methods, pages 327–342. Springer, 2018.
  36. Algorithm selection using reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, pages 511–518, 2000.
  37. Playing FPS games with deep reinforcement learning. In 31st AAAI Conference on Artificial Intelligence, 2017.
  38. Specifying and verifying fault-tolerant systems. In Formal Techniques in real-time and fault-tolerant systems, pages 41–76. Springer, 1994.
  39. The Byzantine generals problem. ACM Transactions on Programming Languages and Systems, 4(3):382–401, 1982.
  40. Deep learning for source code modeling and generation: Models, applications, and challenges. ACM Computing Surveys, 53(3):1–38, 2020.
  41. Douglas B Lenat. The nature of heuristics. Artificial Intelligence, 19(2):189–249, 1982.
  42. Nancy A Lynch. Distributed Algorithms. Morgan Kaufmann, 1996.
  43. Maja J Matarić. Reinforcement learning in the multi-robot domain. In Robot Colonies, pages 73–83. Springer, 1997.
  44. Language and tool support for model checking of fault-tolerant distributed algorithms. In 2008 14th IEEE Pacific Rim International Symposium on Dependable Computing, pages 40–47, 2008. doi:10.1109/PRDC.2008.13.
  45. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
  46. Automatic code generation for embedded systems: From UML specifications to VHDL code. In 8th IEEE International Conference on Industrial Informatics, pages 1085–1090, 2010.
  47. Heiner Müller-Merbach. Heuristics and their design: a survey. European Journal of Operational Research, 8(1):1–23, 1981.
  48. A modern approach. Prentice Hall Upper Saddle River, NJ, USA:, 2002.
  49. SQUARES: a SQL synthesizer using query reverse engineering. Proceedings of the VLDB Endowment, 13(12):2853–2856, 2020.
  50. Morgen E Peck. Blockchains: How they work and why they’ll change the world. IEEE Spectrum, 54(10):26–35, 2017.
  51. Velisarios: Byzantine fault-tolerant protocols powered by coq. In European Symposium on Programming, pages 619–650. Springer, 2018.
  52. Michel Raynal. Fault-Tolerant Message-Passing Distributed Systems. Springer, 2018.
  53. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018.
  54. Reinforcement Learning: An Introduction. MIT Press, 2018.
  55. Q-learning. Machine Learning, 8(3-4):279–292, 1992.
  56. Reinforcement learning. Adaptation, Learning, and Optimization, 12(3), 2012.
  57. Analysis of automatic code generation tools based on machine learning. In 2019 IEEE International Conference on Computer Science and Educational Informatization, pages 263–270, 2019.
  58. Code generation based on deep learning: a brief review. arXiv preprint arXiv:2106.08253, 2021.
  59. Piotr Zielinski. Automatic verification and discovery of byzantine consensus protocols. In 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’07), pages 72–81. IEEE, 2007.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now