MRL-PoS: A Multi-agent Reinforcement Learning based Proof of Stake Consensus Algorithm for Blockchain (2312.09123v1)
Abstract: The core of a blockchain network is its consensus algorithm. Starting with the Proof-of-Work, there have been various versions of consensus algorithms, such as Proof-of-Stake (PoS), Proof-of-Authority (PoA), and Practical Byzantine Fault Tolerance (PBFT). Each of these algorithms focuses on different aspects to ensure efficient and reliable processing of transactions. Blockchain operates in a decentralized manner where there is no central authority and the network is composed of diverse users. This openness creates the potential for malicious nodes to disrupt the network in various ways. Therefore, it is crucial to embed a mechanism within the blockchain network to constantly monitor, identify, and eliminate these malicious nodes. However, there is no one-size-fits-all mechanism to identify all malicious nodes. Hence, the dynamic adaptability of the blockchain network is important to maintain security and reliability at all times. This paper introduces MRL-PoS, a Proof-of-Stake consensus algorithm based on multi-agent reinforcement learning. MRL-PoS employs reinforcement learning for dynamically adjusting to the behavior of all users. It incorporates a system of rewards and penalties to eliminate malicious nodes and incentivize honest ones. Additionally, MRL-PoS has the capability to learn and respond to new malicious tactics by continually training its agents.
- S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” Decentralized business review, 2008.
- F. Saleh, “Blockchain without waste: Proof-of-stake,” The Review of financial studies, vol. 34, no. 3, pp. 1156–1190, 2021.
- M. Castro, B. Liskov et al., “Practical byzantine fault tolerance,” in OsDI, vol. 99, no. 1999, 1999, pp. 173–186.
- S. De Angelis, L. Aniello, R. Baldoni, F. Lombardi, A. Margheri, V. Sassone et al., “Pbft vs proof-of-authority: Applying the cap theorem to permissioned blockchain,” in CEUR workshop proceedings, vol. 2058. CEUR-WS, 2018.
- A. De Vries, “Cryptocurrencies on the road to sustainability: Ethereum paving the way for bitcoin,” Patterns, vol. 4, no. 1, 2023.
- G. Neto, “From single-agent to multi-agent reinforcement learning: Foundational concepts and methods,” Learning theory course, vol. 2, 2005.
- G. Sartoretti, Y. Wu, W. Paivine, T. S. Kumar, S. Koenig, and H. Choset, “Distributed reinforcement learning for multi-robot decentralized collective construction,” in Distributed Autonomous Robotic Systems: The 14th International Symposium. Springer, 2019, pp. 35–49.
- M. Fayaz, W. Yi, Y. Liu, and A. Nallanathan, “Transmit power pool design for grant-free noma-iot networks via deep reinforcement learning,” IEEE Transactions on Wireless Communications, vol. 20, no. 11, pp. 7626–7641, 2021.
- N. Chaudhry and M. M. Yousaf, “Consensus algorithms in blockchain: Comparative analysis, challenges and opportunities,” in 2018 12th International Conference on Open Source Systems and Technologies (ICOSST). IEEE, 2018, pp. 54–63.
- O. Vashchuk and R. Shuwar, “Pros and cons of consensus algorithm proof of stake. difference in the network safety in proof of work and proof of stake,” Electronics and Information Technologies, vol. 9, no. 9, pp. 106–112, 2018.
- L. M. Bach, B. Mihaljevic, and M. Zagar, “Comparative analysis of blockchain consensus algorithms,” in 2018 41st international convention on information and communication technology, electronics and microelectronics (MIPRO). Ieee, 2018, pp. 1545–1550.
- S. King and S. Nadal, “Ppcoin: Peer-to-peer crypto-currency with proof-of-stake,” self-published paper, August, vol. 19, no. 1, 2012.
- A. Kiayias, A. Russell, B. David, and R. Oliynykov, “Ouroboros: A provably secure proof-of-stake blockchain protocol,” in Annual international cryptology conference. Springer, 2017, pp. 357–388.
- T. Kerber, A. Kiayias, M. Kohlweiss, and V. Zikas, “Ouroboros crypsinous: Privacy-preserving proof-of-stake,” in 2019 IEEE Symposium on Security and Privacy (SP). IEEE, 2019, pp. 157–174.
- M. Saad, Z. Qin, K. Ren, D. Nyang, and D. Mohaisen, “e-pos: Making proof-of-stake decentralized and fair,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 8, pp. 1961–1973, 2021.
- A. Poelstra et al., “Distributed consensus from proof of stake is impossible,” 2014.
- I. Bentov, R. Pass, and E. Shi, “Snow white: Provably secure proofs of stake.” IACR Cryptol. ePrint Arch., vol. 2016, no. 919, 2016.
- A. Li, X. Wei, and Z. He, “Robust proof of stake: A new consensus protocol for sustainable blockchain systems,” Sustainability, vol. 12, no. 7, p. 2824, 2020.
- Y. Zou, Z. Jin, Y. Zheng, D. Yu, and T. Lan, “Optimized consensus for blockchain in internet of things networks via reinforcement learning,” Tsinghua Science and Technology, vol. 28, no. 6, pp. 1009–1022, 2023.
- Y. Dai, D. Xu, S. Maharjan, Z. Chen, Q. He, and Y. Zhang, “Blockchain and deep reinforcement learning empowered intelligent 5g beyond,” IEEE network, vol. 33, no. 3, pp. 10–17, 2019.