BLOCKBENCH: A Framework for Analyzing Private Blockchains (1703.04057v1)

Abstract: Blockchain technologies are taking the world by storm. Public blockchains, such as Bitcoin and Ethereum, enable secure peer-to-peer applications like crypto-currency or smart contracts. Their security and performance are well studied. This paper concerns recent private blockchain systems designed with stronger security (trust) assumption and performance requirement. These systems target and aim to disrupt applications which have so far been implemented on top of database systems, for example banking, finance applications. Multiple platforms for private blockchains are being actively developed and fine tuned. However, there is a clear lack of a systematic framework with which different systems can be analyzed and compared against each other. Such a framework can be used to assess blockchains' viability as another distributed data processing platform, while helping developers to identify bottlenecks and accordingly improve their platforms. In this paper, we first describe BlockBench, the first evaluation framework for analyzing private blockchains. It serves as a fair means of comparison for different platforms and enables deeper understanding of different system design choices. Any private blockchain can be integrated to BlockBench via simple APIs and benchmarked against workloads that are based on real and synthetic smart contracts. BlockBench measures overall and component-wise performance in terms of throughput, latency, scalability and fault-tolerance. Next, we use BlockBench to conduct comprehensive evaluation of three major private blockchains: Ethereum, Parity and Hyperledger Fabric. The results demonstrate that these systems are still far from displacing current database systems in traditional data processing workloads. Furthermore, there are gaps in performance among the three systems which are attributed to the design choices at different layers of the software stack.

• The paper introduces BLOCKBENCH as a framework that benchmarks private blockchain systems by measuring throughput, latency, scalability, and fault tolerance.
• It conducts a comparative evaluation of Ethereum, Parity, and Hyperledger Fabric, revealing performance differences linked to design choices.
• The study identifies critical bottlenecks and highlights the need for optimized consensus protocols and improved blockchain architectures.

Analyzing Private Blockchains with BLOCKBENCH

The paper "BLOCKBENCH: A Framework for Analyzing Private Blockchains" presents an innovative benchmarking framework designed specifically for private blockchain systems. Given the broader adoption and enthusiasm for blockchain technologies, particularly private blockchains, this framework addresses a critical gap in assessing and comparing various blockchain platforms.

Key Contributions and Insights

This paper explores several core contributions:

1. Introducing BLOCKBENCH:
   • It introduces BLOCKBENCH, a comprehensive evaluation framework tailored for private blockchains. BLOCKBENCH provides a standardized method to compare different blockchain platforms and facilitates a deeper understanding of design choices across these systems. It measures component-wise performance in terms of throughput, latency, scalability, and fault tolerance.
2. Comprehensive Evaluation:
   • Using BLOCKBENCH, the paper evaluates three prevalent private blockchain platforms: Ethereum, Parity, and Hyperledger Fabric. The findings highlight significant performance discrepancies among these systems attributable to design differences at various layers of the blockchain stack.
3. Metrics and Workloads:
   • BLOCKBENCH implements both macro benchmarks, which test application-layer performance, and micro benchmarks, geared towards evaluating the performance at individual layers such as consensus, data model, and execution.

Detailed Analysis of Key Systems

The paper presents an empirical analysis of three private blockchain systems—Ethereum, Parity, and Hyperledger Fabric. Each system is scrutinized using BLOCKBENCH's suite of benchmarks, offering a nuanced understanding of each platform's strengths and limitations.

1. Throughput and Latency:
   • Hyperledger Fabric exhibits superior throughput across benchmarks but fails to scale beyond 16 nodes.
   • Ethereum shows consistent performance degradation with increasing nodes due to its PoW consensus mechanism.
   • Parity maintains lower latency due to its PoA protocol but suffers from a bottleneck in transaction signing.
2. Scalability:
   • The scalability analysis reveals Hyperledger Fabric's performance sharply declines beyond 16 nodes due to issues in its PBFT consensus implementation.
   • Ethereum and Parity manage more consistent scaling, though they also show performance degradation with larger network sizes.
3. Fault Tolerance and Security:
   • The systems display varying resilience to node failures, with Hyperledger Fabric's performance significantly affected under failure conditions unlike Ethereum and Parity.
   • The security analysis through simulated partition attacks reveals that Ethereum and Parity are vulnerable to blockchain forks, an issue not present in Hyperledger Fabric due to its PBFT consensus.
4. Execution Layer Performance:
   • Ethereum's EVM incurs significant overhead, particularly in memory usage, compared to Hyperledger Fabric's more optimized execution environment.

Implications and Future Directions

The research highlights several practical and theoretical implications:

1. Performance Bottlenecks:
   • Identifying bottlenecks in consensus protocols, data models, and execution layers provides benchmarks for future blockchain development. It emphasizes the need for optimized consensus protocols and efficient execution frameworks.
2. Database Design Principles:
   • The insights suggest integrating design concepts from database systems into blockchain architectures. This includes efficient storage management, leveraging hardware primitives, sharding, and the use of declarative smart contract languages for optimized contract execution.
3. Blockchain Usability:
   • The current state of blockchain systems, as highlighted, necessitates further development before they can disrupt existing database systems. Enhancements in codebase maturity, usability, and application diversity are paramount.

Conclusion

BLOCKBENCH serves as a critical framework for evaluating private blockchain systems, shedding light on the intricate trade-offs and design choices that impact performance. The comprehensive analysis presented in this paper underscores current limitations while charting pathways for future improvements in blockchain architectures. Researchers and developers can leverage these findings to optimize and innovate blockchain platforms, thereby enhancing their viability for large-scale data processing and other enterprise applications.