FastFabric: Scaling Hyperledger Fabric to 20,000 Transactions per Second (1901.00910v2)

Abstract: Blockchain technologies are expected to make a significant impact on a variety of industries. However, one issue holding them back is their limited transaction throughput, especially compared to established solutions such as distributed database systems. In this paper, we re-architect a modern permissioned blockchain system, Hyperledger Fabric, to increase transaction throughput from 3,000 to 20,000 transactions per second. We focus on performance bottlenecks beyond the consensus mechanism, and we propose architectural changes that reduce computation and I/O overhead during transaction ordering and validation to greatly improve throughput. Notably, our optimizations are fully plug-and-play and do not require any interface changes to Hyperledger Fabric.

• The paper presents architectural optimizations that scale Hyperledger Fabric to nearly 20,000 transactions per second, a sevenfold improvement.
• Key methods include separating transaction metadata from data for ordering, leveraging parallelism and caching for validation, and using lightweight data structures.
• These enhancements enable Hyperledger Fabric to support high-demand applications requiring rapid transaction rates, without altering existing system interfaces.

An Examination of "FastFabric: Scaling Hyperledger Fabric to 20,000 Transactions per Second"

This paper proposes a series of architectural optimizations to enhance the throughput of Hyperledger Fabric, a widely recognized permissioned blockchain, increasing its transaction rate from 3,000 to 20,000 transactions per second (TPS). The authors focus on overcoming performance bottlenecks beyond the consensus mechanism, leveraging common system design techniques.

Key Contributions

The authors present four primary optimizations:

1. Separation of Metadata from Data: The consensus layer in Hyperledger Fabric traditionally requires entire transaction data for ordering. The authors propose a redesign where only transaction IDs are used for ordering purposes, leading to reduced I/O overhead and enhanced throughput.
2. Parallelism and Caching: Certain aspects of transaction validation are parallelizable; others can benefit from caching. By aggressively caching unmarshaled data and parallelizing validation steps such as endorsement policy validation, the authors significantly boost throughput.
3. Lightweight Data Structures for Fast Data Access: By replacing the traditional key-value store with a lightweight, in-memory hash table, fast access to transaction-related data on the critical path is achieved. This structural change is further enhanced by utilizing a write-optimized storage cluster for block data.
4. Resource Separation: By decoupling the roles of committers and endorsers across separate hardware, resource contention is minimized, allowing the system to sustain high throughput.

These enhancements are implemented without disrupting existing Hyperledger Fabric interfaces, maintaining plug-and-play compatibility.

Performance and Evaluation

Conducting empirical evaluations, the authors report a successful execution of almost 20,000 TPS, which represents a near sevenfold improvement over the existing systems. This substantial increase in throughput is primarily attributed to the decoupling of transaction IDs and payloads, aggressive caching strategies, and the parallelized architecture.

Implications and Future Directions

The modifications proposed in this paper offer significant implications for scalability in permissioned blockchain systems. By addressing non-consensus related bottlenecks, the authors highlight the path to achieving transaction rates compatible with high-demand applications, like those seen in financial transactions systems akin to Visa, which typically require tens of thousands of TPS.

Future research might explore the application of these optimizations in varied blockchain environments, emphasizing broader compatibility and further performance gains through advanced consensus algorithms. By maintaining a separation between operational and analytic layers, there's also potential to integrate distributed processing frameworks like Apache Spark to handle dynamic, large-scale blockchain workloads.

Conclusion

This paper successfully illustrates how a permissioned blockchain framework like Hyperledger Fabric can undergo a series of architectural redesigns to achieve high scalability, essential for broader industrial adoption. The optimizations explored underscore the importance of performance engineering in blockchain technologies, addressing the critical concerns around transaction throughput and system efficiency.