Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus (2105.11827v4)

Abstract: We propose separating the task of reliable transaction dissemination from transaction ordering, to enable high-performance Byzantine fault-tolerant quorum-based consensus. We design and evaluate a mempool protocol, Narwhal, specializing in high-throughput reliable dissemination and storage of causal histories of transactions. Narwhal tolerates an asynchronous network and maintains high performance despite failures. Narwhal is designed to easily scale-out using multiple workers at each validator, and we demonstrate that there is no foreseeable limit to the throughput we can achieve. Composing Narwhal with a partially synchronous consensus protocol (Narwhal-HotStuff) yields significantly better throughput even in the presence of faults or intermittent loss of liveness due to asynchrony. However, loss of liveness can result in higher latency. To achieve overall good performance when faults occur we design Tusk, a zero-message overhead asynchronous consensus protocol, to work with Narwhal. We demonstrate its high performance under a variety of configurations and faults. As a summary of results, on a WAN, Narwhal-Hotstuff achieves over 130,000 tx/sec at less than 2-sec latency compared with 1,800 tx/sec at 1-sec latency for Hotstuff. Additional workers increase throughput linearly to 600,000 tx/sec without any latency increase. Tusk achieves 160,000 tx/sec with about 3 seconds latency. Under faults, both protocols maintain high throughput, but Narwhal-HotStuff suffers from increased latency.

• The paper introduces a decoupling of transaction dissemination from ordering, significantly enhancing BFT consensus performance.
• Narwhal uses a DAG-based mempool to achieve scalable, reliable transaction sharing, scaling throughput to over 130,000 tx/sec.
• Tusk employs an asynchronous consensus mechanism that delivers high throughput and low latency, paving the way for advanced blockchain systems.

Overview of "Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus"

This paper presents an innovative approach to improving Byzantine fault-tolerant (BFT) consensus performance by decoupling transaction dissemination from ordering. The researchers introduce the Narwhal protocol, a Directed Acyclic Graph (DAG)-based mempool design for efficient transaction sharing, and Tusk, a complementary asynchronous consensus protocol.

Key Contributions

1. Mempool-Consensus Decoupling: The authors propose separating transaction dissemination from consensus ordering. The Narwhal protocol specializes in the high-throughput dissemination of transactions, maintaining performance even during network asynchrony or failures.
2. Narwhal Design and Performance: Narwhal achieves scalable and reliable transaction dissemination via a DAG structure, tolerating network asynchrony. The architecture allows validators to use extra workers, enhancing throughput linearly without increasing latency.
3. Narwhal-HotStuff Integration: When combined with a partially synchronous consensus protocol like HotStuff, Narwhal improves throughput from 1,800 tx/sec to over 130,000 tx/sec while maintaining low latency. Under stress, Narwhal sustains throughput even if latency increases.
4. Tusk - Asynchronous Consensus: Tusk, leveraging Narwhal's structure, provides high performance without extra message overhead. It utilizes a random coin mechanism for asynchronous consensus, achieving 160,000 tx/sec with about 3 seconds latency.

Strong Numerical Results

• Scalability: Narwhal-HotStuff demonstrates over 130,000 tx/sec with less than 2-sec latency.
• Worker Scaling: Additional workers boost throughput linearly up to 600,000 tx/sec.
• Tusk Outcomes: Consistently high throughput of 160,000 tx/sec with about 3 seconds latency, unaffected by faults.

Implications and Future Directions

The decoupling of consensus tasks opens avenues for designing scalable and robust blockchain systems. By separating storage and dissemination of transaction data, Narwhal effectively utilizes available resources, paving the way for scalable distributed ledger technologies.

The Narwhal and Tusk framework suggests potential for future developments in asynchronous BFT consensus mechanisms. As distributed systems increasingly demand scalability without compromising reliability, this work can influence architectural decisions in blockchain technologies and distributed databases.

Looking ahead, further exploration into optimizing the underlying DAG structure and validation mechanics can help refine throughput and latency benefits. Additionally, examining interactions between Narwhal and other consensus protocols could broaden its applicability across diverse distributed system architectures.

In summary, the paper provides a comprehensive solution to current limitations in BFT consensus protocols by rethinking transaction dissemination, offering a foundation for next-generation blockchain systems capable of handling millions of transactions per second.