Shoal++: High Throughput DAG BFT Can Be Fast! (2405.20488v1)

Abstract: Today's practical partially synchronous Byzantine Fault Tolerant (BFT) consensus protocols trade off low latency and high throughput. On the one end, traditional BFT protocols such as PBFT and its derivatives optimize for latency. They require, in fault-free executions, only 3 message exchanges to commit, the optimum for BFT consensus. However, this class of protocols typically relies on a single leader, hampering throughput scalability. On the other end, a new class of so-called DAG-BFT protocols demonstrates how to achieve highly scalable throughput by separating data dissemination from consensus, and using every replica as proposer. Unfortunately, existing DAG-BFT protocols pay a steep latency premium, requiring on average 10.5 message exchanges to commit a transactions. This work aims to soften this tension and proposes Shoal++, a novel DAG-based BFT consensus system that offers the throughput of DAGs while reducing commit latency to an average of 4.5 message exchanges. Our empirical findings are encouraging, showing that Shoal++ achieves throughput comparable to state-of-the-art DAG BFT solutions while reducing latency by up to 60%.

• The paper introduces a novel DAG-based BFT protocol that significantly reduces commit latency using faster direct commit rules.
• It employs dynamic anchor reassignment and parallel DAG instances to lower queuing latency and boost throughput.
• Empirical results show end-to-end latency reductions from 10.5md to 4.5md, indicating strong potential for decentralized applications.

Shoal++: High Throughput DAG-BFT with Reduced Latency

Shoal++ represents a significant advancement in the design of Byzantine Fault Tolerant (BFT) consensus protocols, specifically within the context of Directed Acyclic Graph (DAG)-based structures. This paper presents Shoal++, a novel DAG-based BFT consensus protocol that retains the high throughput characteristics of DAG protocols while substantially reducing latency. It achieves this by integrating innovative methods to address the inherent latency issues in prior DAG-BFT systems like Bullshark and Shoal.

Key Contributions

Shoal++ introduces three main techniques to enhance the latency performance of DAG-BFT protocols:

1. Faster Commit Rules: Shoal++ optimizes the existing Direct Commit rule by leveraging a "Fast Direct Commit Rule." This rule allows for faster commitment of anchor nodes by enabling the commit process as soon as $2f+1$ weak votes (uncertified nodes) link to an anchor, rather than waiting for $f+1$ certified nodes. This reduces the commit latency from 6 message delays (md) to 4md in the common case.
2. Increased Anchor Frequency: Shoal++ dynamically interprets and reassigns anchor nodes to ensure that almost every node can act as an anchor under robust conditions. This minimizes the need for extra synchronization and ensures quicker commitment of transactions.
3. Parallel DAG Instances: Shoal++ operates multiple DAG instances in parallel, each staggered with a small offset. This technique significantly reduces queuing latency by allowing transactions that miss inclusion in one DAG round to be quickly included in the next available round of a different DAG instance.

Latency Analysis and Improvements

In the baseline DAG-BFT protocol, such as Bullshark, the latency comprises three primary components: queuing latency (time taken for a transaction to be proposed), anchoring latency (time taken for a proposal to be referenced by an anchor), and commit latency (time taken to commit an anchor). Shoal++ systematically optimizes each of these components.

• Queuing Latency: By operating multiple DAGs in parallel, Shoal++ can propose new nodes every message delay (md), reducing the average queuing latency from 1.5md to 0.5md.
• Anchoring Latency: The dynamic re-interpretation of anchor assignments and reputation-based candidate selection minimize the time a proposal waits to be referenced by an anchor. This ensures nodes get referenced promptly, leading to an average anchoring latency improvement from 3md (in Shoal) to almost negligible.
• Commit Latency: The introduction of the Fast Direct Commit Rule means that anchors can be committed in 4md instead of the typical 6md.

Overall, the combined effect of these improvements in Shoal++ results in a reduction of average fault-free end-to-end latency from 10.5md (as seen in Shoal) to 4.5md.

Practical Implications and Future Directions

Shoal++ leverages leader reputation, dynamic anchor scheduling, and innovative queuing mechanisms, which contribute to its reduced latency and high throughput. These methods ensure that the system maintains the high throughput characteristic of DAG protocols while achieving latency figures closer to the theoretical optimum of traditional leader-based BFT protocols like PBFT.

In practical deployment, Shoal++ demonstrates significant potential for real-world BFT systems, especially in decentralized applications such as blockchain networks where both throughput and latency are critical. The empirical evaluation showcases Shoal++'s superiority in geo-distributed settings, maintaining low latency and high throughput even under varying network conditions and failure scenarios.

Conclusion

Shoal++ presents a compelling advancement in the evolution of DAG-based BFT consensus protocols. By addressing latency through a combination of faster commit rules, dynamic anchor scheduling, and parallel DAG instances, Shoal++ bridges the gap between high throughput and low latency, a challenge that has historically divided DAG-BFT and traditional BFT protocols. This makes Shoal++ a valuable contribution to both the theoretical landscape and practical deployment of BFT systems.

The techniques introduced in Shoal++ also hold potential for improving queuing latency in broader contexts, suggesting that the principles applied here could be adapted to other BFT protocols aiming to optimize for latency without compromising throughput. This positions Shoal++ as a promising candidate for future developments in fault-tolerant distributed consensus systems.