Finding Nemo-Nemo: CFT DAG-based Consensus in the WAN

Abstract: This paper introduces Nemo-Nemo, a practical crash-fault tolerant (CFT) consensus protocol designed to outperform existing protocols in wide-area networks by bridging design principles from the CFT and Byzantine-fault tolerant (BFT) worlds. By structuring command propagation through a causally ordered DAG, Nemo-Nemo allows all consensus replicas to propose commands with a naturally self-regulating communication regime. By exploiting multi-leader architecture, Nemo-Nemo avoids the performance bottleneck inherent to single-leader protocols. By separating command dissemination from consensus logic, Nemo-Nemo handles challenging network conditions even when consensus commits are stalled. Moreover, leader proposals that miss a deadline are never dropped, but deterministically deferred and executed later, preserving throughput under transient network delays. And by enabling Nemo-Nemo to commit on a DAG in just two network hops, it matches the latency of existing CFT systems, while achieving significantly higher throughput. The result is a robust, deployable system: the first DAG-based CFT consensus protocol proven to exceed state-of-the-art wide-area network performance in both speed and resilience.

• The paper introduces Nemo-Nemo, which adapts DAG-based consensus techniques from BFT to the CFT setting to reduce latency and protocol complexity in WANs.
• It decouples data dissemination from consensus logic with multi-leader scheduling, enabling full parallelism and rapid adaptation under crash failures.
• Experimental evaluations demonstrate up to 800k commands/sec at 600ms median latency, outperforming Multi-Paxos and other legacy protocols.

Nemo-Nemo: DAG-Based CFT Consensus for Wide-Area Networks

Motivation and Theoretical Foundations

Nemo-Nemo addresses a significant performance gap between CFT and BFT consensus: while BFT protocols have achieved high throughput and robustness via DAG-based architectures, CFT protocols have not extensively explored these architectural advances. Nemo-Nemo transfers and adapts foundational concepts from contemporary BFT DAG-based protocols into the CFT setting, leveraging crash-fault assumptions to further reduce protocol complexity and latency. This adaptation enables a new regime of performance and scalability for consensus in wide-area networks (WANs).

Key theoretical pillars are:

• Exploiting a causally ordered, DAG-based mempool for parallel, failure-tolerant command propagation.
• Decoupling the data dissemination layer from the consensus logic, allowing the DAG to grow independently of consensus liveness and commit progress.
• Utilizing multi-leader scheduling, eliminating the single-leader bottleneck and supporting rapid leader rotation without explicit view changes.
• Avoiding cryptographic authentication and block certification, possible in CFT, yielding minimal round complexity and message delay.

Nemo-Nemo Protocol Design

Nemo-Nemo overlays two tightly integrated layers:

1. Data Dissemination Layer: Replicas propose blocks in a causally ordered DAG, referencing at least $f+1$ prior blocks. Blocks are disseminated without certificates or signatures, as equivocation is excluded under crash-faults. Skeleton slots—deterministically allocated per round—define the candidates for direct commitment.
2. Consensus Decision Layer: Consensus commits derive directly from the DAG’s evolving topology. The protocol supports both direct (voting-based) and indirect (causal anchoring) decision rules for slot commitment.

All replicas propose blocks concurrently. Commitment of a block in round $R$ is achieved if $f+1$ blocks in round $R+1$ reference it (direct rule). When this is not achieved, future committed blocks may induce indirect commitment via the unique causal structure. Non-committed proposals are never dropped, only deferred, contrasting sharply with traditional CFT timeouts and view changes, thereby preserving throughput and fairness under asynchrony or leader failure.

Figure 1: Example execution with three replicas and two skeleton slots per round. Skeleton blocks are classified as commit (green), skip (red), or undecided (grey). Thick arrows signify commits via the indirect decision rule.

This structure, exemplified in Figure 1, allows seamless, highly parallel consensus advancement in adverse network conditions.

WAN Performance and Scalability

Under WAN deployment, Nemo-Nemo outperforms classic and state-of-the-art CFT protocols in both throughput and commit latency. Multi-leader DAG topology not only eliminates leader-centric bottlenecks but also supports rapid adaptation to network delays or regional failures.

Figure 2: Performance under normal case WAN execution.

The evaluation in Figure 2 demonstrates Nemo-Nemo’s saturation throughput of 800k commands/sec at 600 ms median latency with 5 replicas—a 2x throughput improvement over Multi-Paxos (400k cmd/sec at 750 ms). EPaxos achieves lower median latency at low load due to partial ordering but reaches only half the throughput at saturation. Nemo-Nemo’s total ordering is thus achieved at marginal coordination cost, with substantially higher throughput due to full parallelization.

Nemo-Nemo’s lack of a single resource bottleneck (commonly the leader in classical protocols) is evident in replication scaling. As the number of replicas increases, throughput increases proportionally, in contrast to increasing load on the leader in Paxos-style protocols.

Figure 3: Scalability with replication factor. Saturation throughput under 500 ms median latency.

Similarly, throughput remains robust with increasing command size, unlike in leader-based protocols where leader’s network bandwidth becomes the bottleneck.

Figure 4: Scalability with command size. Saturation throughput under 500 ms median latency and 5 replicas.

Fault Tolerance and Resource Efficiency

Nemo-Nemo maintains liveness and high throughput during crash failures, in contrast to protocols requiring explicit view changes. When a leader (or any arbitrary replica, for multi-leader systems) crashes, Nemo-Nemo experiences no downtime or system-wide stalls—a direct benefit of its DAG-centric, leaderless commit path and deferral-based proposal management.

Figure 5: Throughput under crash failures with 5 replicas. At 25 seconds, the leader is crashed in Multi-Paxos and QuePaxa, and a random replica in Nemo-Nemo and EPaxos.

Resource utilization is substantially lower in Nemo-Nemo. Median CPU utilization remains near idle well into high throughput regimes, as low message complexity and the absence of cryptographic validation minimize per-replica overhead (see Figure 6).

Figure 6: Average CPU utilization across all 5 replicas.

Robustness Under Random Asynchrony

WAN environments are defined by unpredictable delays and partition shifts. Nemo-Nemo’s decoupling of DAG growth and consensus provides performance stability even under random asynchrony, where classic majority-based CFT protocols degrade substantially as the leader’s quorum members change unpredictably.

Figure 7: Performance under normal conditions and random asynchrony with 5 replicas.

Nemo-Nemo’s throughput is unaffected by moving the system to a randomized asynchronous message schedule, outperforming QuePaxa and Multi-Paxos by 2-3x under the same conditions. This is a key demonstration that the DAG-centric architecture is a superior fit for adversarial or fluctuating WANs.

Comparison to BFT DAG Protocols

Nemo-Nemo’s CFT-optimized variant of uncertified DAG consensus eliminates both explicit block certification and cryptographic signatures, which are required by BFT protocols such as Bullshark and Mysticeti. This architectural reduction yields demonstrably lower commit latency, with equal or superior throughput even at 10 replicas with medium-sized commands.

Figure 8: Comparison against state-of-the-art BFT DAG-based protocols with 10 replicas and 32 B commands.

Specifically, Nemo-Nemo commits in two message delays (fewer than Bullshark’s six and Mysticeti’s three), and avoids all cryptographic cost.

Implications and Future Research Directions

Nemo-Nemo establishes that CFT consensus can, and should, leverage the advanced dissemination and ordering techniques pioneered in modern BFT DAG protocols—directly benefiting from their parallelism and robustness, but with reduced protocol overhead due to the weaker threat model. Practically, Nemo-Nemo supports high-throughput SMR deployments in WANs, cloud-scale key-value storage, and blockchains with strong ordering requirements.

On the theoretical side, Nemo-Nemo reframes the design space for CFT consensus, suggesting that view-change protocols, leader-based bottlenecks, and high-overhead certification can all be made obsolete under the right model. The random asynchronous model used for benchmarking further motivates future work in consensus system evaluation, highlighting the need for robust performance not only under partial synchrony but across realistic network conditions.

Further research directions include integrating partitioning and sharding over the DAG, designing adaptive quorum selection mechanisms, combining with erasure coding or locality optimizations, and rigorously formalizing performance guarantees under the random asynchronous model.

Conclusion

Nemo-Nemo is the first protocol to fully exploit DAG-based consensus architecture for CFT in WAN environments, achieving higher throughput, lower latency, robustness to failures and asynchrony, and reduced resource utilization relative to traditional approaches. Its design philosophy and robust empirical results open new avenues for high-performance, scalable consensus in the crash-fault world.