Making Democracy Work: Fixing and Simplifying Egalitarian Paxos (Extended Version) (2511.02743v1)

Abstract: Classical state-machine replication protocols, such as Paxos, rely on a distinguished leader process to order commands. Unfortunately, this approach makes the leader a single point of failure and increases the latency for clients that are not co-located with it. As a response to these drawbacks, Egalitarian Paxos introduced an alternative, leaderless approach, that allows replicas to order commands collaboratively. Not relying on a single leader allows the protocol to maintain non-zero throughput with up to $f$ crashes of any processes out of a total of $n = 2f+1$. The protocol furthermore allows any process to execute a command $c$ fast, in $2$ message delays, provided no more than $e = \lceil\frac{f+1}{2}\rceil$ other processes fail, and all concurrently submitted commands commute with $c$; the latter condition is often satisfied in practical systems. Egalitarian Paxos has served as a foundation for many other replication protocols. But unfortunately, the protocol is very complex, ambiguously specified and suffers from nontrivial bugs. In this paper, we present EPaxos* -- a simpler and correct variant of Egalitarian Paxos. Our key technical contribution is a simpler failure-recovery algorithm, which we have rigorously proved correct. Our protocol also generalizes Egalitarian Paxos to cover the whole spectrum of failure thresholds $f$ and $e$ such that $n \ge \max{2e+f-1, 2f+1}$ -- the number of processes that we show to be optimal.

• The paper introduces a simplified, fully specified variant of EPaxos that resolves ambiguity and safety issues in leaderless state-machine replication.
• The paper rigorously proves safety and liveness using invariant-based methods and matches the known lower bounds for fast consensus.
• The paper presents a novel recovery mechanism with a validation phase that optimizes fast and slow paths for improved geo-distributed performance.

Fixing and Simplifying Egalitarian Paxos: A Rigorous Approach to Leaderless State-Machine Replication

Introduction and Motivation

The paper addresses the complexity, ambiguity, and correctness issues in the original Egalitarian Paxos (EPaxos) protocol, a leaderless state-machine replication (SMR) protocol designed to overcome the single-leader bottleneck of classic protocols like Paxos and Raft. EPaxos enables any replica to propose and order commands, allowing for lower-latency execution and improved availability, especially in geo-distributed deployments. However, the original EPaxos protocol suffers from intricate and underspecified recovery logic, subtle safety bugs, and a lack of rigorous correctness proofs, particularly for its non-thrifty variant and optimal fast-path quorum sizes.

The authors present a new protocol, also called EPaxos, that retains the democratic, leaderless nature of the original but provides a simpler, fully specified, and provably correct design. The new protocol generalizes the fast-path/slow-path tradeoff, matches known lower bounds for fast consensus, and provides a rigorous proof of safety and liveness.

System Model and Problem Statement

The system consists of $n$ processes, up to $f$ of which may crash, communicating over reliable links in a partially synchronous network. The goal is to implement linearizable SMR, where each process maintains a replica of a deterministic state machine and executes client commands in a way that ensures consistency, integrity, validity, and liveness.

A key property of modern SMR protocols is the ability to execute conflict-free commands quickly (in two message delays) under favorable conditions (few failures, no conflicting concurrent commands). The protocol is parameterized by $e$ (the number of failures tolerated on the fast path) and $f$ (the total number of failures tolerated for liveness). The paper establishes that any $f$-resilient, $e$-fast SMR protocol requires $n \geq \max\{2e+f-1, 2f+1\}$ processes, matching the lower bound for fast consensus.

Protocol Overview

Dependency Graph Abstraction

EPaxos achieves SMR by having processes agree on a dependency graph, where vertices are commands and edges represent ordering constraints due to conflicts. Each process maintains local arrays mapping command identifiers to payloads, dependency sets, and phases (pre-accepted, accepted, committed). Commands are executed in an order consistent with the dependency graph, breaking cycles deterministically.

Commit Protocol

When a client submits a command, the initial coordinator assigns it a unique identifier and computes its initial dependencies (conflicting commands known locally). The coordinator broadcasts a PreAccept message to all processes. Recipients may augment the dependency set with additional conflicts and reply with PreAcceptOK. If the coordinator receives $n-e$ matching responses (a fast quorum), and all dependencies agree, it commits the command immediately (fast path). Otherwise, it enters a slow path, collecting $n-f$ AcceptOK responses before committing.

The protocol ensures that for any two conflicting commands committed at ballot 0, at least one is in the dependency set of the other, preserving consistency.

Execution Protocol

A background task at each process executes committed commands in batches, respecting the dependency graph. Strongly connected components are executed in topological order, with cycles broken by command identifiers. This ensures that all processes execute conflicting commands in the same order.

Recovery Protocol

If a coordinator fails, another process can take over by initiating a recovery protocol. The new coordinator selects a higher ballot and collects state from a quorum. If a slow-path decision was previously made, it resumes from the accepted value. If a fast-path decision is suspected, a novel validation phase is used: the coordinator proposes the candidate dependencies to the quorum, which checks for invalidating or potentially invalidating commands (i.e., conflicting commands that could break consistency if the fast path is taken). If any are found, the recovery is aborted (the command is replaced with a no-op); otherwise, the command is accepted.

The validation phase is a key technical contribution, providing a simple and correct mechanism for safe recovery, in contrast to the original EPaxos's complex and error-prone tentative pre-accept phase.

Optimized Protocol and Lower Bound Matching

The protocol is further optimized to match the lower bound $n \geq \max\{2e+f-1, 2f+1\}$ by refining the recovery logic. In particular, the coordinator aborts recovery if the initial coordinator is in the recovery quorum or if certain intersection properties do not hold, ensuring that the fast path could not have been taken. This allows, for example, $e = \lceil (f+1)/2 \rceil$ for $n = 2f+1$, as originally intended by EPaxos.

Thrifty Variant

A thrifty variant is described, where the coordinator selects a fixed fast quorum a priori and communicates only with it. This allows the fast path even if the rest of the system disagrees, but sacrifices fast-path availability in the presence of failures within the selected quorum.

Correctness and Rigorous Proofs

The paper provides a detailed, invariant-based proof of safety (agreement and visibility invariants) and liveness. The proof covers all protocol variants (baseline, optimized, thrifty) and addresses subtle cases in recovery, including deadlock avoidance and the handling of cycles in dependency waiting. The protocol is shown to be the first to match the optimal fast-path/slow-path tradeoff for SMR.

Comparison with Original EPaxos

The original EPaxos protocol is shown to be ambiguous, incomplete, and buggy, especially in its use of ballot variables and recovery logic. The new protocol avoids these pitfalls by separating the validation phase from dependency commitment, using two ballot variables (as in classic Paxos), and providing a complete specification for both thrifty and non-thrifty variants. The new recovery protocol is both simpler and more robust, avoiding deadlocks and ensuring progress.

Practical and Theoretical Implications

Practically, the new EPaxos protocol provides a drop-in replacement for systems using the original EPaxos, such as distributed databases and transaction systems, with improved correctness and optimal fast-path performance. The protocol is particularly well-suited for geo-distributed deployments with small replica sets, where fast-path availability is critical.

Theoretically, the protocol clarifies the tradeoffs between fast-path and slow-path fault tolerance, matches known lower bounds, and provides a rigorous foundation for future work on leaderless consensus and SMR. The validation phase technique may be applicable to other consensus protocols with fast paths.

Future Directions

Potential future work includes extending the protocol to Byzantine fault tolerance, integrating with reconfiguration and dynamic membership, and further optimizing for wide-area deployments. The validation-based recovery approach may inspire new designs in other leaderless or multi-leader consensus protocols.

Conclusion

The paper presents a new, rigorously specified, and provably correct variant of Egalitarian Paxos, addressing longstanding issues in the original protocol. The new EPaxos matches optimal lower bounds for fast-path consensus, provides a simple and robust recovery mechanism, and is suitable for practical deployment in fault-tolerant, geo-distributed systems. This work establishes a new standard for leaderless SMR protocols, both in terms of theoretical optimality and practical implementability.