---
title: 'ALock: Asymmetric Hierarchical Locks'
url: https://www.emergentmind.com/topics/asymmetric-hierarchical-locks-alock
type: topic
---

# ALock: Asymmetric Hierarchical Locks

Asymmetric Hierarchical Locks (ALock) are a locking primitive specifically designed for RDMA-based (Remote Direct Memory Access) systems, enabling efficient synchronization between local and remote threads accessing shared memory regions. ALock achieves mutual exclusion and starvation-freedom while eliminating RDMA loopback and remote procedure call (RPC) overheads, outperforming traditional RDMA locks in high-locality and mixed-locality workloads. The protocol leverages a two-cohort hierarchy—local and remote—embedded MCS queue locks, and a global Peterson-style arbitration to provide both performance and correctness guarantees [2404.17980].

## 1. Formal Specification

An ALock instance maintains state over two cohorts, local ($L$) and remote ($R$), each managed by an MCS queue. The protocol state is
\[
\mathsf{ALock} = (\tail_r, \; \tail_\ell, \; \victim, \; \{\mathit{desc}_p\}_{p\in P}),
\]
where $\tail_r$ and $\tail_\ell$ denote the tails of the remote and local cohorts (with zero indicating no waiters), $\victim \in \{\mathrm{R},\mathrm{L}\}$ is the Peterson-style global arbitrator, and $\mathit{desc}_p = (\mathit{next}_p, \mathit{budget}_p)$ is the descriptor for each process $p$. Each process $p$ tracks a Boolean $\mathit{passed}_p$ to indicate if it acquired the lock by passing through the cohort or by reacquire.

The locking procedure operates as follows:
- Each operation is dispatched to the respective cohort queue (local or remote) via the $\mathsf{qLock}_c$ function.
- Cohorts arbitrate global lock ownership through a Peterson algorithm: $\mathsf{PetersonWait}(\mathit{cohort}, \victim)$, guaranteeing exclusive progress for one cohort.
- State transitions $\mathsf{qLock}_c$, $\mathsf{qUnLock}_c$, and $\mathsf{pReacquire}$ are responsible for MCS-style acquisition, release, and Peterson-level reacquisition, respectively.

Safety (mutual exclusion) is specified as
\[
\forall p \neq q: \quad \neg(\pc[p]=\mathsf{cs} \wedge \pc[q]=\mathsf{cs}),
\]
and liveness (starvation-freedom) is guaranteed by the budgeted reacquire logic and fair atomic primitives, formally:
\[
(\pc[p]=\mathsf{enter}) \leadsto (\pc[p]=\mathsf{cs}).
\]

## 2. Locking and Unlocking Procedures

The ALock protocol's pseudocode operates in three key modules: the high-level lock/unlock entrypoints, the modified per-cohort MCS queues, and the global Peterson arbitration invoked on budget expiration or missed fast paths.

**Lock Acquisition:**
```latex
\Function{Lock}{lockPtr}
  \State \mathit{isLocal}\gets (current node == lockPtr.nodeID)
  \If{\mathsf{qLock}_{\,\mathit{isLocal}(\mathit{lockPtr})} = \mathsf{False}}
    \State \mathsf{pReacquire}(\mathit{lockPtr})
  \EndIf
\EndFunction
```

**Unlock:**
```latex
\Function{Unlock}{lockPtr}
  \Return \mathsf{qUnLock}_{\,\mathit{isLocal}(\mathit{lockPtr})}
\EndFunction
```

**Per-cohort MCS Lock Acquisition (`qLock_c`) and Release (`qUnLock_c`):** Both operations use local or RDMA-native (remote) CAS and write primitives, depending on cohort type. Fast-path shared-memory passes are enabled for 100% locality cases, with only rare global Peterson reacquire required ($\mathcal{O}(1/B)$ rate).

**Peterson Reacquire:** Upon budget expiry or initial acquisition, $\mathsf{pReacquire}$ synchronizes access between the cohorts, ensuring global fairness.

## 3. Theoretical Performance Analysis

Define $T_{\ell}$ as local atomic primitive latency, $T_{r}$ as RDMA atomic latency, and $T_{spin}$ as the spin-wait cost (negligible).

**Worst-case hold-time:**
\[
T_{\mathrm{hold}}^{(\mathrm{WC})} = 3\,T_{r} + T_{\ell}
\]
This path includes one remote cohort-acquire, one remote cohort-release, plus Peterson reacquire on budget expiration.

**Average-case hold-time:**
\[
T_{\mathrm{hold}}^{(\mathrm{avg})} \approx T_{\ell}
\]
Most requests are local fast-path passes, incurring only local memory operations. Only an $\mathcal{O}(1/B)$ fraction require re-entry to the global Peterson layer, incurring additional $2T_r$ overhead.

Empirical latencies:
\[
T_r \approx 1.2\text{–}2.5\,\mu\mathrm{s}, \qquad T_\ell \approx 50\text{–}100\,\mathrm{ns}
\]
Thus, in locality-dominated conditions, the hierarchical split yields up to $O(10\times)$ speedup over pure RDMA locks.

## 4. Experimental Evaluation

ALock was evaluated on a distributed lock table deployed across 5–20 CloudLab nodes (Intel Xeon E5, Mellanox CX-3). The following table summarizes throughput (lock+unlock ops/sec) and 95th percentile tail-latency for ALock versus MCS-RDMA and RDMA spinlock competitors under various workloads:

| Workload                                  | ALock       | MCS-RDMA    | Spinlock-RDMA | Relative Speedup (ALock/MSCS) |
|-------------------------------------------|-------------|-------------|---------------|-------------------------------|
| 100% locality, 20 locks, 240 threads (high contention) | 1.2M ops/s  | 50k ops/s    | 55k ops/s     | 24×                          |
| 95% locality, 100 locks, 80 threads (medium contention) | 880k ops/s  | 340k ops/s   | 290k ops/s    | 2.6×                         |
| 1000 locks, low contention                | 1.5M ops/s  | 900k ops/s  | 450k ops/s    | 1.7×–3.3×                     |
| Tail-latency (95%), 100% locality         | 120 ns      | 2.1 μs      | 3.9 μs        | —                             |

With increasing locality, ALock's throughput and latency approach those of pure shared-memory locks. As contention or remote operation fractions increase, ALock demonstrates up to 29× throughput and 20× latency improvements under certain settings.

## 5. Protocol Advantages

- **Elimination of RDMA loopback:** For purely local acquisitions, all operations use shared-memory CAS rather than going through the RDMA NIC, yielding minimum latency ($\approx$50 ns).
- **Starvation-freedom and fairness:** Budgeted Peterson reacquire ensures all processes make progress under high contention.
- **Local spinning and precise hand-off:** Favouring per-cohort MCS queues avoids RNIC congestion and supports strict FIFO fairness.
- **Connection reduction:** For local nodes, the protocol reduces the number of required RDMA connections by $1/n$, mitigating QP thrashing.
- **Formal verification:** A TLA+–verified model and open-source implementation are provided, offering strong correctness guarantees.

## 6. Limitations and Integration Considerations

- **Budget tuning:** Cohort budgets $(B_\ell, B_r)$ must be tuned to match workload characteristics; defaults ($5,20$) were empirically validated.
- **Budget expiration overhead:** Occasional global reacquire incurs $O(T_r)$ cost.
- **Descriptor management:** The protocol incurs the overhead of per-thread descriptor allocation and pointer-chasing.
- **Two-cohort limitation:** The present design manages only two cohorts; generalizing beyond local/remote access patterns requires extension.
- **Memory alignment and RDMA constraints:** Metadata (two 8-byte tails, one 4-byte victim, padding) should be 64-byte aligned; pointer encodings must encode home-node for locality. Fencing and ordering for atomic operations must respect RDMA memory model semantics.
- **NIC resource consideration:** The protocol is practical on standard ConnectX-3/4 NICs but requires caution to prevent QP cache thrashing in very large deployments.

## 7. Relation to RDMA Synchronization Landscape

ALock bridges the architectural gap between local-only mutual exclusion and global, RDMA-driven locks. The protocol eliminates slow loopback or RPC fallbacks and provides substantial throughput (up to 29×) and latency (<1/20th) reductions in locality-heavy high-concurrency environments [2404.17980]. Its hierarchical split, combining local fast-paths and global fairness mechanisms, addresses both scalability and performance symmetry, marking a significant step in the evolution of synchronization primitives for high-performance distributed systems.

Source: https://www.emergentmind.com/topics/asymmetric-hierarchical-locks-alock