Transactional WaveCache: Speculative Memory Ordering

• Transactional WaveCache is a memory ordering mechanism that treats each Wave as a nested transaction enabling aggressive out-of-order memory operation execution.
• It leverages metadata structures like the WaveContextTable, MemOp-History, and execution numbers to perform eager conflict detection and precise rollback.
• Performance evaluations show significant speedups in low-hazard scenarios and highlight bandwidth contention challenges that guide future adaptive improvements.

The Transactional WaveCache (TWC) is a memory ordering mechanism for the WaveScalar DataFlow architecture, designed to enable speculative, out-of-order execution of memory operations across control-flow partitions called Waves. TWC leverages concepts from Transactional Memory systems, treating each Wave as a nested transaction: this permits concurrent speculative execution of memory operations across multiple Waves and precise rollback on conflict. The mechanism preserves sequential memory semantics within each Wave while enabling aggressive speculative parallelism between Waves, thus overcoming fundamental serialization bottlenecks of the baseline WaveScalar ordering model (0712.1167).

1. Foundations: WaveScalar and Memory Ordering

WaveScalar is a DataFlow architecture supporting imperative languages by enforcing sequential memory semantics while exposing fine-grain parallelism. Architecturally, the system comprises a 2×2 cluster mesh, each containing four Domains with eight Processing Elements (PEs) apiece, hierarchically backed by L1/L2 caches and a wave‐ordered StoreBuffer per cluster.

Programs are partitioned by the compiler into single-entry, acyclic regions called Waves. These are globally ordered (Wave 0, Wave 1, …), and each operand carries its Wave ID. Within a Wave, memory operations are linked by a ⟨P,C,S⟩ (Predecessor, Current, Successor) triple forming an internal dependence chain. Store instructions are split into address-request (StAddr) and data-request (StData) phases; a partial-store queue per address maintains the chain of accesses. The baseline memory model enforces two constraints:

1. A memory request in Wave $X$ executes only after its in-wave predecessor $P$ completes.
2. All memory operations of waves $0$ through $X-1$ must have executed.

This approach provides sequential consistency but strictly serializes memory between Waves, leading to coarse-grain ordering and limited inter-wave parallelism.

2. Transactional Semantics: Waves as Nested Transactions

Transactional WaveCache reinterprets each Wave as an atomic, nested transaction. New protocol elements are introduced: a Finished bit $F$ per Wave and an integer $lastCommittedWave$ per StoreBuffer.

• Eligibility: A Wave $W$ is eligible for commit when it has issued all memory requests ($W.F = \textrm{TRUE}$).
• Non-speculativeness: $W$ is non-speculative iff $W.\mathrm{Id} = lastCommittedWave + 1$; otherwise, it is speculative.
• Commit: If $W$ is non-speculative and finished, it commits, updating $lastCommittedWave \leftarrow W.\mathrm{Id}$ and discarding transactional metadata.
• Abort: If a conflict is detected between a speculative wave $X$ and an older wave, all Waves $Y \geq X$ are aborted and re-executed, respecting the nesting implied by the global order.

This model enables multiple speculative Waves to execute memory operations out of order with respect to the committed prefix.

3. Memory Ordering, Conflict Detection, and Rollback

Within a single Wave, baseline ordering via $\langle P, C, S \rangle$ chains and partial-store queues is maintained. Across Waves:

• Non-speculative Waves enforce strict program order.
• Speculative Waves may issue loads/stores out of ID order, constrained only by their own in-wave dependencies.

The speculative window, i.e., the number of speculative Waves in flight, may be statically or dynamically limited to control resource utilization.

Hazard detection operates eagerly by tracking all speculative memory accesses in a MemOp-History (MOH) per StoreBuffer. For memory operations $A$ (wave $X$) and $B$ (wave $Y$) accessing the same address, with $X > Y$:

• RAW (Read-After-Write): $A$ is Load, $B$ is Store. Abort all Waves $Z \geq X$.
• WAW (Write-After-Write): Both Stores. Chain $B$'s write to $A$ and defer commit until $A$ commits; no abort.
• WAR (Write-After-Read): $A$ is Store, $B$ is Load. Serve $B$ from $A$'s historical value; no abort.

Rollback utilizes the MOH's undo logs to restore memory and clears all relevant transactional state (WaveContextTable entries, pending memory requests). All aborted Waves' execution numbers are incremented to prevent reordering or reuse of stale operands, ensuring correctness.

4. Transactional Metadata: Structures and Algorithms

To manage speculation and rollback, several metadata structures are employed:

• WaveContextTable (WCT): Per-Wave read-set, holding operands for re-injection on abort.
• MemOp-History (MOH): Per-StoreBuffer log recording (waveId, opType, address, oldValue, ExeN) for conflict detection and undo procedures.
• Execution Number (ExeN): Monotonic tag incremented on abort, ensuring matching tables ignore or discard stale operands.
• Operand-Filtering: Matching tables and Execution Maps filter incoming operands by ExeN, discarding obsolete entries.

These structures enable fast, bounded rollback, eager conflict detection, and precise operand replay during re-execution. Cleanup and commit are handled recursively: when a Wave finishes, commit is attempted for itself and any already-finished successors.

5. Performance Characteristics and Observed Metrics

Evaluation utilized single-threaded, synthetic “kernel” benchmarks stressing memory behaviors:

Kernel	Traffic/Hazards	Observed Speedup
MATRIX, MATRIX-MIN	Low bandwidth, few hazards	≈90%
MATRIX-STORES-MIN	Medium bandwidth	33.1% (∞ window)
MATRIX-STORES-MIN-DEP	Medium, with dependencies	24%
MATRIX-STORES	High bandwidth, few conflicts	–16% (slowdown)
VECTOR-FULL-DEP	Heavy RAW, WAR, WAW hazards	139.7%

Key observations:

• In low-traffic kernels, speculative Waves efficiently utilize memory, realizing significant speedups by exposing inter-wave parallelism.
• As the speculative window grows, more in-flight operations risk saturating memory ports, producing slowdowns when bandwidth is the bottleneck.
• In hazard-rich loops (VECTOR-FULL-DEP), hardware can often exploit useful work even when frequent aborts occur, leading to large net speedups.

6. Architectural Trade-offs and Implementation Implications

The Transactional WaveCache introduces both substantial opportunities and challenges:

• Advantages:
  • Permits aggressive overlapping of memory operations from different Waves, exceeding traditional in-wave decoupling.
  • Nested-transaction rollback is simplified by the ordered structure; eager conflict detection restricts wasted speculative work.
  • Achieves high gains even with heavy hazard profiles.
• Constraints and Bottlenecks:
  • Additional hardware complexity: the MOH, WCT, ExeN tags, operand filtering, and matching table coordination.
  • Risk of memory-system contention: many concurrent speculative requests can exceed L1/L2/store buffer bandwidth, degrading performance.
  • Single-StoreBuffer responsibility: speculative Waves must reside on the same StoreBuffer as their non-speculative predecessor, potentially limiting inter-cluster scaling.
• Future Directions:
  • Hybrid schemes combining in-wave decoupling with inter-wave speculation are plausible enhancements.
  • Adaptive control of the speculative window could mitigate risks of overshooting hardware bandwidth.
  • The metadata infrastructure (WCT, MOH) may act as a foundation for higher-order synchronization primitives.
  • Evaluation of eager versus lazy conflict detection, selective rollback (open/closed nesting), and hardware/software co-design are open for further work.

By conceptualizing each DataFlow Wave as a nested transaction, TWC enables out-of-order, speculative parallelism across control-flow regions at hardware level, subject to the trade-offs in metadata management and bandwidth handling required for correct, high-performance operation (0712.1167).