Filtered Runahead Execution

• Filtered runahead execution is a microarchitectural technique that speculatively executes only the minimal dependence chain of micro-ops required for accurate cache-miss prefetching.
• It utilizes chain extraction and dynamic threshold filtering to reduce redundant speculative execution, cutting energy consumption and avoiding cache pollution.
• Empirical evaluations show significant improvements including up to a 13.1% IPC boost, 62% performance rise in multi-core systems, and reduced effective memory latency.

Filtered runahead execution is a microarchitectural technique designed to reduce effective memory access latency by speculatively—and selectively—executing only those instructions absolutely required to compute addresses for future cache-miss operations. Unlike conventional runahead, which replays the entire post-miss program stream, filtered runahead restricts activity to the minimal dependence chain of micro-operations (µops) that generate independent miss addresses. This selective approach yields more accurate prefetching, substantial energy savings, and improved memory-level parallelism across CPU and matrix processor domains.

1. Architectural Principles and Contrast to Traditional Runahead

Traditional runahead is triggered by retirement stalls due to long-latency loads: the processor checkpoints its register state, poisons the blocking load, and continues to fetch, decode, and replay subsequent µops. While this increases memory-level parallelism (MLP), the broad speculative execution consumes front-end energy and issues many irrelevant instructions. Filtered runahead instead isolates only those µops contributing to the address calculation of independent cache misses. The front end is clock-gated, and only the critical µops—forming a tight dependence chain—are executed in a loop until the blocking load returns (Hashemi, 2016).

In matrix processor domains, as implemented in DARE (Yang et al., 19 Nov 2025), filtered runahead is further augmented to address high redundancy in speculative memory prefetches encountered in sparse computations. A threshold-based filtering classifier ensures that only those speculative memory operations likely to miss in the last-level cache (LLC) are admitted, thus preventing cache pollution and energy overhead endemic to naïve runahead.

2. Hardware Structures and Chain Extraction Algorithms

Filtered runahead requires several specialized on-chip structures:

• Dependence-Chain Generator: Augments reorder buffer (ROB) entries with decoded µop and program counter (PC), and includes two content-addressable memories (CAMs) to match physical register IDs and PCs.
• Runahead Buffer: A 32-entry circular buffer in the rename stage, populated with extracted dependence chains.
• Chain Cache: A tiny 2-entry cache indexed by miss-PC, storing previously filtered chains to amortize chain-generation latency.
• Store Queue CAM: Handles forwarding loads in the chain from preceding stores.

Upon a full-window stall, filtered runahead invokes chain extraction: locating another inflight instance of the same static load in the ROB, recursively enqueueing its source registers, iteratively assembling the dependence chain (up to 32 µops), and caching the result. Runahead buffer execution then fetches µops directly from the buffer, renames to free physical registers, and loops execution until the original miss resolves. A special “MAP” µop ensures correct register mapping at loop boundaries.

In DARE (Yang et al., 19 Nov 2025), the filtered runahead engine comprises a Runahead Issue Queue (RIQ), Vector Matrix Register (VMR) extension for irregular gathers, and the Runahead Filter Unit (RFU). The RFU employs latency histograms to set a dynamic threshold, classifying speculative micro-ops as likely LLC misses and admitting only those prefetches beyond the threshold.

3. Filtering Logic for Matrix and Sparse Workloads

For DNN and matrix workloads, indirect loads due to compressed sparse column/row (CSC/CSR) indexing result in irregular access patterns prone to redundant prefetches. In DARE, every RIQ entry tracks “TentativeSent” and “Granted”—whether a test uop has been issued and whether subsequent dependent uops are authorized.

Filtering operates as follows:

• Issue a “tentative” uop for each entry and measure its latency.
• Upon completion, compare the latency $L_{\mu}$ to a dynamically maintained threshold $\Theta$ (calculated from histogram peaks separated by four bins; $\Theta$ is set to valley latency plus 32 cycles).
• If $L_{\mu} > \Theta$ or the instruction writes to the VMR, update “Granted” and admit further prefetches.

This unsupervised, online approach closely tracks LLC hit/miss distributions without explicit cache probes, suppressing speculative prefetches that are unlikely to yield useful MLP.

4. Analytical Formulas and Performance Metrics

Performance and efficiency are rigorously quantified:

• IPC Speedup: $$\text{Speedup}_{\text{filtered}} = \frac{\text{IPC}_{\text{frun}}}{\text{IPC}_{\text{base}}}$$.
• Average Latency Reduction: $$\text{LatencyReduction} = \frac{L_{\text{base}}-L_{\text{filtered}}}{L_{\text{base}}}\times 100\%$$.
• Prefetch Accuracy and Coverage:
  • $A = \frac{P_{\text{useful}}}{P_{\text{total}}}$
  • $$C = \frac{P_{\text{useful}}}{M_{\text{misses without prefetch}}}$$
  • Effective memory access latency under filtered runahead: $$\overline{L}_{\text{FRE}} = (1-C)\cdot L_{\text{miss}} + C\cdot L_{\text{hit}}$$

In practical systems, filtered runahead executes 36–64% fewer µops per interval and increases MLP by 32% (Hashemi, 2016). In matrix processor implementations, prefetch redundancy is reduced from above 80% to below 15%, and energy efficiency gains up to 22.8× are observed on highly irregular SpMM kernels (Yang et al., 19 Nov 2025).

5. Interaction with Cache Miss Types and Memory Controllers

Filtered runahead explicitly targets independent cache misses—those whose address calculation chains reside entirely on-chip. Dependent misses, requiring off-chip data, are accelerated by offloading their chain to compute-capable memory controllers (EMC), which execute as soon as the data arrives. The EMC can also support filtered runahead for independent misses, further bypassing on-chip cache hierarchy delays and issuing predicted LLC misses directly to DRAM (Hashemi, 2016).

This asymmetry ensures that filtered runahead never speculatively executes dependent chains. Instead, the combined approach achieves a synergistic reduction in latency across both miss categories.

6. Implementation Overhead and Evaluation Methodologies

Filtered runahead entails modest hardware overhead:

• Additional on-chip storage requirements are limited to small buffers and CAMs. For DARE, total state for the filtering logic is approximately 3.05 KB (RIQ, VMR, RFU), which is 3.19× lower than previous broad runahead architectures (e.g., NVR’s 9.72 KB) (Yang et al., 19 Nov 2025).
• It adds about 1.3% area to the baseline MPU datapath.

Evaluation spans x86 cycle-accurate simulation (SPEC CPU2006) and matrix kernel benchmarks (SpMM, SDDMM) under RV64GC. Filtered runahead boosts IPC by 13.1% (single core), yields a 62% performance rise in quad-core systems, and reduces effective memory-access latency by 19% (Hashemi, 2016). For MPUs, DARE achieves throughput improvements up to 4.44× and consistent wins over unfiltered runahead on all tested benchmarks (Yang et al., 19 Nov 2025).

7. Applicability, Insights, and Generalization

Filtered runahead demonstrates maximum benefit in contexts where cache miss rate is moderate (10–50%) and non-trivial reuse exists but naïve runahead would induce cache thrashing. The dynamic threshold classifier embedded in DARE proves robust under varying LLC latencies (20–40 cycles); static thresholds fail when hit/miss latencies converge. For very regular sparse kernels, densification can be selectively disabled, with filtered runahead alone delivering 5–10% performance improvement.

While designed primarily for matrix and DNN workloads, the tentative-uop and latency-histogram approach of filtered runahead can generalize to other irregular compute domains—such as graph processing engines—where a bimodal cache-access latency distribution can be observed (Yang et al., 19 Nov 2025). This suggests filtered runahead is broadly applicable as a precise, low-overhead speculative execution facility in future architectures.

---

Filtered runahead execution fundamentally transforms speculative pre-execution from broad program stream replay into precise microarchitectural loops over minimal dependence chains. This yields substantial reductions in energy, prefetch redundancy, and bandwidth overhead, while maximizing MLP and system throughput in both CPU and matrix accelerator contexts (Hashemi, 2016, Yang et al., 19 Nov 2025).