Reuse Distance Histogram (RDH)

• Reuse Distance Histogram (RDH) is a statistical distribution of reuse distances in memory-access traces that captures temporal locality and is key to cache behavior modeling.
• RDH maps patterns to cache hit/miss ratios and miss-ratio curves, enabling analysis of both fully associative and set-associative cache architectures.
• RDH can be computed via hash map-based methods and estimated using static and learning-based approaches, making it versatile for dynamic and analytical cache assessments.

A Reuse Distance Histogram (RDH) is a histogram or empirical distribution of reuse distances across a memory-access trace. In the literature it is also called a reuse profile, reuse distance profile, or reuse distance distribution, and it serves as a hardware-architecture-independent summary of temporal locality that can be mapped to cache hits, misses, miss-ratio curves, and shared-cache behavior. The precise meaning of reuse distance is not uniform: some works use the classical LRU-stack definition based on distinct intervening references, whereas others use a trace-based count of all intervening accesses and distinguish it from stack distance; one learning-based formulation also introduces forward reuse distance as a prediction target for approximating Belady’s optimal replacement policy (Barai et al., 2019, Razzak et al., 2024, Li et al., 2020).

1. Terminology and formal definitions

For a trace $T=(a_1,a_2,\dots,a_N)$, the classical definition used in several works sets the reuse distance of access $a_t$ to the number of unique memory references between two consecutive references to the same address. If the address has not appeared before, the reuse distance is $\infty$, or a special cold-miss key such as $-1$. This is equivalent to LRU stack distance: the reuse distance is the stack position of the referenced block just before the access. By contrast, one trace-based formulation defines reuse distance as “the number of accesses between two consecutive accesses to the same datum,” and explicitly distinguishes that quantity from stack distance, which counts only unique intervening accesses (Barai et al., 2019, Razzak et al., 23 Sep 2025, Razzak et al., 2024, Li et al., 2020).

Variant	Intervening quantity counted	First-touch or no-future case
Classical reuse distance / stack distance	Unique addresses between consecutive references	$\infty$ or $-1$
Trace-based backward reuse distance	All accesses between consecutive references	$\infty$
Forward reuse distance	All accesses until the next future reference	$\infty$ for last access

Under the classical form, if $t' < t$ is the previous position with $a_{t'}=a_t$, then

$a_t$0

Under the trace-based backward form, the corresponding quantity is

$a_t$1

The forward variant reverses the temporal direction. If $a_t$2 is the next position with $a_t$3, then

$a_t$4

This quantity is a clairvoyant locality metric: $a_t$5 is the number of logical time steps until the next use of the block. A related feature, penultimate reuse distance, is the backward reuse distance of the previous occurrence of the same block. A persistent source of confusion is therefore terminological rather than mathematical: an RDH is only interpretable once the underlying reuse-distance definition has been fixed.

2. Histogram, reuse profile, and locality structure

Given a sequence of reuse distances $a_t$6, the RDH is the mapping from reuse-distance value to frequency,

$a_t$7

or, equivalently, the empirical probability mass function

$a_t$8

Some formulations separate compulsory or cold misses into a dedicated $a_t$9 or $\infty$0 bin; others also use coarser buckets rather than exact distances. In this sense, reuse profile and RDH are interchangeable names for the same object (Barai et al., 2019, Razzak et al., 23 Sep 2025, Razzak et al., 2024).

An RDH summarizes temporal locality in a compact form. High mass at small distances indicates strong temporal locality; substantial mass at large distances or $\infty$1 indicates poor locality, capacity pressure, or single-use data. One work notes that reuse distance is often presented in histogram form and also uses per-access time-series plots of reuse distance versus logical time. Those plots are not themselves histograms, but they can be aggregated into one, and they expose structure that the histogram later compresses: triangle, clouds, gaps and bars, crossing lines, horizontal lines, and clusters. This suggests that an RDH is a summary statistic over structured temporal patterns rather than a replacement for them (Li et al., 2020).

The histogram perspective is especially useful because it preserves locality information while abstracting away from specific hardware parameters. Several works therefore emphasize that RDH is a platform-independent or hardware-architecture-independent characterization of memory behavior. A plausible implication is that the same RDH can be reused across multiple cache-size, associativity, or hierarchy studies, provided that the mapping from reuse distance to hits and misses is modeled appropriately.

3. Mapping RDHs to cache behavior

For a fully associative LRU cache of capacity $\infty$2, the classical mapping is exact: a reference with reuse distance $\infty$3 is a hit if $\infty$4 and a miss if $\infty$5, with $\infty$6 treated as compulsory miss. Consequently,

$\infty$7

This is the foundational reason RDHs are central to cache modeling (Barai et al., 2019, Tsai et al., 2021, Ho et al., 2021).

For realistic set-associative caches, one analytical route is the Stack Distance Cache Model. With reuse distance $\infty$8, associativity $\infty$9, and number of blocks $-1$0, the conditional hit probability is

$-1$1

and the overall hit rate becomes

$-1$2

Other works use the same general decomposition $-1$3 and then instantiate $-1$4 with an analytical cache model (Barai et al., 2019, Razzak et al., 23 Sep 2025, Razzak et al., 2024).

RDHs also underlie miss-ratio curves. One formulation uses the volume fill time model: for a cache of size $-1$5, if reuse distance is larger than the volume fill time $-1$6, the corresponding access is a miss. Counting such accesses across the histogram yields the miss ratio for that size. In a different line of work, a single top-level RDH is generalized to multi-level hierarchies. For exclusive caches, a miss in level $-1$7 occurs when reuse distance exceeds the cumulative capacity $-1$8; for inclusive caches, a miss in level $-1$9 occurs when reuse distance exceeds $\infty$0. These rules enable design-space exploration over cache hierarchies without repeated full simulation (Li et al., 2020, Tsai et al., 2021).

4. Construction, aggregation, and transformation

When reuse distance depends only on past accesses, it can be computed in one pass with a hash map. In the trace-based backward formulation, if last_time[b] stores the last access position of block $\infty$1, then

$\infty$2

Forward reuse distance can be computed offline by scanning the trace backward with a next_time map. Once the distance array has been constructed, the histogram is obtained by counting exact distances or bucketed ranges (Li et al., 2020).

In program-structure-aware modeling, the global RDH can be decomposed into a mixture of per-basic-block RDHs,

$\infty$3

Here $\infty$4 is the probability that a random reference comes from basic block $\infty$5, and $\infty$6 is the conditional reuse-distance distribution for references associated with that block. This linear mixture is the basis of a scalable analytical concurrent reuse profile for OpenMP programs on shared caches (Barai et al., 2019).

For multi-core shared caches, another transformation derives aggregated reuse-distance histograms from per-application RDHs. If application $\infty$7 interleaves additional unique addresses inside the reuse epoch of application $\infty$8, the aggregated reuse distance becomes

$\infty$9

The model estimates $-1$0 from access ratios and per-application $-1$1-$-1$2 tables, shifts each original bin $-1$3 horizontally to $-1$4, and then combines per-application aggregated histograms by summing counts at equal distances. In that framework, the contention effects of concurrent execution are embedded directly in the aggregated RDH (Ho et al., 2021).

RDHs can also be transformed across cache levels. One L2 model takes as input the profiled L1 RDH together with two auxiliary locality summaries: an RST table, which captures the joint distribution of reuse distance and stack distance, and a Hit-RDH, which captures the number of L1 hits inside reuse epochs of a given reuse distance. These quantities support a fast analytical transformation from L1 reuse structure to an L2 RDH and then to L2 miss-rate estimation for LRU or Random L2 caches (Ling et al., 2019).

5. Static, probabilistic, and learning-based RDH estimation

A major research direction is to estimate RDHs without collecting full dynamic traces. One probabilistic method for OpenMP programs instruments only sequential executions on small inputs, labels memory references by basic block, mimics shared and private addresses in synthetic multi-core traces, and then constructs per-basic-block concurrent reuse profiles. This avoids collecting a full multi-threaded trace while preserving a global RDH suitable for shared-cache hit-rate prediction (Barai et al., 2019).

A second direction is static RDH estimation for array programs with nested loops. These methods start from LLVM IR, construct a control-flow graph, derive execution counts, annotate loops and array indices, and generate a compact bracketed representation of symbolic memory accesses. They then unroll small loop bounds statically, compute exact small-case RDHs, and extrapolate to large bounds by predictive equations. For two-level and three-level nests, the predicted frequency of each RDH bin is expressed as a polynomial in loop-bound increments with linear, pairwise, and higher-order interaction terms. One extension adds block separation, loop-block-specific calculation, adjustment of redundant cold misses, and adjustment of array reuses across loop blocks (Razzak et al., 2024, Razzak et al., 23 Sep 2025).

A third direction treats forward reuse distance as a learnable target. In that formulation, each access maps to a feature tensor containing address-delta information, backward reuse distance, penultimate reuse distance, average reuse distance in a sliding window, frequency in a sliding window, and address-delta clustering information. A many-to-one sequence model with 2 stacked LSTM layers of width 256 predicts a scaled forward reuse distance in $-1$5, and the predicted quantity is then used by a pseudo-OPT policy that evicts the block with the largest predicted future time. Conceptually, this turns the RDH from a global marginal distribution into a context-dependent, per-access prediction problem (Li et al., 2020).

These approaches share a common objective: to replace exhaustive trace collection or repeated simulation with reusable locality summaries. The differences lie in what is modeled directly. Static loop methods predict the RDH itself; basic-block probabilistic models predict a mixture distribution $-1$6; learning-based methods predict per-access forward distances whose aggregate would induce a forward-oriented RDH.

6. Empirical findings, design uses, and limitations

Empirical studies consistently show that RDH-based models can be accurate enough for analysis and optimization. In a learning-based cache-replacement study over 13 Microsoft Research storage workloads, pseudo-OPT incurs only 2.3% higher miss ratio than OPT on average, and improves over LRU by 13.7%, over 2Q by 19.2%, and over ARC by 8.6% on average. In several traces, the miss-ratio curve is almost identical to OPT; in others, non-concave or wavy behavior appears, indicating sensitivity to forward-distance prediction errors in certain regions of the effective histogram (Li et al., 2020).

For shared-cache modeling of OpenMP programs, one analytical concurrent reuse-profile model reports that it can predict shared cache hit rate with 97.61% accuracy, with examples such as BFS and MatMul on Core-i7 and Xeon systems showing hit rates close to Valgrind-derived reference values. For early-stage multi-level cache optimization, RDH-based scanning search is reported to be 150 to 250 times faster than traditional simulation-based approaches, with average error below 4% for inclusive hierarchies and essentially exact results for exclusive ones under the stated assumptions. For multi-core shared-cache design, aggregated RDH miss-count estimation has an average error rate of less than 2.4%. For L2 RDH modeling from software traces, the average absolute error is less than 5%, with almost 30X speedup for four L2 candidates. Static reuse-profile estimation for loop-based array programs reports average cache-hit-rate accuracy around 95%, while a later nested-loop framework reports comparable accuracy with orders-of-magnitude improvement in analysis speed relative to PARDA (Barai et al., 2019, Tsai et al., 2021, Ho et al., 2021, Ling et al., 2019, Razzak et al., 2024, Razzak et al., 23 Sep 2025).

The design uses are correspondingly broad. RDHs support miss-ratio curves, cache-size selection, shared-cache performance modeling, static compiler analysis, loop-transformation guidance, and learning-aided replacement policies. Because the histogram is reused across candidate cache parameters, it is well suited to scanning search over size, associativity, or hierarchy configurations. In compiler-oriented settings, the same reuse-profile machinery can guide tiling, fusion, interchange, or near-data-computing decisions (Tsai et al., 2021, Barai et al., 2019, Razzak et al., 23 Sep 2025).

The limitations are equally consistent. Exact RDH-to-miss mappings assume fully associative LRU behavior; set-associative and inclusive hierarchies require approximations. Some shared-cache models assume disjoint data sets across applications or uniform random block-to-set mapping. Static loop-based predictors are targeted at affine array accesses in regular nested loops and do not fully handle complex branching, irregular data structures, pointer-based accesses, or highly dynamic reuse patterns. Learning-based forward-distance predictors depend on future labels generated offline and may produce workload- and cache-size-dependent errors. These caveats do not diminish the central role of RDH; rather, they define the regimes in which the histogram is an exact object, a compressed analytical surrogate, or a target for prediction.