Memory Interleaving Techniques

• Memory interleaving is a technique that distributes memory addresses to increase parallelism and bandwidth while reducing contention.
• It is applied in NUMA, DRAM, and accelerator systems to balance load and optimize performance in heterogeneous memory configurations.
• Interleaving also serves as a defense against side-channel attacks by ensuring access randomness and mitigating leakage risks.

Memory interleaving is a foundational architectural and algorithmic technique for improving aggregate bandwidth, reducing access bottlenecks, and mitigating side-channel vulnerabilities in modern memory systems. By distributing memory accesses—whether at the level of physical pages, cache lines, banks, or within data structures—interleaving seeks to expose maximum concurrency and mitigate locality collapse, contention, or information leakage across diverse hardware-software contexts.

1. Fundamental Concepts and Motivations

Memory interleaving refers to the systematic distribution or transformation of memory addresses, memory pages, banks, or data granules to accomplish one or more of the following objectives:

• Increase available bandwidth by parallelizing access across multiple banks or tiers.
• Alleviate hotspots and controller saturation in shared-memory or multicore environments.
• Restore or augment locality, especially in the presence of throughput-oriented, multi-threaded workloads.
• Enable clash-free, deterministic parallel accesses in hardware accelerators.
• Eliminate information leakage arising from static or repeated memory contents.

Interleaving manifests at multiple hardware and software abstraction layers, including DRAM bank mapping, OS-level page allocation policies, on-chip scratchpad partitioning, and cryptographically motivated data rearrangements. The granularity and mechanism depend on system needs—common units are cache lines, DRAM pages (4 KB), or hardware-defined datawords (8/16 bytes) (Dey et al., 2017, Bhati et al., 2018, Liu et al., 2024, Pätschke et al., 13 Feb 2025).

2. Architectural Interleaving in DRAM and NUMA Systems

In multicore and multi-node architectures (NUMA, DRAM/CXL hybrids), memory interleaving is employed to harness the aggregate bandwidth of several memory controllers by distributing physical pages across available channels. The operating system, using facilities such as Linux's MPOL_INTERLEAVED, cycles through physical nodes for each allocation at page granularity, typically 4 KB. This ensures load balancing and prevents a single link or socket from becoming a bottleneck.

In NUMA contexts, interleaving operates below the TLB and MMU: the logical address space is split in a round-robin manner, preserving contiguity and guaranteeing minimal TLB disruption (Liu et al., 2024). CXL-attached memory introduces further heterogeneity—slower and higher-latency memory (200–400 ns) can be combined with local DRAM, with interleaving ratios selected to balance bandwidth improvement against adverse latency exposure. A central trade-off exists: bandwidth-bound applications benefit from spreading pages across DRAM and CXL, while latency-sensitive applications risk throughput degradation when exposed to tail-latency events in the slower tier.

A decision function R(x) parameterized by the fraction x of remote (CXL) pages is used to minimize predicted slowdowns, weighted for DRAM, cache, and store-induced stalls. The “best-shot” policy selects x* = argmin R(x) based on live performance counters, optimizing the observed workload in a single pass (Liu et al., 2024).

3. Banked Memory Interleaving in Accelerators and DNNs

In specialized hardware, especially sparse DNN accelerators and manycore arrays, interleaving is critical for sustaining z-way parallelism. Here, memories are divided into z single-ported banks. A memory interleaver is a pair of mapping functions b(i) (bank number) and a(i) (address-within-bank) designed so that sets of k indices to be accessed in parallel always map onto distinct banks—termed clash-free interleaving.

This is synthesized algorithmically: for example, weights in a fully-connected, structured-sparse network are mapped using permutation functions π_W such that each parallel group of accesses at cycle k is guaranteed to be bank-disjoint. The construction leverages random permutations and modular arithmetic to provide both maximal spread (minimum close-pair risk) and dispersion (address diversity), with formal proofs of clash-freedom (Dey et al., 2017). Variants (Start Vector Shuffle, Sweep-Starter, Memory-Dither) explore improved randomness at small additional logic or pointer costs. Empirically, clash-free interleaving delivers near-linear throughput scaling with z and obviates the need for expensive multi-port memories.

4. Stream Interleaving, Locality, and Reordering: The MARS Architecture

Throughput-oriented processors, such as modern GPUs, suffer from heavy interleaving of independent data streams at multiple arbitration points (e.g., L1/L2 caches). This can destroy locality at the off-chip memory interface, resulting in scattered accesses across many DRAM rows and diminished row-buffer reuse.

The MARS (Memory-Aware Reordered Source) architecture addresses this by employing a large, on-chip lookahead buffer between the final GPU cache and the DRAM controller. MARS captures pending memory requests, reorganizes them to group accesses by physical page, and forwards all queued requests for the same page consecutively. This increases the row-buffer hit rate (RBH) and maximizes contiguous CAS bursts per activation (ACT), directly lifting effective memory bandwidth. In synthetic benchmarks, MARS raised RBH from ≈20 % to values yielding a net 11 % improvement in achieved bandwidth, with hardware overhead below 0.1 % of GPU die area and negligible latency penalty (Bhati et al., 2018).

System Context	Interleaving Granularity	Key Benefit
NUMA/CXL Hybrid	4 KB pages	Bandwidth scaling
On-chip Accelerator	Data/bank index	Clash-free parallelism
Streaming GPU Memory	DRAM row (4 KB)	Locality restoration

5. Interleaving as a Defense: Preventing Memory-Centric Side-Channels

Software-level interleaving, as realized in cryptographic and confidential-computing contexts, serves primarily as a side-channel mitigation. Memory-centric side-channels, such as ciphertext repetition or silent store elimination, arise when deterministic or redundant stores yield observable constancy in physical DRAM.

“Zebrafix” employs a compiler-driven interleaving algorithm in which every stored value is combined with a freshness-providing counter to produce a unique, never-repeated 128-bit block for encryption. The transformation B_i = (C_i ∥ D), with C_i as a sequential counter and D as the payload, ensures block-level uniqueness and thus defeats equality-based leakage. Implementation is via IR passes in LLVM, converting all protected stores, loads, and memory-altering operations to operate on such interleaved blocks. This nullifies silent-store and ciphertext side-channels with single-digit runtime slowdown, outperforming masking-based alternatives that incur large RNG overheads (Pätschke et al., 13 Feb 2025).

The method is tightly constrained: interleaved blocks must remain 16-byte aligned for atomicity (matching x86-64/AES-NI vectorization), and the transformation is only data-layout, not address-mapping, so hardware memory maps need no change.

6. Performance and Implementation Trade-Offs

Effective memory interleaving imposes trade-offs between throughput, complexity, latency, and hardware/software cost:

• Page-level (OS/NUMA/CXL): Minimal software complexity; dynamic model-based ratio selection (e.g., “best-shot”) achieves near-optimal speedups up to 26 % by leveraging live hardware counters and linear predictors. In practice, skewed ratios that reflect bandwidth asymmetry outperform static 1:1 or manual M:N assignments (Liu et al., 2024).
• Accelerator-bank interleaving: Clash-free hardware schemes minimize bank idle time; addressing complexity is limited to a few small pointer tables and arithmetic operations. Higher “spread” can trade off with “dispersion”—important for accuracy in sensitive low-redundancy ML workloads (Dey et al., 2017).
• MARS-like reordering incurs tens of KBs SRAM and ~1–3 cycles total buffer/forwarding latency—amortized to negligible effect. The technique is orthogonal to memory controller configuration (Bhati et al., 2018).
• Security-centered interleaving: Code size and peak memory usage can increase significantly (code +63 %, memory ×5 worst-case), particularly for aggregate types and pointer-intense programs. However, interleaving complexity is O(1) per store, with hardware unchanged except for atomicity enforcement (Pätschke et al., 13 Feb 2025).

7. Limitations and Context-Dependent Effectiveness

The efficacy and suitability of interleaving are context-dependent:

• Fine-grained (sub-block) interleaving to defeat pointer prefetchers can sharply escalate type-system and memory complexity.
• For latency-sensitive workloads, page-interleaving across slower memory (e.g., CXL) may reduce performance due to tail events; model-based adaptive policies are required to strike optimal trade-offs (Liu et al., 2024).
• In sparse DNN acceleration, high-dispersion but low-spread interleavers may degrade performance on low-redundancy tasks (Dey et al., 2017).
• Secure interleaving requires precise type and pointer tracking; legacy C code with raw casting semantics imposes challenges and potential overheads (Pätschke et al., 13 Feb 2025).

Memory interleaving remains a cornerstone of both memory system performance optimization and confidentiality assurance, with tailored algorithms and implementations emerging for each application and architectural tier. Its theoretical guarantees—clash-freedom, locality enhancement, and leakage nullification—are central to contemporary computer architecture and security research.