Two-Tier Constant-Size Memory

Updated 4 July 2026

Two-tier constant-size memory is a model with fixed capacities, splitting memory into an active working set (e.g., registers) and an exchange tier (e.g., threadgroup memory).
It spans diverse domains—from GPU FFT optimization to tiered main memory and bounded allocation—forcing innovative algorithm designs under strict capacity constraints.
Quantitative insights, such as achieving 138.45 GFLOPS on Apple Silicon and limiting transforms to 4096-point FFTs, highlight its practical impact on performance and system design.

Two-tier constant-size memory denotes a class of designs in which computation is organized around two distinct memory or state tiers whose capacities are fixed rather than scaling with problem size. The most explicit formulation in the cited literature is the Apple Silicon GPU FFT model that treats the 208 KiB register file as the primary data-resident tier and the 32 KiB threadgroup memory as an exchange-only tier, with both capacities fixed architectural constants independent of transform size $N$ (Bergach, 29 Mar 2026). Closely related formulations appear in tiered main memory, virtual-memory-assisted buffer management, bounded concurrent allocation, distributed certification, and memory-restricted search, although in those settings “constant-size” may denote a fixed fast-memory budget, a bounded per-tier resident set, or an $O(1)$ -bit or $O(1)$ -state budget rather than the same hardware abstraction (Chen et al., 2024).

1. Terminology, scope, and recurring structure

Across the literature, the phrase does not name a single universal formalism. It instead recurs as a family of bounded-resource decompositions in which one tier carries the active working set or control state, while a second tier mediates overflow, exchange, migration, or communication. The common structural feature is that these tiers are operationally finite, so algorithms must be decomposed around those bounds rather than assuming memory that scales with input size.

Domain	Two-tier structure	“Constant-size” interpretation
Apple GPU FFT	Register file + threadgroup memory	Fixed on-chip capacities
Tiered main memory	Fast memory + slow memory	Fixed fast-tier capacity
Concurrent allocation	Private pools + shared pool	Bounded local inventory
Distributed verification	Local state + visible labels	$O(1)$ -bit certificates

In the Apple GPU FFT model, “constant-size” means that on-chip capacities are fixed architectural constants—208 KiB of registers and 32 KiB of threadgroup memory—independent of $N$ (Bergach, 29 Mar 2026). In Tuna, the analogous problem is to choose a fixed fast-memory size before deployment so that predicted slowdown stays below a target such as $5\%$ (Chen et al., 2024). In local certification, the constant regime means $O(1)$ -bit labels, and binary certification is already too weak to certify $k$ -colorability for every $k \ge 3$ (Martínez et al., 2022). In memory-restricted Mastermind, constant-size memory means a constant number of stored guess-answer pairs, and the main theorem shows that even one memory cell preserves the optimal $\Theta(n/\log n)$ query complexity for constant alphabet size (Doerr et al., 2011). This suggests that “two-tier constant-size memory” is best understood as a research motif centered on fixed-capacity decomposition rather than a single cross-domain theorem.

2. Accelerator-local memory as the canonical formulation

The most concrete and technically developed instance appears in “Beating vDSP: A 138 GFLOPS Radix-8 Stockham FFT on Apple Silicon via Two-Tier Register-Threadgroup Memory Decomposition” (Bergach, 29 Mar 2026). That work explicitly defines a two-tier local memory model for Apple GPUs. Tier 1 is the register file plus SIMD shuffle: capacity 208 KiB per threadgroup, a per-thread budget of up to 128 32-bit registers, and simd_shuffle communication within a 32-thread SIMD group at roughly $O(1)$ 0 cycles. Tier 2 is the 32 KiB threadgroup memory, shared across threads and used as the only mechanism for exchange between SIMD groups, with measured latency around $O(1)$ 1 cycles. The central interpretive claim is that the useful working set should live in registers, while threadgroup memory should be treated primarily as an exchange fabric.

For complex FP32 data, the paper gives the threadgroup-memory bound

$O(1)$ 2

so a single reusable 32 KiB buffer can hold at most a 4096-point complex transform. The “constant-size” feature is therefore literal: local memory does not scale with $O(1)$ 3, so larger transforms must be factored into subtransforms that fit this fixed budget. The same paper notes a register-resident extension in which 256 threads $O(1)$ 4 32 complex elements per thread can in principle support local FFTs as large as $O(1)$ 5 in FP32, with threadgroup memory used only as a narrow inter-group exchange path (Bergach, 29 Mar 2026).

The empirical measurements are decisive in shaping the model. Sequential threadgroup-memory bandwidth is reported as 688 GB/s, strided bandwidth as 217 GB/s, simd_shuffle throughput as 262 GB/s, and register-threadgroup copy bandwidth as 407–420 GB/s. Combined with the observation that threadgroup barriers cost about 2 cycles, this yields the paper’s key architectural conclusion: barriers are cheap, while scattered threadgroup-memory accesses are expensive. The resulting decomposition rule is therefore not merely “minimize synchronization,” but “minimize bad Tier-2 traffic” (Bergach, 29 Mar 2026).

That principle determines the kernel design. For $O(1)$ 6, the best kernel is a radix-8 Stockham implementation with 512 threads, 4 passes, about 38 GPRs per thread, and sequential stage-to-stage exchange through threadgroup memory. It achieves 138.45 GFLOPS at batch 256, compared with 107.0 GFLOPS for vDSP/Accelerate and 113.6 GFLOPS for the radix-4 Stockham baseline (Bergach, 29 Mar 2026). The same work reports that a shuffle-heavy radix-32 hybrid reaches only 61.5 GFLOPS, despite fewer barriers, because once inter-SIMD-group communication is needed it degenerates into scattered Tier-2 access. For $O(1)$ 7, the implementation switches to a four-step factorization with $O(1)$ 8, preserving the fixed-capacity local design while pushing the unavoidable transpose into device memory.

3. Tiered main memory and storage hierarchies

A second major line of work treats two-tier constant-size memory as a provisioning or system-design problem in which a fast tier is deliberately kept limited and the system relies on migration, placement, or index redesign to make that limit tolerable. Tuna is the clearest sizing formulation. It models a two-tier hierarchy consisting of fast memory and slow memory, then chooses the minimum fast-memory capacity subject to a slowdown constraint. The paper states the optimization objective directly as minimizing fast-memory capacity under a performance-loss target, and for a $O(1)$ 9 performance-loss target Tuna with TPP saves fast memory by 8.5% on average, up to 16%, compared with Microsoft Pond’s reported 5% saving on BFS and SSSP under the same target (Chen et al., 2024). The core technical point is that sizing must model page migration overhead explicitly rather than using locality alone.

Virtual-memory-assisted buffer management generalizes the same bounded-tier logic. “Virtual-Memory Assisted Buffer Management In Tiered Memory” treats DRAM as Tier-0, remote memory as Tier-1, and NVMe disk as Tier-2, while emphasizing that each tier is operationally finite and that capacity pressure drives migration and eviction (Rayhan et al., 3 Mar 2026). In the evaluated system, local memory is fixed at 32 GB and remote memory varies from 8 GB to 128 GB; when DRAM reaches a threshold such as 95% utilization, pages are proactively marked and selected for demotion. The paper introduces move_pages2 to reduce migration bottlenecks and reports that vmcache $O(1)$ 0 can achieve up to $O(1)$ 1 higher query throughput over vmcache for TPC-C, with more specific gains of up to $O(1)$ 2 and $O(1)$ 3 when remote memory is $O(1)$ 4 and $O(1)$ 5 the size of local memory (Rayhan et al., 3 Mar 2026). The central systems lesson is that bounded tiers are useful only when migration cost is controlled.

Jenga addresses a similar fixed-budget regime but focuses on hot-page instability and page-internal heterogeneity. Its fast tier is DRAM, its capacity tier is CXL-attached memory or NVM, and it is evaluated especially when fast-tier capacity approximately matches the application working-set size. The paper defines working-set size as the average number of pages accessed in each 50 ms period times page size, provisions the fast tier to match that WSS, and shows that Jenga runs memory-intensive applications 28% faster across 10 applications than the best previous tiered memory system, at a CPU overhead of less than 3% of a single core and a memory overhead of less than 0.3% (Kadekodi et al., 26 Oct 2025). Its contribution is split between a context-based allocator using 32 allocation regions and smooth hotness cooling that avoids thrashing. Here the “constant-size” issue is not an immutable hardware constant, but a deliberately tight DRAM budget that must be made effective.

The same theme appears in billion-scale vector search. “Characterizing the Dilemma of Performance and Index Size in Billion-Scale Vector Search and Breaking It with Second-Tier Memory” places local DRAM as the first tier and RDMA-attached DRAM, CXL-attached memory, or NVM as a second tier between DRAM and SSD (Cheng et al., 2024). The paper argues that SSD-based graph and cluster indexes improve throughput only by accepting large storage amplification, because SSD coarse-grained access mismatches the fine-grained random reads required by compact vector indexes. Second-tier memory changes that trade-off. On second-tier memory, optimized graph indexes reach optimal performance with 1.7–4.1 $O(1)$ 6 index amplification instead of 4.3–4.9 $O(1)$ 7 on SSD, while optimized cluster indexes can operate with only 0.1–0.4 $O(1)$ 8 index amplification overhead and 1.1–1.4 $O(1)$ 9 total footprint relative to the raw dataset (Cheng et al., 2024). The hierarchy is therefore two-tier in a strict systems sense: a small DRAM-resident routing tier plus a larger fine-grained second tier that avoids SSD-style index inflation.

4. Algorithmic and data-structural analogues

A distinct but related interpretation appears in algorithmic work where the two tiers are not hardware memories but bounded local stores and overflow or shared structures. “Concurrent Fixed-Size Allocation and Free in Constant Time” organizes a concurrent allocator around per-process private pools and a shared global pool (Blelloch et al., 2020). Each private pool contains between 1 and $O(1)$ 0 blocks, with $O(1)$ 1 and $O(1)$ 2, while the shared pool stores blocks in batches. The result is linearizable, wait-free allocate and free for fixed-size blocks of $O(1)$ 3 words with $O(1)$ 4 worst-case time per operation and $O(1)$ 5 additive overhead (Blelloch et al., 2020). The architecture is explicitly two-level: a fast local tier handles the common case, and a slower shared tier is touched only for rebalancing.

“Backyard Cuckoo Hashing” offers an analogous two-level organization for dynamic dictionaries (0912.5424). Its first level is a dense table of small bins that stores most elements; the second level is a “backyard” cuckoo hash table that stores only overflow items. The resulting dictionary uses $O(1)$ 6 memory words while guaranteeing constant-time operations in the worst case with high probability, and the refined succinct version uses only $O(1)$ 7 bits where $O(1)$ 8 (0912.5424). This is not a constant-capacity memory model, but it is a two-tier bounded-overflow design in which one tier is intentionally kept small and most operations remain local to it.

“Playing Mastermind With Constant-Size Memory” gives a more literal constant-memory example (Doerr et al., 2011). There, the codebreaker may store only a constant number of guesses and answers, yet the paper proves that even one memory cell suffices to solve Mastermind with constant alphabet size in $O(1)$ 9 guesses, matching the unrestricted $N$ 0 bound. The mechanism is to use the current guess string itself as an internal data structure that encodes learned code positions, counters, phase markers, sampled substrings, and stored replies. The explicit memory tier is constant-sized, but the guess representation functions as an implicit second tier. This use of structured encoding to simulate richer history recurs across bounded-memory settings.

5. Formal semantics, control, and bounded-state systems

Two-tier constant-size memory also appears in formal semantics and adaptive systems as a separation between an idealized or active tier and a bounded retained tier. “A Two-Phase Infinite/Finite Low-Level Memory Model” introduces an infinite memory model and a finite memory model, connected by an out-of-memory-aware refinement relation (Beck et al., 2024). In the LLVM/VIR instantiation, the infinite model uses Coq’s big-integer $N$ 1 type for physical addresses and the finite model uses 64-bit integers, limiting memory to the 64-bit address space. The finite program is proved to behave identically to the infinite one until the programs terminate in lock-step or the finite program runs out of memory (Beck et al., 2024). Here the second tier is a bounded implementation memory, and “constant-size” is instantiated by fixed-width address representation.

In adaptive identification, “Online Adaptive Identification of Switched Affine Systems Using a Two-Tier Filter Architecture with Memory” uses a dual-layer low-pass filter architecture plus a memory bank that stores filter values at subsystem switch-out times (Patel et al., 2022). The first layer filters regressors and states to produce a relation $N$ 2; the second layer builds $N$ 3 and $N$ 4 with $N$ 5. For each subsystem, the scheme stores the latest switch-out values $N$ 6, together with one stored positive-definite pair once intermittent initial excitation is achieved. The memory is fixed in size with respect to time because each subsystem slot is overwritten rather than grown, and the paper proves exponential convergence of the switched-system parameters under the IIE condition (Patel et al., 2022).

Distributed and mobile-robot settings show that the nature of the constant tier matters as much as its size. “Rendezvous of Two Robots with Constant Memory” splits a previously stronger model into FState and FComm (Flocchini et al., 2013). In FState, a robot can only see its own light, so the light functions as internal persistent state; in FComm, a robot can only see the other robot’s light, so the light functions as constant-size communication. The paper proves that FState robots can rendezvous in SSynch, whereas FComm robots can rendezvous even in ASynch, and it provides constructive protocols using six states for rigid SSynch FState, 12 colors for rigid ASynch FComm, and three colors in several weaker settings (Flocchini et al., 2013). This sharply separates private bounded state from externally visible bounded state.

Finally, lower bounds show that constant-size regimes are not uniform. “Lower bound for constant-size local certification” proves that $N$ 7-colorability for every $N$ 8 cannot be certified by binary certificates in the considered local model (Martínez et al., 2022). Since certification size is a lower bound, and often an upper bound, on the memory needed for silent self-stabilizing constructions, the result demonstrates that the one-bit regime is strictly weaker than slightly larger constant-memory regimes. The paper’s positive binary-certification result for distance-2 3-colorability further shows that some properties remain in the smallest constant tier while others do not (Martínez et al., 2022).

6. Cross-cutting principles and open problems

Taken together, these works suggest several recurring design principles. First, fixed capacities must be treated as first-class algorithmic constraints, not as minor implementation details. On Apple GPUs, the decisive quantity is the 32 KiB threadgroup-memory ceiling and the register-pressure limit around 128 registers per thread, which determines radix choice and decomposition depth (Bergach, 29 Mar 2026). In tiered main memory, the decisive quantity is the fast-tier budget at which migration overhead remains below the target slowdown (Chen et al., 2024). In both cases, the decomposition is driven by the non-scaling tier.

Second, the distinction between a data-resident tier and an exchange or migration tier is usually more important than a simple fast-versus-slow taxonomy. The Apple GPU model makes this explicit by assigning registers to residence and threadgroup memory to orderly exchange (Bergach, 29 Mar 2026). vmcache $N$ 9 and Jenga make the same distinction at page granularity: capacity alone is insufficient if migration becomes the bottleneck or if hot and cold objects are packed into the same page (Rayhan et al., 3 Mar 2026). This suggests that two-tier constant-size memory is often less about raw capacity than about which tier is allowed to hold stable state and which tier is used only for transfer.

Third, bounded memory often remains effective only when the representation itself is redesigned. CSR layout removes SSD-era padding in graph search on second-tier memory (Cheng et al., 2024). Address replication replaces vector replication in clustered ANNS (Cheng et al., 2024). The Mastermind one-cell algorithm stores counters, sampled blocks, and replies inside the current guess string (Doerr et al., 2011). Such examples indicate that representation choice is inseparable from fixed-capacity decomposition.

Several open questions remain explicit in the cited works. Tuna does not solve dynamic online resizing of the fast tier at fine time scales, joint optimization of fast-tier size and migration policy, or multi-tenant contention (Chen et al., 2024). vmcache $5\%$ 0 concludes that page migration overhead remains the bottleneck and that more efficient migration mechanisms are still needed (Rayhan et al., 3 Mar 2026). The two-phase low-level memory model does not track provenance through integers, so inttoptr returns wildcard provenance (Beck et al., 2024). The rendezvous paper leaves open whether FState robots can solve rendezvous in ASynch (Flocchini et al., 2013). These unresolved points are consistent with the broader pattern: once memory is split into two bounded tiers, correctness and performance depend not only on capacity, but on exchange structure, representation, and the semantics of what each tier is allowed to remember.