Allocator/Page Size Sensitivity Insights

• Allocator/Page Size Sensitivity is the study of how allocation mechanisms and memory page sizes shape system performance, efficiency, and correctness across hardware and workload patterns.
• It encompasses economic and performance models, empirical evaluations of TLB miss rates, fragmentation, and dynamic adaptation strategies, highlighting measurable tradeoffs.
• The topic drives future research by recommending adaptive, architecture-aware allocator designs to optimize throughput, reduce overhead, and improve operational reliability.

Allocator/page size sensitivity denotes the non-trivial dependence of system performance, efficiency, and correctness on both the choice of allocation mechanisms (“allocators”) and the memory page sizes they manage. The optimal page size and allocation strategy are dictated by a complex interplay of hardware architecture (e.g., DRAM, GPU, processor TLBs), workload patterns, and software-level policies. Sensitivity manifests in broad phenomena—from translation lookaside buffer (TLB) miss rates and demand paging overheads in GPUs, to inefficiencies from internal fragmentation, to outright operational failures when alignment or contiguity constraints are undersatisfied (notably in processing-in-memory, PUM/PUD, or hybrid model inference). Modern research refines the classic rules-of-thumb (e.g., the Five-Minute Rule) into quantitatively precise models, while experimental work demonstrates that sophisticated allocator/page-size interactions are often the dominant factor for high-performance, scalable, and correct system design.

1. Economic and Performance Theory of Page Size Sensitivity

The canonical analytical framework for determining optimal page size is grounded in a cost-per-unit-time model that weighs RAM holding cost against disk fetch cost, originally formulated in "The Five-Minute Rule Ten Years Later" [9809005]. Let $C_{RAM}$, $B_{RAM}$ denote the cost and bandwidth of RAM respectively, $C_{Disk}$, $B_{Disk}$ for disk, and $T_{life}$ the desired page lifetime. The optimal page size $S^*$ is:

$$S^* = \sqrt{\frac{C_{Disk}\,T_{life}\,B_{Disk}}{C_{RAM}\,B_{RAM}}}$$

Key sensitivities:

• Increasing $C_{RAM}$ or $B_{RAM}$ reduces optimal page size ($-1/2$ power).
• Increasing $B_{RAM}$0, $B_{RAM}$1, or $B_{RAM}$2 enlarges optimal page size ($B_{RAM}$3 power).

Empirical anchor points (circa late 1990s) yielded a baseline $B_{RAM}$4 for randomly accessed pages, with “halving RAM cost or doubling memory bandwidth changes optimal page size by ~30%.” Thus, allocator/page size sensitivity is rooted in economic tradeoffs and amplified by technology shifts impacting these parameters [9809005].

2. Hardware-Software Page Size Interaction in System Allocators

Modern CPU and GPU systems provide multiple hardware-supported page sizes (e.g., 4 KB, 2 MB, 1 GB on x86-64; 4 KB and 2 MB on GPUs), but software allocators often underutilize this flexibility. Trident, for instance, dynamically balances allocator strategies: it extends the Linux buddy allocator to manage free spans up to 1 GB, engages in hierarchical compaction, and employs promotion/demotion policies mediated by empirical TLB walk counters (Ram et al., 2020).

Key results confirm that page-size sensitivity is workload-dependent:

Page Size	Beneficial Scenarios	Limitations/Side Effects
4 KB	Small, irregular	Crippling TLB miss rate for big workloads
2 MB	10MB–500MB hotsets	Moderate bloat, fragmentation
1 GB	>1GB hotsets, stride-1	High bloat if access irregular, allocation failure under fragmentation

Trident empirically delivers an average 18% speedup over Linux’s 2 MB transparent huge pages, with further 5–10% gains from paravirtualized copy-less promotion (Ram et al., 2020). The approach supports dynamically mixed allocation, maximizing TLB reach while minimizing bloat and adaptation cost.

3. Page Size Sensitivity on Emerging Architectures

Architectures with nontraditional operation requirements—particularly PUM/PUD—are acutely sensitive to allocator and page size (Oliveira et al., 2024). PUM substrates require allocation alignment to specific DRAM subarray and row boundaries; standard malloc/posix_memalign cannot meet these requirements, producing 0% in-DRAM operation success, with all work falling back to the CPU. Huge-page allocators (e.g., Linux 2 MB pages), while reducing TLB misses by $B_{RAM}$5, still only achieve 40–60% alignment for large buffers, inducing fallback penalties.

The PUMA allocator resolves this by preallocating huge pages and splitting them into DRAM row-sized subpages ($B_{RAM}$6 KB), retaining 100% alignment, minimal TLB pressure, and full PUD bandwidth (20 GB/s in the experimental prototype):

Allocator	Page Size	Alignment Success	CPU Fallback
malloc/posix_memalign	4 KB	0%	100%
Huge Page Allocator	2 MB	40–60%	40–60%
PUMA	subpage (1 KB)	100%	0%

The conclusion is categorical: allocator design and page size must be tuned to underlying architecture, data layout, and operation type; one-size-fits-all approaches result in either functional incorrectness or orders-of-magnitude slowdowns (Oliveira et al., 2024).

4. Impact of Page Size Sensitivity on Allocator Mechanism and Performance

TLB architecture and demand paging exhibit strong, countervailing page size sensitivities. For example, Mosaic shows that:

• Large pages ($B_{RAM}$7 MB) provide a $B_{RAM}$8–$B_{RAM}$9 order-of-magnitude improvement in TLB reach (e.g., up to $C_{Disk}$0 GB with $C_{Disk}$1 entries), slashing miss rates and corresponding stall penalties.
• However, demand paging with large pages greatly increases page fault latency ($C_{Disk}$2 vs $C_{Disk}$3 over PCIe) and memory bloat due to overfetch.
• Mosaic’s decoupling—demand-paging always on 4 KB base pages, address translation via coalesced 2 MB pages—avoids the fundamental tradeoff and achieves both low paging cost and high TLB efficiency.

Measured in heterogeneous GPU workloads, Mosaic improves speedup over a best-in-class 4 KB allocator by +29.7% to +55.5%, and maintains $C_{Disk}$4 L2-TLB hit rates under increasing application multiplexing (Ausavarungnirun et al., 2018). The core insight is that collaborative, page-size-aware allocation and virtual-to-physical layout is essential for peak system throughput.

5. Fragmentation, Internal Wastage, and Dynamic Adaptation

Allocator/page size sensitivity is directly implicated in fragmentation and memory wastage. In contemporary hybrid (e.g., Mamba-Transformer) inference workloads, a unified pool allocator forces all allocations to the largest natural page size (e.g., KV cache page), resulting in cases where O(1) State Space Model (SSM) allocations are padded excessively. Capacity is then overestimated by up to $C_{Disk}$5, leading to peak VRAM utilization as low as $C_{Disk}$6 (Nguyen, 21 May 2026).

AVMP (Asymmetric Virtual Memory Paging) introduces virtual-handle separation for divergent allocation types, dynamic capacity migration between pools, and deterministic migration policy (batch migration, migration triggers only on allocation failure). Empirical results show:

• A reduction in out-of-memory (OOM) events of $C_{Disk}$7 compared to static pools,
• Goodput multipliers of $C_{Disk}$8 to $C_{Disk}$9 on synthetic workloads,
• Insensitivity of OOM to migration batch size thresholds within broad margins,
• Elimination of pathologies of unified pool padding and maladaptive static pools (Nguyen, 21 May 2026).

This demonstrates that exposing and adapting to page size heterogeneity, while actively combatting fragmentation, is essential for inference efficiency and predictability at scale.

6. Allocator/Page Size Sensitivity in Concurrency and Reclamation

In software memory allocators with thread-local caches (e.g., JEmalloc, TCmalloc), batch-free reclamation interacts pathologically with cache thresholds, exhibiting a pronounced sensitivity not directly to hardware page size, but to the allocator’s batch policy and per-thread thresholds. Large batch frees (as in epoch-based reclamation, EBR) quickly overflow these local caches, triggering high-latency, lock-contended global flushes. This remote batch free (RBF) problem can degrade throughput by an order of magnitude at scale (e.g., 59.5% of time spent in free/flush at 192 threads, compared to 19.2% with amortized freeing) (Kim et al., 2024).

Allocators with per-page free lists (e.g., MImalloc) avoid global contention: contention is limited to per-page granularity, with batch-free performance flat across batch sizes. Thus, software-level allocation structures and their granularity can strongly mediate, or amplify, the underlying page-size sensitivity (Kim et al., 2024).

7. Practical Recommendations and Future Research Directions

Empirical and analytical results across architectures motivate several best practices and ongoing research topics:

• Allocator policies must dynamically orchestrate page sizes to match access locality, physical contiguity, workload hotness, and hardware constraints (Ausavarungnirun et al., 2018, Ram et al., 2020).
• Fragmentation management (e.g., smart compaction, sub-page splitting, in-place page table updates) is a first-class concern and is tightly coupled to allocator/page size design (Ram et al., 2020, Oliveira et al., 2024).
• Large scale systems should periodically retune page size policies as storage price/performance ratios evolve ("reparameterize" as per [9809005]).
• For parallel and lock-free data structures, allocator mechanisms should be batch-size-insensitive or apply amortization to mitigate contention spikes (Kim et al., 2024).
• GPU and hybrid inference systems should expose natural page-size heterogeneity, avoid capacity overestimation from padding, and support dynamic, virtual pool balancing (Nguyen, 21 May 2026).
• Research continues into copy-less promotion, paravirtualized compaction, and transparent multiple page size exposure as means to approach optimal performance envelopes (Ram et al., 2020).

Allocator/page size sensitivity remains a central, evolving theme in systems research, and optimal solutions are necessarily adaptive, architecture-aware, and empirically grounded.