Dependent Memory Allocation

• Dependent Memory Allocation is a memory management paradigm where allocations are determined by external dependencies such as hardware topology, module interdependence, or type invariants.
• It leverages specific constraints—ranging from DRAM subarray residency in PUMA to tensor lifetime planning in DNN inference—to optimize performance and facilitate legal memory usage.
• Techniques vary from hardware-aware placement to type-preserving protocols, highlighting practical trade-offs in efficiency, safety, and dynamic resource management.

Dependent memory allocation denotes allocation regimes in which memory is not assigned solely by size, virtual contiguity, or conventional alignment, but by constraints induced by other objects, future operations, allocator state, or type-level invariants. In the literature represented here, those dependencies arise in several distinct forms: hardware-topology legality for Processing-using-DRAM, internal budget splits among interdependent learning modules, graph- and lifetime-induced reuse plans for deep neural network inference, and typed heap protocols in which initialization state or universe level constrains what may be allocated and when. This suggests that dependent memory allocation is best treated as a family of allocation disciplines unified by one principle: allocation decisions are valid only relative to an external dependency relation, rather than as isolated responses to individual requests (Oliveira et al., 2024, Tamborski et al., 9 Jun 2025, Levental, 2022, Koronkevich et al., 10 Sep 2025, Koronkevich et al., 2024).

1. Defining the dependency relation

In the systems literature, dependency typically means that one object cannot be allocated independently of another because some later operation imposes a joint legality condition. PUMA makes this explicit for Processing-using-DRAM: source and destination operands must reside in the same DRAM subarray and must be aligned to DRAM row boundaries, so allocation of a second operand depends on the physical placement of the first (Oliveira et al., 2024). In the reinforcement-learning setting, the dependency is internal to the agent: model memory $N_{\hat{p}}$ and planning memory $N_{\pi}$ must satisfy $N_{\hat{p}} + N_{\pi} = N$, and allocating more memory to one process can directly degrade another process that depends on it (Tamborski et al., 9 Jun 2025). In dependently typed compilation, the dependency is semantic: the type of a later field can depend on the value stored in an earlier field, so staged heap initialization must preserve that order and its proof obligations (Koronkevich et al., 10 Sep 2025).

The type-theoretic work extends the notion further by making allocation depend on universe structure. In “Type Universes as Allocation Effects,” a universe level does not only stratify types for consistency; it also describes where values of that type may be allocated in the heap, and the universe algebra determines what heap edges are legal (Koronkevich et al., 2024). A broader interpretation therefore emerges: dependent memory allocation is not a single mechanism but an allocation paradigm in which feasibility is determined by relations among operands, modules, metadata, or types.

This broader interpretation also distinguishes dependent allocation from ordinary API-level alignment. posix_memalign can enforce virtual alignment, but it does not express cross-object co-location, type-dependent initialization, or budget-sharing constraints between cooperating subsystems. The recurring design move across these works is to elevate allocation from “size + alignment” to a richer contract that includes legality for future use.

2. Hardware topology and placement legality

The clearest systems instance is PUMA, a kernel-level allocator for PuD memory objects. PUMA begins from a concrete mismatch: malloc, posix_memalign, and huge pages-based allocation do not guarantee that virtual pages are contiguous in physical memory and aligned within a DRAM row, even though RowClone-like and Ambit-like operations require same-subarray residency and row-boundary alignment (Oliveira et al., 2024). PUMA therefore combines internal DRAM mapping information with huge pages and then splits huge pages into finer-grained memory regions that are “aligned to the page address and size” and “virtually contiguous.” It maintains an ordered array indexed by subarray ID, with each entry recording the number of memory regions in a single subarray, and an allocation hashmap keyed by virtual address so that later allocations can be placed relative to earlier ones.

Its user-visible interface exposes the dependency directly. pim_preallocate reserves a number of huge pages for PuD use, pim_alloc allocates the first PuD object, and pim_alloc_align allocates subsequent object(s) aligned to an existing one. For the first allocation, PUMA uses a worst-fit policy over subarrays. For dependent aligned allocation, pim_alloc_align takes a hint pointer, searches the allocation hashmap, iterates over the hint allocation’s memory regions, tries to place each corresponding region of the new object in the same subarray, falls back to worst-fit when necessary, and then performs re-mmap so that regions from different huge pages appear at contiguous virtual addresses. In the reported evaluation, malloc and posix_memalign enable 0% of Ambit operations to execute in the PuD substrate, huge-page allocation reaches only “up to 60%” for large allocations such as 32 Kb, and PUMA significantly outperforms the baseline allocators for all evaluated microbenchmarks and allocation sizes (Oliveira et al., 2024).

A related but distinct constrained-allocation line appears in host-based allocation for device memory. There the defining dependency is not same-subarray legality but the allocator’s inability to read the memory it manages. Because “the allocator can't read the memory it is allocating,” boundary tags and in-band free-list metadata are unavailable, so correctness depends on host-resident shadow metadata structures such as Hybrid Array Lists, bitmasks, and hash tables (Bell et al., 2024). This is still a form of dependent allocation: each allocation and free operation depends on an external authoritative model of device memory rather than on heap-local metadata. The paper’s comparison table makes the trade-off explicit: HALs have worst-case overhead $50\%$, allocation $O(1)$, and coalescence $O(n/m + m)$; bitmasks have worst-case overhead $1.5\%$, allocation $O(n)$, and coalescence $O(s)$; hash tables have worst-case overhead $87\%$, allocation $N_{\pi}$0, and coalescence $N_{\pi}$1 (Bell et al., 2024).

3. Execution-structure-aware memory planning

Dependent allocation also arises when allocation requests are induced by a repetitive computation graph rather than by arbitrary program behavior. MemoMalloc treats DNN inference as such a workload. Its key observation is that intermediate tensor allocations are determined by graph structure, operator identity, kernel behavior, tensor lifetimes, aliasing relationships, and multi-threaded execution context, so the requests are not independent calls to a general-purpose allocator (Levental, 2022). The system therefore profiles one forward pass, captures all allocations, uniquely associates them with their high-level source operation, reconstructs exact lifetimes, sizes, and aliasing relationships, and then computes a replayable memory plan.

The planning problem is formalized as offline dynamic storage allocation. Given a size $N_{\pi}$2 and a live interval for each allocation, the planner chooses a slab size $N_{\pi}$3 and offsets $N_{\pi}$4 so that overlapping lifetimes cannot overlap in address space. MemoMalloc evaluates bump_allocation, mip, Gergov’s $N_{\pi}$5 approximation, greedy_by_size, and mincost_flow, and defaults to greedy_by_size because it gives near-optimal peak memory with much lower planning time (Levental, 2022). The runtime then rewrites TorchScript IR to insert prim::AllocateSlab and prim::AllocateTensor, turning repeated dynamic allocation into deterministic slab-offset replay.

This is dependency-aware allocation in a stronger sense than ordinary region allocation. The plan is bound to operator provenance, not merely to the order of requests, because order-based memorization is too brittle under operator reordering. It is also not reducible to graph-visible tensor scheduling, because profiling reveals many hidden kernel allocations that TorchScript IR does not expose. MemoMalloc reports that it outperforms state-of-the-art general purpose memory allocators with respect to DNN inference latency by as much as 40\%, while accepting moderate increases in peak or average memory usage depending on the workload (Levental, 2022).

4. Budget allocation among interdependent modules

A second major interpretation of dependent memory allocation treats memory as an internal budget distributed among cooperating components. In resource-constrained reinforcement learning, the central question is not only how much memory an agent has, but how that memory is divided among dependent internal processes. The model-based formulation uses a total budget $N_{\pi}$6 and a split $N_{\pi}$7, where $N_{\pi}$8 stores transitions for the estimated model $N_{\pi}$9 and $N_{\hat{p}} + N_{\pi} = N$0 stores planning structure such as MCTS nodes (Tamborski et al., 9 Jun 2025). With $N_{\hat{p}} + N_{\pi} = N$1, the experiments report an inverse-U relationship between planning memory allocation and return: very small $N_{\hat{p}} + N_{\pi} = N$2 yields near-zero returns, performance peaks roughly around $N_{\hat{p}} + N_{\pi} = N$3, and excessive planning memory hurts because model memory becomes too small. In PT-DQN, where $N_{\hat{p}} + N_{\pi} = N$4, the best-performing configuration allocates only 10% of hidden units to the permanent value function and reaches about 0.3 reward per step, whereas the canonical 50–50 split reaches only about 0.2 reward per step (Tamborski et al., 9 Jun 2025). The dependence here is structural: one module’s value depends on the adequacy of another module that shares the same finite budget.

Reasoning-model deployment exhibits a closely related byte-budget problem. “Not All Bits Are Equal” formulates total inference memory as the sum of weight memory and KV-cache memory, with the latter growing roughly linearly in model scale, generation length $N_{\hat{p}} + N_{\pi} = N$5, and parallel sampling group size $N_{\hat{p}} + N_{\pi} = N$6 unless eviction caps it (Kim et al., 13 Oct 2025). The concrete example in the paper is Qwen3-4B with 4-bit weights at 2.49 GB and a KV cache for 32k tokens at 4.42 GB. Across over 1,700 inference scenarios on AIME25 and GPQA-Diamond, the paper reports a scale-dependent threshold around an effective size of 8-bit 4B, approximately 4.19–4.2 GB of weight memory: below that threshold, additional memory is better spent on more or higher-fidelity weights; at or above it, additional memory is better spent on longer generation until saturation, and then on parallel scaling for sufficiently large budgets (Kim et al., 13 Oct 2025). The paper also reports that, for smaller effective models, KV eviction is preferred, whereas for larger ones KV quantization becomes increasingly competitive, with 2-bit KV quantization generally too aggressive.

A broader dynamic-allocation theory appears in SPII, where memory is not the allocated commodity but the allocator’s internal state. There the capacity factor

$N_{\hat{p}} + N_{\pi} = N$7

measures how much of the full-information capacity region can be retained under a noisy channel when the encoder has $N_{\hat{p}} + N_{\pi} = N$8 bits of memory and the allocator has $N_{\hat{p}} + N_{\pi} = N$9 bits (Xu et al., 2019). The main theorem establishes that, with memory-feedback, $50\%$0, $50\%$1, and, for $50\%$2-majorizing channels under the stated assumptions, $50\%$3 for all finite $50\%$4 (Xu et al., 2019). This shows that allocator memory, rather than encoder memory, is the critical state variable. In a broader sense, it extends dependent allocation from heap placement to dynamic control: allocation quality depends on what the allocator can remember about past observations.

5. Type-preserving and universe-indexed allocation

In type-theoretic work, dependent memory allocation refers literally to heap allocation whose legality is governed by dependent types. “Dependent-Type-Preserving Memory Allocation” introduces a target intermediate language, CC-CC$50\%$5, with explicit heap operations for two-word tuples: allocation $50\%$6, first-field assignment $50\%$7, second-field assignment $50\%$8, and closure tagging $50\%$9 (Koronkevich et al., 10 Sep 2025). Dependent pairs carry initialization flags. Allocation yields

$O(1)$0

first-field assignment produces

$O(1)$1

and second-field assignment requires the first field to be initialized and the assigned value to have type $O(1)$2, yielding

$O(1)$3

Projection is correspondingly constrained: fst requires the first field initialized, and snd requires both fields initialized. The main theorem states a type-preservation property for compilation from CC-CC to CC-CC$O(1)$4: if a source term is well typed, then its translation is well typed under the translated context and type (Koronkevich et al., 10 Sep 2025). The point is not only memory safety inside the compiler pass; it is preservation of allocation-related invariants so that linking against components that provide uninitialized memory can be treated as ill typed.

“Type Universes as Allocation Effects” shifts the emphasis from initialization protocols to heap structure. In its main predicative system, base types inhabit $O(1)$5, references shift universe level by one,

$O(1)$6

and function types $O(1)$7 require $O(1)$8, so closure environments contribute directly to allocation level (Koronkevich et al., 2024). The result is a stratified heap in which references point downward. This blocks Landin’s Knot because a closure that captures a level-1 reference must itself live at level 1, and therefore cannot be stored in a location that expects a level-0 function. For the standard ramified hierarchy, the paper proves termination using a logical relation indexed by universe level rather than by a step index (Koronkevich et al., 2024).

These two papers use “dependent memory allocation” in the strongest formal sense. Allocation is not merely influenced by future use; it is typed by that use. Initialization flags, dependent pair structure, and universe levels are all first-class constraints on what may be allocated, stored, projected, or linked.

6. Misconceptions, limits, and open directions

A recurring misconception is that dependent allocation is just a stronger form of alignment. The PUMA work shows why that is inadequate: same-subarray residency, DRAM-row-boundary alignment, and physical contiguity are cross-object and topology-aware constraints that malloc and posix_memalign do not express (Oliveira et al., 2024). Another misconception is that more memory is uniformly beneficial. Both the RL experiments and the reasoning-model study report non-monotonic effects: excessively favoring planning over model memory, or longer generation over model capacity at the wrong scale, can reduce performance (Tamborski et al., 9 Jun 2025, Kim et al., 13 Oct 2025).

The surveyed systems also remain limited in scope. PUMA is allocation-time placement control, not dynamic remapping; it assumes access to DRAM interleaving information and depends on huge-page pools reserved at boot time (Oliveira et al., 2024). MemoMalloc assumes traced inference without control flow and fixed intermediate tensor sizes, and its plan is tied to a profiled regime rather than arbitrary dynamic behavior (Levental, 2022). The RL study evaluates static per-run splits rather than an adaptive allocator that changes the split online (Tamborski et al., 9 Jun 2025). The reasoning-model threshold around 8-bit 4B is empirical, derived from the Qwen3 family and the tested quantization and KV-compression methods, not a theorem of universal scope (Kim et al., 13 Oct 2025). The typed-allocation compiler pass is explicitly ongoing work: it does not yet provide a complete end-to-end compiler correctness theorem, a full linking calculus, or explicit free (Koronkevich et al., 10 Sep 2025). Host-based device allocators, meanwhile, exchange classical in-band metadata for host-side shadow structures, making metadata overhead and granularity central design parameters (Bell et al., 2024).

Taken together, these limitations suggest that dependent memory allocation is best understood not as one settled technique but as a general design stance. The common claim is that memory management should account for operand pairings, execution structure, module interdependence, allocator state, or type-level evidence whenever those relations determine whether later computation is legal, efficient, or even typable. The surveyed literature uses the term for both systems-level placement control and type-preserving heap allocation; the unifying idea is that allocation legality is defined relative to dependencies that ordinary size-based allocators do not model.