SpeedMalloc: Efficient Memory Allocation

• SpeedMalloc is a regime-sensitive memory allocation design that optimizes performance by tailoring fast paths, metadata placement, and reclamation policies to different workload profiles.
• It employs techniques like lazy initialization, fixed-size pooling, and hardware-assisted support cores to minimize latency, cache interference, and synchronization overhead.
• Empirical evaluations show significant speedups—up to 10× in fixed-size implementations and improved scalability in multi-threaded and single-threaded environments.

Searching arXiv for the cited allocator papers to ground the article in current preprint metadata. SpeedMalloc denotes a family of memory-allocation strategies centered on reducing allocator-induced latency, cache disruption, synchronization overhead, and fragmentation. In the cited literature, the name is used both for a fixed-size memory pool manager built around an in-place free list and lazy initialization (Kenwright, 2022) and for a multi-threaded allocation architecture that offloads allocator work to a lightweight support-core (Li et al., 27 Aug 2025). Taken together, related work on single-threaded specialization, scalable multicore allocation, allocator behavior in analytical query engines, and head-first best-fit with space-fitting suggests that “SpeedMalloc” is best understood as a regime-sensitive design objective: make the fast path short, keep metadata from interfering with user data, and shape reuse and reclamation policies to the actual concurrency and size-distribution profile of the workload (Li et al., 11 Oct 2025, Aigner et al., 2015, Durner et al., 2019, Hakarsa, 2024).

1. Performance significance of allocator design

Memory allocation is treated in this literature not as incidental runtime plumbing but as a system-level performance determinant. One line of work frames allocator inefficiency as a “datacenter tax”: memory allocators sit on the hot path of almost every runtime, and even a 1% improvement in allocator efficiency can translate to millions of dollars in savings and measurable reductions in datacenter energy consumption (Li et al., 11 Oct 2025). A second line shows a “butterfly effect” in multi-threaded systems: allocator-related instructions may account for only around 2–5% of execution, yet allocator choice can produce up to a 2.71× variation across state-of-the-art allocators, with allocator metadata conflicts contributing 28.3% of all cache misses for BFS in TCMalloc with 16 threads and allocator metadata synchronization consuming 22.8% of total cycles in Larson (Li et al., 27 Aug 2025). A third line demonstrates application-level consequences in analytical DBMS: on a 4-socket Intel Xeon server, the right allocator increased TPC-DS (SF 100) performance by 2.7× (Durner et al., 2019).

These results establish a common premise. Allocation policy changes only a small fraction of dynamic instructions, but it reshapes cache occupancy, TLB behavior, remote-memory traffic, lock and atomic pressure, and reclamation timing. A plausible implication is that allocator research should be organized around the dominant bottleneck in each setting—deterministic latency for fixed-size pools, metadata minimization for single-threaded binaries, synchronization elimination for multi-threaded programs, or fragmentation control for best-fit heaps—rather than around a single universal design.

2. Recurrent design primitives

Across the papers, several primitives recur. In fixed-size pools, the pool is one contiguous region partitioned into $N$ equal-sized blocks; when a block is free, its first 4 bytes store the index of the next free block, yielding no separate header and no per-block metadata outside the block itself. Pointer/index conversion is direct,

$$\text{index} = \frac{(\text{ptr} - \text{base})}{\text{block\_size}}, \qquad \text{capacity} = \left\lfloor \frac{\text{pool\_bytes}}{\text{block\_size}} \right\rfloor,$$

and allocation/free are $O(1)$. Lazy initialization avoids pre-linking the entire pool and initializes at most one additional block per allocation, eliminating loops from initialization and hot-path allocation (Kenwright, 2022).

Single-threaded general-purpose specialization uses a different primitive set. Exgen-Malloc organizes memory into 4 MiB segments, with page organizations of 64 pages per segment for the smallest page type, 8 for a medium type, and 1 for the largest type; very large allocations use huge segments sized to the object. Each page serves one block-size class, uses 8-byte granularity for small sizes, and maintains compact per-page metadata together with a single free-block list. The design removes per-thread/per-core caches, global sharding, multi-list management, locks, atomics, and cross-thread coordination, while retaining fine-grained size classes, aggregated metadata, bitmaps, a page/segment hierarchy, and inline fast paths (Li et al., 11 Oct 2025).

Scalable multicore allocators emphasize frontend/backend separation and distributed reuse. scalloc allocates within 2 MB virtual spans drawn from a 32 TB arena; each virtual span contains a real span sized to the size class, leaving the remainder unbacked under on-demand paging. The frontend uses hot spans, reusable spans, local and remote free lists, while the backend uses a real-span-size–segregated span-pool implemented as distributed Treiber stacks. Empty spans are returned eagerly, and larger real spans can be reclaimed with madvise(MADV_DONTNEED) once the real-span size reaches the default MADVISE_THRESHOLD of 32 KB (Aigner et al., 2015).

Best-fit allocators pursue a different balance. Head-first allocation on best-fit with space-fitting privileges the top chunk at the high-address end of an arena, attempts to satisfy allocations from that region first in $O(1)$ time, and otherwise falls back to a best-fit search. Space-fitting donates excess bytes to a free right neighbor first, else to a free left neighbor, and splits only when the remainder is at least $3 \times \text{overhead}$, in order to avoid creating tiny fragments (Hakarsa, 2024).

Taken together, these mechanisms suggest three broad SpeedMalloc motifs: constant-time object retrieval, metadata placement that minimizes interference with user data, and reclamation policies that trade off locality, fairness, and fragmentation in workload-specific ways.

3. Fixed-size pool SpeedMalloc

In "Fast Efficient Fixed-Size Memory Pool: No Loops and No Overhead" (Kenwright, 2022), SpeedMalloc is a fixed-size, zero-overhead allocator for uniform objects. The pool state contains num_blocks, block_size, num_free, num_initialized, a base pointer, and a head pointer. Initialization allocates contiguous memory for $N$ blocks of size $B$, sets num_free = N, num_initialized = 0, and head = base, but does not pre-link the pool. Each allocation first checks exhaustion, then performs lazy initialization if num_initialized < num_blocks by writing the next index into the block being initialized, returns the current head, decrements num_free, and advances head using the stored index. Free computes the block index, prepends the block to the singly linked free list, and increments num_free. Reset restores the initial state without touching each block, so the free list is rebuilt on demand.

The central claim of “no memory overhead” is precise rather than absolute. Per-block overhead is $H = 0$: an allocated block has no extra header, and the first 4 bytes are reused for bookkeeping only while the block is free. Global overhead remains a few integers and pointers, typically 32–48 bytes on a 64-bit platform. The hot path touches a constant number of cache lines, avoids iteration, and is therefore suited to time-critical systems such as games.

The reported measurements were run on Windows 7 64-bit with an Intel i7-2600 @ 3.4 GHz and 16 GB RAM, using Visual Studio Release builds, with tested allocation sizes 10, 100, 1000, 10000, and 100000 bytes. In release standalone runs, the fixed-size pool achieved about 10× speedup over system malloc across tested sizes; inside the debugger, it was up to approximately 1000× faster, reflecting debugger overhead on malloc. The paper expresses speedup and throughput as

$$S = \frac{T_{\text{malloc}}}{T_{\text{pool}}}, \qquad \tau = \frac{N}{t}.$$

Its limitations are equally explicit. The design is fixed-size only, cannot serve requests larger than block_size without fallback, requires one contiguous region, and is not thread-safe in its basic form. Optional robustness mechanisms—bounds checks, debug-only bitsets for double-free detection, or guard patterns—remain outside the hot path.

4. Single-threaded specialization and Exgen-Malloc

"Old is Gold: Optimizing Single-threaded Applications with Exgen-Malloc" (Li et al., 11 Oct 2025) argues that modern allocators such as jemalloc, tcmalloc, and mimalloc embed costs that are unnecessary in single-threaded programs. Those allocators rely on per-thread or per-core caches, shared and local free lists, synchronization or control logic, and balancing across threads. In single-threaded services and pipelines, these mechanisms become overhead: extra metadata causes cache pollution and false sharing risks, locks and atomics add latency, and fast paths traverse more structures than necessary.

Exgen-Malloc answers this by specializing for single-threaded execution. Its core design features are a centralized heap, a single free-block list per page, and a balanced strategy for memory commitment and relocation. Allocation rounds requests up to the size class, uses fixed-size blocks within a page, and follows a first-fit policy within the page’s singly linked free list. If page->free is non-null, the allocator pops the head and returns it in $O(1)$ time; otherwise it obtains a new page from the current segment, then from the segment cache, then from mmap, initializes the page by carving it into blocks, and returns the first block. Free writes the current page->free into the freed block’s next pointer, updates page->free, and increments counters, with no synchronization. Coalescing is unnecessary for fixed-size blocks inside a page, and calloc rather than malloc performs zero-initialization.

The OS policy is also specialized. The first segment defers committing physical memory page-by-page on demand; when a segment becomes free, it enters a small cache with one cached segment per page type; and if the cache is full, the segment is released via munmap. Build-time selection is used to link Exgen-Malloc only for single-threaded binaries, and the implementation is ABI-compatible with malloc/free/realloc/calloc and new/delete.

Evaluation on two Intel Xeon systems reports a 1.17× geometric-mean speedup over dlmalloc on SPEC CPU2017, 1.10× on redis-benchmark, and 1.93× on mimalloc-bench. Against mimalloc, the reported speedups are 1.05× on SPEC CPU2017, 1.01× on redis-benchmark, and 1.02× on mimalloc-bench; memory savings versus mimalloc are 6.2%, 0.1%, and 25.2%, respectively. Microarchitecturally, Exgen-Malloc reduces L1D MPKI by 18.0% versus dlmalloc and 2.9% versus mimalloc, L2 MPKI by 25.2% versus dlmalloc and 9.4% versus mimalloc, LLC MPKI by 24.4% versus dlmalloc and 7.8% versus mimalloc, and DTLB MPKI by 98.6% versus dlmalloc and 87.1% versus mimalloc. The design is explicitly unsupported for multi-threaded programs.

5. Support-core SpeedMalloc for multi-threaded allocation

"SpeedMalloc: Improving Multi-threaded Applications via a Lightweight Core for Memory Allocation" (Li et al., 27 Aug 2025) defines SpeedMalloc as a hardware-software co-design for multi-threaded applications. A lightweight, programmable support-core processes all user-level malloc/free work, houses allocator metadata in its own caches, and communicates with application cores through four new ISA instructions: mallocstart(), freestart(), mallocend(), and freeend(). mallocstart() sends a request and waits for an end signal that writes the returned pointer into a destination register; freestart() is asynchronous and retires without waiting. The support-core is an AArch64 “little” in-order core with 16 KB 4-way L1d and 16 KB 4-way L1i caches, plus hardware message queues with 128-entry dispatch queues for malloc() and free() and a 128-entry response queue.

The architectural claim is that application cores should never touch allocator metadata. Metadata for size classes and free lists resides only in the support-core caches, which eliminates cache interference between allocator metadata and user data and removes all cross-core metadata synchronization. Requests and responses travel over the coherent network with an 8-cycle main-to-support-core latency, and the support-core scheduler prioritizes malloc requests because they lie on the critical path, serving free requests when malloc queues are empty. free is centralized rather than being redirected into another thread’s cache, eliminating the remote-free handoff problem of conventional multi-threaded allocators.

The reported implementation is transparent for malloc/free/new/delete, requires no source changes, and is selected at link time; a post-compilation pass inserts the new instructions and target PCs. No kernel modifications are required, though the ISA extensions and the support-core attached to the coherent fabric are hardware requirements.

In gem5 full-system evaluation with 1–16 big out-of-order AArch64 application cores, SpeedMalloc achieves average 16-thread speedups of 1.75× over Jemalloc, 1.18× over TCMalloc, 1.15× over Mimalloc, 1.23× over Mallacc, and 1.18× over Memento+. The geo-mean speedups versus Jemalloc across 1, 2, 4, 8, and 16 threads are 1.39×, 1.40×, 1.58×, 1.73×, and 1.75×. Example 16-thread results versus Jemalloc include 3.57× in BFS and 3.19× in Larson. L2 miss cycles are reduced by 42.36% versus Jemalloc, 18.76% versus TCMalloc, and 22.80% versus Mimalloc, while atomic and synchronization cycles are eliminated by approximately 3.57% of total time versus Jemalloc, 11.99% versus TCMalloc, and 11.73% versus Mimalloc. Energy savings at 16 threads are reported as 1.69× versus Jemalloc, 1.15× versus TCMalloc, 1.12× versus Mimalloc, 1.26× versus Mallacc, and 1.22× versus Memento+. The support-core itself accounts for approximately 1.50% area and 2.25% power in a 16-core system.

6. Comparative results, limitations, and research scope

The literature reports materially different “SpeedMalloc” benefits depending on workload regime and allocator structure (Kenwright, 2022, Li et al., 11 Oct 2025, Li et al., 27 Aug 2025, Aigner et al., 2015, Durner et al., 2019, Hakarsa, 2024).

Design	Context	Reported result
Fixed-size SpeedMalloc	Fixed-size pool vs system malloc	About 10× faster in release standalone runs
Exgen-Malloc	Single-threaded SPEC CPU2017 vs dlmalloc	1.17× geometric mean
Exgen-Malloc	Single-threaded mimalloc-bench vs dlmalloc	1.93× geometric mean
Support-core SpeedMalloc	16-thread geo-mean vs Jemalloc	1.75×
scalloc distributed span-pool	Span-pool scaling stress	Up to 2.7× better performance
scalloc eager reclamation	39-thread reclamation comparison	About 25% lower memory consumption
Head-first best-fit with space-fitting	Best-fit baseline comparison	Approximately 34.86% faster
Allocator choice in analytical DBMS	TPC-DS (SF 100) on 4-socket Xeon	2.7× performance improvement

These results should not be collapsed into a single universal claim. Fixed-size SpeedMalloc is exacting and deterministic, but only for uniform object sizes. Exgen-Malloc improves single-threaded performance and memory usage by removing tiers of metadata and coordination logic, yet it is not thread-safe and can use slightly more memory than dlmalloc and glibc on workloads such as x264 with medium-to-large allocations. The support-core SpeedMalloc removes cache pollution and cross-core synchronization in multi-threaded software, but it requires ISA additions and a hardware support-core, and its gains are smaller on single-threaded workloads. scalloc scales well through virtual spans, a constant-time frontend, and a distributed backend, but allocations larger than 1 MB fall back to mmap/munmap, and the design explicitly benefits from Transparent Huge Pages being disabled. Head-first best-fit improves execution time and preserves low external fragmentation by maintaining the top free region, but it still relies on general free structures and coalescing for non-top requests.

The papers also clarify several common misconceptions. “No memory overhead” in the fixed-size pool refers to per-block overhead $$\text{index} = \frac{(\text{ptr} - \text{base})}{\text{block\_size}}, \qquad \text{capacity} = \left\lfloor \frac{\text{pool\_bytes}}{\text{block\_size}} \right\rfloor,$$0, not to the absence of any global allocator state. “Fast” does not mean “metadata-free”: Exgen-Malloc and scalloc both use compact metadata aggressively, but they aggregate or isolate it to improve locality. Nor does “SpeedMalloc” imply that a single allocator dominates every setting. In the analytical DBMS study, jemalloc was the best overall choice on the tested multi-socket platform, while TCMalloc had excellent memory fairness but poor scalability; this suggests that allocator evaluation must remain workload- and topology-specific rather than slogan-driven.

A plausible synthesis is that SpeedMalloc is less a single allocator than a research agenda. Its recurring principles are constant-time fast paths, metadata placement that avoids conflict with hot user data, reclamation policies that do not destroy locality, and specialization by regime: fixed-size pools for deterministic object lifecycles, single-threaded heaps without thread-scale machinery, and multi-threaded architectures that eliminate cross-core metadata synchronization altogether.