---
title: 'SpeedMalloc: Efficient Memory Allocation'
url: https://www.emergentmind.com/topics/speedmalloc
type: topic
---

# SpeedMalloc: Efficient Memory Allocation

Searching arXiv for the cited allocator papers to ground the article in current preprint metadata.
SpeedMalloc denotes a family of memory-allocation strategies centered on reducing allocator-induced latency, cache disruption, synchronization overhead, and fragmentation. In the cited literature, the name is used both for a fixed-size memory pool manager built around an in-place free list and lazy initialization [2210.16471] and for a multi-threaded allocation architecture that offloads allocator work to a lightweight support-core [2508.20253]. Taken together, related work on single-threaded specialization, scalable multicore allocation, allocator behavior in analytical query engines, and head-first best-fit with space-fitting suggests that “SpeedMalloc” is best understood as a regime-sensitive design objective: make the fast path short, keep metadata from interfering with user data, and shape reuse and reclamation policies to the actual concurrency and size-distribution profile of the workload [2510.10219], [1503.09006], [1905.01135], [2409.03488].

## 1. Performance significance of allocator design

Memory allocation is treated in this literature not as incidental runtime plumbing but as a system-level performance determinant. One line of work frames allocator inefficiency as a “datacenter tax”: memory allocators sit on the hot path of almost every runtime, and even a 1% improvement in allocator efficiency can translate to millions of dollars in savings and measurable reductions in datacenter energy consumption [2510.10219]. A second line shows a “butterfly effect” in multi-threaded systems: allocator-related instructions may account for only around 2–5% of execution, yet allocator choice can produce up to a 2.71× variation across state-of-the-art allocators, with allocator metadata conflicts contributing 28.3% of all cache misses for BFS in TCMalloc with 16 threads and allocator metadata synchronization consuming 22.8% of total cycles in Larson [2508.20253]. A third line demonstrates application-level consequences in analytical DBMS: on a 4-socket Intel Xeon server, the right allocator increased TPC-DS (SF 100) performance by 2.7× [1905.01135].

These results establish a common premise. Allocation policy changes only a small fraction of dynamic instructions, but it reshapes cache occupancy, TLB behavior, remote-memory traffic, lock and atomic pressure, and reclamation timing. A plausible implication is that allocator research should be organized around the dominant bottleneck in each setting—deterministic latency for fixed-size pools, metadata minimization for single-threaded binaries, synchronization elimination for multi-threaded programs, or fragmentation control for best-fit heaps—rather than around a single universal design.

## 2. Recurrent design primitives

Across the papers, several primitives recur. In fixed-size pools, the pool is one contiguous region partitioned into $N$ equal-sized blocks; when a block is free, its first 4 bytes store the index of the next free block, yielding no separate header and no per-block metadata outside the block itself. Pointer/index conversion is direct,
$$
\text{index} = \frac{(\text{ptr} - \text{base})}{\text{block\_size}}, \qquad
\text{capacity} = \left\lfloor \frac{\text{pool\_bytes}}{\text{block\_size}} \right\rfloor,
$$
and allocation/free are $O(1)$. Lazy initialization avoids pre-linking the entire pool and initializes at most one additional block per allocation, eliminating loops from initialization and hot-path allocation [2210.16471].

Single-threaded general-purpose specialization uses a different primitive set. Exgen-Malloc organizes memory into 4 MiB segments, with page organizations of 64 pages per segment for the smallest page type, 8 for a medium type, and 1 for the largest type; very large allocations use huge segments sized to the object. Each page serves one block-size class, uses 8-byte granularity for small sizes, and maintains compact per-page metadata together with a single free-block list. The design removes per-thread/per-core caches, global sharding, multi-list management, locks, atomics, and cross-thread coordination, while retaining fine-grained size classes, aggregated metadata, bitmaps, a page/segment hierarchy, and inline fast paths [2510.10219].

Scalable multicore allocators emphasize frontend/backend separation and distributed reuse. scalloc allocates within 2 MB virtual spans drawn from a 32 TB arena; each virtual span contains a real span sized to the size class, leaving the remainder unbacked under on-demand paging. The frontend uses hot spans, reusable spans, local and remote free lists, while the backend uses a real-span-size–segregated span-pool implemented as distributed Treiber stacks. Empty spans are returned eagerly, and larger real spans can be reclaimed with `madvise(MADV_DONTNEED)` once the real-span size reaches the default `MADVISE_THRESHOLD` of 32 KB [1503.09006].

Best-fit allocators pursue a different balance. Head-first allocation on best-fit with space-fitting privileges the top chunk at the high-address end of an arena, attempts to satisfy allocations from that region first in $O(1)$ time, and otherwise falls back to a best-fit search. Space-fitting donates excess bytes to a free right neighbor first, else to a free left neighbor, and splits only when the remainder is at least $3 \times \text{overhead}$, in order to avoid creating tiny fragments [2409.03488].

Taken together, these mechanisms suggest three broad SpeedMalloc motifs: constant-time object retrieval, metadata placement that minimizes interference with user data, and reclamation policies that trade off locality, fairness, and fragmentation in workload-specific ways.

## 3. Fixed-size pool SpeedMalloc

In "Fast Efficient Fixed-Size Memory Pool: No Loops and No Overhead" [2210.16471], SpeedMalloc is a fixed-size, zero-overhead allocator for uniform objects. The pool state contains `num_blocks`, `block_size`, `num_free`, `num_initialized`, a base pointer, and a head pointer. Initialization allocates contiguous memory for $N$ blocks of size $B$, sets `num_free = N`, `num_initialized = 0`, and `head = base`, but does not pre-link the pool. Each allocation first checks exhaustion, then performs lazy initialization if `num_initialized < num_blocks` by writing the next index into the block being initialized, returns the current `head`, decrements `num_free`, and advances `head` using the stored index. Free computes the block index, prepends the block to the singly linked free list, and increments `num_free`. Reset restores the initial state without touching each block, so the free list is rebuilt on demand.

The central claim of “no memory overhead” is precise rather than absolute. Per-block overhead is $H = 0$: an allocated block has no extra header, and the first 4 bytes are reused for bookkeeping only while the block is free. Global overhead remains a few integers and pointers, typically 32–48 bytes on a 64-bit platform. The hot path touches a constant number of cache lines, avoids iteration, and is therefore suited to time-critical systems such as games.

The reported measurements were run on Windows 7 64-bit with an Intel i7-2600 @ 3.4 GHz and 16 GB RAM, using Visual Studio Release builds, with tested allocation sizes 10, 100, 1000, 10000, and 100000 bytes. In release standalone runs, the fixed-size pool achieved about 10× speedup over system `malloc` across tested sizes; inside the debugger, it was up to approximately 1000× faster, reflecting debugger overhead on `malloc`. The paper expresses speedup and throughput as
$$
S = \frac{T_{\text{malloc}}}{T_{\text{pool}}}, \qquad
\tau = \frac{N}{t}.
$$

Its limitations are equally explicit. The design is fixed-size only, cannot serve requests larger than `block_size` without fallback, requires one contiguous region, and is not thread-safe in its basic form. Optional robustness mechanisms—bounds checks, debug-only bitsets for double-free detection, or guard patterns—remain outside the hot path.

## 4. Single-threaded specialization and Exgen-Malloc

"Old is Gold: Optimizing Single-threaded Applications with Exgen-Malloc" [2510.10219] argues that modern allocators such as jemalloc, tcmalloc, and mimalloc embed costs that are unnecessary in single-threaded programs. Those allocators rely on per-thread or per-core caches, shared and local free lists, synchronization or control logic, and balancing across threads. In single-threaded services and pipelines, these mechanisms become overhead: extra metadata causes cache pollution and false sharing risks, locks and atomics add latency, and fast paths traverse more structures than necessary.

Exgen-Malloc answers this by specializing for single-threaded execution. Its core design features are a centralized heap, a single free-block list per page, and a balanced strategy for memory commitment and relocation. Allocation rounds requests up to the size class, uses fixed-size blocks within a page, and follows a first-fit policy within the page’s singly linked free list. If `page->free` is non-null, the allocator pops the head and returns it in $O(1)$ time; otherwise it obtains a new page from the current segment, then from the segment cache, then from `mmap`, initializes the page by carving it into blocks, and returns the first block. Free writes the current `page->free` into the freed block’s next pointer, updates `page->free`, and increments counters, with no synchronization. Coalescing is unnecessary for fixed-size blocks inside a page, and `calloc` rather than `malloc` performs zero-initialization.

The OS policy is also specialized. The first segment defers committing physical memory page-by-page on demand; when a segment becomes free, it enters a small cache with one cached segment per page type; and if the cache is full, the segment is released via `munmap`. Build-time selection is used to link Exgen-Malloc only for single-threaded binaries, and the implementation is ABI-compatible with `malloc/free/realloc/calloc` and `new/delete`.

Evaluation on two Intel Xeon systems reports a 1.17× geometric-mean speedup over dlmalloc on SPEC CPU2017, 1.10× on `redis-benchmark`, and 1.93× on `mimalloc-bench`. Against mimalloc, the reported speedups are 1.05× on SPEC CPU2017, 1.01× on `redis-benchmark`, and 1.02× on `mimalloc-bench`; memory savings versus mimalloc are 6.2%, 0.1%, and 25.2%, respectively. Microarchitecturally, Exgen-Malloc reduces L1D MPKI by 18.0% versus dlmalloc and 2.9% versus mimalloc, L2 MPKI by 25.2% versus dlmalloc and 9.4% versus mimalloc, LLC MPKI by 24.4% versus dlmalloc and 7.8% versus mimalloc, and DTLB MPKI by 98.6% versus dlmalloc and 87.1% versus mimalloc. The design is explicitly unsupported for multi-threaded programs.

## 5. Support-core SpeedMalloc for multi-threaded allocation

"SpeedMalloc: Improving Multi-threaded Applications via a Lightweight Core for Memory Allocation" [2508.20253] defines SpeedMalloc as a hardware-software co-design for multi-threaded applications. A lightweight, programmable support-core processes all user-level `malloc/free` work, houses allocator metadata in its own caches, and communicates with application cores through four new ISA instructions: `mallocstart()`, `freestart()`, `mallocend()`, and `freeend()`. `mallocstart()` sends a request and waits for an end signal that writes the returned pointer into a destination register; `freestart()` is asynchronous and retires without waiting. The support-core is an AArch64 “little” in-order core with 16 KB 4-way L1d and 16 KB 4-way L1i caches, plus hardware message queues with 128-entry dispatch queues for `malloc()` and `free()` and a 128-entry response queue.

The architectural claim is that application cores should never touch allocator metadata. Metadata for size classes and free lists resides only in the support-core caches, which eliminates cache interference between allocator metadata and user data and removes all cross-core metadata synchronization. Requests and responses travel over the coherent network with an 8-cycle main-to-support-core latency, and the support-core scheduler prioritizes `malloc` requests because they lie on the critical path, serving `free` requests when `malloc` queues are empty. `free` is centralized rather than being redirected into another thread’s cache, eliminating the remote-free handoff problem of conventional multi-threaded allocators.

The reported implementation is transparent for `malloc/free/new/delete`, requires no source changes, and is selected at link time; a post-compilation pass inserts the new instructions and target PCs. No kernel modifications are required, though the ISA extensions and the support-core attached to the coherent fabric are hardware requirements.

In gem5 full-system evaluation with 1–16 big out-of-order AArch64 application cores, SpeedMalloc achieves average 16-thread speedups of 1.75× over Jemalloc, 1.18× over TCMalloc, 1.15× over Mimalloc, 1.23× over Mallacc, and 1.18× over Memento+. The geo-mean speedups versus Jemalloc across 1, 2, 4, 8, and 16 threads are 1.39×, 1.40×, 1.58×, 1.73×, and 1.75×. Example 16-thread results versus Jemalloc include 3.57× in BFS and 3.19× in Larson. L2 miss cycles are reduced by 42.36% versus Jemalloc, 18.76% versus TCMalloc, and 22.80% versus Mimalloc, while atomic and synchronization cycles are eliminated by approximately 3.57% of total time versus Jemalloc, 11.99% versus TCMalloc, and 11.73% versus Mimalloc. Energy savings at 16 threads are reported as 1.69× versus Jemalloc, 1.15× versus TCMalloc, 1.12× versus Mimalloc, 1.26× versus Mallacc, and 1.22× versus Memento+. The support-core itself accounts for approximately 1.50% area and 2.25% power in a 16-core system.

## 6. Comparative results, limitations, and research scope

The literature reports materially different “SpeedMalloc” benefits depending on workload regime and allocator structure [2210.16471], [2510.10219], [2508.20253], [1503.09006], [1905.01135], [2409.03488].

| Design | Context | Reported result |
|---|---|---|
| Fixed-size SpeedMalloc | Fixed-size pool vs system `malloc` | About 10× faster in release standalone runs |
| Exgen-Malloc | Single-threaded SPEC CPU2017 vs dlmalloc | 1.17× geometric mean |
| Exgen-Malloc | Single-threaded `mimalloc-bench` vs dlmalloc | 1.93× geometric mean |
| Support-core SpeedMalloc | 16-thread geo-mean vs Jemalloc | 1.75× |
| scalloc distributed span-pool | Span-pool scaling stress | Up to 2.7× better performance |
| scalloc eager reclamation | 39-thread reclamation comparison | About 25% lower memory consumption |
| Head-first best-fit with space-fitting | Best-fit baseline comparison | Approximately 34.86% faster |
| Allocator choice in analytical DBMS | TPC-DS (SF 100) on 4-socket Xeon | 2.7× performance improvement |

These results should not be collapsed into a single universal claim. Fixed-size SpeedMalloc is exacting and deterministic, but only for uniform object sizes. Exgen-Malloc improves single-threaded performance and memory usage by removing tiers of metadata and coordination logic, yet it is not thread-safe and can use slightly more memory than dlmalloc and glibc on workloads such as `x264` with medium-to-large allocations. The support-core SpeedMalloc removes cache pollution and cross-core synchronization in multi-threaded software, but it requires ISA additions and a hardware support-core, and its gains are smaller on single-threaded workloads. scalloc scales well through virtual spans, a constant-time frontend, and a distributed backend, but allocations larger than 1 MB fall back to `mmap/munmap`, and the design explicitly benefits from Transparent Huge Pages being disabled. Head-first best-fit improves execution time and preserves low external fragmentation by maintaining the top free region, but it still relies on general free structures and coalescing for non-top requests.

The papers also clarify several common misconceptions. “No memory overhead” in the fixed-size pool refers to per-block overhead $H = 0$, not to the absence of any global allocator state. “Fast” does not mean “metadata-free”: Exgen-Malloc and scalloc both use compact metadata aggressively, but they aggregate or isolate it to improve locality. Nor does “SpeedMalloc” imply that a single allocator dominates every setting. In the analytical DBMS study, jemalloc was the best overall choice on the tested multi-socket platform, while TCMalloc had excellent memory fairness but poor scalability; this suggests that allocator evaluation must remain workload- and topology-specific rather than slogan-driven.

A plausible synthesis is that SpeedMalloc is less a single allocator than a research agenda. Its recurring principles are constant-time fast paths, metadata placement that avoids conflict with hot user data, reclamation policies that do not destroy locality, and specialization by regime: fixed-size pools for deterministic object lifecycles, single-threaded heaps without thread-scale machinery, and multi-threaded architectures that eliminate cross-core metadata synchronization altogether.

Source: https://www.emergentmind.com/topics/speedmalloc