Semantic Lookaside Buffer (SLB) in Aeon

• Semantic Lookaside Buffer (SLB) is a software-managed cache that stores recently accessed semantic locations (Atlas nodes) to expedite hierarchical semantic searches.
• It dramatically reduces retrieval latency by using a small contiguous ring buffer and SIMD-based brute-force scanning, achieving sub-millisecond performance under conversational workloads.
• The design draws an analogy to CPU TLBs by caching semantic entry points, distinguishing itself from conventional OS page caches and enabling predictive prefetching.

The Semantic Lookaside Buffer (SLB) is, in its explicit and named form, a small software-managed cache of recently relevant semantic locations in a long-term memory index. In Aeon, it stores a compact set of recently accessed Atlas nodes so that the next conversational query can often begin from a semantically nearby point instead of re-traversing the full index from the root (Arslan, 14 Jan 2026). In a broader, interpretive sense, the same phrase is useful for mechanisms that cache or consume higher-level structural or contextual metadata rather than only page-by-page translations. Under that broader reading, MESC is an SLB-like design because it exploits contiguity semantics encoded in page tables, whereas PCAX is best understood as a related context-based translation mechanism rather than an SLB by name (Yu et al., 2021, Murthy et al., 2024).

1. Definition and conceptual scope

In Aeon, the SLB is introduced to address the retrieval pathologies of long-horizon LLM systems. The motivating diagnosis is fourfold: quadratic self-attention cost prevents arbitrarily long histories from remaining in-context; Lost in the Middle degrades reasoning as context windows expand; Flat RAG treats memory as an unstructured “bag of vectors”; and this produces Vector Haze, namely retrieval of semantically similar but episodically disjoint facts lacking continuity (Arslan, 14 Jan 2026). The SLB addresses the latency and continuity side of that problem by exploiting conversational locality rather than treating every query as an isolated vector lookup.

Aeon presents the SLB as a systems primitive and makes the analogy to CPU memory systems explicit. Atlas is the persistent structured memory substrate; loading relevant semantic clusters into the SLB is likened to paging; and the SLB is described as “a direct parallel to the Translation Lookaside Buffer (TLB) in CPU architecture” (Arslan, 14 Jan 2026). The analogy is functional rather than literal: embeddings are not addresses in the hardware sense, but repeated semantic lookup in a large hierarchy exhibits locality in a way that resembles repeated address translation.

A common misconception is to equate the SLB with an ordinary OS page cache. Aeon distinguishes them sharply. Atlas is memory-mapped via mmap, so the operating system manages file-backed pages transparently, whereas the SLB is an application-level semantic cache that stores meaningful access points in semantic space, not file pages (Arslan, 14 Jan 2026). This distinction is central to the term: the “lookaside” object is semantic or structural, not merely physical.

2. Position within the Aeon architecture

Aeon uses a Core-Shell model. The Core (C++23) provides low-latency vector search, tree traversal, and memory management; the Shell (Python) provides LLM orchestration, graph or topology management, and higher-level cognition (Arslan, 14 Jan 2026). Within that decomposition, Atlas is the spatial semantic memory kernel, Trace is the episodic or causal DAG, and the SLB is a fast semantic cache in front of Atlas retrieval.

The retrieval path is structurally simple. A new query vector arrives; the system probes the SLB first; if the SLB yields a sufficiently close cached semantic location, retrieval starts there or directly returns its associated Atlas node pointer; otherwise the system performs the normal Atlas search from the root; and the resulting node is inserted into the SLB for future turns (Arslan, 14 Jan 2026). The SLB is therefore neither a replacement for Atlas nor a cache of final answer strings. It is a fast path that short-circuits or shortens hierarchical semantic search.

The cached unit is specified exactly as

$e_k = \{ \mathbf{c}_{node}, \text{ptr}_{atlas} \},$

where $\mathbf{c}_{node}$ is the centroid or representative vector of a cached Atlas node and $\text{ptr}_{atlas}$ is a direct pointer or offset to that node in the memory-mapped Atlas structure (Arslan, 14 Jan 2026). This makes the SLB a cache of Atlas node access points: a semantic representative for similarity testing plus a direct reference into the authoritative memory structure.

The granularity is node-level, not document-level or conversation-level. Aeon sometimes speaks of “semantic clusters” or “relevant semantic clusters,” which suggests that Atlas nodes may correspond to clustered regions rather than only atomic chunks, but the concrete implementation described for the SLB is node-centric (Arslan, 14 Jan 2026). The paper also states what the SLB does not cache: it is not a store of conversation transcripts, latent model states, Python objects, or explicit Trace neighborhoods.

The interaction with Trace is architectural rather than algorithmic. Trace records chronology, causal lineage, and backtracking, while the SLB provides fast semantic entry points into Atlas during ongoing dialogue. The paper does not describe the SLB as caching Trace subgraphs or symbolic paths directly (Arslan, 14 Jan 2026). Similarly, the zero-copy C++/Python bridge does not define the SLB, but it preserves the latency benefits of the SLB by avoiding serialization or copying once Atlas nodes have been located.

3. Locality model and lookup mechanics

The theoretical basis of the SLB is Semantic Inertia. Aeon formalizes it as

$$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$

where $\mathbf{q}_i$ and $\mathbf{q}_{i+1}$ are consecutive query vectors and $\text{dist}$ is cosine distance (Arslan, 14 Jan 2026). The operational claim is that human dialogue tends to evolve gradually, so the best starting point for query $i+1$ is often not the Atlas root but a node retrieved for query $i$.

The buffer organization is correspondingly minimal. The SLB is a software-managed cache implemented as a small contiguous ring buffer of fixed size $K$, “typically $\mathbf{c}_{node}$0,” chosen to fit within L1/L2 CPU cache (Arslan, 14 Jan 2026). Aeon emphasizes this implementation choice because it avoids pointer chasing, benefits from CPU prefetching, and permits a brute-force SIMD scan.

For a query vector $\mathbf{c}_{node}$1, the SLB computes similarity to each cached centroid:

$\mathbf{c}_{node}$2

selects

$\mathbf{c}_{node}$3

and compares the result to a threshold $\mathbf{c}_{node}$4 (Arslan, 14 Jan 2026). On a hit, the associated Atlas pointer is returned immediately. On a miss, the system falls back to Atlas search from the root.

Aeon states that for $\mathbf{c}_{node}$5, a brute-force scan is faster than building another ANN index because the working set is tiny, contiguous, and SIMD-friendly (Arslan, 14 Jan 2026). The hit path is therefore intentionally simple: scan all cached entries, keep the best-scoring slot, and return its pointer if the threshold test succeeds. The evaluation further clarifies that a hit provides “an entry point within the target neighborhood,” after which the system may require only 1–2 additional hops to reach the optimal node.

The miss path falls back to Atlas, whose retrieval is described as Greedy SIMD Descent. For a current node $\mathbf{c}_{node}$6 with children $\mathbf{c}_{node}$7, the score is

$\mathbf{c}_{node}$8

and the next node is

$\mathbf{c}_{node}$9

repeating until a local optimum or leaf is reached (Arslan, 14 Jan 2026). The stated complexity is

$\text{ptr}_{atlas}$0

where $\text{ptr}_{atlas}$1 is the effective branching factor and $\text{ptr}_{atlas}$2 is the total number of Atlas nodes.

Admission is straightforward: every query result, whether obtained through an SLB hit or an Atlas miss-then-search, is inserted into the SLB (Arslan, 14 Jan 2026). Replacement is described as a simplified Least Recently Used (LRU) eviction policy, implemented “via the ring buffer pointer or a lightweight timestamp.” Aeon also describes predictive prefetching as an architectural capability and future optimization: after a successful SLB hit, the system can speculatively load children of the hit node into the SLB during idle periods. The paper presents this as a natural extension enabled by the hierarchy, not as a fully benchmarked current mechanism.

4. Empirical behavior and systems properties

Aeon evaluates the SLB on an Apple M4 Max workstation using synthetic 768-d vectors and datasets of up to $\text{ptr}_{atlas}$3 nodes. The relevant workloads are Uniform Random, described as low locality and worst-case for caching, and Conversational Walk, a random walk on the semantic graph intended to mimic dialogue with semantic continuity (Arslan, 14 Jan 2026). The comparison set includes Flat search, HNSW baseline (FAISS), Aeon Cold (Atlas without SLB), and Aeon Warm (Atlas with SLB).

Metric	Result	Context
SLB hit rate	$\text{ptr}_{atlas}$4	Conversational Walk
Hit latency	$\text{ptr}_{atlas}$5 ms	SLB hit, nearby entry point
Miss latency	$\text{ptr}_{atlas}$6 ms	Full Atlas traversal
Effective average latency	$\text{ptr}_{atlas}$7 ms	Weighted 85% hit / 15% miss
HNSW latency	$\text{ptr}_{atlas}$8 ms	Locality-insensitive baseline
Speedup vs HNSW	$\text{ptr}_{atlas}$9 average	Conversational workload
Speedup on hits vs HNSW	$$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$0	$$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$1 ms vs $$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$2 ms
Focused-session latency	under $$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$3 ms	When hit rate approaches 95%

The weighted average reported for conversational retrieval is

$$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$4

which is the paper’s headline demonstration of sub-millisecond retrieval latency under high-locality conversational workloads (Arslan, 14 Jan 2026). The latency distribution is explicitly described as bimodal: roughly 85% of queries fall under 0.1 ms, while the remaining 15% extend to 2.5 ms.

The implementation details matter directly to the SLB’s viability. Aeon reports about 50 ns per 768-dimensional comparison, estimating a 64-entry scan at roughly 3.2 microseconds (Arslan, 14 Jan 2026). The design uses AVX-512 intrinsics, ARM NEON portability through SIMDe on the Apple M4 Max, 4× loop unrolling, explicit _mm_prefetch hints in the math kernel, and a contiguous ring buffer layout to avoid pointer chasing. These details explain why the SLB can rely on brute-force SIMD rather than on another approximate index.

The broader system preserves these gains with a zero-copy bridge between the C++ Core and the Python Shell. Python receives read-only views; the bridge wraps C++ pointers in Python capsules, exposes them as NumPy buffers, and marks them writeable=False (Arslan, 14 Jan 2026). The measured overhead is only 2 microseconds for 10 MB transfer, versus tens of milliseconds for JSON or pickle serialization. This is not an SLB mechanism proper, but it prevents the semantic cache’s latency gains from being erased downstream.

Two precision points matter. First, Aeon defines “cache hit rate” in one part of the paper as the percentage of queries where the SLB provided a valid starting node closer to the target than the root node, while the algorithmic section operationalizes a hit through the threshold test $$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$5 (Arslan, 14 Jan 2026). The two notions are related but not formally reconciled. Second, the paper does not report a separate quantified memory overhead for the SLB, and it does not provide separate ablations for threshold sensitivity, replacement variants, or predictive prefetching.

5. Translation-centric analogues and broader SLB-like designs

Outside Aeon, the term Semantic Lookaside Buffer is not standardized. Two recent address-translation mechanisms are particularly relevant because they substitute structural or contextual metadata for purely address-indexed lookup, even though neither uses the exact term in the same way.

Mechanism	Cached or consumed information	Relation to SLB
Aeon SLB	Atlas node centroids and pointers	Explicit SLB
MESC	Contiguity bits, MSC adjacency bitmap, subregion TLB entries	SLB-like semantic translation coalescing
PCAX	PC-indexed VPN-PTE and metadata	Context-based translation assistance

MESC is proposed for GPU address translation in a shared virtual memory setting and is explicitly described as highly relevant to an SLB-style design if SLB is interpreted broadly as a structure that stores semantic metadata about address-space structure rather than only exact cached translations (Yu et al., 2021). The mechanism divides each 2MB virtual large-page frame into 8 fixed-size subregions of 64 × 4KB pages = 256KB, stores subregion contiguity bits $$P\left( \text{dist}(\mathbf{q}_{i+1}, \mathbf{q}_i) < \epsilon \right) \approx 1,$$6 and an AC bit in unused L2PTE fields, and uses a Memory Subregion Cache (MSC) with a 7-bit bitmap to record adjacency contiguity between neighboring subregions. During a page-table walk, the PTW consumes these semantics to synthesize subregion TLB entries that cover 1 to 8 consecutive subregions, for a maximum of 512 pages = 2MB without requiring large pages or changes to demand paging. For translation-sensitive workloads, MESC reports 77.2% performance improvement, 76.4% reduction in dynamic translation energy, and 93.5% of THP performance. The paper therefore characterizes the design as a page-table-encoded semantic translation coalescing scheme with SLB-like behavior.

PCAX provides a narrower but conceptually related form of context-based translation assistance. Rather than indexing by the data virtual address alone, it uses the program counter (PC) of a load instruction to retrieve a likely-needed page table entry (PTE) before the effective address is fully formed (Murthy et al., 2024). Its central structure, the PC-Indexed Address Translation Table (PCAT), stores tag bits derived from the load PC or instruction-block address, a Valid bit, a Useful bit, an STLBM bit, the VPN, and the corresponding PTE. Access occurs alongside instruction fetch; the result is placed in the Recent PCAT Accesses (RPA) buffer; and at address-generation time the machine checks whether the predicted VPN matches the actual base-register VPN and whether the offset addition crosses a page boundary. The paper reports that a small subset of static loads causes most DTLB misses, that for such problem loads the same PTE is often reused by successive dynamic instances, and that PCAX can cut the effective miss rate of a conventional DTLB by roughly 2–3× in many cases. The SLB connection is therefore interpretive: PCAX is best categorized as PC-correlated translation prediction/caching or context-based translation assistance, not as an SLB by name.

Taken together, these mechanisms show three distinct meanings of “lookaside” behavior. In Aeon, the cached object is a semantic entry point in a memory hierarchy. In MESC, it is contiguity semantics that allow arithmetic reconstruction of multiple base-page translations. In PCAX, it is instruction context that predicts a useful VPN-PTE pair. The commonality is that all three move beyond a simple lookup keyed only by the final address being dereferenced.

6. Limitations, misconceptions, and scope boundaries

The most important limitation of Aeon’s SLB is explicit: it depends on locality. Its strongest results occur under Conversational Walk, where semantic continuity is high; under Uniform Random, semantic locality disappears, so the SLB should be much less effective (Arslan, 14 Jan 2026). Even in the favorable case, misses still incur the full 2.50 ms Atlas path, producing a long tail. The paper also states that predictive prefetching is more an architectural direction than a fully evaluated mechanism, and it does not specify a detailed invalidation algorithm for stale SLB pointers when Atlas is updated. No explicit SLB concurrency model is given.

A related misconception is to treat the term SLB as a settled systems label across the literature. The evidence does not support that reading. Aeon uses the term directly and centrally; MESC is presented as SLB-like only under a broader interpretation; and PCAX is explicitly framed as not a paper about an SLB by name (Arslan, 14 Jan 2026, Yu et al., 2021, Murthy et al., 2024). The scope of the term is therefore partly architectural and partly interpretive.

The limitations of the translation-centric analogues are also specific. MESC depends on naturally occurring contiguity in ordinary 4KB-page allocation, requires OS support to scan page tables and maintain contiguity metadata, assumes unused L2PTE bits are available, and only coalesces structure within a 2MB virtual frame and its fixed 8 subregions (Yu et al., 2021). PCAX works best when a repeated static load often misses in the DTLB yet successive dynamic instances still access the same virtual page; it is less effective when page affinity is unstable, instruction locality is poor, or baseline DTLB miss rate is already low. Correctness is maintained conservatively through PCAT Shootdown, which invalidates the entire PCATs on relevant page-table updates (Murthy et al., 2024).

A plausible implication is that “semantic” in a lookaside design is layer-dependent rather than absolute. In Aeon it means semantic neighborhoods in vector space; in MESC it means contiguity structure latent in the page table; in PCAX it means instruction identity and repeated page affinity. The common design principle is consistent across these systems: expensive authoritative search remains in place, but a small hot structure caches a more informative starting point than a conventional exact-match lookup would provide.