Large-Granularity Memory Systems (LGMS)

• LGMS is a memory organization concept where storage is managed in larger units, such as huge pages or entire trajectories, to reduce translation overhead and maintain semantic fidelity.
• LGMS architectures leverage hardware and software innovations, including huge pages, modular memory systems, and adaptive page migration, to address scaling, signaling, and persistence challenges.
• Research on LGMS demonstrates improved performance metrics, with strategies like multi-tiered management yielding up to 42% gains over traditional fine-grained memory solutions.

Large-Granularity Memory Systems (LGMS) are memory organizations in which the effective unit of storage, placement, transfer, migration, retrieval, or reasoning is deliberately larger than a conventional fine-grained unit. In systems research, the relevant unit may be a 2 MB superpage, a 1 GB page, a DRAM row, a tile-major block, or a compiler-visible remote-memory object; in LLM and agent research, it may be an entire trajectory, session, episode summary, topic track, or user profile. Across these literatures, LGMS arise when fine-grained mechanisms either impose excessive translation, control, or I/O overheads, or fail to preserve higher-level semantic structure (Ren et al., 2023, Ju et al., 4 Jul 2026, Zhang et al., 21 Dec 2025, Xu et al., 7 Jul 2026).

1. Scope and terminology

The term LGMS is used in at least two closely related ways in the cited literature. One usage concerns hardware and operating-system memory systems whose efficient access granularity is intrinsically large. Examples include multi-tiered memories with huge pages in the 2 MB or 1 GB range, row-granularity DRAM interfaces, and GPU-attached memories whose minimum access granularity is measured in kilobytes (Ren et al., 2023, Ram et al., 2020, Nam et al., 1 Dec 2025). A second usage appears in agent memory systems, where the stored unit is not limited to a single transition or turn, but can include entire trajectories, plans, reusable tools, topic tracks, summaries, and profiles (Zhang et al., 21 Dec 2025, Xu et al., 26 May 2025, Xu et al., 7 Jul 2026).

These usages share a common structural feature: the memory unit is chosen to match either the underlying hardware transfer size or the semantic structure of the workload. In MemEvolve, large-granularity memory systems can store “entire trajectories, complex plans, tool APIs, reasoning templates, and higher-level procedural knowledge,” formalized through a modular memory system

$$\Omega = (\mathcal{E}, \mathcal{U}, \mathcal{R}, \mathcal{G}),$$

with encode, store, retrieve, and manage components (Zhang et al., 21 Dec 2025). In MemGAS, each conversational memory unit stores multiple abstractions of the same session,

$M_i = \{ S_i, T_i, U_i, K_i \} \in \mathcal{M},$

where $S_i$ is the session, $T_i$ the turns, $U_i$ a summary, and $K_i$ keywords (Xu et al., 26 May 2025).

This dual usage matters because it prevents a narrow interpretation of LGMS as merely “very large memory capacity.” In the cited work, granularity is a first-class design variable, affecting translation overhead, migration cost, bandwidth utilization, semantic fidelity, and the control policies needed to operate the system.

2. Architectural motivations: scaling, signaling, and persistence

Several system papers argue that LGMS cannot be understood solely through the lens of capacity expansion. “The Future of Memory: Limits and Opportunities” states that proposed architectures with huge shared memories among large numbers of CPUs are limited by two practical engineering challenges: scaling and signaling (Liu et al., 28 Aug 2025). The paper reports that DRAM cost per GB has been flat for over a decade, SRAM cell sizes are no longer shrinking further, and per-core DRAM bandwidth for server processors has been stagnant. It therefore argues against “large, shared, homogenous memories” and proposes “smaller slices more tightly coupled with compute elements,” using 2.5D/3D integration, private local memory, on-package shared memory, and DRAM for large working sets and cold data (Liu et al., 28 Aug 2025).

A concise statement of the signaling problem appears in the energy and bandwidth data reported for different integration technologies:

Integration	Energy/bit	BW/chip
On-die (SRAM)	5 fJ	131 TB/s
Hybrid Bonding	~600 fJ	2.5 TB/s
Microbump (HBM)	~2,000 fJ	1.2 TB/s
C4 Solder Bump	~10,000 fJ	0.1 TB/s

These values quantify the paper’s principle that tighter integration yields higher bandwidth and lower energy, while remote or board-level access rapidly worsens both properties (Liu et al., 28 Aug 2025).

A complementary perspective is given by “The First Principle of Big Memory Systems,” which identifies persistence as “the first principle of big memory systems” and analyzes both vertical and horizontal extensions of the memory hierarchy (Hua, 2023). Vertical extension combines volatile and non-volatile memories within a node; horizontal extension aggregates resources through mechanisms such as RDMA and CXL. The same paper describes “full-stack persistence,” “moving persistence,” and the use of ADR and eADR to move the persistent boundary closer to the CPU (Hua, 2023).

Taken together, these results show that LGMS design is shaped by physical distance, packaging technology, persistence semantics, and software visibility of memory capacities and distances. A plausible implication is that future LGMS will continue to replace the abstraction of a single flat memory with explicitly layered, distance-aware hierarchies.

3. Operating-system and page-management mechanisms

At the OS and virtual-memory level, LGMS research addresses a recurring conflict: large units improve translation efficiency, but small units are often needed for migration, copy-on-write, and hot-data placement. MTM, proposed for multi-tiered large memory systems, is an application-transparent page management system based on three principles: coupling profiling overhead control with the profiling mechanism, building a universal page migration policy for complex multi-tiered memory, and introducing huge page awareness (Ren et al., 2023). MTM dynamically adjusts profiling granularity and frequency to maintain overhead within a user-specified bound, uses adaptive thresholds and utility-based ranking for promotion and demotion across any number of tiers, and performs hierarchical profiling for huge pages so that hot subpages can be selectively migrated while preserving huge-page benefits (Ren et al., 2023). Evaluated on big-data applications with working sets from hundreds of GB to 1 TB, MTM outperforms seven state-of-the-art solutions by up to 42% and by 17% on average (Ren et al., 2023).

Trident