MemExchange: Cluster Memory Trading

• MemExchange is a decentralized, cluster-wide memory management system that reallocates cache memory dynamically using RDMA and online Miss Ratio Curve estimation.
• It integrates with Memcached to create a logical aggregated memory pool, reducing rigid tenant quotas and lowering miss rates by up to 63.1%.
• By separating a decentralized control-plane from a fast RDMA data-plane, MemExchange optimizes memory usage and minimizes latency in cloud environments.

Searching arXiv for recent and related papers on MemExchange and adjacent systems. MemExchange is a cluster-wide, multi-tenant memory management system for in-memory caching in cloud environments. It dynamically right-sizes tenants according to workload demand by redistributing idle memory between tenants across physical nodes with RDMA, using marginal-utility-based allocation derived from online Miss Ratio Curve (MRC) estimation. The system is designed to convert dedicated cache memory scattered across servers into a logically aggregated pool, so that remote DRAM can act as a transparent overflow tier without centralized coordination or forced tenant co-location (Seyri et al., 13 Jul 2026).

1. Problem setting and operating model

Cloud providers typically provision memory against peak demand and service-level objectives, while steady-state working sets are often smaller. The result is conservative over-provisioning, rigid per-tenant quotas, and stranded memory capacity across clusters. The measurements discussed for MemExchange cite cluster-wide memory utilization that is typically 40–60%, alongside external reports of around 24% in Alibaba HPC, 42–65% in Alibaba datacenters, and below 60% in Google Cloud. In parallel, memory-constrained tenants can experience elevated miss rates even when unused capacity remains elsewhere in the infrastructure. For cache workloads, this imbalance is consequential because backend miss penalties can be on the order of milliseconds, whereas local-versus-remote DRAM differences are measured in microseconds (Seyri et al., 13 Jul 2026).

The target environment is multi-tenant in-memory caching, specifically Memcached deployments in cloud clusters. MemExchange is organized around three goals: transforming per-tenant, per-node cache DRAM into a logically aggregated pool; redistributing capacity across nodes without centralized coordination or forced co-location; and using RDMA-backed remote memory as an overflow tier while maintaining performance and security isolation. This suggests that MemExchange is not framed as a replacement for caching semantics, but as an adaptive capacity-management layer that changes where cached state resides while preserving the application-level cache abstraction.

2. System architecture and decentralized control

MemExchange separates a decentralized control-plane from an RDMA data-plane. Each server runs a lightweight Tracker process that manages a shared-memory region and mediates local tenant allocation through mmap and munmap. The Tracker maintains metadata including shared memory size, allocated and unallocated regions, per-tenant limits, tenant-reported scores, active remote mappings, and RDMA connections. Tenants are Memcached instances extended with shadow queues for online MRC estimation, marginal-utility scoring, remote pages and items, and RDMA verbs support (Seyri et al., 13 Jul 2026).

The control-plane is implemented through the MemExchange Tracker Communication (MTC) protocol. MTC is event-driven and decentralized: it discovers candidate victim–victor pairs, coordinates page reallocation, ensures atomicity of each transfer, and handles timeouts and retries. The data-plane then serves or places cache items in remote memory using one-sided RDMA READ and WRITE operations, without involving remote CPUs. Persistent RDMA connections are maintained between tenants, and remote memory regions are accessed through address and rkey metadata communicated during setup. A central architectural distinction is that logical aggregation does not require tenant co-location on the same host; tenants may borrow memory from any server in the cluster (Seyri et al., 13 Jul 2026).

This organization differentiates control from fast-path access. The control-plane decides when and where to trade memory, while the data-plane turns the resulting remote allocations into an operational overflow tier. A plausible implication is that MemExchange treats cross-node remote memory not as a general-purpose shared address space, but as a coordinated extension of tenant-specific cache capacity.

3. Utility model and online Miss Ratio Curve estimation

MemExchange bases its allocation logic on marginal utility derived from online Miss Ratio Curve estimation. For tenant $i$, the MRC is denoted $m_i(B)$, the miss rate as a function of cache size $B$. Marginal utility is defined as the slope of miss improvement per unit memory,

$MU_i(B) = -\frac{d\,m_i(B)}{dB},$

with the discrete approximation

$$MU_i(B) \approx \frac{m_i(B) - m_i(B + \Delta)}{\Delta}.$$

The conceptual allocation objective is to choose per-tenant allocations $B_i$ to maximize total utility subject to total cluster memory capacity, or equivalently to minimize weighted miss cost under the same capacity constraint. The paper describes the resulting intuition as a water-filling process: pages are reallocated toward tenants with the highest marginal utility until marginal utilities balance or tenants saturate (Seyri et al., 13 Jul 2026).

In practice, Trackers do not exchange full MRCs. Tenants compute scalar scores locally and share only the final score:

$$S_1 = S_0 \cdot \left(\frac{N_r}{N_g}\right) \cdot MR_{ins} \cdot t \cdot \frac{1}{M}.$$

Here, $S_0$ is the extreme marginal utility from shadow versus main queues, $N_r/N_g$ is the ratio of cumulative pages released and gained, $MR_{ins}$ is recent per-second miss rate, $m_i(B)$0 encodes temporal recency, and $m_i(B)$1 is current allocation. High $m_i(B)$2 indicates a likely victor tenant; low $m_i(B)$3 indicates a likely victim (Seyri et al., 13 Jul 2026).

Online estimation is built around shadow queues. When an item is evicted from the main LRU, a lightweight shadow entry containing key and flags, but no value, is inserted into a shadow queue mirroring the slab page layout. On a main-cache miss, a shadow hit indicates that the item would have hit with a larger cache, and its position encodes the extra memory required. Counters are maintained over reuse-distance intervals, with segmented LRU per slab class and parallel shadow pages, to estimate the local slope of $m_i(B)$4. Because shadow entries omit payloads, the estimation path is designed to remain lightweight and adaptive (Seyri et al., 13 Jul 2026).

4. Memory trading protocol and RDMA data path

MTC proceeds in three phases. In candidate discovery, an initiating Tracker sends a UDP multicast memory request containing a tenant identifier, score, and message identifier. Peer Trackers reply by unicast with their best victim candidate and lowest score. In victim selection and notification, the initiator collects responses within a bounded window, chooses a victim—preferring local candidates if scores are comparable—and notifies the selected Tracker, which acknowledges. In RDMA setup, both sides reuse or establish RDMA connections; the victim prepares a page by evicting items, clearing contents, and registering a memory region; then the victim transfers the memory-region metadata to the victor through a two-sided RDMA SEND. Thereafter, the victor accesses the page through one-sided RDMA READ and WRITE operations, and the remote CPU is not involved in the steady-state data path (Seyri et al., 13 Jul 2026).

The implementation uses one-sided RDMA READ on the GET path and one-sided RDMA WRITE on the SET path; ATOMICS are not required. Queue pairs and completion queues are established once and reused through persistent connections indexed by network endpoint. The victim registers pages for RDMA, producing lkey and rkey; the victor stores remote page mappings and per-item address offsets. The victor also maintains a small 2 MB registered region split between reads and writes so that threads can use disjoint buffers during thousands of concurrent operations (Seyri et al., 13 Jul 2026).

This design is explicitly page-granular and atomic. Reallocation completes before a newly chosen victor can preempt the same page, buffers are non-overlapping, and completion is enforced before reuse. Lost UDP messages are handled by timeout and retry, while message identifiers avoid confusion. Tracker crashes stop new cross-node reallocations only for local tenants; existing mappings remain valid. Victim crashes cause future RDMA hits to fail, after which the victor treats the access as a miss and refills through the normal path. Victor crashes can leave orphaned remote pages registered, and automatic reclamation is left as future work (Seyri et al., 13 Jul 2026).

5. Memcached integration and empirical results

MemExchange is implemented in Memcached. Tenants allocate from a Tracker-managed shared region, while preserving Memcached’s slab and page layout, 1 MB pages, per-class chunks, and segmented LRU structure. The system distinguishes local and remote items through two hash tables: a local table and a remote table. GET probes the local table first and then the remote table, issuing an RDMA READ on a remote hit. SET places data locally when possible, spills into remote slots through asynchronous RDMA WRITE when local space is full, and only evicts local cold items when both local and remote slots are full. Remote items are treated as overflow capacity so that hot local data remain protected (Seyri et al., 13 Jul 2026).

Evaluation spans microbenchmarks, medium-scale experiments, and rack-scale deployment on up to 100 CloudLab servers. In a two-server microbenchmark with a 1,074 MB working set and a victor holding 64 MB locally, local GET median latency was approximately 22 $m_i(B)$5, while RDMA remote GET was approximately 31 $m_i(B)$6; TCP-based remote GET was approximately 73 $m_i(B)$7, and RXE approximately 62 $m_i(B)$8. The reported steady-state result is up to 2.3x lower remote-access overhead for RDMA relative to TCP, with a larger reduction at the $m_i(B)$9 tail. In a resizing microbenchmark, the victor’s average latency decreased from 26.9 $B$0 during active resizing to 23.9 $B$1 after convergence, while throughput remained around 25K requests per second and control-plane UDP traffic dropped from about 5 packets per second to near zero (Seyri et al., 13 Jul 2026).

In a 10-server experiment with two tenants per server, MemExchange raised the final hit rate on a Twitter Zipf workload to 95.2%, compared with 92.0% for MemSweeper and 86.9% for Memcached; the miss-rate reduction versus Memcached was reported as up to 63.1%. It also reclaimed and reallocated about 10 GB to constrained tenants, corresponding to about a 50% utilization increase in that setting. At rack scale, using 100 servers and 200 tenants over a 12-hour run, MemExchange reallocated 53 GB in total, including 32 GB served through RDMA, increased cluster-wide memory utilization by 13.25%, and brought 99 of 104 under-provisioned tenants to sustained 100% hit rates, with 4 additional tenants above 90% (Seyri et al., 13 Jul 2026).

These results delimit the operating regime in which remote memory is beneficial. The system is presented as especially effective for skewed workloads with a clear hot set and large cold overflow, where modest RDMA latency is outweighed by the reduction in backend misses. Less favorable conditions include uniformly random access, more frequent remote GETs, and tail-sensitive service-level objectives with minimal tolerance for moderate $B$2 inflation.

6. Relation to adjacent systems and terminological ambiguity

MemExchange is positioned against several classes of prior systems. Relative to Memtrade, it is described as fully distributed rather than broker-based, with marginal utility computed from shadow queues and RDMA used as the remote-memory substrate. Relative to Infiniswap, it does not expose remote memory as swap and does not rely on page-fault and block-layer machinery; instead, it right-sizes cache tenants and accesses remote cache state through one-sided RDMA. Relative to MemSweeper, DynaCache, and MemShare, it extends marginal-utility-based resizing beyond a single server by adding cross-node capacity exchange and a distributed coordination mechanism (Seyri et al., 13 Jul 2026).

A common misconception is to equate MemExchange with any page migration or page exchange mechanism. In the hybrid-memory literature, the term is also used descriptively for exchanging or migrating pages across DRAM and non-volatile memory tiers. The MNEME framework, for example, exploits inter-memory and intra-memory asymmetries in hybrid DRAM–NVM systems, combines predictive initial placement with channel-free intra-memory migration, and is described as enabling “MemExchange” in that narrower sense of page movement across tiers rather than cloud-scale tenant memory trading (Song et al., 2020). This distinction is important because the named system MemExchange refers specifically to decentralized, cluster-wide memory trading for multi-tenant caches, whereas MNEME addresses OS- and architecture-level placement and migration inside hybrid tiered memories.

The two lines of work nevertheless share a structural theme: both seek to reassign memory capacity toward accesses with higher utility while avoiding unnecessary movement overhead. This suggests a broader interpretation of memory exchange as a resource-allocation problem spanning device tiers, hosts, and tenants, but the mechanisms and operating assumptions differ materially between hybrid-memory page migration and RDMA-based cloud cache resizing.