Remote Access Memory Channels (RAMC)

• Remote Access Memory Channels (RAMC) are persistent, uni-directional channels for one-sided remote memory access that decouple physical placement from logical communication.
• RAMC employs explicit initiator and target roles with bulletin-board discovery and phased synchronization via status values and hardware-visible counters.
• Evaluation indicates RAMC offers higher bandwidth and lower latency for mid-to-large messages on HPE Cray Slingshot, while facing challenges with small message latency.

Remote Access Memory Channels (RAMC) denotes a channel-centric approach to one-sided remote memory access in which the fundamental abstraction is not a monolithic shared window but a persistent, uni-directional communication relation between an initiator and a target buffer. In its explicit published form, RAMC is a communication library built for HPE Cray Slingshot and organized around reusable point-to-point channels, target-side and initiator-side status state, and hardware-visible completion through memory region counters (Schonbein et al., 3 Jun 2026). Its conceptual lineage is older: earlier work proposed treating communication endpoints in parallel machines analogously to virtual addresses in virtual-memory systems, so that virtual communication channel references remain stable even when computing units are allocated, moved, or swapped (Schirmer, 2013).

1. Conceptual foundations and historical lineage

The conceptual precursor to RAMC is the proposal to virtualize communication endpoints rather than shared memory addresses. In that model, the relevant machine is composed of many identical computing units, each with its own local memory, connected through communication channels rather than load/store access to a shared address space. The allocatable resource is therefore a computing unit, not a memory page. The proposed abstraction is a translation from a virtual channel end address to a physical channel end address, directly paralleling virtual-memory translation from virtual addresses to physical memory locations (Schirmer, 2013).

This earlier formulation defines several terms that remain useful for RAMC-oriented systems: channel end address, virtual channel end address, physical channel end address, and interconnect node. The preferred translation architecture is distributed implicit translation, where the upper part of a virtual channel endpoint address identifies the responsible interconnect node and the lower part indexes that node’s local translation table. A connection request therefore follows a stable virtual reference even when the physical endpoint changes. The practical consequence is that connectivity is preserved at the virtual level across allocation and swapping, with translation failure handled analogously to a page-fault path (Schirmer, 2013).

For RAMC, this history matters because it shifts the design center from globally shared memory exposure to stable communication relationships. A plausible implication is that RAMC’s persistent initiator–target channel can be read as an operational specialization of this broader idea: remote access is mediated through a maintained channel relation, while physical placement, registration state, and endpoint discovery are treated as system concerns rather than application-visible addressing details.

2. Channel abstraction and programming model

RAMC, as introduced for Slingshot, is an explicit one-sided communication library whose core abstraction is a persistent uni-directional communication channel between exactly two roles: an initiator and a target. The target exposes a buffer; the initiator issues remote reads or writes against that buffer; and the relation is established once and then reused repeatedly (Schonbein et al., 3 Jun 2026).

The programming model is built around three pieces of state: the target data buffer, a target status value, and an initiator-side local status value. The target status is a user-controlled unsigned integer, and its semantics are explicitly user-defined. The example protocol in the paper uses even values for OK_TO_READ and odd values for OK_TO_WRITE, but that convention is presented only as an example rather than as a fixed semantic requirement. Synchronization is therefore phased rather than collective: both sides advance status values, and operations proceed only when the relevant phase condition is satisfied (Schonbein et al., 3 Jun 2026).

The target-side interface includes ramc_tgt_create_window, posting and activation routines for discovery, synchronization routines such as await_win_ops and test_win_ops, status-management routines such as increment_win_status and set_win_status, and destroy_window. The initiator-side interface includes bulletin-board discovery and state inspection routines such as check_bb_status, get_bb_posting, check_win_status, and get_win_status, together with local status update routines. The communication interface exposes blocking and non-blocking ramc_put and ramc_get operations, plus completion waits for outstanding puts and gets (Schonbein et al., 3 Jun 2026).

This model is explicitly designed to make remote access look more like persistent point-to-point communication without giving up RDMA-style dynamic access. The stated target use cases include halo exchanges, stencil computations, collectives built from repeated peer-to-peer exchanges, and applications that repeatedly access the same peer buffers (Schonbein et al., 3 Jun 2026).

3. Discovery, synchronization, and completion semantics

RAMC avoids collective window creation through a bulletin board (BB) mechanism. Each process allocates a BB and exchanges BB addressing information with all others at initialization using PMI. A target registers its status and data buffers, fills in BB metadata and a tag, marks the BB as active, waits until expected initiators read it, and then deactivates the BB. An initiator reads a target’s BB status and tag; if the BB is active and the tag matches, it reads the target buffer information and can then create or use the channel relation (Schonbein et al., 3 Jun 2026).

The channel’s synchronization logic is deliberately lightweight. The initiator compares its local status with the target status according to the paper’s explicit rule: $$tgt\_status < init\_status,\qquad tgt\_status > init\_status,\qquad tgt\_status = init\_status.$$ In the example interpretation, tgt_status < init_status means the target is behind and the initiator should wait or not write; tgt_status > init_status means the target is ahead and there may be a dangerous mismatch; and tgt_status = init_status means the operation may proceed. check_win_status therefore returns RAMC_SUCCESS if equal, RAMC_AHEAD if target $>$ initiator, and RAMC_BEHIND if target $<$ initiator (Schonbein et al., 3 Jun 2026).

A distinctive feature of RAMC is its use of Slingshot hardware counters for completion. Endpoint counters are used to track completion of local FI_WRITE and FI_READ operations. For writes, an endpoint-counter increment means the source buffer can be reused because the target NIC has acknowledged receipt; for reads, it means the read data is visible to the application. More distinctive are memory region (MR) counters, associated with a registered memory region and able to count the number of remote operations on that region or the number of bytes manipulated. RAMC uses these MR counters as operation counters so that the target can determine that remote writes or reads have occurred without a second explicit notification message or notification write (Schonbein et al., 3 Jun 2026).

For data transfer, RAMC uses fi_inject_write when the message fits the Slingshot inject limit and otherwise uses fi_write; the paper states that the inject threshold is 192B. The example execution flow therefore becomes: initialization, target window creation, BB posting and discovery, status alignment, ramc_put or ramc_get, initiator-side completion wait through endpoint counters, target-side completion wait through the MR counter, and then phase advancement through status updates (Schonbein et al., 3 Jun 2026).

4. Position relative to other one-sided models

RAMC is framed as a response to the limitations of window-centric or monolithic shared-memory one-sided interfaces. The motivating claim is that existing one-sided communication frameworks, such as MPI RMA and OpenSHMEM, rely on monolithic shared memory models that introduce scalability and usability challenges, especially when applications need point-to-point or dynamically evolving communication patterns (Schonbein et al., 3 Jun 2026).

Model	Characteristics stated in the literature	RAMC contrast
MPI RMA	Collectively created windows; group-centric synchronization	Pair-wise channel; BB-based non-blocking setup
OpenSHMEM	Symmetric heap; monolithic shared-memory abstraction	No symmetric heap; granular channel abstraction
PGAS / UPC	Unified local and remote memory view	Explicit communication; RDMA semantics remain visible
MPI partitioned communication	Persistent point-to-point; counts and datatypes; partition boundaries	Byte-addressed buffers; arbitrary offsets; dynamic usage
RAMC	Persistent uni-directional initiator–target relation	Slingshot MR-counter completion and pair-wise synchronization

Against MPI RMA, the paper emphasizes that RAMC avoids an “everyone participates in the same window” model. Its stated differences are point-to-point relation instead of monolithic window, non-blocking channel setup through the bulletin board, a persistent remote-access relation, and hardware-based completion notification through MR counters. The paper also notes that MPI RMA lacks a standardized “RMA with notification” semantic, although such a proposal exists in the MPI ecosystem (Schonbein et al., 3 Jun 2026).

Against OpenSHMEM, the contrast is the absence of a symmetric heap and the refusal to impose a group-wide exposed-memory model. Against PGAS / UPC, RAMC does not present local and remote memory as a unified region; it leaves communication explicit and requires the user to manage synchronization and transfer conditions. Against MPI partitioned communication, the paper stresses that partitioned communication uses counts and datatypes, constrains offsets to partition boundaries, and does not offer a general RDMA-style dynamic memory access model, whereas RAMC works with byte-addressed buffers, arbitrary buffer offsets, and dynamic usage patterns (Schonbein et al., 3 Jun 2026).

The broader significance is methodological. RAMC treats one-sided access as a stable communication relation rather than as an access into a globally exposed memory object. This suggests a re-centering of one-sided programming around explicit channel state, reusable discovery metadata, and hardware completion observability.

5. Experimental evaluation and observed trade-offs

The published RAMC evaluation was carried out on Sandia’s Eldorado supercomputer, described as an HPE Cray EX4000 with 384 AMD MI-300A nodes, 4 Slingshot NICs per node, and 84 cores per node available to user processes after reserving some cores for OS and background use. For MPI comparisons, the study used Cray MPICH 9.0.1. Two libfabric versions were evaluated: libfabric 1.15.2 and libfabric 2.3.1. All microbenchmark results were averaged over ten runs, with standard deviation as error bars (Schonbein et al., 3 Jun 2026).

For application-scale evidence, the paper implemented a heat diffusion code using RAMC for a 5-point stencil. In each iteration, a process updates its interior temperature, increments local status, checks four neighbors, writes when the neighbor is ready, waits for four writes to its own target buffer, and then computes the next step. The reported result is that RAMC had no difficulty scaling this application to 19.6 thousand processes across 250 nodes, with up to 79 processes per node (Schonbein et al., 3 Jun 2026).

The microbenchmark results are split between bandwidth and latency. Under libfabric 1.15.2, RAMC achieved approximately 100% to 130% higher bandwidth than Cray MPI for message sizes from 1B to 4KiB, with the gains diminishing for larger messages and approaching parity by 32KiB. Under libfabric 2.3.1, RAMC still outperformed MPI for small messages, with bandwidth approximately 30% to 45% higher for 1B to 8KiB, while MPI slightly led by about 2% to 4% for 64KiB and 128KiB (Schonbein et al., 3 Jun 2026).

Latency results were more mixed. Under libfabric 1.15.2, MPI beat RAMC by roughly 430–540 ns for 1–64B messages, but by the 192B inject threshold that gap disappeared; for larger messages RAMC showed latency reductions of up to 32% for 16–512KiB messages. Under libfabric 2.3.1, MPI remained faster for small messages by about 480–500 ns, and this advantage extended into medium-sized messages, with MPI averaging about 339 ns faster than RAMC, while RAMC again achieved up to 32% lower latency in the 16–512KiB range (Schonbein et al., 3 Jun 2026).

The paper also evaluated an explicit-notification variant that replaced MR counters with an extra notification buffer and an atomic increment after the payload write. Under 1.15.2, explicit notification was almost indistinguishable from standard RAMC for 1–128B messages, but at 256B latency increased by 86%. Under 2.3.1, the explicit-notification variant incurred about 13–24% extra latency for 1–64KiB messages, and the jump at the inject limit became a 128% increase. These measurements are used to support the claim that Slingshot MR counters provide a lighter notification path than an explicit follow-up notification transfer (Schonbein et al., 3 Jun 2026).

The authors also note several practical limitations: current RAMC is passive target, so initiators poll target status and can generate significant network traffic; endpoint completion counters are less granular than target MR counters; and RAMC does not beat Cray MPI on very small messages. Several tuning attempts, including reducing provider capabilities, matching Cray MPI CXI defaults, disabling receiver-side ordering constraints, and using a low-latency traffic class endpoint, did not improve RAMC latency and sometimes made it worse (Schonbein et al., 3 Jun 2026).

6. Related RAMC-like mechanisms in remote-memory systems

Although the term “RAMC” is explicit only in the Slingshot paper, closely related mechanisms appear in adjacent literatures. A study of distributed memory access on the Ares and Helios HPC clusters compares local memory access, a VFS-backed distributed memory access over shared storage using Lustre and LD_PRELOAD, and an MPI-based one-way remote read mechanism based purely on RDMA. Its stated conclusion is that the MPI-backed approach is similar to local memory access and most closely approximates a high-performance remote memory channel, while the VFS-backed approach is substantially slower and more sensitive to storage and interconnect limits (Meizner et al., 2 Dec 2025).

Remote-memory resilience work extends the RAMC design space from access efficiency to availability. Hydra is a resilience mechanism for remote memory that operates over one-sided RDMA READ/WRITE using reliable connections and maps each remote page onto $(k+r)$ remote slabs with Reed–Solomon erasure coding. It reports single-digit $\mu s$ read/write latency for erasure-coded remote memory, 1.6× lower memory overhead than replication, and a CodingSets placement algorithm that reduces the probability of data loss by about 10× under correlated failures (Lee et al., 2019). This is not a channel library in the RAMC sense, but it shows how a remote memory channel can be made failure-aware without abandoning microsecond-scale access paths.

Fault-tolerance work for Remote Memory Access (RMA) programming models provides the formal resilience side of the same problem. That work argues that resilience for one-sided access is fundamentally different from message-passing resilience because the target is passive, accesses may be asynchronous, and completion depends on synchronization such as flush, unlock, or gsync. It develops a formal RMA execution model, RMA-consistent coordinated checkpointing, transparent logging of remote memory accesses, and transparent recovery of failed processes, with reported checkpointing overheads of about 1–5% in the best configuration and topology-aware failure reductions of 1–3 orders of magnitude in the probability analysis (Besta et al., 2020). For RAMC-like systems, this literature is directly relevant because it treats the remote access itself, rather than a two-sided message, as the object that must be logged and recovered.

A more aggressive line of related work treats the RDMA path not merely as a transport but as a remote execution substrate. RedN uses self-modifying RDMA work-request chains and the verbs CAS, Wait, and Enable to claim that RDMA is Turing complete, enabling conditionals, loops, linked-list traversal, and Memcached key lookup without NIC hardware modification. In the Memcached evaluation, RedN is reported as up to 1.7× faster than a one-sided baseline, 2.6× faster than a two-sided baseline, and with 99th-percentile latency 35× lower under contention (Reda et al., 2021). This does not define RAMC, but it materially extends the idea of a remote memory channel into a programmable offload channel.

7. Terminology, scope, and common sources of confusion

A recurring source of confusion is acronym overlap. In the memory-device coding literature, “RAM channels” may refer to resistive random-access memory (ReRAM) channels rather than Remote Access Memory Channels. Those ReRAM papers study sneak-path interference, selector failures, data shaping, and coding strategies for crossbar arrays. They are concerned with a physical storage channel, not with one-sided remote access on networked systems (Song et al., 2020, Song et al., 2024).

Within distributed-systems research, by contrast, RAMC refers to a channel abstraction for remote memory access over interconnect hardware. The named RAMC system is explicitly built for HPE Cray Slingshot and is defined by persistent initiator–target channels, bulletin-board-based discovery, target and initiator status values, and completion through endpoint counters and memory region counters (Schonbein et al., 3 Jun 2026). Closely related work may discuss distributed memory access, RMA, RDMA offload, or remote memory without using the RAMC term at all. The literature therefore separates into three categories: the explicit RAMC library, precursor channel-virtualization concepts, and adjacent one-sided remote-memory systems that implement RAMC-like behavior under different names.

The resulting research picture is coherent. RAMC is not simply a synonym for generic one-sided communication. It identifies a specific architectural choice: persistent, pair-wise, channel-based remote access with explicit synchronization state and hardware-observable completion. Earlier channel-virtualization work provides a conceptual foundation for stable endpoint references (Schirmer, 2013), while recent system papers supply performance, resilience, and offload mechanisms that clarify where channel-centric remote memory access can outperform shared-window or storage-mediated alternatives (Meizner et al., 2 Dec 2025, Lee et al., 2019, Reda et al., 2021).