NetDAM: Network Direct Attached Memory

• NetDAM is a packet-based memory and compute architecture that directly attaches DRAM/HBM to an Ethernet controller with programmable logic, enabling efficient memory disaggregation and in-network computation.
• It bridges intra-host protocols like PCIe and CXL with inter-host networking via Ethernet/IP/UDP to reduce DMA overhead and support high-throughput operations such as MPI_Allreduce.
• Its flexible instruction set and SIMD processing allow offloaded arithmetic, memory pooling, and security functions, making it ideal for distributed AI training and other data-intensive workloads.

Searching arXiv for the primary NetDAM paper and closely related terms to ground the article. NetDAM, short for Network Direct Attached Memory, is a packet-based memory-and-compute architecture in which DRAM/HBM is directly attached to the Ethernet controller and augmented with programmable logic, so that the resulting device can function both as a memory pool / memory disaggregation node and as a programmable in-memory / in-network compute endpoint (Fang et al., 2021). The design is motivated by data-intensive workloads, especially distributed AI training, which are described as requiring multi-terabytes memory capacity with multi-terabits bandwidth. NetDAM is presented as a software and hardware friendly programmable architecture that seeks to bridge intra-host interconnects such as PCIe, CXL, AXI, CHI and inter-host networking based on Ethernet/IP/UDP, while avoiding the overheads that arise when either domain’s semantics are extended directly into the other (Fang et al., 2021).

1. Architectural position and design rationale

NetDAM is introduced as a response to a systems boundary problem: intra-host links such as PCIe/CXL/AXI/CHI are optimized for shared-memory style access, low latency, and cache-coherent communication, whereas inter-host Ethernet/RDMA is optimized for packet/message transport, with associated congestion, ordering, loss, and coherence complications (Fang et al., 2021). The paper argues that neither simply exporting intra-host protocols over the network nor importing network semantics into the host yields a clean unification. The former introduces congestion/coherence overhead; the latter can reduce programming flexibility and increase complexity.

Within that framing, NetDAM is positioned as a “Dam” at the barrier of the host: a unified memory-access layer that can absorb bursts and transition between batch-mode large flits for message passing and small flits for coherent or shared-memory-style access (Fang et al., 2021). This suggests that NetDAM is not merely a NIC enhancement, but an attempt to define an intermediate systems layer with explicit protocol, memory, and execution semantics.

The architecture is described as a hardware template rather than a fixed-function network endpoint. Its intended scope includes memory pooling, in-memory computing, in-network computing, and DPU-style offloads (Fang et al., 2021). A plausible implication is that the paper treats memory disaggregation and communication offload not as separate accelerators, but as specializations of one packetized memory-and-compute substrate.

2. Core organization: directly attached memory and programmable logic

The central architectural move is explicit: “We directly attach the memory to the ethernet controller with some programable logic...” (Fang et al., 2021). In NetDAM, the Ethernet-facing device contains attached DRAM/HBM, programmable logic, memory queues, and the ability to execute memory operations and simple compute operations close to memory. The stated intent is to reduce CPU-mediated DMA overhead, PCIe overhead, and cache-coherence traffic (Fang et al., 2021).

The programmable logic interprets NetDAM instructions and supports basic memory operations such as READ, WRITE, CAS, and MEMCOPY, as well as SIMD-style arithmetic and user-defined extensions (Fang et al., 2021). The same logic also accommodates reorder logic, security logic, and routing-related functions. The paper further lists application-specific offloads including reduce-scatter, all-gather, compression, crypto, hash, and longest-prefix match (Fang et al., 2021). This programmable substrate is what allows NetDAM to be described simultaneously as a memory node and a compute endpoint.

NetDAM also defines a connectivity model in which CPU / accelerator / storage components attach via AXI, CHI, or PCIe/CXL and share a unified memory pool (Fang et al., 2021). Multiple NetDAM devices connected through a switch can form a global memory pool with multi-terabytes memory capacity and multi-terabits bandwidth (Fang et al., 2021). The pool can be accessed through a translation layer/IOMMU-like mapping, and a device may operate either in standalone mode or as part of a switched fabric.

The architecture explicitly supports both in-memory computing and in-network computing. In-memory operations execute on or near the attached memory via on-device ALUs, while in-network behavior is enabled through packet forwarding/routing and chained computation driven by segment routing and packet instructions (Fang et al., 2021). The paper emphasizes that the instruction field and routing model enable dataflow-like computation over multiple nodes.

3. Packet model, transport semantics, and programmable ISA

NetDAM is defined as a packet-based protocol in which packets carry both instructions and data (Fang et al., 2021). The packet format contains the following fields:

Field	Role
Sequence	Ordering and reliable transmit
Segment Routing Header	Multipath routing and chained computation across nodes
Instruction	READ, WRITE, CAS, MEMCOPY, and user-defined extensions
Address	Data memory address for the operation
Data	Variable-length payload

A distinctive property of the protocol is payload size. The paper states that the data field can be 9000B, enabling SIMD processing across approximately 2048 × float32 values in parallel (Fang et al., 2021). This is used to motivate throughput-oriented fused memory/compute execution close to the network-attached memory.

For inter-host communication, NetDAM uses IP/UDP over Ethernet (Fang et al., 2021). The transport discussion emphasizes four properties. First, deterministic latency is attributed to a fixed pipeline and the elimination of PCIe DMA and cache snooping. Second, reliable transmit is available as an option. Third, relaxed ordering is allowed for commutative/idempotent operations. Fourth, multi-path is supported through the Segment Routing Header in UDP (Fang et al., 2021). The reported wire-to-wire SIMD read of 32 × float32 from DRAM measured 618 ns average latency, 39 ns jitter, and 920 ns maximum latency (Fang et al., 2021).

The instruction model is described as a kind of remote procedure call (RPC) suitable for hardware/software-friendly programming (Fang et al., 2021). NetDAM exposes a Request Queue and a Complete Queue. Software may either write a NetDAM packet to the Request Queue memory address and fetch completion from the Complete Queue, or send the packet directly over a UDP socket to the NetDAM device for inter-host communication (Fang et al., 2021). The programming flow is described as: packet placement or UDP send, packet parsing, instruction dispatch, memory address resolution, optional SIMD ALU processing, result writeback and/or forwarding, and completion notification via the Complete Queue or ACK (Fang et al., 2021).

The basic instruction template includes READ, WRITE, MEMCOPY, and Atomic operations including CAS, all in SIMD mode (Fang et al., 2021). Reserved bits in the instruction field permit user-defined operations such as ADD, SUB, MUL, XOR, MIN, MAX, along with collective operations such as Reduce-Scatter and All-Gather, and DPU-style functions such as compress, crypto, hash, longest prefix match (Fang et al., 2021). The ISA is therefore explicitly extensible rather than closed.

4. Addressing, memory pooling, virtualization, and security

Each NetDAM device has its own memory address space and can map into the intra-host network (Fang et al., 2021). The paper describes a local IOMMU for virtual/physical translation, including the ability to map remote memory to local virtual address and to reserve a special address pool for Request and Complete Queues (Fang et al., 2021). This is presented as making the architecture suitable for virtualization and VM/container deployments.

At the fabric level, the translation function can convert a Global Virtual Address to a NetDAM device IP address with NetDAM Local Address, and this translation can be implemented on a programmable switch (Fang et al., 2021). Within the memory-pooling story, the paper also proposes block interleaved mode as a mechanism to avoid incast. In this mode, many-to-one communication is load-balanced across multiple NetDAM devices, after which the receiver pulls data back using rate-limited READ commands (Fang et al., 2021). The text states that this avoids incast without complex congestion control.

Security is treated as an important but only partially elaborated design area. Because memory is shared and instructions are remotely executed, the paper suggests that an SDN controller can act like an MMU, applying malloc/free, access control lists, and policy attachment to each NetDAM or switch (Fang et al., 2021). It also proposes encryption-write and decryption-read instructions for secure computing (Fang et al., 2021). These are presented as architectural possibilities rather than a fully evaluated security subsystem.

A recurring semantic theme is that NetDAM is most naturally suited to operations that can exploit idempotence, relaxed ordering, and SIMD-style fused memory/compute execution (Fang et al., 2021). This matters because several of the claimed transport and offload advantages rely on those assumptions.

5. FPGA prototype and the MPI_Allreduce case study

The paper reports an FPGA prototype built from 2 × Xilinx Alveo U55N FPGA blades, with 2 independent NetDAM devices per blade (Fang et al., 2021). Each NetDAM device includes one 100G Ethernet port and 2 GB HBM memory, giving a total of 4 NetDAM devices connected to a Cisco Nexus 93180FX switch (Fang et al., 2021). For baseline comparison, the authors also built a 4-node RoCEv2 platform with Mellanox CX516A NIC, 2 × Intel Xeon Gold 6230R CPU, 12 × DDR4 32G @ 2933 MHz DRAM, and Mellanox HPC-X and OFED installed (Fang et al., 2021).

The principal case study is MPI Allreduce, described as a critical bottleneck for distributed AI training (Fang et al., 2021). NetDAM implements Ring Reduce-Scatter instruction and Ring All-Gather instruction in its programmable ISA, corresponding to the ring-allreduce algorithm used in distributed training (Fang et al., 2021). For reduce-scatter, the paper describes a chain in which Node1 sends $A_1$ to Node2, Node2 computes $A_1 + B_1$ and sends to Node3, Node3 computes $A_1 + B_1 + C_1$ and sends to Node4, and Node4 writes $A_1 + B_1 + C_1 + D_1$ into local memory (Fang et al., 2021). In the NetDAM implementation, data is fetched directly from DRAM, placed in packet buffer SRAM, reduced by on-device ALUs, and forwarded using the Segment Routing Header (Fang et al., 2021).

The contrast with RoCEv2 is explicit. The paper states that RoCEv2 requires multiple DMA operations, multiple load/store operations on CPU, temporary sums in separate memory space, and synchronization barriers between iterations, whereas NetDAM fuses reduction with data movement inside the memory/network device (Fang et al., 2021). It also states that while a traditional CPU with AVX-512 may support 32 × float32 adds per cycle, NetDAM can use multiple ALUs to support 2048 × float32 adds with one instruction (Fang et al., 2021).

The Allreduce experiment uses 536,870,912 × float32 elements (Fang et al., 2021). Reported times are 2.8 seconds for native MPI Allreduce, 2.1 seconds for ring-based allreduce, and “xxx microseconds” for NetDAM, accompanied by the note that the initial testing result is 400ms, that the authors were fixing some bug, and that it might be even faster (Fang et al., 2021). This is one of the paper’s most notable caveats: the intended performance claim is clear, but the final prototype number is explicitly not cleanly finalized in the text.

6. Idempotence, limitations, and broader significance

The Allreduce case also exposes a subtle point about NetDAM’s execution semantics. The paper notes that reduce-scatter is idempotent for intermediate nodes because computation modifies the packet buffer without side effects on local memory, but that the last node writes to local memory, which is not idempotent (Fang et al., 2021). To address this, NetDAM introduces a block-hash instruction: each block may contain 2048 × float32, the hash is carried with the packet, and the destination writes only if the local memory hash matches the instruction hash; otherwise the packet is dropped (Fang et al., 2021). This is described as supporting error handling and last-hop idempotence.

The paper’s stated contributions are architectural and demonstrative rather than standardizing. It claims: a new architectural template, unified support for memory pooling and compute, a programmable instruction set, a packet format with segment routing, a prototype FPGA implementation, and an MPI_Allreduce case study (Fang et al., 2021). At the same time, it explicitly describes the ISA and architecture as a template/prototype, not a fully standardized system (Fang et al., 2021).

Several limitations are identified in the text itself. The reported results are preliminary; some of the strongest performance claims remain in-progress, including the unresolved NetDAM allreduce timing (Fang et al., 2021). The protocol depends on substantial hardware support, including FPGA/programmable logic, attached memory, and routing support for segment routing and optional IOMMU-like translation (Fang et al., 2021). Security, ordering, reliable transmit, and virtualization are discussed, but not fully evaluated (Fang et al., 2021). The architecture also assumes workloads that can benefit from relaxed ordering, idempotent operations, and SIMD-style fused memory/compute execution (Fang et al., 2021).

Within those constraints, NetDAM’s overarching implication is to make memory itself a networked, programmable, poolable resource rather than something managed only through host-centric DMA and cache-coherent abstractions (Fang et al., 2021). The paper’s central claim is that future data-intensive systems, especially distributed AI training, need a device that can simultaneously provide high-capacity memory, extremely high bandwidth, deterministic low-latency transport, and a programmable ISA for offloading both communication and computation (Fang et al., 2021). This suggests a convergence of memory disaggregation, near-memory computing, and network offload into one packet-executed substrate.

7. Terminological scope and disambiguation

The name “NetDAM” is not unique across the broader literature summarized alongside this topic. Distinct works associate similar naming with darknet traffic analysis, including a framework for characterizing Internet background radiation using deep representation learning, clustering, interpretable decision trees, and EMD-based change detection (Kallitsis et al., 2021), and with a darknet traffic analysis and network management framework centered on Weight Agnostic Neural Networks, Predictive Power Score, and SHAP for real-time malicious intent detection (Demertzis et al., 2021). Another separate line uses damper-based terminology for Large-scale Deterministic IP Networks, focusing on bounded end-to-end delay and bounded jitter via dampers and gate-controlled queues (Naghmouchi et al., 2022).

These usages are conceptually distinct from Network Direct Attached Memory. The NetDAM of (Fang et al., 2021) is a memory-and-compute architecture based on directly attaching memory to the Ethernet controller and executing packet-carried memory/compute instructions near the memory substrate. The presence of acronym overlap in adjacent literature suggests a naming collision rather than a shared technical lineage.