---
title: 'NetDAM: Network Direct Attached Memory'
url: https://www.emergentmind.com/topics/netdam
type: topic
---

# NetDAM: Network Direct Attached Memory

Searching arXiv for the primary NetDAM paper and closely related terms to ground the article.
NetDAM, short for **Network Direct Attached Memory**, is a **packet-based memory-and-compute architecture** in which **DRAM/HBM is directly attached to the Ethernet controller** and augmented with **programmable logic**, so that the resulting device can function both as a **memory pool / memory disaggregation node** and as a **programmable in-memory / in-network compute endpoint** [2110.14902]. The design is motivated by data-intensive workloads, especially **distributed AI training**, which are described as requiring **multi-terabytes memory capacity with multi-terabits bandwidth**. NetDAM is presented as a **software and hardware friendly programmable architecture** that seeks to bridge intra-host interconnects such as **PCIe, CXL, AXI, CHI** and inter-host networking based on **Ethernet/IP/UDP**, while avoiding the overheads that arise when either domain’s semantics are extended directly into the other [2110.14902].

## 1. Architectural position and design rationale

NetDAM is introduced as a response to a systems boundary problem: **intra-host links** such as **PCIe/CXL/AXI/CHI** are optimized for **shared-memory style access, low latency, and cache-coherent communication**, whereas **inter-host Ethernet/RDMA** is optimized for **packet/message transport**, with associated **congestion, ordering, loss, and coherence complications** [2110.14902]. The paper argues that neither simply exporting intra-host protocols over the network nor importing network semantics into the host yields a clean unification. The former introduces **congestion/coherence overhead**; the latter can **reduce programming flexibility and increase complexity**.

Within that framing, NetDAM is positioned as a **“Dam” at the barrier of the host**: a **unified memory-access layer** that can absorb bursts and transition between **batch-mode large flits** for message passing and **small flits** for coherent or shared-memory-style access [2110.14902]. This suggests that NetDAM is not merely a NIC enhancement, but an attempt to define an intermediate systems layer with explicit protocol, memory, and execution semantics.

The architecture is described as a **hardware template** rather than a fixed-function network endpoint. Its intended scope includes **memory pooling**, **in-memory computing**, **in-network computing**, and **DPU-style offloads** [2110.14902]. A plausible implication is that the paper treats memory disaggregation and communication offload not as separate accelerators, but as specializations of one packetized memory-and-compute substrate.

## 2. Core organization: directly attached memory and programmable logic

The central architectural move is explicit: **“We directly attach the memory to the ethernet controller with some programable logic...”** [2110.14902]. In NetDAM, the Ethernet-facing device contains **attached DRAM/HBM**, **programmable logic**, **memory queues**, and the ability to execute **memory operations** and **simple compute operations close to memory**. The stated intent is to reduce **CPU-mediated DMA overhead**, **PCIe overhead**, and **cache-coherence traffic** [2110.14902].

The programmable logic interprets NetDAM instructions and supports basic memory operations such as **READ**, **WRITE**, **CAS**, and **MEMCOPY**, as well as **SIMD-style arithmetic** and user-defined extensions [2110.14902]. The same logic also accommodates **reorder logic**, **security logic**, and **routing-related functions**. The paper further lists application-specific offloads including **reduce-scatter**, **all-gather**, **compression**, **crypto**, **hash**, and **longest-prefix match** [2110.14902]. This programmable substrate is what allows NetDAM to be described simultaneously as a memory node and a compute endpoint.

NetDAM also defines a connectivity model in which **CPU / accelerator / storage components** attach via **AXI, CHI, or PCIe/CXL** and share a **unified memory pool** [2110.14902]. Multiple NetDAM devices connected through a switch can form a **global memory pool** with **multi-terabytes memory capacity** and **multi-terabits bandwidth** [2110.14902]. The pool can be accessed through a **translation layer/IOMMU-like mapping**, and a device may operate either in standalone mode or as part of a switched fabric.

The architecture explicitly supports both **in-memory computing** and **in-network computing**. In-memory operations execute on or near the attached memory via on-device ALUs, while in-network behavior is enabled through **packet forwarding/routing** and **chained computation** driven by **segment routing** and packet instructions [2110.14902]. The paper emphasizes that the instruction field and routing model enable **dataflow-like computation over multiple nodes**.

## 3. Packet model, transport semantics, and programmable ISA

NetDAM is defined as a **packet-based protocol** in which packets carry both **instructions** and **data** [2110.14902]. The packet format contains the following fields:

| Field | Role |
|---|---|
| Sequence | Ordering and reliable transmit |
| Segment Routing Header | Multipath routing and chained computation across nodes |
| Instruction | READ, WRITE, CAS, MEMCOPY, and user-defined extensions |
| Address | Data memory address for the operation |
| Data | Variable-length payload |

A distinctive property of the protocol is payload size. The paper states that the data field can be **9000B**, enabling SIMD processing across approximately **2048 × float32** values in parallel [2110.14902]. This is used to motivate throughput-oriented fused memory/compute execution close to the network-attached memory.

For inter-host communication, NetDAM uses **IP/UDP over Ethernet** [2110.14902]. The transport discussion emphasizes four properties. First, **deterministic latency** is attributed to a **fixed pipeline** and the elimination of **PCIe DMA** and **cache snooping**. Second, **reliable transmit** is available as an option. Third, **relaxed ordering** is allowed for **commutative/idempotent operations**. Fourth, **multi-path** is supported through the **Segment Routing Header in UDP** [2110.14902]. The reported wire-to-wire SIMD read of **32 × float32** from DRAM measured **618 ns** average latency, **39 ns** jitter, and **920 ns** maximum latency [2110.14902].

The instruction model is described as a kind of **remote procedure call (RPC)** suitable for hardware/software-friendly programming [2110.14902]. NetDAM exposes a **Request Queue** and a **Complete Queue**. Software may either write a NetDAM packet to the Request Queue memory address and fetch completion from the Complete Queue, or send the packet directly over a UDP socket to the NetDAM device for inter-host communication [2110.14902]. The programming flow is described as: packet placement or UDP send, packet parsing, instruction dispatch, memory address resolution, optional SIMD ALU processing, result writeback and/or forwarding, and completion notification via the Complete Queue or ACK [2110.14902].

The basic instruction template includes **READ**, **WRITE**, **MEMCOPY**, and **Atomic** operations including **CAS**, all in **SIMD mode** [2110.14902]. Reserved bits in the instruction field permit user-defined operations such as **ADD, SUB, MUL, XOR, MIN, MAX**, along with collective operations such as **Reduce-Scatter** and **All-Gather**, and DPU-style functions such as **compress, crypto, hash, longest prefix match** [2110.14902]. The ISA is therefore explicitly extensible rather than closed.

## 4. Addressing, memory pooling, virtualization, and security

Each NetDAM device has its own memory address space and can map into the intra-host network [2110.14902]. The paper describes a local **IOMMU** for **virtual/physical translation**, including the ability to **map remote memory to local virtual address** and to reserve a special address pool for **Request** and **Complete Queues** [2110.14902]. This is presented as making the architecture suitable for **virtualization** and **VM/container deployments**.

At the fabric level, the translation function can convert a **Global Virtual Address** to a **NetDAM device IP address with NetDAM Local Address**, and this translation can be implemented on a **programmable switch** [2110.14902]. Within the memory-pooling story, the paper also proposes **block interleaved mode** as a mechanism to avoid incast. In this mode, **many-to-one communication is load-balanced across multiple NetDAM devices**, after which the receiver **pulls data back using rate-limited READ commands** [2110.14902]. The text states that this avoids incast **without complex congestion control**.

Security is treated as an important but only partially elaborated design area. Because memory is shared and instructions are remotely executed, the paper suggests that an **SDN controller can act like an MMU**, applying **malloc/free**, **access control lists**, and policy attachment to each NetDAM or switch [2110.14902]. It also proposes **encryption-write** and **decryption-read** instructions for secure computing [2110.14902]. These are presented as architectural possibilities rather than a fully evaluated security subsystem.

A recurring semantic theme is that NetDAM is most naturally suited to operations that can exploit **idempotence**, **relaxed ordering**, and **SIMD-style fused memory/compute execution** [2110.14902]. This matters because several of the claimed transport and offload advantages rely on those assumptions.

## 5. FPGA prototype and the MPI_Allreduce case study

The paper reports an **FPGA prototype** built from **2 × Xilinx Alveo U55N FPGA blades**, with **2 independent NetDAM devices** per blade [2110.14902]. Each NetDAM device includes **one 100G Ethernet port** and **2 GB HBM memory**, giving a total of **4 NetDAM devices** connected to a **Cisco Nexus 93180FX switch** [2110.14902]. For baseline comparison, the authors also built a **4-node RoCEv2 platform** with **Mellanox CX516A NIC**, **2 × Intel Xeon Gold 6230R CPU**, **12 × DDR4 32G @ 2933 MHz DRAM**, and **Mellanox HPC-X and OFED** installed [2110.14902].

The principal case study is **MPI Allreduce**, described as a critical bottleneck for distributed AI training [2110.14902]. NetDAM implements **Ring Reduce-Scatter instruction** and **Ring All-Gather instruction** in its programmable ISA, corresponding to the ring-allreduce algorithm used in distributed training [2110.14902]. For reduce-scatter, the paper describes a chain in which Node1 sends \(A_1\) to Node2, Node2 computes \(A_1 + B_1\) and sends to Node3, Node3 computes \(A_1 + B_1 + C_1\) and sends to Node4, and Node4 writes \(A_1 + B_1 + C_1 + D_1\) into local memory [2110.14902]. In the NetDAM implementation, data is fetched directly from DRAM, placed in packet buffer SRAM, reduced by on-device ALUs, and forwarded using the Segment Routing Header [2110.14902].

The contrast with RoCEv2 is explicit. The paper states that RoCEv2 requires **multiple DMA operations**, **multiple load/store operations on CPU**, **temporary sums in separate memory space**, and **synchronization barriers between iterations**, whereas NetDAM fuses reduction with data movement inside the memory/network device [2110.14902]. It also states that while a traditional CPU with **AVX-512** may support **32 × float32** adds per cycle, NetDAM can use multiple ALUs to support **2048 × float32** adds with one instruction [2110.14902].

The Allreduce experiment uses **536,870,912 × float32** elements [2110.14902]. Reported times are **2.8 seconds** for **native MPI Allreduce**, **2.1 seconds** for **ring-based allreduce**, and **“xxx microseconds”** for **NetDAM**, accompanied by the note that the **initial testing result is 400ms**, that the authors were **fixing some bug**, and that it **might be even faster** [2110.14902]. This is one of the paper’s most notable caveats: the intended performance claim is clear, but the final prototype number is explicitly not cleanly finalized in the text.

## 6. Idempotence, limitations, and broader significance

The Allreduce case also exposes a subtle point about NetDAM’s execution semantics. The paper notes that **reduce-scatter is idempotent for intermediate nodes** because computation modifies the **packet buffer** without side effects on local memory, but that the **last node writes to local memory**, which is **not idempotent** [2110.14902]. To address this, NetDAM introduces a **block-hash instruction**: each block may contain **2048 × float32**, the hash is carried with the packet, and the destination writes only if the local memory hash matches the instruction hash; otherwise the packet is dropped [2110.14902]. This is described as supporting error handling and last-hop idempotence.

The paper’s stated contributions are architectural and demonstrative rather than standardizing. It claims: **a new architectural template**, **unified support for memory pooling and compute**, **a programmable instruction set**, **a packet format with segment routing**, **a prototype FPGA implementation**, and **an MPI_Allreduce case study** [2110.14902]. At the same time, it explicitly describes the ISA and architecture as a **template/prototype**, not a fully standardized system [2110.14902].

Several limitations are identified in the text itself. The reported results are **preliminary**; some of the strongest performance claims remain in-progress, including the unresolved NetDAM allreduce timing [2110.14902]. The protocol depends on substantial hardware support, including **FPGA/programmable logic**, **attached memory**, and routing support for **segment routing** and optional **IOMMU-like translation** [2110.14902]. Security, ordering, reliable transmit, and virtualization are discussed, but **not fully evaluated** [2110.14902]. The architecture also assumes workloads that can benefit from **relaxed ordering**, **idempotent operations**, and **SIMD-style fused memory/compute execution** [2110.14902].

Within those constraints, NetDAM’s overarching implication is to make memory itself a **networked, programmable, poolable resource** rather than something managed only through host-centric DMA and cache-coherent abstractions [2110.14902]. The paper’s central claim is that future data-intensive systems, especially distributed AI training, need a device that can simultaneously provide **high-capacity memory**, **extremely high bandwidth**, **deterministic low-latency transport**, and a **programmable ISA** for offloading both communication and computation [2110.14902]. This suggests a convergence of memory disaggregation, near-memory computing, and network offload into one packet-executed substrate.

## 7. Terminological scope and disambiguation

The name “NetDAM” is not unique across the broader literature summarized alongside this topic. Distinct works associate similar naming with **darknet traffic analysis**, including a framework for characterizing Internet background radiation using deep representation learning, clustering, interpretable decision trees, and EMD-based change detection [2108.00079], and with a **darknet traffic analysis and network management framework** centered on **Weight Agnostic Neural Networks**, **Predictive Power Score**, and **SHAP** for real-time malicious intent detection [2102.08411]. Another separate line uses damper-based terminology for **Large-scale Deterministic IP Networks**, focusing on **bounded end-to-end delay** and **bounded jitter** via **dampers** and **gate-controlled queues** [2209.12052].

These usages are conceptually distinct from **Network Direct Attached Memory**. The NetDAM of [2110.14902] is a **memory-and-compute architecture** based on **directly attaching memory to the Ethernet controller** and executing packet-carried memory/compute instructions near the memory substrate. The presence of acronym overlap in adjacent literature suggests a naming collision rather than a shared technical lineage.

Source: https://www.emergentmind.com/topics/netdam