Proximal Data Accelerator (PDA)

• PDA is a class of programmable accelerators placed near data-access paths to offload operations constrained by latency and bandwidth, with applications in both near-NIC and near-memory contexts.
• On BlueField‑3, PDA (or DPA) leverages a many‑core RISC‑V fabric to process packets inline, reducing processing latency via proximity to the NIC pipeline.
• In near-memory settings like DX100, PDA reorders and coalesces irregular memory accesses using DRAM-aware policies and compiler co-design to boost effective throughput.

Searching arXiv for the cited papers and related PDA terminology. Proximal Data Accelerator (PDA) denotes a class of programmable accelerators placed proximally to a critical data-access path in order to offload operations that are bottlenecked by latency, bandwidth, or both. In the BlueField‑3 context, PDA is terminologically aligned with the programmable datapath accelerator (DPA), a many‑core on‑path RISC‑V compute fabric tightly coupled to the NIC pipeline and able to access host and Arm resources (Chen et al., 2024). In a broader memory-system context, PDA refers to a programmable, request-level accelerator placed ideally near memory controllers and coherence directories, with visibility over bulk access windows and the ability to reorder, coalesce, and interleave requests while respecting correctness (Khadem et al., 29 May 2025). The term therefore spans two closely related but distinct deployment regimes: near-NIC inline packet processing and near-memory data-access optimization.

1. Conceptual scope and terminology

The term PDA is used in the broader NVIDIA ecosystem and community to emphasize proximity to the NIC’s packet datapath and the intent of near-NIC, inline processing. On BlueField‑3, this use maps exactly to what the paper on DPA-enhanced SmartNICs calls the “programmable datapath accelerator (DPA)” (Chen et al., 2024). Functionally, PDA and DPA refer to the same BF3 many-core on-path compute fabric in that setting.

A second, more general use appears in the context of irregular memory access acceleration. There, a PDA is defined as a programmable, request-level accelerator placed proximally to the data-access path, ideally near memory controllers and coherence directories. Its defining properties are visibility over thousands of future indices or addresses, the ability to reorder, coalesce, and interleave requests across channels and bank-groups, ISA and compiler support for indirect loops and conditionals, and direct coordination with memory and coherence systems to bypass core structural bottlenecks (Khadem et al., 29 May 2025).

These two uses share a common architectural idea: proximity is exploited to convert structural advantages into system-level performance. In the BF3 setting, proximity is to the NIC pipeline; in DX100, proximity is to the memory and coherence path. This suggests that PDA is best understood as an architectural pattern rather than a single fixed implementation.

2. PDA on BlueField‑3: near-NIC on-path compute

On BlueField‑3, the PDA/DPA sits in the network critical path and can process packets inline, contrasting the off-path Arm complex that provides look-aside acceleration (Chen et al., 2024). It is a many-core RISC‑V subsystem, exposed via DOCA as DPA threads. Hardware provides 256 DPA threads, although DOCA v2.5.0 limits concurrent use to 190 threads in the reported experiments. The DPA cores run at 1.8 GHz; the comparison platform uses a host Xeon at 2.5 GHz and BF3 Arm cores at 2.134 GHz.

The cache hierarchy is unusually small and asymmetric. The reported organization is L1D: 1 KB × 256 threads, L1I: 1 KB × 16, L2: 1.5 MB × 1 shared, and L3: 3 MB × 1 shared. DOCA DPA APIs expose event-driven packet processing and memory load/store semantics in DPA threads; applications use DOCA queues and rings for Rx/Tx and rely on NIC steering rules to feed work. The host and Arm side use DPDK with RSS in the reported experiments (Chen et al., 2024).

The memory hierarchy is central to PDA behavior on BF3. No direct DDR is attached to DPA. Instead, a 1‑GB region in Arm DDR5 is carved out for exclusive DPA access as “DPA memory.” DPA can also access host and Arm memory through a near-DPA aperture that converts loads and stores to PCIe transactions. DPA memory accesses traverse the NIC switch and are cached by Arm LLC and DPA L1/L2/L3, whereas aperture-based accesses to host or Arm memory go through DPA L1 only and bypass DPA L2/L3 (Chen et al., 2024).

The networking datapath further differentiates these memory placements. If NetBuf is in DPA memory, NIC places packets directly into DPA L2/L3. If NetBuf is in Arm memory, NIC places packets into Arm L3 via the NIC switch. If NetBuf is in host memory, NIC places packets into host L3 via the NIC switch and PCIe. When NetBuf is in DPA memory, recently received packets preferentially occupy DPA L2, then DPA L3, then Arm L3, then DRAM, and a consistent 128 KB of most recent packets is guaranteed in DPA L2 (Chen et al., 2024).

3. Architectural characteristics on BF3

The BF3 study identifies three unique architectural characteristics of PDA/DPA: computing, networking, and memory (Chen et al., 2024).

The first is a compute profile of many threads but wimpy per-thread performance. Under equal core counts of 16, Arm provides Gops comparable to host. By contrast, with full hardware availability, DPA’s achievable Gops is 7.5× lower than host and 4.7× lower than Arm, while single-thread compute is up to 26× lower than host or Arm. DPA nevertheless scales linearly with thread count for simple operations such as the multiplication microbenchmark. The study attributes the weak per-thread behavior to the 1 KB L1D per thread, a cache hierarchy whose L1 latency is 10.5× the host L1 latency, and lower uncore frequency (Chen et al., 2024).

The second characteristic is networking proximity and direct cache insertion. Because PDA sits on the NIC’s critical path, it has the lowest network latency among BF3 compute units. In the L2 reflector application, DPA→DPA memory yields the lowest RTT, Arm is moderate, and host is highest, consistent with physical proximity: DPA shares the chip with NIC, Arm has one NIC switch hop, and host adds NIC switch plus PCIe. However, this latency advantage degrades as per-packet compute or memory access increases, because DPA memory and single-thread compute are comparatively weak (Chen et al., 2024).

The third characteristic is a memory system with no directly attached DDR and complex cache interactions. DPA memory uses DPA L1/L2/L3 but suffers higher cache latencies and limited bandwidth; host and Arm memory accesses bypass DPA L2/L3 and therefore interact differently with the hierarchy. Among DPA’s options, DPA→Arm memory reads have lower latency than DPA→DPA memory reads despite the latter using DPA L2/L3, because aperture-based loads to Arm memory bypass the high-latency DPA L2/L3. DPA→Host memory is highest latency due to PCIe plus NIC switch traversal (Chen et al., 2024).

These measurements lead to a precise characterization of BF3 PDA as proximity-optimized rather than compute-dominant. A plausible implication is that algorithm selection must be governed less by nominal programmability than by how effectively an application can translate proximity into reduced end-to-end path length.

4. Performance envelopes and programming guidance on BF3

The BF3 characterization proposes three programmer guidelines to unleash PDA/DPA (Chen et al., 2024).

First, offload latency-sensitive and simple workloads to DPA, and use DPA memory for NetBuf. The documented examples include clock synchronization, fast path telemetry updates, header tweaks, and ack reflection with minimal compute or memory touch. The recommended method is to place packet rings in DPA memory so that NIC places payloads into DPA L2/L3, keep the working set tiny, and ensure minimal parsing or compute per packet. In the reported clock synchronization case study, time uncertainty bound $\epsilon$ improves by up to 2.0× on average and 2.3× at p999 versus host or Arm when both client and master run on DPA, especially with NetBuf in DPA memory (Chen et al., 2024).

Second, offload easy-to-parallelize workloads with small working sets to exploit the many-core parallelism of DPA. The examples include stateless packet functions such as L2 reflector, IP header checks, and simple per-packet filtering or aggregation. The guidance is to scale threads aggressively across the 190–256 thread budget, ensure data locality per thread, avoid sharing large footprints, and keep the working set under DPA L2, that is, at most 1.5 MB. The measured outcome is that DPA single-thread throughput is much lower than host or Arm, but with many threads DPA matches host or Arm throughput for 64‑B and 1‑KB NF workloads, subject to memory-placement caveats (Chen et al., 2024).

Third, carefully select memory buffers for DPA applications and combine memory types to boost bandwidth. For NetBuf, the study recommends Arm or host memory to hit high Rx/Tx rates; for state such as AggBuf, DPA or Arm memory is preferred for higher effective bandwidth and cache locality. Splitting DPA threads to simultaneously access different memory types improves aggregate bandwidth up to 2.4× relative to using a single memory type. In key-value aggregation, the best DPA memory combination, “Net‑Arm + Agg‑DPA,” achieves the highest throughput among DPA variants, and the best versus worst DPA memory combination yields speedup up to 4.3× according to $S = T_{optimized} / T_{baseline}$ (Chen et al., 2024).

Several negative results are equally important. Per-thread sequential read/write bandwidth is up to 205× lower than host or Arm; all-thread bandwidth is up to 7.6× lower and below 400 Gbps line rate. Random read bandwidth drops by up to 25× when the working set exceeds DPA L2 at 1.5 MB. With 1‑KB payloads, DPA reaches line rate only when NetBuf is in Arm or host memory; placing NetBuf in DPA memory caps throughput near approximately 100 Gbps send and approximately 50 Gbps receive. With 64‑B packets, none of host, Arm, or DPA reaches line rate, and DPA needs many more threads to achieve comparable throughput (Chen et al., 2024).

5. DX100 as a near-memory PDA

DX100 provides a concrete instantiation of PDA for indirect memory access acceleration (Khadem et al., 29 May 2025). It addresses workloads in which indirect memory accesses are ubiquitous and memory bound, including sparse linear algebra, adaptive mesh refinement, unstructured meshes, hash joins, and graph analytics. The reported motivation is that indirection destroys locality, limits memory-level parallelism, and interacts poorly with core-side structures such as ROB, LSQ, MSHRs, and register spills or fills. In the evaluated baseline, only 54 of 224 ROB entries are loads or stores; average outstanding loads are about 19, but only about 10 target data arrays, and the DRAM request buffer occupancy is only about 2 out of 32 per-channel slots (Khadem et al., 29 May 2025).

DX100 is shared across cores and memory-mapped into the coherent fabric. It sits proximally to caches and memory controllers and exposes two interfaces: a Cache Interface for streaming accesses with high locality and a DRAM Interface for indirect accesses likely to miss in LLC. A Coherency Agent snoops the directory during fill to determine cache-line presence via an H bit and tracks scratchpad cache lines to invalidate on instruction dispatch. A Core Interface exposes memory-mapped regions for scratchpad, registers, ready and finish bits, tile size, and instruction ingress, with three 64b stores per 192b instruction (Khadem et al., 29 May 2025).

Its principal hardware blocks are a 2 MB scratchpad with 32 tiles of 16K elements, a controller with dispatch, issue, execute, and retire stages, a 32-scalar register file, a Stream Access Unit, an Indirect Access Unit, a Range Fuser Unit, and a 16-lane ALU. The Indirect Access Unit is the distinctive component. It uses Row Table slices per DRAM bank, with a BCAM tracking 64 target rows and SRAM tracking up to 8 columns per row, plus a Word Table implementing per-word linked lists. Fill, request, and response stages jointly transform indirect accesses into a reordered and partially coalesced stream of cache-line requests (Khadem et al., 29 May 2025).

The request-generation policy is explicitly DRAM-aware. Address translation maps requests to channel, rank, bank-group, bank, row, and column. Within each slice, all columns for a row are emitted consecutively to maximize row-buffer hits. Across slices, the Request Generator uses a predetermined interleaving order across channels and bank-groups to maximize BGI and CHI, improving effective throughput because $t_{CCDS} \approx 0.5 t_{CCDL}$ (Khadem et al., 29 May 2025). DX100 therefore targets memory bandwidth utilization rather than merely prefetch latency hiding.

6. ISA, compiler co-design, and execution model

DX100 includes an 8-opcode ISA comprising indirect operations, stream operations, ALU operations, and a range operation (Khadem et al., 29 May 2025). The indirect instructions are ILD, IST, and IRMW; stream instructions are SLD and SST; ALU instructions are ALUV and ALUS; the range instruction is RNG. Example sequences in the paper show gather, scatter, filtered read-modify-write, and CSR traversal using these primitives.

Compiler support is provided through an MLIR-based pipeline. The reported stages are: raising C/C++ to MLIR via Polygeist; loop tiling to create bulk regions; DFS from loop induction variables over use-def chains to detect indirect patterns and legality; hoisting indirect loads and sinking indirect stores or RMW to packed operations outside the inner loop; and lowering packed operations to DX100 API calls, including PTE transfers into DX100’s 256-entry TLB for huge pages. Synchronization is inserted with dx100_wait before core reads scratchpad or after DX100 writes global memory (Khadem et al., 29 May 2025).

Legality constraints are strict. There must be no stores to DX100-accessed arrays within the transformed region, and no loop-carried dependencies. Gauss–Seidel is rejected unless non-aliasing can be proven. Pointer-chasing with random bases is unsupported, though array-based indirection is supported. For very small frontiers, where tile utilization is poor, the compiler falls back to the baseline; in the evaluated GAP applications, more than 99% of nodes were accelerated (Khadem et al., 29 May 2025).

This ISA-plus-compiler structure is significant because it makes PDA programmable at the request level rather than at the level of fixed-function prefetching. A plausible implication is that the main barrier to wider PDA deployment is less hardware feasibility than the availability of legality analyses and runtime interfaces that preserve correctness under aggressive reordering.

7. Evaluation, limitations, and broader significance

The BF3 DPA study and the DX100 study describe complementary performance envelopes. On BF3, PDA’s principal advantages derive from lowest network latency and inline processing, while its major constraints are weak per-thread compute, slow caches, lack of direct DDR, and memory placement sensitivity (Chen et al., 2024). In DX100, PDA’s advantages derive from bulk visibility into future accesses, request-level reordering and coalescing, bypass of core-side MLP limits, and explicit exploitation of DRAM geometry (Khadem et al., 29 May 2025).

DX100 reports a geometric mean speedup of 2.6× over a multicore baseline across 12 benchmarks and 2.0× over the state-of-the-art indirect prefetcher DMP. Average bandwidth utilization improves 3.9× over baseline; row-buffer hit rate increases 2.7× on average, and UME kernels improve from 15% to 91%, a 6.1× increase. DRAM request buffer occupancy rises 12.1×. In all-miss microbenchmarks, DX100 sustains 82–85% utilization across index orders, whereas the baseline tops out at about 65% even with 100% RBH+CHI+BGI, and the worst-case pattern sees speedup up to 9.9× (Khadem et al., 29 May 2025).

The principal BF3 case studies illustrate where near-NIC PDA is effective. In clock synchronization, DPA→DPA memory delivers the lowest average and tail $\epsilon$, but under increased per-packet compute or memory touches, DPA latency rises quickly and can lose its edge. In stateless NFV workloads, DPA can match host or Arm throughput only with sufficient threading and suitable NetBuf placement. In key-value aggregation, “Net‑Arm + Agg‑DPA” is consistently best among DPA variants, but throughput drops sharply once the working set exceeds DPA L2/L3 and Arm L3. Even in best case, DPA throughput under the Yelp trace remains lower than host by 2.5× and Arm by 1.3×, although optimal memory placement narrows the gap substantially (Chen et al., 2024).

Both studies also define hard limits. BF3 PDA is a poor fit for heavy floating-point or integer compute, serial or branch-heavy logic, and large per-flow state that cannot be sharded. DX100 underperforms for extremely low locality where reordering cannot increase row-buffer locality, for strong loop-carried dependencies or aliasing stores, and for non-commutative RMW updates requiring strict serialization (Chen et al., 2024, Khadem et al., 29 May 2025).

Taken together, these results establish PDA as a heterogeneous systems abstraction whose efficacy depends on where proximity is exploited. Near the NIC, PDA accelerates latency-sensitive, low-touch packet processing and trivially parallel small-footprint functions. Near memory, PDA accelerates indirect data access by transforming irregular request streams into DRAM-friendly orderings. The common principle is that proximity creates privileged visibility and control over a bottlenecked path, but the realized benefit depends on careful matching between workload structure and the specific architectural substrate (Chen et al., 2024, Khadem et al., 29 May 2025).