---
title: Coalesced DMA Optimization
url: https://www.emergentmind.com/topics/coalesced-dma
type: topic
---

# Coalesced DMA Optimization

Coalesced Direct Memory Access (DMA) is a system and hardware architecture design pattern focused on efficiently grouping small, logically contiguous or compatible data transfers into larger, single DMA requests. By reducing the number of independent transfers, coalesced DMA amortizes startup latency, improves sustained bandwidth utilization, and alleviates contention between compute and data-movement operations. Modern distributed and heterogeneous computing environments—ranging from deep learning clusters leveraging GPU peer-to-peer communication to near-memory accelerators and high-throughput persistent memory systems—employ this pattern to approach hardware throughput limits otherwise unattainable with fine-grained transfer patterns.

## 1. Formal Definition and Core Performance Model

Coalesced DMA refers to merging multiple logically adjacent, same-destination (or compatible) data fragments into a single hardware DMA transaction. In GPU-based multicompute systems, this is implemented (e.g., using hipMemcpyDtoDAsync) such that, instead of launching $n$ DMA requests for $n$ fragments, a single coalesced DMA copy is issued for a larger, contiguous region [2512.10236].

The time to transfer $m$ bytes via a DMA is modeled by a Hockney-style equation:
\[
T_\text{DMA}(m) = \alpha_\text{DMA} + \frac{m}{B_\text{DMA}}
\]
where $\alpha_\text{DMA}$ is the per-transfer launch or latency overhead, and $B_\text{DMA}$ is the sustainable bandwidth (e.g., 64 GB/s for AMD Infinity Fabric).

Fragmentation into $n$ transfers of size $\frac{M}{n}$ causes a total transfer time:
\[
T_\text{total\_fragmented} = n \cdot \alpha_\text{DMA} + \frac{M}{B_\text{DMA}}
\]
relative to the ideal single transfer, coalescing recovers the inefficiency term:
\[
\Delta T_\text{fragment} = (n - 1) \alpha_\text{DMA}
\]
Thus, coalescing drives $n \to 1$, minimizing latency and maximizing link utilization.

## 2. Architectural and Algorithmic Implementations

### GPU Compute/Communication Overlap (FiCCO)
In ML parallelization, FiCCO (Finer-Grain Compute-Communication Overlap) schedules exploit coalesced DMA to offload fine-grained peer-to-peer sharded communication onto DMA engines, merging same-destination sub-shards wherever possible. The architecture supports various communication patterns (1D/2D sharding) and fuses compute kernels where beneficial. Four key schedules control the decomposition and overlapping strategy, with coalesced DMA central to minimizing startup costs [2512.10236].

### Accelerator–CPU Transfers (AXI4MLIR)
For custom hardware accelerator driver stacks, coalesced DMA is implemented by a compiler-level pass (e.g., in AXI4MLIR) that merges adjacent accel.send operations (targeting consecutive regions) into single, variadic send calls. The backend emits a single dma_submit covering the contiguous span, bounded by alignment and maximum burst size constraints [2402.19184].

```python
# Pseudocode for DMA coalescing in AXI4MLIR [2402.19184]
function coalesceDMA(mlirFunc):
    for op in mlirFunc:
        if consecutive and sum(size) <= MAX_BURST:
            group += op
        else:
            dma_submit(chan, group[0].addr, sum(group.size))
```

### Near-Memory Indirect Access (AXI-Pack/SpMV)
In irregular streaming workloads such as SpMV, near-memory hardware units buffer and coalesce narrow requests (e.g., 64b words) into wide DRAM bursts (e.g., 512b lines), using a windowed scheduler to merge all accesses within a DRAM block before issuance. The AXI-Pack protocol facilitates this by defining "indirect burst" descriptors; the Request Coalescer block issues composite reads, sustaining high memory bandwidth even for non-contiguous application-level accesses [2311.10378].

## 3. Quantitative Impact and Performance Results

### Distributed ML Systems
FiCCO evaluation on 8× AMD MI300X systems shows:
- Non-coalesced peer-to-peer sharding can slow down performance by up to $-3.9\times$ vs. ideal.
- Non-coalesced DMA with FiCCO yields geometric mean $1.34\times$ speedup (up to $1.6\times$).
- Emulated 2D coalesced DMA achieves $1.47\times$ geomean speedup (up to $1.7\times$).
- DMA-offload (hardware-engine driven) surpasses GPU-core-driven communication by up to $1.15\times$ [2512.10236].

### Custom Accelerator Data Movement
In the AXI4MLIR MatMul accelerator case study, baseline core utilization is $<10\%$. Incorporating coalescing (alongside DMA allocation and pipelining) increases compute-core utilization up to $60\%$, and standalone coalescing reduces per-tile DMA setup overhead by $20–30\%$ [2402.19184].

### Sparse-Memory Vector Multiplication (SpMV)
With a 256-window coalescing buffer, the near-memory adapter achieves $25 \text{ GB/s}$ bandwidth (over $8.6\times$ improvement relative to no coalescer), attaining $78\%$ of DRAM peak. System-level SpMV speedup is $3–4\times$, with off-chip traffic and utilization approaching ideal levels [2311.10378].

### RDMA and Persistent Memory Hashing
Continuity hashing groups all lookup positions into one segment, enabling every lookup to complete in exactly one RDMA read regardless of load factor. This leads to $1.4–2.4\times$ throughput speedup and $1.2–2.2\times$ latency reduction compared to level hashing or P-FaRM-KV, with fewer persistent memory writes per operation [2107.06836].

## 4. Design Trade-offs, Inefficiencies, and Bounds

Coalesced DMA introduces several design trade-offs:
- **Alignment and Burst Size Constraints:** Coalesced blocks must typically align to DMA/DRAM controller requirements (e.g., 64B or 512B). Exceeding the hardware max ($L_\text{max}$) causes further splitting [2402.19184, 2311.10378].
- **Latency vs. Concurrency:** Over-coalescing (very large bursts) may block access to other flows or increase latency; hardware schedulers balance window size $W$ and maximum in-flight requests (MLP).
- **Decomposition and Contention Loss:** In FiCCO, decomposing large kernel operations (both compute and communication) introduces decomposition inefficiency loss (DIL) and contention inefficiency loss (CIL). The optimal schedule—whether to coalesce for larger kernels (favoring low DIL) or to stagger for lower CIL—depends on operation intensity, data movement, and available hardware parallelism. Heuristic selection based on op-to-byte ratio (OTB) and predicted memory traffic selects among four FiCCO schedules to best exploit coalesced DMA [2512.10236].

| Schedule              | DIL Degree   | CIL Degree | Coalesced DMA Role                   |
|-----------------------|-------------|------------|--------------------------------------|
| uniform-fused-1D      | low         | high       | All-to-all, largest fusable regions  |
| hetero-fused-1D       | medium      | medium     | Mixed: some early, some staggered    |
| hetero-unfused-1D     | high        | low        | Fine granularity, minimal overlap    |
| uniform-fused-2D      | lowest      | medium     | Largest 2D patches, high utilization |

## 5. Protocols, System Integration, and Hardware Considerations

Implementation of coalesced DMA is dependent on integration at several architectural levels:

- **Software/Compiler:** Compiler passes (e.g., MLIR) generate coalesced transfer requests automatically during code emission.
- **Bus Protocols:** Standards like AXI4MLIR and AXI-Pack provide protocol-level support for variadic and indirect bursts, with sideband metadata for mapping logical to physical accesses [2402.19184, 2311.10378].
- **Hardware Schedulers:** Request Coalescers implement multi-entry windows and metadata queues, with tight design cost trade-offs (e.g., 27 kB SRAM and $0.2–0.3$ mm² area for a 256-entry coalescer at 1 GHz) [2311.10378].
- **Atomic Transactionality:** Coalesced RDMA designs for consistency (e.g., continuity hashing) place atomic indicator words at the head of each coalesced region to provide log-free, crash-consistent state transitions with a single atomic operation per transfer [2107.06836].

## 6. Broader Applications and Measured System-Level Effects

Coalesced DMA is a central optimization across a range of system domains:
- In distributed model parallelism, coalescing communication shards unlocks performance on multi-GPU/accelerator clusters by enabling higher compute-communication overlap and improved steady-state kernel throughput [2512.10236].
- For sparse, irregular applications, a hardware coalescer with effective bandwidth utilization ($\eta \approx 0.9$) recovers near-theoretical DRAM throughput, directly accelerating algorithms such as SpMV by $3–4\times$ end-to-end [2311.10378].
- In persistent memory systems, one-round-trip, one-store coalesced protocols eliminate the RDMA access amplification and logging overhead prevalent in naive designs, improving both latency and memory endurance [2107.06836].
- Compiler-generated coalesced DMA in host drivers provides seamless scalability as custom accelerator workloads expand, directly coupling with pipelining and staging to maximize arithmetic resource occupancy [2402.19184].

## 7. Summary and Future Outlook

The coalesced DMA pattern—grouping fine-grained, logically adjacent transfers into hardware-efficient bursts—is now widely recognized as a prerequisite for bandwidth-saturating operation in high-performance computing, storage, and accelerator systems. It eliminates per-transfer startup overhead, reduces control-path contention, and enables principled trade-offs between kernel launch efficiency and memory system utilization. As systems and workloads continue to fragment both spatially (across devices) and temporally (into finer compute/communication tiles), further advances in coalescing algorithms, hardware schedulers, and protocol-level integration will remain essential to closing the gap between effective and peak attainable performance [2512.10236, 2107.06836, 2402.19184, 2311.10378].

Source: https://www.emergentmind.com/topics/coalesced-dma