---
title: 'M4BRAM: In-BRAM Mixed-Precision DNN Acceleration'
url: https://www.emergentmind.com/topics/m4bram
type: topic
---

# M4BRAM: In-BRAM Mixed-Precision DNN Acceleration

M4BRAM is a compute-in-block RAM (BRAM) architecture designed to accelerate mixed-precision matrix-matrix multiplication operations in field-programmable gate arrays (FPGAs), specifically targeting deep neural network (DNN) inference workloads. M4BRAM extends conventional BRAM tiles to deliver bit-serial multiply-accumulate (MAC) computing directly within the memory array, supporting flexible weight and activation quantization, and efficient data reuse schemes. The architecture facilitates high hardware utilization for mixed-precision neural network models and maintains full memory functionality, allowing simultaneous memory access and in-memory computation, thereby complementing existing digital signal processing (DSP) resources for DNN inference acceleration [2311.02758].

## 1. Microarchitecture and Mixed-Precision Support

M4BRAM augments each standard 20 Kb M20K (Intel) or 36 Kb RAMB36E1 (Xilinx) BRAM tile with four “dummy” BRAM arrays—one per in-BRAM processing element (BPE)—a lightweight microsequencer (eFSM), and a “duplication shuffler” capable of selectively replicating and shuffling 32-bit weight data across the four BPEs. Port A serves both as a conventional memory write port and, in compute mode, as the instruction port feeding operands into the eFSM and BPEs. Port B remains available as a read port for simultaneous access by other compute resources during M4BRAM operations.

M4BRAM supports weights ($P_w$) at 2, 4, or 8 bits and activations ($P_i$) with 2 to 8 bits precision. Weight precision is fixed at configuration, while activation precision is dynamically selectable on each compute-in-memory (CIM) instruction via a 4-bit byte-enable field in the data word. Each BPE implements a bit-serial multiplier-accumulator, running for $(P_i + 2)$ cycles in synchronous (single-clock) mode or $(\frac{P_i}{2} + 2)$ cycles if double-pumped at twice the BRAM clock.

This configuration enables true mixed-precision operation, permitting per-layer or even per-inference flexibility in quantization, responding to DNN requirements without idle resource cycling [2311.02758].

## 2. In-BRAM Data Duplication and Parallelism

M4BRAM employs a configurable in-BRAM data duplication scheme to exploit both weight-sharing ($N_W$) and activation-sharing ($N_I$) per processing element, tailored to diverse DNN layer reuse patterns. For a given $P_w$, each 32-bit BRAM word packs $L = 32/P_w$ weights, sliced and assigned to the four BPEs via the duplication shuffler. By choosing a duplication factor $D \in \{1, 2, 4\}$, any slice may be broadcast to 1, 2, or all 4 BPEs, setting $N_I$. For $P_w = 8$, selectable $(N_W, N_I) \in \{(4,1), (2,2), (1,4)\}$ (doubled in the “large-dummy” variant).

The total multiply-accumulate operations per 32-bit load is:
$$
R_{\text{total}} = \#\text{BPEs} \times (N_W \cdot N_I) = 4 \times L = 4 \times \frac{32}{P_w}.
$$

This duplication scheme enables the architecture to flexibly match DNN layers characterized by either high weight or high activation reuse, optimizing utilization and supporting both convolutional and transformer models.

## 3. Accelerator Dataflow and System-Level Integration

In a tiled DNN accelerator employing M4BRAM, the dominant dataflow is weight-stationary, activation-broadcast—mirroring typical FPGA direct learning accelerator (DLA) protocols. Tiled DNN computation proceeds as follows:

1. A 32-bit weight vector is loaded from DRAM into main BRAM.
2. Four $P_i$-bit activations are sent in a 32-bit Port A write, cueing the eFSM to initiate a CIM instruction.
3. The eFSM latches instruction data, configures duplication, and streams activations to BPEs.
4. Over $(P_i + 2)$ cycles, each BPE performs bit-serial MAC operations, accumulating partial sums.
5. Port B remains available during this operation so other fabric (e.g., DSPs) can fetch from BRAM concurrently.
6. On completion, Port B reads retrieve the computed results for buffer or further reduction. 

Workload partitioning between M4BRAM (bit-serial MAC) and conventional DSP (bit-parallel) engines follows the $Q_\text{VEC}$ output dimension, enabling heterogeneous parallel execution and output merging.

M4BRAM occupies only Port A during compute (two instruction cycles plus $P_i + 2$ compute cycles); Port B access is decoupled and unaffected, ensuring full BRAM utility and supporting double-buffered dataflows, unlike previous compute-in-BRAM (CIM) approaches.

## 4. Performance Modeling and Hardware Efficiency

Theoretical throughput for a single M4BRAM tile operating at BRAM frequency $f$ is:
$$
\mathrm{Throughput}_{\rm MAC} = \frac{f \times 4 \left(\frac{32}{P_w}\right)}{P_i + 2} \quad \text{[MACs/s]}
$$
For double-pumped BPEs:
$$
\mathrm{Throughput}_{\rm MAC}^{\rm DP} = \frac{2f \times 4\left(\frac{32}{P_w}\right)}{P_i/2 + 2}
$$

Scaling to $T$ tiles yields total throughput $T \cdot \mathrm{Throughput}_{\rm MAC}$, contingent on off-chip bandwidth $B_{\text{DRAM}} \geq T \cdot (f \times 32\, \text{bits})$ plus output reads. M4BRAM supports concurrent double-buffering and ping-pong arbitration, as Port A is only briefly occupied during eFSM operation. 

When compared with DSP-only and prior CIM designs (BRAMAC, CCB, CoMeFa), M4BRAM-S introduces a +19.6% BRAM area overhead per tile, but only Port A is reserved for CIM. Full-chip replacement increases core area by approximately 5.6% on a 28%-BRAM-heavy FPGA, more than doubling mixed-precision MAC throughput. BRAMAC supports only uniform-precision operation and occupies both ports, restricting flexibility and throughput (<1.7× DSP baseline), whereas M4BRAM yields $>2.1\times$.

## 5. Experimental Evaluation

Across standard ImageNet classification networks (e.g., AlexNet, VGG-16, ResNet-18, ViT-Base self-attention), inclusion of M4BRAM in a tiled DNN accelerator delivers an average 2.16× speedup relative to 8-bit DSP-based DLA, maintaining top-1 accuracy degradation below 0.5% for $P_i \geq 6$. Versus prior BRAMAC‐1DA, M4BRAM provides 1.43× greater performance.

Table: Representative Performance Metrics

| Model         | BRAM Type      | Logic LUTs | DSPs Used | BRAMs   | $f_{\max}$ | Tile Latency (μs) | Top-1 Loss | Energy Eff. vs. DSP |
|---------------|---------------|------------|-----------|---------|------------|-------------------|------------|---------------------|
| AlexNet       | DLA (8/8)     | 55K        | 648       | 2×1K    | 300 MHz    | 1.2               | 0.00%      | 1×                  |
|               | M4BRAM-S      | 60K        | 400       | 2×1K    | 280 MHz    | 0.55              | 0.28%      | 2.3×                |
| ResNet-18     | DLA (8/8)     | 63K        | 648       | 2×1K    | 300 MHz    | 2.4               | 0.00%      | 1×                  |
|               | M4BRAM-L      | 68K        | 500       | 2×1K    | 275 MHz    | 1.1               | 0.45%      | 2.2×                |
| ViT-Base      | DLA (8/8)     | 70K        | 648       | 2×1K    | 300 MHz    | 3.1               | 0.00%      | 1×                  |
| (self-attn)   | M4BRAM-S (DP) | 75K        | 300       | 2×1K    | 260 MHz    | 1.4               | 0.32%      | 2.5×                |

Key observations include greater than 2× latency reduction for minimal accuracy drop (<0.5% top-1).

## 6. Design Trade-offs and Implications

Reducing activation bit-width $P_i$ yields near-linear gains in bit-serial M4BRAM throughput ($P_i+2$ cycle depth), particularly when $P_i \geq 6$ dominates serial overhead. Lowering weight width $P_w$ increases packing $L=32/P_w$, and thus multiply-accumulate parallelism ($R_{\text{total}}$), which DSP architectures cannot readily exploit for sub-8 bit weights.

Per-tile area overhead for M20K $\rightarrow$ M4BRAM-S is +19.6%, but for FPGAs with 28% of core area comprising M20K, net core impact is ~5.6% for 2×–2.3× MAC throughput benefit. A plausible implication is that M4BRAM can efficiently address DNN workloads with dynamic, heterogeneous quantization requirements at minimal resource premium. 

## 7. Architectural Innovations and Future Prospects

M4BRAM is the first compute-in-BRAM design to support true mixed-precision operation with runtime precision switches: $P_w \in \{2,4,8\}, P_i \in [2,8]$. The duplication shuffler mechanism enables efficient weight- and activation-sharing, matching $(N_W,N_I)$ to the prevailing reuse in DNN layers for maximized utilization. The split-port architecture makes M4BRAM the only CIM approach to leave Port B uninterrupted, solving long-standing data sharing and double-buffering issues in FPGA DNN engines.

*This suggests* M4BRAM forms an ideal building block for next-generation FPGA AI accelerators requiring fine-grained, layer-wise quantization control and robust hardware resource sharing. Its integration transforms passive on-chip memory to dynamically configurable compute arrays, delivering 2–3× DNN performance boost over conventional designs with modest area increases [2311.02758].

Source: https://www.emergentmind.com/topics/m4bram