Matrix–Vector–Threshold Unit (MVTU) Overview

• Matrix–Vector–Threshold Unit (MVTU) is a hardware accelerator for QNNs on FPGAs that maps fully connected and convolutional layers to general matrix–vector multiplications.
• Its architecture offers configurable parallelism and quantization settings, enabling precise tradeoffs between resource usage, latency, and synthesis efficiency.
• Comparative evaluations show RTL-based MVTU implementations deliver lower FF/BRAM usage and faster synthesis times than HLS, favoring edge deployment scenarios.

A Matrix–Vector–Threshold Unit (MVTU), also known as a Matrix–Vector Unit (MVU), is a hardware accelerator component for implementing a single layer of a quantized neural network (QNN) on FPGAs. The MVTU executes either fully connected or convolutional layers converted to general matrix–vector multiplication (GEMV), supporting high parallelism, custom quantization, and tailored dataflow for low-latency deep neural network (DNN) inference. The architecture and empirical analyses of the MVTU underpin frameworks such as FINN and hls4ml, enabling configurable dataflow execution via register-transfer level (RTL) or high-level synthesis (HLS) approaches, with rigorous tradeoffs in resource usage, critical path latency, and synthesis efficiency (Alam et al., 2022).

1. Functional and Architectural Overview

The MVTU implements one QNN layer per hardware instance, mapping convolutions to GEMV by lowering the kernel operation and streaming data in and out to align with the FPGA dataflow paradigm. The core compute variant accepts input tiles of size $K_d^2I_c \times O_d^2$ and multiplies them with a weight matrix of size $O_c \times K_d^2I_c$, producing an output matrix $O_d^2 \times O_c$. Each column vector, length $K_d^2I_c$, is streamed in and yields a length-$O_c$ output vector, sustaining parallel vectorization.

The typical dataflow for layer $L$ is orchestrated as follows: Sliding-Window Unit (SWU) → Input Buffer → MVTU → Threshold & Binarize → Next Layer. Inside the MVTU, established major pipeline stages—each targeting initiation interval (II) = 1—comprise the following workflow:

1. Fetching a $K_d^2I_c$ input slice into a FIFO.
2. Concurrently fetching $S$ weight values per processing element (PE) from on-chip RAM.
3. Computation of $S$ parallel multiply/XNOR operations per PE, generating partial products.
4. Tree-accumulation of these partial products to produce $P$ parallel sums.
5. Application of a per-output threshold (e.g., using a small lookup table or comparator).
6. Emitting $O_c \times K_d^2I_c$0 thresholded outputs on an AXI-Stream interface.

Thresholding, in the FINN framework, is achieved by appending a fixed threshold and sign bit per output channel, generally negligible in resource cost.

2. Key Configurable Parameters

The MVTU exposes several design degrees of freedom, allowing tunable performance–resource tradeoffs:

• $O_c \times K_d^2I_c$1: Number of input channels.
• $O_c \times K_d^2I_c$2: Number of output channels (weight matrix rows).
• $O_c \times K_d^2I_c$3: Kernel dimension (with $O_c \times K_d^2I_c$4 for fully connected layers).
• $O_c \times K_d^2I_c$5: Number of parallel PEs determining how many output channels are processed concurrently.
• $O_c \times K_d^2I_c$6: Number of Single-Instruction Multiple Data (SIMD) lanes per PE determines dots computed in parallel within a PE.
• $O_c \times K_d^2I_c$7: Bit-width of weights; $O_c \times K_d^2I_c$8 for activations.
• On-chip RAM depth per PE: $O_c \times K_d^2I_c$9.
• RAM read word-width: $O_d^2 \times O_c$0 bits.

When $O_d^2 \times O_c$1 or $O_d^2 \times O_c$2, folding is used: rows or columns are time-multiplexed over multiple cycles.

3. Analytical Performance Expressions

The MVTU's data-parallel architecture permits analytical modeling of both latency and throughput:

• Total MACs (multiply-accumulate operations) per layer:

$O_d^2 \times O_c$3

• Compute cycles per image tile:

$O_d^2 \times O_c$4

• Throughput (output vectors per second):

$O_d^2 \times O_c$5

• Throughput (MACs per second):

$O_d^2 \times O_c$6

• Pipeline latency (cycles until first valid output):

$O_d^2 \times O_c$7

$O_d^2 \times O_c$8 (pipeline depth) is 10–20 cycles for HLS-generated MVTUs, 5–10 for RTL.

• Real-time latency (nanoseconds):

$O_d^2 \times O_c$9

4. Comparative Resource Utilization and Synthesis Efficiency

Empirical evaluation on Xilinx XC7Z020 (Pynq-Z1) using Vivado 2020.1 demonstrates concrete tradeoffs between RTL- and HLS-based MVTUs (all figures from (Alam et al., 2022)):

MVTU Size / Bitwidth	LUTs (HLS)	LUTs (RTL)	FFs (HLS)	FFs (RTL)	BRAMs (HLS)	BRAMs (RTL)	Synth. Time HLS+RTL	Synth. Time RTL
Small, 1-bit (P=2,S=2)	872	642	—	—	2	0	30–50 min	2–5 min
Small, 4-bit (P=2,S=2)	1901	1424	3152	604	2	0	30–50 min	2–5 min
Large, 4-bit (P=16,S=16)	7528	7572	—	—	4	2	30–50 min	2–5 min

• LUT usage: RTL is ~25–26% lower for small designs; HLS converges or is marginally better for large ($K_d^2I_c$0) designs.
• Flip-Flops (FFs): HLS consistently uses 2–5× more; e.g., small 4-bit MVTU HLS uses 3152 FFs versus 604 for RTL.
• BRAM: HLS consumes $K_d^2I_c$1 more, often utilizing BRAM inefficiently.
• Synthesis time: RTL out-of-context synthesis is 2–5 min per instance, HLS+RTL requires 30–50 min (10× slower, with HLS time scaling superlinearly).
• Critical path and $K_d^2I_c$2: RTL achieves 44–80% faster cycles; small RTL (1-bit) $K_d^2I_c$3 MHz versus HLS's 392 MHz; small 4-bit RTL $K_d^2I_c$4 MHz ($K_d^2I_c$5 ns) vs HLS $K_d^2I_c$6 MHz ($K_d^2I_c$7 ns).

5. Empirical Evaluation: Use Case in Network Intrusion Detection

A practical instantiation evaluates a 4-layer MLP on UNSW-NB15 with 2-bit quantization; layers sized [600 → 64 → 64 → 64 → 1], folding (P,S) selected per layer for resource matching. For the largest layer (0):

• HLS: 30,744 LUTs, 21,159 FFs, 0 BRAMs, $K_d^2I_c$8 ns, synth=38′45″, cycles=17
• RTL: 43,894 LUTs, 12,965 FFs, 0 BRAMs, $K_d^2I_c$9 ns, synth=5′21″, cycles=17

In this scenario, HLS uses 30% fewer LUTs but 63% more FFs, with $O_c$0 longer synthesis and a clock that is $O_c$1 slower. Other layers follow similar patterns: larger designs favor HLS in LUTs only, while RTL maintains compelling advantages in FF/BRAM footprint, synthesis time, and clock speed.

6. Design, Implementation, and Selection Guidelines

For DNN accelerator architectures targeting FPGAs, the choice of MVTU backend (RTL vs HLS) yields substantive implications:

• RTL is optimal where peak $O_c$2, minimal FF/BRAM budgets, and fast synthesis (iteration) are prioritized (notably, small devices or edge deployments). It delivers up to 2× speed-up, 2–5× FF reduction, 2× less BRAM, and 10× faster synthesis.
• For very large MVTUs ($O_c$3), LUT counts converge; HLS can marginally surpass RTL in LUT metrics at the expense of higher FF/BRAM and protracted synthesis.
• HLS mainly benefits rapid design-space exploration through ease of code change, but this can be negated in large variants due to exponentially increasing synthesis times.
• In code-generated, non-handwritten hardware flows (e.g., FINN, hls4ml), adopting an RTL MVTU library accelerates iteration and confers improved resource/performance tradeoffs. HLS should only be considered if the workflow is strictly limited to C++/OpenCL, extensive synthesis times are tolerable, and additional FF/BRAM usage is acceptable.

A recommended practice is to supply a compact, RTL-derived MVTU primitive that achieves II=1 and consistently outperforms HLS alternatives in every principal hardware metric, integrating smoothly with modern dataflow compilers (Alam et al., 2022).