Soft Datapath Vectorization (SDV)

• Soft Datapath Vectorization (SDV) is a technique that packs multiple low-bitwidth operands into a DSP multiplier to perform several MAC operations concurrently.
• It leverages the DSP's pre-adder and specialized overflow monitoring to efficiently process signed and unsigned low-precision arithmetic, reducing both LUT and DSP usage.
• Integrated in frameworks like AMD’s FINN, SDV enhances throughput and resource efficiency, achieving significant improvements in FPS/DSP and overall hardware utilization.

Soft Datapath Vectorization (SDV) is a hardware technique designed to maximize the utilization of wide fixed-width digital signal processing (DSP) slices in field-programmable gate arrays (FPGAs) when operating on quantized low-bitwidth arithmetic. By strategically "packing" multiple low-precision operands into one side of a DSP multiplier datapath, SDV enables each DSP to concurrently compute several multiply–accumulate (MAC) operations within its native datapath width. This approach is particularly effective for deep neural network (DNN) inference at reduced bitwidths (1–8 bits), a regime where conventional DSPs exhibit significant underutilization. SDV, in conjunction with optimized overflow control and exploitation of the DSP’s internal features, delivers substantial resource savings and throughput gains for neural network workloads mapped to FPGAs, and is natively supported within platforms such as AMD’s FINN framework (Bornträger et al., 9 Jun 2026).

1. Motivation and Context

Modern FPGA DSP slices (e.g., Xilinx DSP48E2, AMD Versal DSP58) natively implement wide multiplier–accumulator datapaths—commonly 27×18 bits. In low-bitwidth ML workloads dominated by 1–8 bit arithmetic, performing a single 4×4 bit multiply on such wide units leaves most of the silicon idle. SDV addresses this inefficiency by packing several b-bit operands into a single N-bit input, allowing multiple MACs to execute in parallel on one DSP per cycle. For quantized DNN inference, this not only increases throughput (in TOPS/DSP) but also reduces the number of required logic look-up tables (LUTs) and DSP blocks, an essential consideration for edge FPGAs operating under tight resource constraints (Bornträger et al., 9 Jun 2026).

2. Dynamic Arithmetic Packing Technique

2.1 General Packed-Operand Model

The principal mechanism of SDV is arithmetic operand packing. For $k$ input values $x_0, x_1, \ldots, x_{k-1}$ of bitwidths $b_0, b_1, \ldots, b_{k-1}$, non-overlapping shift offsets $s_0, s_1, \ldots, s_{k-1}$ are chosen such that the packed N-bit input is

$P = \sum_{i=0}^{k-1} x_i \cdot 2^{s_i}.$

When one side of the multiplier receives $P$ and the other receives a value $y$, the DSP computes

$$P \cdot y = \sum_{i=0}^{k-1} (x_i \cdot y) \cdot 2^{s_i},$$

allowing each constituent product to occupy a distinct slice in the output word and be extracted without bit overlap.

2.2 Handling Signed and Unsigned Inputs

A key challenge is efficient packing of signed operands; naïve concatenation of two’s-complement signed values leads to overlapping sign extensions, corrupting the results. SDV resolves this by leveraging the DSP’s internal pre-adder:

• Each $b$-bit signed word $x_i$ is split into $x_0, x_1, \ldots, x_{k-1}$0-bit magnitudes and a sign bit.
• All magnitudes are concatenated into a wide word $x_0, x_1, \ldots, x_{k-1}$1, while sign bits are collected into a second word $x_0, x_1, \ldots, x_{k-1}$2, with each bit appropriately shifted.
• The DSP’s pre-adder computes $x_0, x_1, \ldots, x_{k-1}$3 internally, producing the correct packed signed operand.

This approach eliminates the need for external adder trees or extra LUT logic; signed packing is realized without additional fabric resources.

2.3 Lane-Size Constraints and Overflow Monitoring

To prevent overflow across packed operand "lanes" in the accumulator, each lane of size $x_0, x_1, \ldots, x_{k-1}$4 must satisfy

$x_0, x_1, \ldots, x_{k-1}$5

where $x_0, x_1, \ldots, x_{k-1}$6 and $x_0, x_1, \ldots, x_{k-1}$7 are the bitwidths of operands multiplied in that lane. Overflow is detected by a small fabric monitor that recomputes the least significant two bits of each partial product and tracks inter-lane carries modulo 4, utilizing a minimal LUT footprint for reliable accumulation.

3. SDV-Based FPGA Architectures

3.1 Matrix–Vector Multiplication Architecture

In the SDV matrix–vector multiply architecture, multiple input vector elements are packed and processed per cycle:

1. For each cycle, $x_0, x_1, \ldots, x_{k-1}$8 vector elements $x_0, x_1, \ldots, x_{k-1}$9 are extracted and their sign bits separated.
2. $b_0, b_1, \ldots, b_{k-1}$0 (concatenation of magnitudes) and $b_0, b_1, \ldots, b_{k-1}$1 (aligned sign bits) are formed.
3. The DSP’s pre-adder computes $b_0, b_1, \ldots, b_{k-1}$2; the second multiplier input is the corresponding weight $b_0, b_1, \ldots, b_{k-1}$3.
4. Post-multiplier, the specialized overflow monitor computes low bits of each lane, and the accumulator tracks carries.
5. Each output is reconstructed as $b_0, b_1, \ldots, b_{k-1}$4, where $b_0, b_1, \ldots, b_{k-1}$5 is lane size.

Design-time parameters include the number of lanes $b_0, b_1, \ldots, b_{k-1}$6 (constrained by $b_0, b_1, \ldots, b_{k-1}$7) and lane size $b_0, b_1, \ldots, b_{k-1}$8 ($b_0, b_1, \ldots, b_{k-1}$9 for signed lanes). Arbitrary $s_0, s_1, \ldots, s_{k-1}$0 are supported up to $s_0, s_1, \ldots, s_{k-1}$1.

3.2 Convolution via Binary Segmentation (BSEG)

The BSEG approach extends SDV principles to convolutions, packing both kernel coefficients and input patch data into the two multiplier inputs. For $s_0, s_1, \ldots, s_{k-1}$2 kernel elements and $s_0, s_1, \ldots, s_{k-1}$3 patch inputs (e.g., each 4 bits):

• $s_0, s_1, \ldots, s_{k-1}$4
• $s_0, s_1, \ldots, s_{k-1}$5

The DSP computes

$s_0, s_1, \ldots, s_{k-1}$6

Design constraints ensure no overlap and are parameterized as follows:

• $s_0, s_1, \ldots, s_{k-1}$7
• $s_0, s_1, \ldots, s_{k-1}$8
• $s_0, s_1, \ldots, s_{k-1}$9

Typical 4×4 configurations achieve $P = \sum_{i=0}^{k-1} x_i \cdot 2^{s_i}.$0, $P = \sum_{i=0}^{k-1} x_i \cdot 2^{s_i}.$1 (9 MACs/DSP). Guard-bit injection via the DSP’s C-port or RND mode prevents overflow.

4. Integration and Workflow within FINN

SDV and BSEG architectures have been fully integrated into AMD’s FINN framework. They provide automated packing strategies, hardware module generation, and resource-aware scheduling for DNN deployments. The embedding of SDV yields reduced fabric logic via minimized LUT usage, and increased performance density on available DSP blocks, thus benefiting edge artificial intelligence accelerators. Integration into FINN ensures alignment with established toolflows for quantized DNNs, facilitating transparent adoption in advanced FPGA design pipelines (Bornträger et al., 9 Jun 2026).

5. Quantitative Evaluation and Comparative Results

Evaluation on the UltraNet model with 416×416 input, using the FINN reference pipeline as a baseline, demonstrates:

• 21% reduction in overall LUT count.
• FPS per DSP increase from 1.1 to 1.5 (36% improvement).
• 28% reduction in DSP allocation at constant frames per second.

Layer-wise analysis (first 5 convolutional layers) shows BSEG employs 27% fewer LUTs than HiKonv in convolution, and raises FPS/DSP by 25%. At maximum frequency for a large 1×1500×16 input and 128 1×8×16, 4-bit kernels: baseline achieves 580 MHz, 17.8k LUTs, 256 DSPs; SDV+BSEG design achieves 590 MHz, 6.5k LUTs, 192 DSPs—yielding 63% fewer LUTs, 25% fewer DSPs, and a ±2% frequency variance (Bornträger et al., 9 Jun 2026).

6. Summary and Impact

Soft Datapath Vectorization enables efficient execution of multiple MACs per DSP even with arbitrary, low-precision operand widths, using only the DSP pre-adder and a lightweight overflow monitor. When combined with Binary Segmentation for convolution, these methods yield up to 9 MACs per DSP, reduce LUT requirements by 21%, increase FPS per DSP by 36%, and either maintain or slightly exceed baseline operating frequencies. All methods are compatible with open-source toolchains such as AMD’s FINN, representing a highly efficient datapath utilization strategy for quantized neural network acceleration on modern FPGAs (Bornträger et al., 9 Jun 2026).