FINN-R: FPGA QNN Framework

• FINN-R is an end-to-end framework that maps quantized neural networks onto FPGAs, automating design-space exploration and custom accelerator generation.
• It supports quantization from 1-bit to 8-bit by employing streamlining techniques and analytic cost models to balance accuracy and resource efficiency.
• The framework leverages hybrid dataflows and both HLS and RTL backends to deliver high throughput and low resource utilization across diverse FPGA platforms.

FINN-R is an end-to-end framework and code generator for implementing quantized neural networks (QNNs) on FPGAs. Designed for fast design-space exploration and deployment, FINN-R enables researchers and engineers to automatically derive custom, highly efficient inference accelerators for a wide range of neural architectures and quantization schemes. FINN-R is built as a Python-based domain-specific toolchain atop Vivado HLS, combining an expressive cost model, dataflow analysis, automated parameter selection, and deep integration with FPGA-specific features such as BRAM partitioning and custom parallel architectures. It supports quantizations as low as 1-bit (binarized) up to 8 bits, multilevel tiling, streaming and weight-stationary dataflows, and both HLS- and RTL-based backend flows. FINN-R achieves up to 50 TOPS on Xilinx VU9P FPGAs with ultra-low LUT and BRAM requirements and minimal to zero DSP usage for sub-8-bit inference, enabling extreme throughput and power efficiency on both data center and embedded FPGA devices (Blott et al., 2018, Li, 13 May 2025, Alam et al., 2022).

1. Tool Flow and Architectural Principles

FINN-R ingests a quantization-aware trained neural network description (e.g., ONNX, Caffe, or TensorFlow with explicit bitwidths) and emits a hardware bitstream and host driver for the target FPGA. The tool flow consists of four main stages:

• Frontend Parsing and IR Construction: The parser translates the network topology and per-layer quantization into an intermediate representation (IR), a directed acyclic graph with explicit bitwidth annotations and operator parameters.
• Transform and Analysis Passes: DirectQuant rewrites floating-point or 32-bit layers to 8-bit fixed-point where needed; streamlining fuses batch normalization, scaling, and quantization into single multi-threshold comparators; resource analysis enumerates per-layer compute requirements, estimating the required processing elements (PEs), SIMD width, and multi-vector (MVU) duplication to satisfy throughput or resource constraints.
• Backend Code Generation: Two backend architectures are supported:
  • Dataflow (DF): Each layer is mapped to a dedicated pipelined engine tailored to its own parallelism and resource parameters.
  • Multilayer Offload (MO): A shared compute engine is instantiated for time-multiplexed execution of multiple layers, with layer weights and parameters streamed as needed.
• Platform Integration and Bitstream Generation: The backend emits parameterized HLS C++ code and TCL scripts that synthesize the architecture for PYNQ, Ultra96, or AWS F1 platforms, integrating host interface and memory controllers as appropriate.

This end-to-end flow is fully automated, with analytic and empirical models enabling rapid iteration and retargeting to different devices and performance goals (Blott et al., 2018).

2. Quantization Strategies and Streamlined Inference

FINN-R generalizes the original FINN’s binary streaming dataflow to support networks with integer precisions from 1 to 8 bits, covering binarized (W^{1,1}), ternary (W^{2,a}), and multibit (e.g., W^{4,4}, W^{8,8}) settings. Key aspects include:

• Per-layer quantization: Notation W^{w,a} reflects w-bit weights and a-bit activations. I/O layers typically retain higher precision (e.g., W^{8,8}), while hidden layers are more aggressively quantized.
• Accuracy trade-off: Binarized networks suffer only modest accuracy losses under suitable training, with ResNet-50 showing an FP16 top-5 accuracy of 93.2% versus 85.9% for W^{1,1} (Blott et al., 2018).
• Quantization-aware training: Straight-through estimators and batch normalization fusion are employed so that “streamlining” can merge quantize and scale operations into a single threshold comparator, eliminating run-time multipliers.
• Layer mapping: All neural operators (convolution, fully-connected, pooling, activation) are mapped to hardware primitives with bitwidth-matched LUT/BRAM resources. The architectures are deeply pipelined and resource-scaled for the precision specified.

FINN-R automates the entire quantization-to-hardware mapping, allowing direct exploration of throughput, resource, and accuracy trade-offs (Blott et al., 2018, Li, 13 May 2025).

3. Dataflow Styles, Tiling, and Memory Hierarchy

FINN-R employs a hybrid streaming/weight-stationary dataflow optimized for quantized inference:

• Streaming across layers: Each layer is formed as a pipeline stage, passing activations by stream (AXI-Stream FIFOs) from one layer to the next without off-chip roundtrips.
• Weight-stationary within layers: Convolutional and dense blocks preload weights into on-chip BRAM/URAM/LUTRAM, maximizing weight reuse as activations are streamed through.
• Two-level tiling: Outer tiles (large spatial or channel blocks) are transferred from DRAM/HBM; inner tiles are staged in on-chip buffers, processed in compute clusters, and consumed fully before the next tile is fetched.
• Auto-selected unrolling and buffering: The code generator chooses spatial (T_y, T_x), input-channel (T_c_in), and output-channel (T_c_out) tile sizes, and corresponding loop unrolling via an analytic cost model to balance resource fit and bandwidth saturation.
• Buffer footprints and reuse factors:
  • Weight buffer: $$B_w = T_{c\_out}\cdot T_{c\_in}\cdot K\cdot K\cdot b_w$$
  • Input activation buffer: $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$
  • Output buffer: $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$

Typical FINN-R hardware cores implement high weight reuse ($R_w = T_y \cdot T_x \cdot T_{c\_in}$) and activation reuse ($R_a = K\cdot K \cdot T_{c\_out}$) (Li, 13 May 2025).

4. Resource Modeling, Performance Prediction, and Automation

FINN-R adopts formal parametric models for resource and performance estimation, allowing cost-driven design-space exploration:

• Dot-product cost: For operations of length $N$ with weight and activation bitwidths $W$ and $A$, $C = N \cdot W \cdot A$. LUTs per bit-product are empirically fit as $\alpha \simeq 0.10$ (handwritten RTL), or $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$0 higher in HLS due to control overhead (Blott et al., 2018).
• BRAM and LUT models: Sliding window (input buffering), weight memory, and MVU logic are costed as explicit functions of tile parameters and bitwidths, e.g.,

$$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$1

$$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$2

• Performance estimation: For a convolutional layer, total MAC ops $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$3, effective concurrency $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$4, latency

$$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$5

End-to-end pipeline throughput approaches $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$6.

• Design-space exploration: Dataflow balancing algorithm iteratively increases $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$7 (the MVU parallelism for each layer $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$8) for the layer with highest compute-to-provision ratio, until resource budgets are met. If no configuration fits, falls back to MO.
• Automation and productivity: The toolflow reduces manual design time by $$B_{a\_in} = (T_y+K-1)\cdot(T_x+K-1)\cdot T_{c\_in}\cdot b_a$$9 versus RTL (Li, 13 May 2025). HLS accelerates iteration at the cost of $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$0–$$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$1 LUT/FF/DSP overhead but enables parameter co-optimization in minutes.

This formalism allows near-analytic selection of architecture parameters to satisfy application throughput and resource constraints, with measured performance predictions within 30% of actual post-synthesis results (Blott et al., 2018, Li, 13 May 2025).

5. Backend Realizations: HLS and RTL MVU Implementations

FINN-R supports both HLS-based and RTL-based backend implementations for Matrix Vector Units (MVUs):

• HLS backend: Generates parameterized C++ kernels with pipelines (II = 1), auto-generated AXI-Stream control, and FSMs. Favorable for rapid prototyping and iteration, but incurs up to $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$2 LUT and up to $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$3 BRAM/FF overhead for small designs.
• RTL backend: Handwritten Verilog modules with minimized control logic and precisely placed pipelining. Achieves 30–70% higher f_max, 10×–15× lower synthesis time, and up to an order of magnitude lower FF usage in small designs (Alam et al., 2022).

Method	Design Time	LUTs (small)	Flip-Flops (FFs)	BRAMs	f_max (RTL/HLS)	Synthesis Speed
HLS	Fast prototyping	Higher	1.5–3× higher	$$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$4 more	$$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$5	Slow
RTL	Manual	Lower	Lower	Lower	1.3–1.7× faster	$$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$610× faster

For design-space exploration involving many parameter sweeps, the RTL backend is preferable due to shorter synthesis turnaround and more accurate resource control, especially for production deployments (Alam et al., 2022). For large designs HLS overheads become a smaller fraction of total resources.

6. Empirical Results and Comparison

FINN-R demonstrates state-of-the-art performance and energy efficiency across diverse FPGA platforms:

• Throughput:
  • AWS F1 DF designs achieve up to $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$7 (binarized MLP), $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$8 (CNN).
  • Embedded Ultra96 achieves $$B_{a\_out} = T_y\cdot T_x\cdot T_{c\_out}\cdot b_{out}$$9 (MLP), PYNQ-Z1 $R_w = T_y \cdot T_x \cdot T_{c\_in}$0.
• Power efficiency: Up to $R_w = T_y \cdot T_x \cdot T_{c\_in}$1 for Ultra96 MLP-4.
• Resource utilization:
  • VU9P: $R_w = T_y \cdot T_x \cdot T_{c\_in}$2k LUTs (75%), $R_w = T_y \cdot T_x \cdot T_{c\_in}$3 DSPs for $R_w = T_y \cdot T_x \cdot T_{c\_in}$4-bit, $R_w = T_y \cdot T_x \cdot T_{c\_in}$5–$R_w = T_y \cdot T_x \cdot T_{c\_in}$6 MB BRAM/URAM.
  • Embedded: $R_w = T_y \cdot T_x \cdot T_{c\_in}$7k–$R_w = T_y \cdot T_x \cdot T_{c\_in}$8k LUTs, $R_w = T_y \cdot T_x \cdot T_{c\_in}$9–$R_a = K\cdot K \cdot T_{c\_out}$0 BRAMs.
• Network case studies:
  • MLP-4 (MNIST, W^{1,1}), CNV-6 (CIFAR-10, W^{1,1} hidden), Tincy YOLO (VOC2007, W^{1,3} hidden), DoReFa-Net/PF (ImageNet, W^{1,2} hidden) (Blott et al., 2018).
• Silicon results confirm: Near-linear scaling with quantization bitwidth and parallelism; high measured energy efficiency ($R_a = K\cdot K \cdot T_{c\_out}$1 GOPS/W), especially in zero-DSP configurations.
• Comparison with other architectures: FINN-R uses a weight-stationary hybrid dataflow as opposed to output/row-stationary; uniquely achieves full zero-DSP mapping and extremely high weight reuse (Li, 13 May 2025).

7. Lessons Learned, Limitations, and Outlook

Key practical lessons from the literature:

• Automation through HLS and a Python DSL sharply shortens design cycles but incurs resource/timing overhead; RTL offers superior resource efficiency and clock rates at greater design effort (Alam et al., 2022).
• FINN-R’s cost models, derived analytically and validated empirically, facilitate rapid tuning of pipeline depth, unrolling, and tiling factors to match throughput/resource targets.
• The pipeline-oriented, streaming architecture scales across both edge and data center FPGAs with only high-level parameter retargeting.
• Limitations include the need for manual RTL IP packaging to fully leverage the RTL backend and more intricate extension for novel compute primitives (e.g., Winograd, sparse kernels).
• Future directions highlighted in surveys call for integration with partial reconfiguration, hybrid dataflows, and the development of domain-specific compiler flows for even tighter coupling with next-generation FPGA fabrics (Li, 13 May 2025).

FINN-R remains a foundational reference architecture and toolchain for implementing QNN inference on FPGAs, leveraging streaming-weight-stationary pipelining, two-level tiling, and formal resource modeling to maximize efficiency and usability (Blott et al., 2018, Li, 13 May 2025, Alam et al., 2022).