---
title: 'On-FPGA Workflow: End-to-End Design'
url: https://www.emergentmind.com/topics/on-fpga-workflow
type: topic
---

# On-FPGA Workflow: End-to-End Design

On-FPGA workflow denotes the end-to-end process by which an algorithm, neural network, or application kernel is transformed from a high-level software or graph description into an FPGA-resident implementation, then integrated into a larger system, validated, deployed, and, in some systems, reconfigured at runtime. Across recent work, this workflow spans model import, quantization or pruning, intermediate-representation construction, analytical or search-based design-space exploration, code or RTL generation, vendor-tool synthesis, host-side orchestration, and, in some cases, virtualization, migration, or online adaptation [2103.05579] [1711.08740] [2004.04641] [2504.08534].

## 1. Front ends, inputs, and problem formulation

The entry point of an on-FPGA workflow depends on the application domain and compiler stack. In machine-learning flows, the input is often a trained model in a standard framework: hls4ml provides converters for “(Q)Keras, TensorFlow, PyTorch, and ONNX model formats,” fpgaConvNet expects a Caffe model description, CNN2Gate consumes ONNX exported from frameworks such as Keras, PyTorch, TensorFlow, Caffe, or Caffe2, and fpgaHART ingests a high-level network description from frameworks such as PyTorch or ONNX [2103.05579] [1711.08740] [2004.04641] [2305.19896]. In software-centric and systems workflows, the input can instead be general-purpose C++, OpenCL kernels, Verilog, LLVM IR, CSV topology specifications, or workflow-level task descriptions [2602.06085] [1509.00036] [2109.02484] [1609.00130] [2409.20099] [2402.12612].

A common feature is that the source description is paired with explicit deployment intent. fpgaConvNet requires a target FPGA specification and an optimization goal such as throughput, latency, or a multiobjective criterion; CNN2Gate requires a target FPGA board and user-provided fixed-point quantization values; EVEREST treats the application as a larger workflow and marks only selected kernels for acceleration; the adaptive CNN compiler in [2504.08534] takes user constraints such as latency bounds and resource limits [1711.08740] [2004.04641] [2402.12612] [2504.08534]. This framing is technically important because the workflow is constrained from the beginning by device capacity, interface assumptions, and application-level objectives rather than by a purely algorithmic notion of correctness.

The resulting diversity of front ends can be summarized as follows.

| Workflow | Primary input abstraction | FPGA-side output form |
|---|---|---|
| hls4ml | Trained model from (Q)Keras, TensorFlow, PyTorch, or ONNX | HLS project, IP core, or Vitis kernel [2103.05579] |
| fpgaConvNet | Caffe CNN, target FPGA, optimization goal | Streaming accelerator, single flexible bitstream, or partition-specific bitstreams [1711.08740] |
| CNN2Gate | ONNX model, quantization values, target Intel FPGA | OpenCL project, RTL/bitstream, executable FPGA implementation [2004.04641] |
| FastFlow + Vitis | `proc.csv`, `circuit.csv`, hardware kernel sources | `host.cpp`, `connectivity.cfg`, `.xclbin` [2409.20099] |
| ODE/ODR compiler | Pretrained CNN plus latency/resource constraints | RTL/HDL plus runtime-selectable execution modes [2504.08534] |

## 2. Intermediate representations and compile-time reasoning

The central abstraction in most on-FPGA workflows is an intermediate representation that separates frontend semantics from backend synthesis. In hls4ml, imported models are transformed into a common internal graph representation called an “HLSModel,” to which hardware-related configuration is attached; this graph then undergoes simplifications such as batch-normalization fusion and compile-time folding of constant operations [2103.05579]. fpgaConvNet uses **Synchronous Dataflow (SDF)** as the common formalism for both CNN workload representation and hardware mapping, with the implementation represented by a topology matrix
\[
\boldsymbol{\Gamma} \in \mathbb{R}^{(M \times N)},
\]
and a workload matrix \(\boldsymbol{W}\) that encodes the total amount of work performed by each hardware block during inference [1711.08740]. fpgaHART extends the same idea to 3D CNNs by lowering a network DAG into an SDFG and factorizing the topology as
\[
\Gamma = S \times R,
\]
where \(S\) captures stream multiplicity and \(R\) captures normalized rates [2305.19896].

Other systems use different IRs for the same purpose. CNN2Gate relies on ONNX as an interchange graph and maintains an intermediate hardware configuration through linked layer structures plus generated OpenCL configuration data [2004.04641]. EVEREST converges multiple front ends into MLIR dialects such as `ekl`, `dfg`, `teil`, `esn`, and `jabbah`, so that big-data, HPC, and ML abstractions can be optimized in a common compiler pipeline [2402.12612]. FOS uses JSON descriptors for both shell and accelerator metadata, decoupling software tooling from vendor-generated implementation artifacts [2001.09990]. These differences in representation encode a shared design principle: on-FPGA workflow is feasible only when the compiler or runtime can reason about structure, data movement, and resource use before RTL is frozen.

Analytical modeling is the second compile-time pillar. fpgaConvNet attaches throughput, latency, and resource models to the SDF graph and explores graph partitioning, coarse-grained folding, fine-grained folding, and weights reloading without implementing every candidate [1711.08740]. fpgaHART computes an initiation-interval matrix by
\[
II = W / \Gamma
\]
and estimates partition execution time and throughput from the largest matrix entry \(II_{max}\), batch size, and reconfiguration cost [2305.19896]. The adaptive CNN workflow in [2504.08534] derives explicit conv, FC, and pipeline latency/resource equations and uses a multi-objective genetic algorithm to search feasible mappings. SqueezeJet-2 similarly derives an HLS-informed analytical model from pipeline depths, trip counts, and function-call overheads and reports that its model stays within 8.3% maximum absolute percentage error from hardware-trace latency, whereas Vivado HLS worst-case performance estimation reaches 46.1% error and C/RTL co-simulation reaches 33.4% [1902.03192]. Such models do not eliminate synthesis, but they change the workflow from blind compilation to guided exploration.

## 3. FPGA-side realization patterns

On-FPGA workflows do not converge to a single microarchitectural template. hls4ml generates model-specific firmware in which each layer becomes a customized module specialized to the specific dimensions, precision, and often the specific trained weights; the full model is synthesized into an IP core rather than a generic programmable matrix engine [2103.05579]. fpgaConvNet and fpgaHART instead generate streaming accelerators composed of hardware building blocks connected by streams, with SDF actors corresponding to layer fragments or layer-level engines [1711.08740] [2305.19896]. CNN2Gate maps ONNX operators to an OpenCL kernel library comprising memory read, memory write, convolution, pooling, and fully connected functions, interconnected by OpenCL pipes implemented as FIFOs on FPGA [2004.04641].

A second class of workflow maps computations onto reusable accelerator fabrics. Gemmini is a systolic-array-based accelerator with load, execute, and store controllers attached to a Rocket core, and the workflow in [2408.07404] lowers quantized CNN operators to Gemmini instructions through TVM and microTVM. SqueezeJet-2, by contrast, is a streaming convolution/max-pool accelerator with line buffers, double-buffered windows, and \(16\) PEs \(\times 16\) MACs/PE on a small Zynq SoC [1902.03192]. Transparent Live Code Offloading uses a pre-programmed **Data Flow Engine (DFE)** overlay that can be reconfigured by software place-and-route at the overlay level, avoiding full bitstream regeneration [1609.00130]. This suggests that “on-FPGA” may mean direct synthesis of fixed datapaths, placement onto an intermediate overlay, or lowering onto a preexisting accelerator ISA.

The same diversity appears outside neural inference. The matrix-element study on an AMD Alveo U250 organizes the FPGA as a four-stage `hls::stream` dataflow pipeline—input loader, phase-space generation, compute stage, output writer—and maps either the full \(e^+e^- \to \mu^+\mu^-\) event-evaluation chain or an isolated color-reduction kernel for \(gg \to t\bar t + X\) [2605.23785]. The FastFlow+Vitis workflow, in contrast, leaves computation kernels in user-provided HDL, C++, or SystemC and automates only host-side orchestration, memory connectivity, and multi-FPGA routing [2409.20099]. The architectural consequence is that an on-FPGA workflow may be centered on hardware generation, host control generation, or the boundary between the two.

## 4. Verification, software integration, and deployment mechanics

Verification is typically staged, with increasingly hardware-faithful checkpoints before board execution. hls4ml supports bit-accurate emulation of the generated HLS-synthesizable inference code directly from Python, allowing comparison against the original floating-point or quantized model and inspection of hidden-layer outputs before vendor synthesis [2103.05579]. CNN2Gate provides CPU emulation mode through Intel’s OpenCL toolchain, validating functionality and accuracy much faster than full FPGA synthesis, followed by full-flow compilation to board-specific RTL and bitstream artifacts [2004.04641]. LAAFD performs host compilation, functional testing against the original C++ kernel, then HLS co-simulation and synthesis-report analysis in a closed loop with specialized compile-fixer, runtime-fixer, judge, and optimizer agents [2602.06085]. In the SqueezeJet-2 flow, the hardware is benchmarked with SDSoC hardware tracing, Vivado HLS performance estimation, C/RTL co-simulation, and the custom analytical model [1902.03192].

System integration is equally central. Xilinx-centric flows commonly expose generated accelerators as IP blocks for integration into Vivado block designs or as Vitis kernels accompanied by host code. hls4ml supports both direct RTL/IP use and Vitis Accel packaging, where host code manages host-FPGA communication “either through DMA transfers or AXI streams” [2103.05579]. The QICK workflow wraps the hls4ml-generated `NN_hls4ml` core inside an `NN_axi` HLS module exposing AXI4-Lite configuration, AXI-Stream input, BRAM-backed output, and a direct trigger from the tProc; the integrated design on a Xilinx ZCU216 stores predictions in a 128 KB PL BRAM buffer accessible from Python through functions such as `reset_classifier`, `configure_classifier`, and `get_classifier_prediction` [2501.14663]. In MLPerf Tiny deployments, hls4ml and FINN accelerators are attached to AXI buses on Pynq-Z2 or to MicroBlaze/MIG subsystems on Arty A7-100T, and the processor loads data, starts execution, polls completion, and checks outputs [2206.11791].

A recurrent misconception is that “end-to-end” implies every stage of an application resides on the FPGA. The literature is more selective. In the Gemmini-based YOLOv7 deployment, the quantized main CNN body runs on Gemmini in programmable logic, while floating-point NMS remains on the Zynq ARM processing system via TVM runtime [2408.07404]. In the matrix-element study, the full event-evaluation numerical chain is on device only for \(e^+e^- \to \mu^+\mu^-\); for \(gg \to t\bar t + X\), only the color-reduction kernel is accelerated and the `jamp` amplitudes are precomputed externally [2605.23785]. “End-to-end” in FPGA workflow therefore usually means end-to-end within a chosen acceleration boundary, not elimination of the host.

## 5. Runtime orchestration, reconfiguration, and virtualization

Runtime control ranges from simple host invocation to explicit virtualization. fpgaConvNet offers two distinct deployment styles: graph partitioning with full FPGA reconfiguration between SDF subgraphs for throughput-driven execution, and weights reloading into a single flexible architecture for latency-sensitive execution [1711.08740]. The adaptive CNN compiler in [2504.08534] implements Online Design Reconfiguration by enabling and disabling layer blocks or filter subsets with clock gating and enable signals, supporting both depth-wise and width-wise execution modes without redeployment. Transparent Live Code Offloading changes functionality by downloading overlay configurations to a preloaded DFE, with one measured configuration download of 2.1 ms, rather than by generating a new FPGA fabric bitstream [1609.00130]. These mechanisms are technically distinct: full reconfiguration, overlay reconfiguration, and intra-design mode selection should not be conflated.

Multi-FPGA orchestration introduces another level of workflow complexity. FastFlow in FPGA stacks lets a programmer specify `FPGA-ID`, `Src`, `Dst`, and `Kernel Name` in `proc.csv`, plus interface and memory-slot data in `circuit.csv`; a script then generates `connectivity.cfg`, `host.cpp`, and the `.xclbin`, binding each `ff_node_fpga` to a specific FPGA in a stack and reducing manual coding effort by 96–97% across the paper’s examples [2409.20099]. FOS treats the FPGA as an operating-system-managed substrate with a static shell, partial-reconfiguration regions, relocatable modules, JSON metadata, generic drivers, and resource-elastic scheduling across space and time [2001.09990]. EVEREST extends this toward cluster execution by combining compiler-generated kernels and subsystem infrastructure with QEMU-KVM, libvirt, SR-IOV virtual functions, LEXIS workflow deployment, and a Dask-like runtime API [2402.12612].

SYNERGY generalizes virtualization further by transforming Verilog so that the running hardware can yield to software at sub-clock-tick granularity, enabling suspend/resume, migration, and spatial/temporal multiplexing on commodity FPGA platforms [2109.02484]. In this sense, an on-FPGA workflow can extend beyond compilation and invocation into lifecycle management of hardware jobs. A plausible implication is that FPGA deployment is increasingly treated as a runtime systems problem as much as a hardware-generation problem.

## 6. Optimization objectives, trade-offs, and persistent limitations

The dominant optimization axes are precision, parallelism, data movement, and structural specialization. hls4ml makes this explicit through layer-wise fixed-point configuration, reuse factor, sparse compression, quantization-aware training, and quantization-aware pruning. On a Xilinx Virtex UltraScale+ VU9P at 200 MHz, its jet-classification case study shows that a 16-bit PTQ model reaches 76.4% accuracy with 50 ns latency and uses 1,852 DSPs, whereas a 6-bit QAT model reaches 76.2% accuracy with 45 ns latency and uses only 38 DSPs [2103.05579]. fpgaConvNet tunes coarse-grained and fine-grained folding, graph partitioning, and weights reloading under FPGA resource and memory-bandwidth constraints [1711.08740]. fpgaHART adjusts stream counts \(s_i, s_o\), dot-product parallelism \(p_{mac}\), and partition boundaries for throughput-oriented 3D CNN deployment [2305.19896]. The adaptive CNN compiler [2504.08534] searches PE allocations per layer under DSP, LUT, and BRAM constraints, then uses width-wise or depth-wise operating modes at runtime.

These optimization freedoms come with limits. Multiple papers explicitly restrict operator coverage or model families. CNN2Gate supports convolution, max-pooling, ReLU, GEMM, and softmax, but not arbitrary modern operators [2004.04641]. hls4ml emphasizes dense networks, quantized models, pruned models, and references support for convolutional and graph neural networks, while noting that Xilinx Vivado HLS is its most advanced backend [2103.05579]. fpgaHART supports a useful subset of 3D CNN operations and is strongest on throughput-oriented HAR models rather than arbitrary neural workloads [2305.19896]. EVEREST frames acceleration at the level of selected kernels in larger workflows rather than complete monolithic applications [2402.12612]. Automation therefore reduces manual effort but does not remove the need to match the model class to the toolflow.

Another recurring controversy concerns the relation between automation and efficiency. The literature is explicit that a toolflow may trade peak efficiency for generality or runtime flexibility. fpgaHART reports competitive throughput across modern 3D CNNs but does not claim to universally beat hand-tuned accelerators [2305.19896]. CNN2Gate emphasizes automation, portability through ONNX, and automatic fitting across FPGA sizes rather than the highest raw performance [2004.04641]. Transparent Live Code Offloading is slower than its software baseline in the prototype video application—31 frames/s with FPGA offloading versus almost 83 frames/s in pure software—because of overlay overhead, PCIe transfer inefficiency, and software place-and-route cost [1609.00130]. The evidence therefore does not support a simple equation between “end-to-end workflow” and “best absolute performance.”

## 7. Representative application domains and broader significance

The application spread is unusually wide. In scientific low-power ML, hls4ml positions near-sensor processing as a way to improve experimental design and accelerate discovery [2103.05579]. In quantum control, the QICK workflow embeds a ternary \(800 \times 4 \times 1\) MLP directly into the FPGA-resident readout path on a Xilinx ZCU216 RFSoC, achieving 96% single-shot fidelity, 32 ns post-window latency, and less than 16% FPGA look-up table resource utilization for the classifier IP [2501.14663]. In edge vision, the Gemmini-based workflow deploys YOLOv7 on a Xilinx ZCU102 with real-time performance and 36.5 GOP/s/W, and then integrates that detector into a traffic-monitoring pipeline using Zephyr RTOS and ROS2 [2408.07404]. In embedded benchmark settings, open-source FPGA-ML codesign for MLPerf Tiny reports latencies as low as 20 \(\mu s\) and energy consumption as low as 30 \(\mu J\) per inference on Pynq-Z2 and Arty A7-100T [2206.11791].

Outside mainstream ML inference, the same workflow vocabulary appears in HPC and scientific computing. The matrix-element study on the AMD Alveo U250 reports 0.075 s for \(3\times 10^7\) \(e^+e^- \to \mu^+\mu^-\) events with 8 compute units, corresponding to \(4.01\times 10^8\) events/s, and \(0.18\ \mu\mathrm{J}\) per event for the best full-workflow FPGA configuration [2605.23785]. EVEREST extends the notion of on-FPGA workflow to big-data and cluster settings, where kernels from WRF, map matching, or PTDR are compiled, packaged, and scheduled across FPGA-based clusters with virtualization support [2402.12612]. These case studies show that the term now refers not only to board-level acceleration of a kernel, but to a broader stack that spans compilation, integration, orchestration, and application-level deployment.

Taken together, the literature defines on-FPGA workflow as a systems concept rather than a single compiler pass. It includes frontend import, graph or program normalization, hardware-aware optimization, generation of FPGA-resident compute structures, host or cluster orchestration, and, increasingly, runtime adaptation. What distinguishes current research is not merely that it places computation on an FPGA, but that it tries to make that placement inspectable, automatable, and operationally manageable across scientific instruments, embedded devices, datacenter stacks, and heterogeneous clusters [2103.05579] [2409.20099] [2109.02484].

Source: https://www.emergentmind.com/topics/on-fpga-workflow