On-FPGA Workflow: End-to-End Design

Updated 5 July 2026

On-FPGA workflow is an end-to-end process that transforms high-level software or model descriptions into FPGA-resident implementations with integrated system orchestration.
It encompasses stages such as model import, quantization, intermediate representation construction, analytical design-space exploration, code generation, vendor-tool synthesis, and host-side management.
The workflow enables rapid prototyping, efficient design-space optimization, and dynamic runtime adaptations across applications in machine learning, HPC, and embedded systems.

On-FPGA workflow denotes the end-to-end process by which an algorithm, neural network, or application kernel is transformed from a high-level software or graph description into an FPGA-resident implementation, then integrated into a larger system, validated, deployed, and, in some systems, reconfigured at runtime. Across recent work, this workflow spans model import, quantization or pruning, intermediate-representation construction, analytical or search-based design-space exploration, code or RTL generation, vendor-tool synthesis, host-side orchestration, and, in some cases, virtualization, migration, or online adaptation (Fahim et al., 2021, Venieris et al., 2017, Ghaffari et al., 2020, Mazouz et al., 11 Apr 2025).

1. Front ends, inputs, and problem formulation

The entry point of an on-FPGA workflow depends on the application domain and compiler stack. In machine-learning flows, the input is often a trained model in a standard framework: hls4ml provides converters for “(Q)Keras, TensorFlow, PyTorch, and ONNX model formats,” fpgaConvNet expects a Caffe model description, CNN2Gate consumes ONNX exported from frameworks such as Keras, PyTorch, TensorFlow, Caffe, or Caffe2, and fpgaHART ingests a high-level network description from frameworks such as PyTorch or ONNX (Fahim et al., 2021, Venieris et al., 2017, Ghaffari et al., 2020, Toupas et al., 2023). In software-centric and systems workflows, the input can instead be general-purpose C++, OpenCL kernels, Verilog, LLVM IR, CSV topology specifications, or workflow-level task descriptions (Moraru et al., 4 Feb 2026, Qin et al., 2015, Landgraf et al., 2021, Rigamonti et al., 2016, Paul et al., 2024, Pilato et al., 2024).

A common feature is that the source description is paired with explicit deployment intent. fpgaConvNet requires a target FPGA specification and an optimization goal such as throughput, latency, or a multiobjective criterion; CNN2Gate requires a target FPGA board and user-provided fixed-point quantization values; EVEREST treats the application as a larger workflow and marks only selected kernels for acceleration; the adaptive CNN compiler in (Mazouz et al., 11 Apr 2025) takes user constraints such as latency bounds and resource limits (Venieris et al., 2017, Ghaffari et al., 2020, Pilato et al., 2024, Mazouz et al., 11 Apr 2025). This framing is technically important because the workflow is constrained from the beginning by device capacity, interface assumptions, and application-level objectives rather than by a purely algorithmic notion of correctness.

The resulting diversity of front ends can be summarized as follows.

Workflow	Primary input abstraction	FPGA-side output form
hls4ml	Trained model from (Q)Keras, TensorFlow, PyTorch, or ONNX	HLS project, IP core, or Vitis kernel (Fahim et al., 2021)
fpgaConvNet	Caffe CNN, target FPGA, optimization goal	Streaming accelerator, single flexible bitstream, or partition-specific bitstreams (Venieris et al., 2017)
CNN2Gate	ONNX model, quantization values, target Intel FPGA	OpenCL project, RTL/bitstream, executable FPGA implementation (Ghaffari et al., 2020)
FastFlow + Vitis	`proc.csv`, `circuit.csv`, hardware kernel sources	`host.cpp`, `connectivity.cfg`, `.xclbin` (Paul et al., 2024)
ODE/ODR compiler	Pretrained CNN plus latency/resource constraints	RTL/HDL plus runtime-selectable execution modes (Mazouz et al., 11 Apr 2025)

2. Intermediate representations and compile-time reasoning

The central abstraction in most on-FPGA workflows is an intermediate representation that separates frontend semantics from backend synthesis. In hls4ml, imported models are transformed into a common internal graph representation called an “HLSModel,” to which hardware-related configuration is attached; this graph then undergoes simplifications such as batch-normalization fusion and compile-time folding of constant operations (Fahim et al., 2021). fpgaConvNet uses Synchronous Dataflow (SDF) as the common formalism for both CNN workload representation and hardware mapping, with the implementation represented by a topology matrix

$\boldsymbol{\Gamma} \in \mathbb{R}^{(M \times N)},$

and a workload matrix $\boldsymbol{W}$ that encodes the total amount of work performed by each hardware block during inference (Venieris et al., 2017). fpgaHART extends the same idea to 3D CNNs by lowering a network DAG into an SDFG and factorizing the topology as

$\Gamma = S \times R,$

where $S$ captures stream multiplicity and $R$ captures normalized rates (Toupas et al., 2023).

Other systems use different IRs for the same purpose. CNN2Gate relies on ONNX as an interchange graph and maintains an intermediate hardware configuration through linked layer structures plus generated OpenCL configuration data (Ghaffari et al., 2020). EVEREST converges multiple front ends into MLIR dialects such as ekl, dfg, teil, esn, and jabbah, so that big-data, HPC, and ML abstractions can be optimized in a common compiler pipeline (Pilato et al., 2024). FOS uses JSON descriptors for both shell and accelerator metadata, decoupling software tooling from vendor-generated implementation artifacts (Vaishnav et al., 2020). These differences in representation encode a shared design principle: on-FPGA workflow is feasible only when the compiler or runtime can reason about structure, data movement, and resource use before RTL is frozen.

Analytical modeling is the second compile-time pillar. fpgaConvNet attaches throughput, latency, and resource models to the SDF graph and explores graph partitioning, coarse-grained folding, fine-grained folding, and weights reloading without implementing every candidate (Venieris et al., 2017). fpgaHART computes an initiation-interval matrix by

$II = W / \Gamma$

and estimates partition execution time and throughput from the largest matrix entry $II_{max}$ , batch size, and reconfiguration cost (Toupas et al., 2023). The adaptive CNN workflow in (Mazouz et al., 11 Apr 2025) derives explicit conv, FC, and pipeline latency/resource equations and uses a multi-objective genetic algorithm to search feasible mappings. SqueezeJet-2 similarly derives an HLS-informed analytical model from pipeline depths, trip counts, and function-call overheads and reports that its model stays within 8.3% maximum absolute percentage error from hardware-trace latency, whereas Vivado HLS worst-case performance estimation reaches 46.1% error and C/RTL co-simulation reaches 33.4% (Mousouliotis et al., 2019). Such models do not eliminate synthesis, but they change the workflow from blind compilation to guided exploration.

3. FPGA-side realization patterns

On-FPGA workflows do not converge to a single microarchitectural template. hls4ml generates model-specific firmware in which each layer becomes a customized module specialized to the specific dimensions, precision, and often the specific trained weights; the full model is synthesized into an IP core rather than a generic programmable matrix engine (Fahim et al., 2021). fpgaConvNet and fpgaHART instead generate streaming accelerators composed of hardware building blocks connected by streams, with SDF actors corresponding to layer fragments or layer-level engines (Venieris et al., 2017, Toupas et al., 2023). CNN2Gate maps ONNX operators to an OpenCL kernel library comprising memory read, memory write, convolution, pooling, and fully connected functions, interconnected by OpenCL pipes implemented as FIFOs on FPGA (Ghaffari et al., 2020).

A second class of workflow maps computations onto reusable accelerator fabrics. Gemmini is a systolic-array-based accelerator with load, execute, and store controllers attached to a Rocket core, and the workflow in (Peccia et al., 2024) lowers quantized CNN operators to Gemmini instructions through TVM and microTVM. SqueezeJet-2, by contrast, is a streaming convolution/max-pool accelerator with line buffers, double-buffered windows, and $16$ PEs $\times 16$ MACs/PE on a small Zynq SoC (Mousouliotis et al., 2019). Transparent Live Code Offloading uses a pre-programmed Data Flow Engine (DFE) overlay that can be reconfigured by software place-and-route at the overlay level, avoiding full bitstream regeneration (Rigamonti et al., 2016). This suggests that “on-FPGA” may mean direct synthesis of fixed datapaths, placement onto an intermediate overlay, or lowering onto a preexisting accelerator ISA.

The same diversity appears outside neural inference. The matrix-element study on an AMD Alveo U250 organizes the FPGA as a four-stage hls::stream dataflow pipeline—input loader, phase-space generation, compute stage, output writer—and maps either the full $e^+e^- \to \mu^+\mu^-$ event-evaluation chain or an isolated color-reduction kernel for $\boldsymbol{W}$ 0 (Arance et al., 22 May 2026). The FastFlow+Vitis workflow, in contrast, leaves computation kernels in user-provided HDL, C++, or SystemC and automates only host-side orchestration, memory connectivity, and multi-FPGA routing (Paul et al., 2024). The architectural consequence is that an on-FPGA workflow may be centered on hardware generation, host control generation, or the boundary between the two.

4. Verification, software integration, and deployment mechanics

Verification is typically staged, with increasingly hardware-faithful checkpoints before board execution. hls4ml supports bit-accurate emulation of the generated HLS-synthesizable inference code directly from Python, allowing comparison against the original floating-point or quantized model and inspection of hidden-layer outputs before vendor synthesis (Fahim et al., 2021). CNN2Gate provides CPU emulation mode through Intel’s OpenCL toolchain, validating functionality and accuracy much faster than full FPGA synthesis, followed by full-flow compilation to board-specific RTL and bitstream artifacts (Ghaffari et al., 2020). LAAFD performs host compilation, functional testing against the original C++ kernel, then HLS co-simulation and synthesis-report analysis in a closed loop with specialized compile-fixer, runtime-fixer, judge, and optimizer agents (Moraru et al., 4 Feb 2026). In the SqueezeJet-2 flow, the hardware is benchmarked with SDSoC hardware tracing, Vivado HLS performance estimation, C/RTL co-simulation, and the custom analytical model (Mousouliotis et al., 2019).

System integration is equally central. Xilinx-centric flows commonly expose generated accelerators as IP blocks for integration into Vivado block designs or as Vitis kernels accompanied by host code. hls4ml supports both direct RTL/IP use and Vitis Accel packaging, where host code manages host-FPGA communication “either through DMA transfers or AXI streams” (Fahim et al., 2021). The QICK workflow wraps the hls4ml-generated NN_hls4ml core inside an NN_axi HLS module exposing AXI4-Lite configuration, AXI-Stream input, BRAM-backed output, and a direct trigger from the tProc; the integrated design on a Xilinx ZCU216 stores predictions in a 128 KB PL BRAM buffer accessible from Python through functions such as reset_classifier, configure_classifier, and get_classifier_prediction (Guglielmo et al., 24 Jan 2025). In MLPerf Tiny deployments, hls4ml and FINN accelerators are attached to AXI buses on Pynq-Z2 or to MicroBlaze/MIG subsystems on Arty A7-100T, and the processor loads data, starts execution, polls completion, and checks outputs (Borras et al., 2022).

A recurrent misconception is that “end-to-end” implies every stage of an application resides on the FPGA. The literature is more selective. In the Gemmini-based YOLOv7 deployment, the quantized main CNN body runs on Gemmini in programmable logic, while floating-point NMS remains on the Zynq ARM processing system via TVM runtime (Peccia et al., 2024). In the matrix-element study, the full event-evaluation numerical chain is on device only for $\boldsymbol{W}$ 1; for $\boldsymbol{W}$ 2, only the color-reduction kernel is accelerated and the jamp amplitudes are precomputed externally (Arance et al., 22 May 2026). “End-to-end” in FPGA workflow therefore usually means end-to-end within a chosen acceleration boundary, not elimination of the host.

5. Runtime orchestration, reconfiguration, and virtualization

Runtime control ranges from simple host invocation to explicit virtualization. fpgaConvNet offers two distinct deployment styles: graph partitioning with full FPGA reconfiguration between SDF subgraphs for throughput-driven execution, and weights reloading into a single flexible architecture for latency-sensitive execution (Venieris et al., 2017). The adaptive CNN compiler in (Mazouz et al., 11 Apr 2025) implements Online Design Reconfiguration by enabling and disabling layer blocks or filter subsets with clock gating and enable signals, supporting both depth-wise and width-wise execution modes without redeployment. Transparent Live Code Offloading changes functionality by downloading overlay configurations to a preloaded DFE, with one measured configuration download of 2.1 ms, rather than by generating a new FPGA fabric bitstream (Rigamonti et al., 2016). These mechanisms are technically distinct: full reconfiguration, overlay reconfiguration, and intra-design mode selection should not be conflated.

Multi-FPGA orchestration introduces another level of workflow complexity. FastFlow in FPGA stacks lets a programmer specify FPGA-ID, Src, Dst, and Kernel Name in proc.csv, plus interface and memory-slot data in circuit.csv; a script then generates connectivity.cfg, host.cpp, and the .xclbin, binding each ff_node_fpga to a specific FPGA in a stack and reducing manual coding effort by 96–97% across the paper’s examples (Paul et al., 2024). FOS treats the FPGA as an operating-system-managed substrate with a static shell, partial-reconfiguration regions, relocatable modules, JSON metadata, generic drivers, and resource-elastic scheduling across space and time (Vaishnav et al., 2020). EVEREST extends this toward cluster execution by combining compiler-generated kernels and subsystem infrastructure with QEMU-KVM, libvirt, SR-IOV virtual functions, LEXIS workflow deployment, and a Dask-like runtime API (Pilato et al., 2024).

SYNERGY generalizes virtualization further by transforming Verilog so that the running hardware can yield to software at sub-clock-tick granularity, enabling suspend/resume, migration, and spatial/temporal multiplexing on commodity FPGA platforms (Landgraf et al., 2021). In this sense, an on-FPGA workflow can extend beyond compilation and invocation into lifecycle management of hardware jobs. A plausible implication is that FPGA deployment is increasingly treated as a runtime systems problem as much as a hardware-generation problem.

6. Optimization objectives, trade-offs, and persistent limitations

The dominant optimization axes are precision, parallelism, data movement, and structural specialization. hls4ml makes this explicit through layer-wise fixed-point configuration, reuse factor, sparse compression, quantization-aware training, and quantization-aware pruning. On a Xilinx Virtex UltraScale+ VU9P at 200 MHz, its jet-classification case study shows that a 16-bit PTQ model reaches 76.4% accuracy with 50 ns latency and uses 1,852 DSPs, whereas a 6-bit QAT model reaches 76.2% accuracy with 45 ns latency and uses only 38 DSPs (Fahim et al., 2021). fpgaConvNet tunes coarse-grained and fine-grained folding, graph partitioning, and weights reloading under FPGA resource and memory-bandwidth constraints (Venieris et al., 2017). fpgaHART adjusts stream counts $\boldsymbol{W}$ 3, dot-product parallelism $\boldsymbol{W}$ 4, and partition boundaries for throughput-oriented 3D CNN deployment (Toupas et al., 2023). The adaptive CNN compiler (Mazouz et al., 11 Apr 2025) searches PE allocations per layer under DSP, LUT, and BRAM constraints, then uses width-wise or depth-wise operating modes at runtime.

These optimization freedoms come with limits. Multiple papers explicitly restrict operator coverage or model families. CNN2Gate supports convolution, max-pooling, ReLU, GEMM, and softmax, but not arbitrary modern operators (Ghaffari et al., 2020). hls4ml emphasizes dense networks, quantized models, pruned models, and references support for convolutional and graph neural networks, while noting that Xilinx Vivado HLS is its most advanced backend (Fahim et al., 2021). fpgaHART supports a useful subset of 3D CNN operations and is strongest on throughput-oriented HAR models rather than arbitrary neural workloads (Toupas et al., 2023). EVEREST frames acceleration at the level of selected kernels in larger workflows rather than complete monolithic applications (Pilato et al., 2024). Automation therefore reduces manual effort but does not remove the need to match the model class to the toolflow.

Another recurring controversy concerns the relation between automation and efficiency. The literature is explicit that a toolflow may trade peak efficiency for generality or runtime flexibility. fpgaHART reports competitive throughput across modern 3D CNNs but does not claim to universally beat hand-tuned accelerators (Toupas et al., 2023). CNN2Gate emphasizes automation, portability through ONNX, and automatic fitting across FPGA sizes rather than the highest raw performance (Ghaffari et al., 2020). Transparent Live Code Offloading is slower than its software baseline in the prototype video application—31 frames/s with FPGA offloading versus almost 83 frames/s in pure software—because of overlay overhead, PCIe transfer inefficiency, and software place-and-route cost (Rigamonti et al., 2016). The evidence therefore does not support a simple equation between “end-to-end workflow” and “best absolute performance.”

7. Representative application domains and broader significance

The application spread is unusually wide. In scientific low-power ML, hls4ml positions near-sensor processing as a way to improve experimental design and accelerate discovery (Fahim et al., 2021). In quantum control, the QICK workflow embeds a ternary $\boldsymbol{W}$ 5 MLP directly into the FPGA-resident readout path on a Xilinx ZCU216 RFSoC, achieving 96% single-shot fidelity, 32 ns post-window latency, and less than 16% FPGA look-up table resource utilization for the classifier IP (Guglielmo et al., 24 Jan 2025). In edge vision, the Gemmini-based workflow deploys YOLOv7 on a Xilinx ZCU102 with real-time performance and 36.5 GOP/s/W, and then integrates that detector into a traffic-monitoring pipeline using Zephyr RTOS and ROS2 (Peccia et al., 2024). In embedded benchmark settings, open-source FPGA-ML codesign for MLPerf Tiny reports latencies as low as 20 $\boldsymbol{W}$ 6 and energy consumption as low as 30 $\boldsymbol{W}$ 7 per inference on Pynq-Z2 and Arty A7-100T (Borras et al., 2022).

Outside mainstream ML inference, the same workflow vocabulary appears in HPC and scientific computing. The matrix-element study on the AMD Alveo U250 reports 0.075 s for $\boldsymbol{W}$ 8 $\boldsymbol{W}$ 9 events with 8 compute units, corresponding to $\Gamma = S \times R,$ 0 events/s, and $\Gamma = S \times R,$ 1 per event for the best full-workflow FPGA configuration (Arance et al., 22 May 2026). EVEREST extends the notion of on-FPGA workflow to big-data and cluster settings, where kernels from WRF, map matching, or PTDR are compiled, packaged, and scheduled across FPGA-based clusters with virtualization support (Pilato et al., 2024). These case studies show that the term now refers not only to board-level acceleration of a kernel, but to a broader stack that spans compilation, integration, orchestration, and application-level deployment.

Taken together, the literature defines on-FPGA workflow as a systems concept rather than a single compiler pass. It includes frontend import, graph or program normalization, hardware-aware optimization, generation of FPGA-resident compute structures, host or cluster orchestration, and, increasingly, runtime adaptation. What distinguishes current research is not merely that it places computation on an FPGA, but that it tries to make that placement inspectable, automatable, and operationally manageable across scientific instruments, embedded devices, datacenter stacks, and heterogeneous clusters (Fahim et al., 2021, Paul et al., 2024, Landgraf et al., 2021).