Tensil: FPGA Accelerator for Edge AI
- Tensil is an open-source framework that integrates a parameterizable systolic-array accelerator with a full software stack for mapping neural networks onto FPGAs.
- It streamlines the workflow from model training and ONNX conversion to Dockerized compilation and Vivado synthesis, targeting applications in biomedical and few-shot learning.
- Performance evaluations on platforms like PYNQ-Z1 and ZCU104 demonstrate low latency, high throughput, and power efficiency, though precision flexibility remains a challenge.
In the FPGA and edge-AI literature, Tensil denotes an open-source framework for running machine learning models on custom accelerator architectures. Across the cited works, it is presented as a combined hardware-and-software stack: a parameterizable accelerator instantiated in FPGA programmable logic, a compiler that maps trained neural networks onto that accelerator, and a runtime/driver layer that integrates execution with Zynq-based systems such as the PYNQ-Z1 and ZCU104. Tensil is used to compile and deploy convolutional backbones for biomedical inference and few-shot learning, and it also serves as a baseline against which later arbitrary-bit-width FPGA design flows are compared (Ribeiro et al., 2024, Inadagbo et al., 2023, Isik et al., 2023).
1. Architectural concept and terminology
Tensil is described as both an open-source ML inference accelerator and an open-source framework for running machine learning models on custom accelerator architectures. In the biomedical deployment literature, the hardware side appears as a Tensor Compute Unit (TCU), a custom accelerator block synthesized for the PYNQ-Z1 and placed in the programmable logic region of the Zynq-7000 SoC. In the few-shot-learning literature, the same overall role is expressed through a parameterizable systolic-array accelerator specified by an architecture description file, .tarch, which captures the number of Processing Elements, data format, and memory size (Alici et al., 2023, Ribeiro et al., 2024).
The software side is correspondingly structured around model compilation. Tensil takes a trained network, compiles it to an internal execution format, and emits deployment artifacts such as .tmodel, .tprog, and .tdata. In the PEFSL workflow, the model interchange path is PyTorch to ONNX to ONNX simplifier to Tensil; in the ECG workflow, the model is trained in a standard deep-learning framework and then compiled inside a Dockerized Tensil toolchain for deployment onto PYNQ. The cited papers consistently treat Tensil as an end-to-end deployment stack rather than as an isolated kernel library (Alici et al., 2023, Ribeiro et al., 2024).
This architecture is intentionally heterogeneous. The ARM processing system executes orchestration, data transfer, and host-side logic, while the Tensil accelerator in programmable logic executes the backbone tensor operations. This division is central in both the ECG and few-shot-learning deployments: convolutional inference runs in Tensil, whereas ancillary logic such as support-set handling or Nearest Class Mean classification remains on the CPU side in the current PEFSL pipeline (Ribeiro et al., 2024).
2. Compilation, synthesis, and deployment workflow
The Tensil deployment flow is described at two levels of granularity. In the biomedical FPGA studies, the workflow begins with a Dockerized Tensil toolchain, followed by selection of a TCU architecture appropriate for the PYNQ-Z1 resource budget, synthesis and implementation in Vivado, PS–PL configuration through AXI interfaces and a smart interconnect, and finally model compilation into .tmodel, .tprog, and .tdata for runtime execution in the PYNQ Python environment (Alici et al., 2023).
In PEFSL, the same logic is organized into a more explicit three-part pipeline. Part A trains the backbone in PyTorch, exports it to ONNX, simplifies the graph with ONNX simplifier, and compiles the model with Tensil using a .tarch file. Part B compiles the architecture itself and generates RTL files for the Tensil accelerator IP. Part C integrates that RTL IP into a Vivado SoC design and uses the PYNQ driver for data transfer between CPU and FPGA. The Tensil compiler also produces cycle-count or latency estimates, and these estimates are used for design-space exploration of backbone variants before deployment (Ribeiro et al., 2024).
The deployment boundary conditions are therefore unusually explicit for an open-source FPGA inference flow. Tensil is not only a backend for executing a precompiled graph; it also participates in architecture selection, RTL generation, memory sizing, and PS–PL interfacing. This suggests that Tensil occupies an intermediate position between fixed accelerator overlays and fully custom RTL development: it exposes architecture-level parameters while hiding most low-level HDL design.
3. Hardware organization, memory hierarchy, and execution model
The papers do not provide a unified microarchitectural specification of Tensil, but they do identify recurring structural elements. In the PEFSL line of work, Tensil is associated with a systolic-array matrix engine whose architecture is parameterized through .tarch. In the ZCU104 optimization study, the instantiated design contains a 32×32 systolic MAC array, local memory, accumulators, and AXI-connected DRAM and instruction interfaces; the data ports are mapped to Zynq AXI HP ports and the instruction stream is supplied through AXI DMA (Isik et al., 2023, Ribeiro et al., 2024).
A concise summary of representative published configurations is useful:
| Context | Platform | Salient Tensil configuration |
|---|---|---|
| ECG inference | PYNQ-Z1 | TCU in PL, compiled .tmodel/.tprog/.tdata, 100 MHz |
| Few-shot backbone deployment | PYNQ-Z1 | .tarch-defined systolic array, 12×12 in demonstrator, 16-bit fixed-point with 8 integer bits |
| ResNet20 optimization study | ZCU104 | 32×32 systolic MAC array, DRAM0/DRAM1 plus instruction DMA, URAM-enhanced local memory |
The execution model is memory-sensitive. In the ZCU104 study, the default Tensil compiler is characterized as weight-stationary: weights are kept in local memory while activations are reloaded across partitions when local memory is insufficient. The baseline BRAM-centric design uses 16 KV local memory and 4 KV accumulators; an optimized variant redirects local memory to UltraRAM and accumulators to BRAM, yielding about 48 KV local memory and about 20 KV accumulators. This reduces partitioning and DRAM traffic by allowing more layers or larger layer partitions to remain resident on chip (Isik et al., 2023).
The same study also introduces a dual-clock solution in which the Tensil RTL block remains at 100 MHz while the AXI side runs at a higher frequency, increasing effective data bandwidth between processing system memory and programmable logic. The reported effect is a rise from 133.54 FPS to 152.04 FPS before the larger memory and compiler optimizations are added. This exposes one of the central properties of Tensil as used in practice: performance is governed not only by MAC-array size but by the interaction between compilation strategy, local memory capacity, and PS–PL transfer bandwidth (Isik et al., 2023).
4. Workloads and published deployments
Tensil has been used most concretely for convolution-dominated workloads. In the ECG studies, four model families are trained and evaluated in software—LSTM, CNN, RNN, and DBN—but only the CNN, specifically a ResNet-20 variant, is actually mapped to Tensil’s TCU and deployed on FPGA. The deployed model uses 2-D convolution, pooling, fully connected layers, and ReLU activation, and the authors explicitly identify Tensil as the hardware mechanism that executes the MAC-dominated convolutional and dense computations (Alici et al., 2023).
The ECG deployment is technically specific. The workload is derived from the MIT-BIH Arrhythmia Database, with 187 input samples per beat in the FPGA experiments. For the reported PYNQ-Z1 implementation, the published comparison table gives a 2-D convolutional model with ReLu activation, 47,560 MACs, a 100 MHz clock, 99.1% accuracy, and 1.53 W power. Resource utilization on the Zynq-7000 SoC is reported as 23.75% LUT, 18.85% FF, 41.64% BRAM, 24% IO, and 53.13% DSP. The same paper positions FPGAs as “ideal for real-time processing and large data transfers” and “high-throughput, power-efficient computing applications,” with Tensil serving as the concrete implementation vehicle for that claim (Alici et al., 2023).
In PEFSL, the mapped workload is different but structurally similar. Tensil accelerates ResNet-9 or ResNet-12 backbones for few-shot learning on FPGA SoCs, while the Nearest Class Mean classifier remains on the CPU. The published demonstrator on PYNQ-Z1 uses a ResNet-9 backbone with 16 feature maps, a 12×12 systolic array, and 16-bit fixed-point with 8 bits designated for the integer part; it reports 30 ms backbone latency and 6.2 W for the full system including SoC, camera, and screen, with a live frame rate of about 16 FPS (Ribeiro et al., 2024).
A plausible implication is that Tensil has been most effective in the cited literature where the dominant computation can be expressed as convolutions and dense operations, and where host-side orchestration can tolerate leaving some classifier or preprocessing stages outside the accelerator.
5. Numeric formats, quantization constraints, and design-space pressure
A major theme in the later literature is that Tensil constrains the numeric design space. In the fixed-point few-shot-learning study, the hardware path using Tensil supports only two fixed-point formats in the current design flow: 8 integer bits plus 8 fractional bits, and 16 integer bits plus 16 fractional bits, i.e. and . Brevitas is introduced as a software-side quantization engine so that quantization-aware training and post-training quantization can mirror the fixed-point behavior that Tensil actually implements in hardware (Kanda et al., 1 Feb 2026).
That study also reports that software validation at lower precisions remains competitive with floating-point baselines. On miniImageNet and CIFAR-FS, both QAT and PTQ at 5+5 or 6+6 bits produce accuracies close to float32, whereas 3+3 bits cause severe degradation, especially for PTQ. The stated conclusion is that accuracy comparable to floating-point operations can be maintained even with 6 bits or 5 bits each, indicating potential for further reduction in computational resources. In other words, Tensil’s current precision options are treated as conservative rather than intrinsically necessary (Kanda et al., 1 Feb 2026).
The next step in this line of work is explicitly to move beyond Tensil’s precision rigidity. The FINN-based successor environment characterizes the Tensil-based conventional design environment as fixed to 16-bit or 32-bit implementations and DRAM-centric, and adopts FINN to enable arbitrary fixed-point bit-widths. In the reported CIFAR-10 few-shot benchmark on PYNQ-Z1, the Tensil baseline uses 16-bit precision, achieves 62.78% accuracy, 35.9 ms backbone latency, and consumes 159 DSPs, whereas the FINN implementation uses 6-bit convolution and 4-bit ReLU, achieves 59.70% accuracy, and reduces backbone latency to 16.3 ms while using 22 DSPs; the paper summarizes the throughput gain as approximately twice that of the conventional realization (Kanda et al., 17 Feb 2026).
These papers do not imply that Tensil is unusable for low-power edge AI; rather, they argue that Tensil’s current fixed-point granularity and DRAM-centric organization limit more aggressive co-design between quantization and hardware. That critique is technical rather than rhetorical, and it is grounded in direct accuracy and latency comparisons.
6. Performance profile, limitations, and subsequent trajectory
The Tensil literature spans low-power embedded deployments and more performance-oriented FPGA studies. On ZCU104, a Tensil-based ResNet20 design optimized through a 32×32 systolic MAC array, UltraRAM use, dual-clock data movement, and revised compiler strategy is reported to achieve a frame rate of 293.58 FPS, about 90% accuracy on CIFAR, and 21.12 GOP/s with 5.21 W on-chip power consumption at 100 MHz (Isik et al., 2023). On PYNQ-Z1, the PEFSL demonstrator reports 30 ms backbone latency and 6.2 W system power, while the ECG deployment reports 99.1% accuracy at 1.53 W with moderate LUT/FF usage and substantially higher BRAM and DSP occupancy (Ribeiro et al., 2024, Alici et al., 2023).
Several limitations recur across the papers. First, Tensil is usually treated as a black-box accelerator/compiler framework; the cited works do not go deeply into its microarchitecture or instruction set. Second, current deployments cover only a subset of the models explored in software: in the ECG case, only CNN/ResNet-20 is mapped to Tensil, whereas LSTM, RNN, and DBN remain CPU/GPU workloads. Third, the current few-shot-learning hardware path restricts fixed-point choice to and , which later studies identify as over-provisioned for the accuracy actually required (Alici et al., 2023, Kanda et al., 1 Feb 2026).
The development trajectory therefore proceeds in two directions. One is internal optimization of Tensil itself: improved memory hierarchy, better scheduling, dual-clock transfer schemes, Dynamic Partial Reconfiguration, broader model compatibility, and enhanced usability are all named as future directions in the ECG and ZCU104 studies. The other is migration to more flexible frameworks when arbitrary bit-widths or fully streaming BRAM-centric dataflows are required; the FINN-based design environment is the clearest example of this second path (Isik et al., 2023, Kanda et al., 17 Feb 2026).
Taken together, the cited literature presents Tensil as a significant open-source FPGA inference stack that made end-to-end deployment on modest Zynq platforms practical, especially for convolutional workloads. At the same time, the same literature documents the pressure points that emerge as workloads shift toward tighter edge budgets and finer quantization co-design: precision flexibility, memory locality, and graph-level customization become decisive, and those requirements increasingly define the boundary of Tensil’s present envelope.