LeFlow: Enabling Flexible FPGA High-Level Synthesis of Tensorflow Deep Neural Networks (1807.05317v1)

Abstract: Recent work has shown that Field-Programmable Gate Arrays (FPGAs) play an important role in the acceleration of Machine Learning applications. Initial specification of machine learning applications are often done using a high-level Python-oriented framework such as Tensorflow, followed by a manual translation to either C or RTL for synthesis using vendor tools. This manual translation step is time-consuming and requires expertise that limit the applicability of FPGAs in this important domain. In this paper, we present an open-source tool-flow that maps numerical computation models written in Tensorflow to synthesizable hardware. Unlike other tools, which are often constrained by a small number of inflexible templates, our flow uses Google's XLA compiler which emits LLVM code directly from a Tensorflow specification. This LLVM code can then be used with a high-level synthesis tool to automatically generate hardware. We show that our flow allows users to generate Deep Neural Networks with very few lines of Python code.

• The paper presents LeFlow’s automated toolchain that converts TensorFlow models to FPGA-compatible designs using high-level synthesis.
• It leverages XLA’s LLVM-based conversion to streamline the development of deep neural networks, demonstrated through MLP and CNN benchmarks.
• The study highlights trade-offs in performance metrics like clock frequency, power, and area, promoting FPGA adoption in machine learning.

An Overview of LeFlow: High-Level Synthesis from TensorFlow to FPGAs

LeFlow offers a significant advancement in the synthesis of deep neural networks (DNNs) on Field-Programmable Gate Arrays (FPGAs) by facilitating the translation from high-level TensorFlow models directly to hardware descriptions. This paper introduces an open-source toolchain leveraging Google's Accelerated Linear Algebra (XLA) compiler to convert TensorFlow code to LLVM intermediate representation (IR), which can then be processed by high-level synthesis (HLS) tools to produce FPGA-compatible hardware schematics.

High-Level Synthesis Challenge & LeFlow's Contributions

FPGA implementations of DNNs provide power efficiency and speed advantages over traditional software-based implementations. Despite these benefits, translating state-of-the-art machine learning models to FPGAs poses challenges due to the intricate manual coding required, often necessitating hardware design expertise. This gap often limits FPGA utilization in this domain.

LeFlow addresses these issues by automating the hardware generation procedure using TensorFlow specifications. The paper highlights several key contributions:

1. Tool-kit Description: Decouples hardware generation from manual coding efforts through a Python interface, leveraging XLA's LLVM code generation capabilities, ultimately streamlining rapid prototyping of DNNs on FPGAs.
2. Application Evaluation: Provides examples demonstrating LeFlow's implementation, such as a multilayer perceptron (MLP) for digit recognition using the MNIST dataset and a convolutional neural network (CNN).
3. Performance Benchmarks: Establishes benchmarks to evaluate the efficiency of LeFlow, offering a suite of tests that gauge performance metrics and functionality.
4. Access to the Tool: Facilitates community engagement by offering access to LeFlow's code repository, encouraging further development and customization.

Strong Numerical Results and Tool Integration

The integration of LeFlow with LegUp HLS tools demonstrates notable throughput and area efficiency. The paper's performance evaluations reveal that generated circuits from LeFlow can achieve competitive clock frequencies and resource utilization, albeit with some trade-offs in power and throughput compared to bespoke hardware designs. The examples provided, including MLP and CNN implementations, illustrate how these models can be prototyped quickly with minimal coding effort.

Implications and Future Directions

The implications of deploying LeFlow are noteworthy:

• Practical Applications: LeFlow paves the way for FPGA usage in domains typically restricted to CPU or GPU implementations, expanding FPGA applications in machine learning for rapid prototyping and potentially real-time processing tasks.
• Theoretical Impact: Simplifies the transition from algorithm design to hardware implementation, which could encourage broader adoption of hardware-specific optimizations in machine learning workflows.

Looking forward, enhancements to LeFlow could involve improvements in automatic memory partitioning algorithms and support for fixed-point arithmetic, which are crucial for optimizing FPGA designs further. Additionally, the development of an FPGA-specific kernel within XLA could enhance performance gains attained through hardware acceleration.

Conclusions

LeFlow stands as a pivotal advancement in lowering the barrier for hardware acceleration of DNNs on FPGAs for designers without hardware expertise. The integration of TensorFlow and LLVM within this toolkit opens substantial opportunities for tool evolution and adoption in research and industry. As more DNN kernels and optimization strategies are supported by XLA and HLS frameworks, tools like LeFlow will play pivotal roles in the intersection of machine learning and hardware development.