FINN: A Framework for Fast, Scalable Binarized Neural Network Inference (1612.07119v1)

Abstract: Research has shown that convolutional neural networks contain significant redundancy, and high classification accuracy can be obtained even when weights and activations are reduced from floating point to binary values. In this paper, we present FINN, a framework for building fast and flexible FPGA accelerators using a flexible heterogeneous streaming architecture. By utilizing a novel set of optimizations that enable efficient mapping of binarized neural networks to hardware, we implement fully connected, convolutional and pooling layers, with per-layer compute resources being tailored to user-provided throughput requirements. On a ZC706 embedded FPGA platform drawing less than 25 W total system power, we demonstrate up to 12.3 million image classifications per second with 0.31 {\mu}s latency on the MNIST dataset with 95.8% accuracy, and 21906 image classifications per second with 283 {\mu}s latency on the CIFAR-10 and SVHN datasets with respectively 80.1% and 94.9% accuracy. To the best of our knowledge, ours are the fastest classification rates reported to date on these benchmarks.

FINN: A Framework for Fast, Scalable Binarized Neural Network Inference

The paper introduces Finn, a flexible, high-performance framework designed to implement Binarized Neural Networks (BNNs) on Field Programmable Gate Arrays (FPGAs). The authors present a detailed exploration of the challenges and methodologies involved in optimizing BNN inference while emphasizing the benefits of using FPGAs for such tasks.

Overview of Contributions

Finn is remarkable for multiple reasons:

1. Efficient Mapping of BNNs: The framework utilizes a set of novel optimizations to efficiently map BNNs onto hardware, which includes fully connected layers, convolutional layers, and pooling layers.
2. Scalability: The architecture allows for tailoring compute resources per layer, adapting to user-provided throughput requirements.
3. Performance Metrics: Finn on the ZC706 FPGA platform consumes less than 25 W total system power, achieving up to 12.3 million classifications per second with sub-microsecond latency and high accuracy on the MNIST dataset.
4. Architecture: Finn employs a heterogeneous streaming architecture designed to optimize classification throughput and latency.

Key Technical Details

Binarized Neural Networks (BNNs)

BNNs utilize binary values for weights and activations, significantly reducing the resource requirements compared to floating-point computations. This reduction offers advantages:

• Memory Efficiency: Binary parameters eliminate the off-chip memory bottleneck, enabling larger networks to fit into on-chip memory.
• Binary Operations: FPGAs excel at binary operations, providing a theoretical peak performance that surpasses floating-point operations.

Hardware Implementation

Finn makes use of several key FPGA-specific optimizations:

• Popcount for Accumulation: BNNs leverage the inherent properties of binary arithmetic to replace multiply-accumulate operations with popcount, conserving FPGA resources.
• Threshold-Based Activations: Instead of traditional batch normalization and activation, thresholds are precomputed, converting activations into simple threshold comparisons.
• Boolean OR for Max-Pooling: Max-pooling operations are simplified to Boolean OR operations, further reducing computational complexity.

The implementation involves several hardware units:

• Matrix-Vector-Threshold Unit (MVTU): Core computational unit for fully connected layers, employing parallel processing elements (PEs) and SIMD lanes to perform dot products followed by thresholding.
• Sliding Window Unit (SWU): Implements 2D convolutions by transforming input images into an appropriate format for the MVTU.
• Pooling Unit (PU): Executes max-pooling operations using optimized Boolean logic.

Performance and Resource Utilization

The authors provide a comprehensive performance evaluation featuring various BNN topologies, such as:

• SFC and LFC: Fully connected networks targeting the MNIST dataset, achieving high accuracy and exceptional classification rates.
• CNN Topologies: Convolutional networks for CIFAR-10 and SVHN, demonstrating competitive accuracy and significant throughput.

Key findings include:

• Throughput and Latency: Finn achieves unprecedented classification throughput, e.g., 12.3 million FPS for MNIST, with minimal latency (as low as 0.31 µs for certain configurations).
• Resource Efficiency: The framework's designs efficiently leverage available FPGA resources, consistently achieving high runtime efficiency and competitive resource allocation metrics.
• Energy Efficiency: Prototypes like SFC-max demonstrate impressive energy efficiency, achieving FPS per Watt values much superior to existing solutions.

Implications and Future Directions

The work lays a solid groundwork for deploying BNNs on FPGAs, highlighting the potential for real-time, energy-efficient applications in fields like robotics, augmented reality, and autonomous systems. The results also indicate areas for future research such as:

• Support for Non-Binary Precision: Exploring ternary or higher-precision quantization to balance between performance gains and accuracy.
• Scaling and Multi-FPGA Configurations: Addressing scenarios where BNN parameters exceed the available on-chip memory, potentially through external memory integration or distributed FPGA systems.
• Advanced Architectural Changes: Enhancing the SWU and other bottleneck components to further elevate the performance ceiling.

Conclusion

Finn exemplifies a methodical approach to optimizing BNNs for FPGA platforms, the architectural design, and practical implementations discussed in this paper provide valuable insights for ongoing research and development in efficient, scalable neural network inference. The authors bolster their claims with rigorous evaluation, comparing favorably against existing works while suggesting promising avenues for future advancements.