FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review (1901.00121v1)

Abstract: Due to recent advances in digital technologies, and availability of credible data, an area of artificial intelligence, deep learning, has emerged, and has demonstrated its ability and effectiveness in solving complex learning problems not possible before. In particular, convolution neural networks (CNNs) have demonstrated their effectiveness in image detection and recognition applications. However, they require intensive CPU operations and memory bandwidth that make general CPUs fail to achieve desired performance levels. Consequently, hardware accelerators that use application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and graphic processing units (GPUs) have been employed to improve the throughput of CNNs. More precisely, FPGAs have been recently adopted for accelerating the implementation of deep learning networks due to their ability to maximize parallelism as well as due to their energy efficiency. In this paper, we review recent existing techniques for accelerating deep learning networks on FPGAs. We highlight the key features employed by the various techniques for improving the acceleration performance. In addition, we provide recommendations for enhancing the utilization of FPGAs for CNNs acceleration. The techniques investigated in this paper represent the recent trends in FPGA-based accelerators of deep learning networks. Thus, this review is expected to direct the future advances on efficient hardware accelerators and to be useful for deep learning researchers.

• The paper presents FPGA-based acceleration methods that reduce CNN convolution time, which is a major computational bottleneck.
• It details techniques such as parallel multiply-accumulate operations, bit-width reduction, and advanced buffering to enhance throughput and resource efficiency.
• The review recommends automated synthesis frameworks and optimization algorithms, paving the way for scalable and resource-efficient deep learning systems.

Review of Recent Developments in FPGA-based Acceleration of Convolutional Neural Networks (CNNs)

The paper offers an extensive survey on the leveraging of Field Programmable Gate Arrays (FPGAs) to accelerate Convolutional Neural Networks (CNNs). These networks are pivotal in deep learning domains such as image detection and speech recognition, but they require significant computational and memory resources. The authors analyze numerous approaches to optimize CNN performance on FPGAs, which inherently support parallel computation.

The document highlights convolution as the primary computational burden in CNNs—often accounting for over 90% of the computation time—thereby necessitating its acceleration. Several optimization techniques are discussed, including parallel multiply-accumulate operations, data reuse to minimize memory bandwidth usage, and bit-width reduction for feature maps and weights to conserve memory resources. The paper also explores the use of singular value decomposition (SVD) to reduce memory constraints in fully connected layers.

Furthermore, the review comprehensively examines various FPGA-based acceleration frameworks. For instance, the Roofline model is deployed to identify the optimal convolution layer design, while DeepBurning automates hardware synthesis using a compiled library, and ALAMO offers a modularized RTL compiler alternative for both ASIC and FPGA platforms. Tools like these showcase the efforts in harnessing FPGA capabilities by providing configurable, scalable templates that account for diverse CNN characteristics and FPGA constraints.

The paper also discusses innovative buffering strategies to overlap data processing and memory transfer, contributing to throughput enhancements. Techniques such as double-buffering and customized data access patterns are proven to be effective. Notably, Catapult by Microsoft integrates FPGA boards with datacenter applications, achieving remarkable speed increases by utilizing multi-banked input buffers and kernel weight buffers.

In the software domain, advanced methodologies in data quantization and layer optimization are pivotal. These include runtime adjustments of bit precision for weights and activations, ensuring minimal performance degradation while optimizing hardware usage. The paper underscores the significance of a flexible architecture, which can be efficiently reused across different CNN models while maximizing the FPGA's computational resources.

The discussion extends to the potential application of metaheuristic algorithms like Genetic Algorithms and Particle Swarm Optimization (PSO) to solve combinatorial optimization problems inherent in CNN design. These algorithms can optimize CNN model parameters to minimize computational costs and enhance accuracy, signifying a paradigm shift from human expertise-driven network design to algorithm-guided architecture optimization.

For practical implications, the authors recommend the development of a framework incorporating an intuitive interface for specifying CNN models and target FPGA platforms. This framework should facilitate automatic optimizations, such as data bit-width minimization and resource distribution, based on configurable error tolerances. Additionally, the framework should offer performance projections and suggest suitable FPGA hardware configurations to meet user-specified performance criteria.

In conclusion, the paper articulates the necessity for and promises in leveraging FPGAs for CNN acceleration. It underscores the potential of FP-GAs to revolutionize CNN implementations by enhancing computational efficiency and reducing resource constraints through systematic optimization strategies. By offering a comprehensive survey and proposing strategic recommendations, the authors provide a foundational guide for future research and development aimed at maximizing the potential of FPGAs in deep learning applications.