HG-PIPE: Vision Transformer Acceleration with Hybrid-Grained Pipeline (2407.17879v2)

Abstract: Vision Transformer (ViT) acceleration with field programmable gate array (FPGA) is promising but challenging. Existing FPGA-based ViT accelerators mainly rely on temporal architectures, which process different operators by reusing the same hardware blocks and suffer from extensive memory access overhead. Pipelined architectures, either coarse-grained or fine-grained, unroll the ViT computation spatially for memory access efficiency. However, they usually suffer from significant hardware resource constraints and pipeline bubbles induced by the global computation dependency of ViT. In this paper, we introduce HG-PIPE, a pipelined FPGA accelerator for high-throughput and low-latency ViT processing. HG-PIPE features a hybrid-grained pipeline architecture to reduce on-chip buffer cost and couples the computation dataflow and parallelism design to eliminate the pipeline bubbles. HG-PIPE further introduces careful approximations to implement both linear and non-linear operators with abundant Lookup Tables (LUTs), thus alleviating resource constraints. On a ZCU102 FPGA, HG-PIPE achieves 2.78 times better throughput and 2.52 times better resource efficiency than the prior-art accelerators, e.g., AutoViTAcc. With a VCK190 FPGA, HG-PIPE realizes end-to-end ViT acceleration on a single device and achieves 7118 images/s, which is 2.81 times faster than a V100 GPU.

• The paper presents a hybrid-grained pipeline design that optimizes resource usage and minimizes bottlenecks in Vision Transformer acceleration.
• It employs LUT-based optimizations, including Power-of-Two index approximation and GeLU-ReQuant fusion, to achieve high accuracy with reduced hardware resources.
• Results on FPGAs demonstrate up to 2.81x throughput improvement over a V100 GPU, highlighting its potential for real-time, energy-efficient applications.

HG-PIPE: Vision Transformer Acceleration with Hybrid-Grained Pipeline

The Vision Transformer (ViT) models have gained significant traction in recent years due to their superior performance in various computer vision tasks. However, their computational and parametric complexity poses a significant challenge for hardware acceleration, especially on platforms like Field-Programmable Gate Arrays (FPGAs). In response to these challenges, the paper introduces HG-PIPE, an FPGA accelerator designed specifically for ViT models, leveraging a hybrid-grained pipeline architecture.

Key Contributions

1. Hybrid-Grained Pipeline Design: The hybrid-grained approach synthesizes the strengths of both fine-grained and coarse-grained pipelining. It addresses the extensive hardware resource constraints and pipeline bubbles typically induced by the global computation dependency inherent in ViTs. By strategically managing the computational workload across different granularities, HG-PIPE mitigates bottlenecks and optimized memory access efficiencies.
2. Efficient Use of LUT-Based Non-Linear Functions: The accelerator makes extensive use of Lookup Tables (LUTs) to implement both linear and non-linear operators. Techniques such as Power-of-Two (PoT) index approximation and joint table range calibration ensure that these functions are resource-efficient while maintaining high computational accuracy.
3. Throughput and Resource Efficiency: On a ZCU102 FPGA, HG-PIPE achieves a throughput of 2.78 times and a resource efficiency of 2.52 times better than prior-art accelerators like AutoViTAcc. On a more advanced VCK190 FPGA, HG-PIPE demonstrates an impressive processing speed of 7118 images per second, surpassing the throughput of a V100 GPU by a factor of 2.81.

Methodology

The architecture of HG-PIPE revolves around the hybrid-grained pipeline, designed to balance the computational load and optimize memory usage:

• Hybrid-Grained Pipelining: This approach combines deep buffers and FIFOs with coarse-grained and fine-grained pipelines, ensuring low off-chip memory access and minimal pipeline bubbles. The key insight is to manage the varying data locality requirements across different stages of the self-attention mechanism inherent in ViTs.
• Parallelism Design: Tiled matrix multiplication is employed using an Output Stationary (OS) dataflow to optimize data locality and memory use. The paper details a meticulous parallelism design approach, ensuring pipeline balance and efficient BRAM utilization.
• LUT-Based Optimizations: Several specific techniques were incorporated:
  • Power-of-Two Index Approximation: Simplifies the computation of LUT indices to reduce DSP usage.
  • GeLU-ReQuant Fusion: Combines the GeLU activation function and ReQuant operations into a single LUT operation to save resources.
  • Joint Table Range Calibration: Dynamically adjusts LUT ranges to better fit data distributions, thus reducing redundancy.
  • Segmented Table for High Dynamic Range: Enhances the implementation of functions like Recip with high dynamic ranges, ensuring precision without excessive LUT usage.

Implications and Future Directions

HG-PIPE demonstrates significant improvements in both throughput and resource efficiency for ViT acceleration on FPGAs. The implications are profound for edge computing environments where energy efficiency and real-time processing capabilities are critical:

• Practical Applications: The accelerator's high throughput and low latency are particularly beneficial for applications such as autonomous vehicles, real-time video analysis, and augmented reality, where rapid and accurate processing of visual data is imperative.
• Theoretical Advancements: The hybrid-grained pipeline concept could inspire future research in hardware acceleration across various deep learning models, potentially extending beyond ViTs to models with similar computational characteristics.
• Future Developments: Further research could explore the hybrid-grained pipeline's adaptability to other types of transformers and beyond. Integrating more advanced quantization techniques and exploring the scalability of the design to even larger FPGAs or custom ASICs could yield additional gains in performance and efficiency.

Conclusion

The paper presents a comprehensive and innovative approach to accelerating Vision Transformers on FPGA platforms through the HG-PIPE architecture. By addressing fundamental challenges associated with global computation dependency and resource constraints, the proposed hybrid-grained pipeline architecture sets a new benchmark in FPGA-based ViT acceleration. The work not only improves the performance and efficiency of existing ViT accelerators but also opens new avenues for future research in hardware-accelerated deep learning.