Post-Training Quantization for Vision Transformer (2106.14156v1)

Abstract: Recently, transformer has achieved remarkable performance on a variety of computer vision applications. Compared with mainstream convolutional neural networks, vision transformers are often of sophisticated architectures for extracting powerful feature representations, which are more difficult to be developed on mobile devices. In this paper, we present an effective post-training quantization algorithm for reducing the memory storage and computational costs of vision transformers. Basically, the quantization task can be regarded as finding the optimal low-bit quantization intervals for weights and inputs, respectively. To preserve the functionality of the attention mechanism, we introduce a ranking loss into the conventional quantization objective that aims to keep the relative order of the self-attention results after quantization. Moreover, we thoroughly analyze the relationship between quantization loss of different layers and the feature diversity, and explore a mixed-precision quantization scheme by exploiting the nuclear norm of each attention map and output feature. The effectiveness of the proposed method is verified on several benchmark models and datasets, which outperforms the state-of-the-art post-training quantization algorithms. For instance, we can obtain an 81.29\% top-1 accuracy using DeiT-B model on ImageNet dataset with about 8-bit quantization.

An Overview of "Post-Training Quantization for Vision Transformer"

The paper "Post-Training Quantization for Vision Transformer" by Zhenhua Liu, Yunhe Wang, Kai Han, Siwei Ma, and Wen Gao introduces a post-training quantization approach specifically tailored for vision transformers. This research addresses the challenges associated with deploying transformer-based models on resource-limited devices by compressing their architectures to reduce memory and computational demands. Here, we present an analytical review of the methodologies and findings reported in the paper, along with a consideration of their implications for future AI research.

Problem Context and Methodological Framework

Vision transformers have exhibited notable prowess in various computer vision tasks, surpassing the capabilities of traditional convolutional neural networks. However, the architectural complexity and the extensive parameter space of these models pose significant obstacles for their implementation on devices with limited computational capabilities, such as mobile phones or IoT devices.

This paper outlines a post-training quantization scheme that efficiently reduces these models' computational and memory overhead without necessitating additional training or access to extensive datasets. The core idea revolves around optimizing quantization intervals to maintain high performance despite reduced bit representation for weights and inputs.

Key Components of the Proposed Approach

The authors propose a multifaceted solution framework that integrates several innovative components:

1. Similarity-Aware Quantization: The goal is to maximize the similarity between the outputs of full-precision and quantized models. This is achieved by optimizing quantization intervals for weights and inputs to minimize the loss of information during the conversion process.
2. Ranking-Aware Quantization: Recognizing the importance of the self-attention mechanism in vision transformers, the authors introduce a ranking loss added to the quantization objective to preserve the relative order of attention scores post-quantization. This ensures the essential functionality of the attention mechanism, which is vital for maintaining model accuracy in a quantized state.
3. Mixed-Precision Quantization: By analyzing the feature diversity through the nuclear norm, the authors propose a mixed-precision scheme. This method allocates different bit-widths to various layers based on their respective sensitivity, leading to an optimal balance between computational efficiency and performance.
4. Bias Correction: To mitigate cumulative quantization errors, the authors employ a bias correction technique. This adjustment helps stabilize the mean distribution of the outputs by subtracting expected errors calculated from a calibration dataset.

Experimental Results and Comparative Analysis

The paper presents a detailed empirical evaluation using comprehensive experiments on datasets such as CIFAR-10, CIFAR-100, ImageNet, and COCO2017 for image classification and object detection tasks. The proposed methodology was compared against existing techniques such as EasyQuant and Bit-Split across multiple transformer models, including ViT and DeiT.

In these evaluations, the proposed method consistently outperformed traditional post-training quantization techniques. For the DeiT-B model on ImageNet, the proposed quantization achieved an impressive 81.29% top-1 accuracy using mixed-precision 8-bit quantization, showing minimal degradation from the baseline accuracy of full-precision models.

Implications and Prospective Directions

The advancements detailed in this paper significantly contribute to the evolving methods for deploying complex AI models in real-time, constrained environments. The ability to apply effective post-training quantization to vision transformers may facilitate broader adoption in industrial and commercial applications, particularly those requiring on-device processing.

Future work might explore extending these quantization techniques to other transformer architectures and application domains, such as natural language processing, where similar efficiency gains could be realized. Additionally, further research could focus on automated techniques for determining layer sensitivity and optimizing bit-width allocation, potentially using advanced search algorithms or machine learning techniques to enhance performance.

In summary, this paper presents a robust framework for overcoming the challenges associated with vision transformer deployment in limited-resource settings, further pushing the boundaries of efficient AI model deployment.