Data-Free Quantization Through Weight Equalization and Bias Correction (1906.04721v3)

Abstract: We introduce a data-free quantization method for deep neural networks that does not require fine-tuning or hyperparameter selection. It achieves near-original model performance on common computer vision architectures and tasks. 8-bit fixed-point quantization is essential for efficient inference on modern deep learning hardware. However, quantizing models to run in 8-bit is a non-trivial task, frequently leading to either significant performance reduction or engineering time spent on training a network to be amenable to quantization. Our approach relies on equalizing the weight ranges in the network by making use of a scale-equivariance property of activation functions. In addition the method corrects biases in the error that are introduced during quantization. This improves quantization accuracy performance, and can be applied to many common computer vision architectures with a straight forward API call. For common architectures, such as the MobileNet family, we achieve state-of-the-art quantized model performance. We further show that the method also extends to other computer vision architectures and tasks such as semantic segmentation and object detection.

• The paper introduces a data-free quantization method that combines weight equalization and bias correction to retain near-FP32 accuracy in 8-bit models.
• It leverages the scale-equivariant properties of ReLU and uses batch normalization parameters for analytical bias correction to optimize quantization.
• Extensive evaluations on architectures like MobileNet and ResNet demonstrate its efficacy for efficient AI inference without retraining or data usage.

Data-Free Quantization Through Weight Equalization and Bias Correction

The paper presents a novel approach to quantizing deep neural networks without requiring data, fine-tuning, or hyperparameter selection, achieving near-original model performance in 8-bit fixed-point quantization. This technique is particularly relevant for efficient inference on modern deep learning hardware and addresses the challenges inherent in quantizing models without compromising performance or increasing engineering effort.

Methodology Overview

The primary innovation lies in equalizing weight ranges across the network by leveraging the scale-equivariance properties of activation functions like ReLU. Additionally, the method applies bias correction to the error introduced during quantization, enhancing the quantization accuracy significantly.

• Weight Equalization: By adjusting the weights so that they are more amenable to quantization, the technique exploits positive scaling equivariance. This is particularly useful for piecewise linear activation functions, where scaling reparameters the model to better utilize quantization range without altering network performance in FP32 settings.
• Bias Correction: The quantization process introduces biased error on the outputs, with non-trivial effects on subsequent network layers. The approach uses batch normalization parameters to analytically determine and correct these biases, ensuring that the mean outputs remain stable post-quantization. This method benefits from being data-free, which improves its practicality in various deployment scenarios.

Results and Implications

The results demonstrate state-of-the-art performance across various architectures, notably achieving significant improvements on MobileNet family models, which have historically been difficult to quantize without fine-tuning. The paper also highlights success in extending the method to more complex computer vision tasks such as semantic segmentation and object detection.

• Performance Metrics: For MobileNetV2, the method achieved 71.19% accuracy, closely matching the full precision performance. This represents a marked improvement over previous unsupervised per-channel quantization methods and competes well against more complex approaches that require data and retraining.
• Broader Applicability: The method is evaluated across several architectures including ResNet18 and MobileNetV1, showing consistent performance improvements without the need for data. These results extend to both classification and detection tasks, indicating broad generalizability.

Practical and Theoretical Implications

This research introduces a practical solution for stakeholders such as cloud-based inference providers and edge-device manufacturers, allowing direct conversion of FP32 models to INT8 without data usage or model retraining. The automation potential saves engineering time and resources while maintaining performance.

Theoretically, this work provokes a re-examination of quantization noise and error correction in neural networks, suggesting future research directions in model reparameterization and activation function design for better quantization compatibility.

Future Directions

As AI models become more prevalent in edge and mobile applications, further developments could explore this method's applicability to other model architectures or investigate integration with more diverse hardware setups. Additional research might enhance the bias correction mechanism's efficacy for networks with non-standard activation functions or those employing more complex layers. Overall, the paper sets a foundational step in achieving efficient and data-free model deployment for real-world applications in AI.