Adaptive Quantization for Deep Neural Network (1712.01048v1)

Abstract: In recent years Deep Neural Networks (DNNs) have been rapidly developed in various applications, together with increasingly complex architectures. The performance gain of these DNNs generally comes with high computational costs and large memory consumption, which may not be affordable for mobile platforms. Deep model quantization can be used for reducing the computation and memory costs of DNNs, and deploying complex DNNs on mobile equipment. In this work, we propose an optimization framework for deep model quantization. First, we propose a measurement to estimate the effect of parameter quantization errors in individual layers on the overall model prediction accuracy. Then, we propose an optimization process based on this measurement for finding optimal quantization bit-width for each layer. This is the first work that theoretically analyse the relationship between parameter quantization errors of individual layers and model accuracy. Our new quantization algorithm outperforms previous quantization optimization methods, and achieves 20-40% higher compression rate compared to equal bit-width quantization at the same model prediction accuracy.

Insights on Adaptive Quantization for Deep Neural Networks

Quantization in deep neural networks (DNNs) is essential for optimizing these models for deployment on mobile platforms with limited computational resources and memory constraints. The paper "Adaptive Quantization for Deep Neural Network" by Zhou and colleagues introduces a novel optimization framework that selects layer-wise bit-widths for quantization, targeting minimal accuracy degradation.

Overview

The paper begins by acknowledging the critical issue faced by mobile platforms—the computational and memory overhead associated with deploying complex DNN architectures. It addresses this problem by proposing an optimization framework for adaptive quantization. The authors develop a metric to evaluate the effect of parameter quantization errors at the layer level on the overall model performance. This metric is utilized to determine an optimal bit-width for each layer, considering both the compression rate of the model and the accuracy retention.

Methodology

1. Quantization Noise: The methodology is rooted in the analysis of quantization noise, modeled as uniform noise added to the weights of the network layers. The paper defines expected noise norms, establishing a theoretical relationship between bit-width reduction and increase in expected noise norm.
2. Measurement of Quantization Effect: Zhou et al. propose a measurement model that estimates the impact of quantization noise on the last feature map of the network. This model considers both the linear response of layers to added noise and its subsequent propagation through network layers, effectively capturing how layer-specific quantization influences overall network prediction accuracy.
3. Optimization Framework: The authors present an optimization problem minimizing model size under a constraint of negligible accuracy loss. The framework ensures that the sum of layer-wise quantization effects remains beneath a certain threshold.
4. Adaptive Bit-width Determination: The paper provides a procedure to determine optimal bit-widths for each layer, leveraging parameters derived from the error measurement model and the robustness of each layer to quantization-induced noise.

Findings

The paper's experimental results with various models like AlexNet, VGG-16, GoogleNet, and ResNet-50 demonstrate that the proposed method significantly outperforms both uniform bit-width quantization and existing SQNR-based optimization methods. Key findings include:

• A 20-40% improvement in compression rates compared to equal bit-width quantization while maintaining the same prediction accuracy.
• Enhanced effectiveness on models with more diverse layer sizes and structures.

Implications

The theoretical groundwork laid by the authors promises a more nuanced understanding of quantization noise, its propagation, and effect on DNN accuracy. Practically, the adaptive layer-wise quantization method enhances the deployment of high-performing DNN models on constrained platforms—crucial for real-world applications requiring efficient inference in environments like mobile devices and embedded systems.

The paper suggests potential future directions, such as combining this quantization framework with other compression techniques or fine-tuning the models further post-quantization to enhance performance. Additionally, adopting this bit-width optimization in conjunction with techniques like knowledge distillation may foster superior model performance without overwhelming computational overhead.

Conclusion

"Adaptive Quantization for Deep Neural Network" presents a robust framework for understanding and implementing quantization in DNNs effectively. The numerical improvements documented in their findings position this method as a significant step forward in deploying complex models on limited-resource devices, expanding both practical applications and theoretical development of quantization techniques in DNNs.