Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks (1908.05033v1)

Abstract: Hardware-friendly network quantization (e.g., binary/uniform quantization) can efficiently accelerate the inference and meanwhile reduce memory consumption of the deep neural networks, which is crucial for model deployment on resource-limited devices like mobile phones. However, due to the discreteness of low-bit quantization, existing quantization methods often face the unstable training process and severe performance degradation. To address this problem, in this paper we propose Differentiable Soft Quantization (DSQ) to bridge the gap between the full-precision and low-bit networks. DSQ can automatically evolve during training to gradually approximate the standard quantization. Owing to its differentiable property, DSQ can help pursue the accurate gradients in backward propagation, and reduce the quantization loss in forward process with an appropriate clipping range. Extensive experiments over several popular network structures show that training low-bit neural networks with DSQ can consistently outperform state-of-the-art quantization methods. Besides, our first efficient implementation for deploying 2 to 4-bit DSQ on devices with ARM architecture achieves up to 1.7$\times$ speed up, compared with the open-source 8-bit high-performance inference framework NCNN. [31]

Overview of Differentiable Soft Quantization for Neural Network Optimization

The paper "Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks" presents a novel approach to network quantization, referred to as Differentiable Soft Quantization (DSQ). This concept is designed to transcend the limitations of traditional binary and uniform quantization methods, which often suffer from performance degradation due to their discrete nature. DSQ serves as a differentiable intermediary that enables a smoother transition between full-precision and low-bit quantized networks, thereby preserving network performance while leveraging the computational advantages of quantization.

Key Contributions

The paper introduces several notable contributions:

1. DSQ Functionality: The DSQ method utilizes a differentiable approximation to standard quantization functions. This approach employs hyperbolic tangent functions to smoothly approximate binary or uniform quantization, maintaining differentiability and allowing for gradient descent optimization.
2. Automatic Evolution: A significant innovation of DSQ is its capacity to evolve during training. By controlling a characteristic variable, DSQ dynamically adjusts the approximation level, allowing the quantization process to gradually become more discrete as training progresses.
3. Balancing Quantization Errors: The method addresses both clipping and rounding errors inherent in quantization. By jointly optimizing the quantization range and approximation level, DSQ achieves a balance between these two types of errors, leading to improved model accuracy.
4. Implementation Efficiency: The authors have implemented DSQ for 2 to 4-bit quantization on ARM architectures, demonstrating substantial improvements in inference speed, up to 1.7 times faster than the 8-bit NCNN inference framework.

Experimental Evaluation

The authors conduct extensive experiments to validate DSQ, measuring its performance on popular datasets like CIFAR-10 and ImageNet with network architectures such as VGG-Small, ResNet, and MobileNetV2. The experiments reveal that DSQ consistently outperforms state-of-the-art quantization methods across various bit-widths and model architectures.

One strong numerical outcome highlighted in the paper is a nearly 5% performance improvement on CIFAR-10 in a fully quantized setting compared to prior methods. Additionally, DSQ achieves a significant accuracy increase in low-bit quantizations, providing practical benefits in terms of model deployment on resource-constrained hardware.

Implications and Future Prospects

The theoretical and practical implications of DSQ are noteworthy. Theoretically, DSQ advances the understanding of quantization in deep learning by introducing smooth, differentiable transitions from full-precision to quantization levels. Practically, it facilitates the deployment of neural networks on mobile and embedded devices by reducing performance loss typically associated with low-bit quantization.

The capability for DSQ to adaptively evolve during training opens new avenues for future research. One potential direction is the exploration of DSQ in conjunction with advanced optimization algorithms and search techniques to further minimize quantization loss. Additionally, refining the hardware implementations of DSQ for broader platforms may significantly impact real-time AI applications, especially in edge computing environments.

In summary, Differentiable Soft Quantization presents a compelling approach to neural network quantization, offering both theoretical innovation and tangible improvements in performance and deployment. As the field of AI continues to advance, methods like DSQ will likely play a critical role in the efficient and effective execution of neural networks across diverse hardware platforms.