Bit-pragmatic Deep Neural Network Computing (1610.06920v1)

Abstract: We quantify a source of ineffectual computations when processing the multiplications of the convolutional layers in Deep Neural Networks (DNNs) and propose Pragmatic (PRA), an architecture that exploits it improving performance and energy efficiency. The source of these ineffectual computations is best understood in the context of conventional multipliers which generate internally multiple terms, that is, products of the multiplicand and powers of two, which added together produce the final product [1]. At runtime, many of these terms are zero as they are generated when the multiplicand is combined with the zero-bits of the multiplicator. While conventional bit-parallel multipliers calculate all terms in parallel to reduce individual product latency, PRA calculates only the non-zero terms using a) on-the-fly conversion of the multiplicator representation into an explicit list of powers of two, and b) hybrid bit-parallel multplicand/bit-serial multiplicator processing units. PRA exploits two sources of ineffectual computations: 1) the aforementioned zero product terms which are the result of the lack of explicitness in the multiplicator representation, and 2) the excess in the representation precision used for both multiplicants and multiplicators, e.g., [2]. Measurements demonstrate that for the convolutional layers, a straightforward variant of PRA improves performance by 2.6x over the DaDiaNao (DaDN) accelerator [3] and by 1.4x over STR [4]. Similarly, PRA improves energy efficiency by 28% and 10% on average compared to DaDN and STR. An improved cross lane synchronication scheme boosts performance improvements to 3.1x over DaDN. Finally, Pragmatic benefits persist even with an 8-bit quantized representation [5].

• The paper introduces Pragmatic (PRA), a Deep Neural Network accelerator architecture that avoids ineffectual computations in convolutional layers by processing only non-zero multiplications.
• Numerical results show that PRA achieves speed improvements of up to 3.1x and energy efficiency gains of up to 28% over the DaDianNao accelerator, even at 8-bit quantization.
• By significantly decreasing computational overhead, PRA enables more energy-efficient designs, potentially extending battery life in portable devices and lowering data center costs.

An Overview of Bit-Pragmatic Deep Neural Network Computing

The paper "Bit-Pragmatic Deep Neural Network Computing," ventures into optimizing the performance and energy efficiency of Deep Neural Network (DNN) accelerators by targeting ineffectual computations that occur when processing the convolutional layers. The proposed architecture, Pragmatic (PRA), innovatively bypasses the unnecessary computations typically carried out by conventional bit-parallel multipliers, allowing only the non-zero multiplications to be processed. This strategic shift enhances the processing speed and energy consumption, which are critical in the context of resource-intensive DNN operations.

Core Concepts and Proposed Architecture

The primary issue tackled in the paper revolves around ineffectual computations induced by zero-product terms inherent in traditional multiplier designs. These inefficiencies are primarily due to non-explicit multiplier representations and excessive representation precision. Pragmatic addresses these through a two-fold approach: firstly, leveraging on-the-fly conversion of neuron representations into explicit lists of significant terms; secondly, employing bit-serial neuron/multiplier processors to handle these terms effectively. This methodological shift is presented as an advancement over the Stripes (STR) architecture, which already attempts some level of optimization by avoiding extra precision bits.

PRA specifically capitalizes on both the lack of explicitness in multiplier representations and unnecessary precision in neuron or synapse data to achieve speed improvements of up to 3.1x and energy efficiency gains by as much as 28% over the established DaDianNao (DaDN) accelerator. A distinguishing feature of PRA is its ability to uphold performance enhancements even at an 8-bit quantization level, which signifies the architecture's robustness and adaptability to variations in data representation.

Numerical Results and Comparative Efficacy

The experimental setup elucidated in the paper quantifies PRA's effectiveness using standard DNN models, including high computational burden examples such as VGG and AlexNet. The numerical findings reveal a notable reduction in the number of terms processed compared to DaDN and STR, highlighting substantial computational savings. For instance, under a 16-bit fixed-point representation, PRA slashed the processed terms to just 10% compared to DaDN's baseline, showcasing its potential for substantial efficiency improvements.

Implications and Future Directions

By decreasing the computational overhead significantly, PRA not only provides immediate improvements in throughput but also sets a precedent for more energy-efficient designs in neural network accelerators. The ripple effects of such efficiency gains could be profound, extending battery life in portable AI devices, lowering operational costs in data centers, and enabling deployment of more sophisticated AI models across various domains.

Future research directions naturally spring from this work, inviting further exploration into the integration of PRA's methods into existing architectures and across other parts of neural networks beyond convolutional layers. Additionally, its compatibility with emerging low-precision arithmetic methods such as quantization introduces a fertile ground for further performance optimizations.

Moreover, while this paper demonstrates the high effectiveness of PRA for specific architectures, broader performance evaluations in diverse hardware environments could reinforce its applicability. As the demand for robust and resource-efficient AI systems continues to grow, Bit-Pragmatic approaches like those explored here will likely play a crucial role in shaping the next generation of artificial intelligence technology.