Channel Gating Neural Networks (1805.12549v2)

Abstract: This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the input channels for these ineffective regions. Unlike static network pruning, channel gating optimizes CNN inference at run-time by exploiting input-specific characteristics, which allows substantially reducing the compute cost with almost no accuracy loss. We experimentally show that applying channel gating in state-of-the-art networks achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and 2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy loss on CIFAR-10. Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We further demonstrate that channel gating can be realized in hardware efficiently. Our approach exhibits sparsity patterns that are well-suited to dense systolic arrays with minimal additional hardware. We have designed an accelerator for channel gating networks, which can be implemented using either FPGAs or ASICs. Running a quantized ResNet-18 model for ImageNet, our accelerator achieves an encouraging speedup of 2.4$\times$ on average, with a theoretical FLOP reduction of 2.8$\times$.

• The paper presents channel gating, a novel dynamic pruning scheme that optimizes convolutional neural network inference by selectively skipping computations on ineffective input channels.
• The methodology divides layers into base and conditional paths using a gate function to predict computation necessity, achieving 2.7-8.0x FLOP and 2.0-4.4x memory reductions with negligible accuracy loss.
• Channel gating is shown to be hardware-efficient, fitting dense systolic arrays and achieving a 2.4x speedup on a specialized accelerator, enabling efficient deployment on resource-constrained devices.

An Overview of Channel Gating Neural Networks

This paper presents the introduction of a novel dynamic pruning scheme called channel gating, aimed at optimizing convolutional neural network (CNN) inference by reducing computational and memory requirements. Channel gating exploits spatial structures within feature maps to selectively skip computations on certain input channels deemed ineffective for the target classification, thus presenting a fine-grained approach compared to traditional static pruning methods.

Methodology and Results

Channel gating divides CNN layers into two distinct paths: a base path and a conditional path. The base path performs convolution on a subset of input channels, generating partial sums which contribute to an activation-wise gate function. This function predicts the necessity of continuing computation on the remaining input channels, effectively skipping those that do not contribute to meaningful activations. Empirical evidence reveals substantial correlation between partial and full sums, allowing the approach to maintain high prediction accuracy while reducing computation.

Experimental results reflect impressive reductions in computational demands, achieving a 2.7-8.0x decrease in FLOPs and 2.0-4.4x reduction in off-chip memory accesses on CIFAR-10, with negligible impact on accuracy. Notably, integrating channel gating with knowledge distillation on ResNet-18 dataset showcases a 2.6x reduction in computation cost without compromising accuracy, illustrating robustness against inference tasks on ImageNet.

Hardware Implications

Channel gating is demonstrated to be hardware-efficient, fitting well with dense systolic array architectures typical of AI accelerators like Google's TPU. The design of a specialized hardware accelerator for CGNet models exhibits a speedup of 2.4x, validating the practical efficiency gains alongside theoretical FLOP reductions of 2.8x. This reflects the ability to leverage dynamic sparsity in CNN computations without necessitating significant hardware alterations, promoting both computational and energy efficiencies.

Comparison with Existing Techniques

Compared to existing pruning approaches, channel gating presents superior performance in minimizing computational resource requirements, outperforming both static and other dynamic pruning techniques. It achieves a finer granularity in pruning, reducing dense activation-level computations effectively, which is highlighted by lower accuracy drop metrics against competing methods in scenarios with substantial FLOP reduction.

Theoretical and Practical Implications

The introduction of channel gating expands the spectrum of computational efficiency strategies within the CNN landscape, offering a potentially significant reduction in inference cost particularly suitable for resource-constrained environments like mobile or embedded devices. It complements existing approaches by employing a dynamic and trainable methodology without necessitating additional weights or extensive training overhead.

Potential future directions for research include extending channel gating techniques to broader machine learning domains such as object detection tasks. Furthermore, exploration into optimizing the design space through varied configurations of group sizes and target thresholds may yield further improvements in balancing accuracy and compute efficiency.

In conclusion, channel gating represents a significant step forward in enhancing CNN inference efficiency by dynamically adjusting computation based on real-time input characteristics. Its integration within current architectures provides a promising avenue for enhancing the deployment and scalability of deep learning models across a range of applications.