Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition (1412.6553v3)

Abstract: We propose a simple two-step approach for speeding up convolution layers within large convolutional neural networks based on tensor decomposition and discriminative fine-tuning. Given a layer, we use non-linear least squares to compute a low-rank CP-decomposition of the 4D convolution kernel tensor into a sum of a small number of rank-one tensors. At the second step, this decomposition is used to replace the original convolutional layer with a sequence of four convolutional layers with small kernels. After such replacement, the entire network is fine-tuned on the training data using standard backpropagation process. We evaluate this approach on two CNNs and show that it is competitive with previous approaches, leading to higher obtained CPU speedups at the cost of lower accuracy drops for the smaller of the two networks. Thus, for the 36-class character classification CNN, our approach obtains a 8.5x CPU speedup of the whole network with only minor accuracy drop (1% from 91% to 90%). For the standard ImageNet architecture (AlexNet), the approach speeds up the second convolution layer by a factor of 4x at the cost of $1\%$ increase of the overall top-5 classification error.

• The paper demonstrates that combining low-rank CP-Decomposition with fine-tuning can significantly speed up CNNs, achieving up to 8.5x acceleration with minimal accuracy loss.
• It introduces a two-step methodology where the convolutional layers are decomposed into rank-one tensors and then fine-tuned to recover performance.
• The proposed approach reduces computational demands and memory footprint, making CNNs more feasible for real-time applications on mobile and embedded devices.

Speeding-up Convolutional Neural Networks Using Fine-Tuned CP-Decomposition

The paper "Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition" by Lebedev et al. presents a method for accelerating the performance of Convolutional Neural Networks (CNNs) using a combination of tensor decomposition and discriminative fine-tuning. The approach employs a two-step process where a low-rank CP-Decomposition is first applied to the convolutional layers, and the network is subsequently fine-tuned using standard backpropagation.

Overview

Methodology

The proposed methodology involves two primary steps:

1. Tensor Decomposition: The authors utilize a non-linear least squares (NLS) algorithm to compute a low-rank CP-Decomposition of the 4D convolution kernel tensor. This results in a sum of a small number of rank-one tensors that approximate the original kernel tensor.
2. CNN Fine-tuning: Post decomposition, the original convolutional layer is replaced by a sequence of four convolutional layers with smaller kernels. The entire network is then fine-tuned on the training data to adjust the weights of the newly introduced layers as well as the existing layers, maintaining accuracy.

Numerical Results

The results are evaluated on two network architectures: a character classification CNN and AlexNet.

Character Classification CNN

• Speedup: The approach achieved an 8.5x CPU speedup on the character classification CNN with only a 1% drop in accuracy (from 91% down to 90%).
• Parameter Reduction: The second and third convolutional layers accounted for 90% of the original model's processing time and achieved significant compression.
• Fine-tuning: Fine-tuning was shown to aid in recovering most of the lost accuracy due to tensor approximation.

AlexNet

• Speedup for AlexNet: A 4x speedup was observed in the second convolutional layer of AlexNet with a 1% increased overall top-5 classification error.
• Rank Comparison: NLS-based CP-Decomposition proved superior to the greedy approach in providing more accurate approximations with fewer parameters.

Technical Insights

The technical strength of the paper lies in the use of CP-Decomposition, a well-established tool in tensor algebra, and the application of non-linear least squares optimization to achieve better approximations compared to greedy rank-1 tensor methods. A fundamental observation is that the combination of CP-Decomposition and global fine-tuning can often yield better speed-accuracy trade-offs than existing methods.

Implications and Future Directions

The implications of this research are significant for deploying CNNs on resource-constrained devices such as mobile processors and embedded systems in robotics. The method effectively reduces the memory footprint and computational burden, making real-time operation of CNNs more feasible on low-end processors. Theoretical insights suggest that the modern CNNs are over-parameterized and can still retain competitive performance with significantly fewer parameters due to intelligent decomposition.

Future developments in this area could focus on:

1. Exploring modifications and improvements to the CP-decomposition approach, especially for layers with spatially-varying kernels.
2. Extending this methodology to more complex architectures and larger scale datasets.
3. Integration with other optimization techniques to address the instability issues observed during low-rank decompositions.

Conclusion

This paper provides a well-substantiated method for accelerating CNNs through CP-Decomposition and discriminative fine-tuning. The approach not only reduces computational complexity but also maintains accuracy, making it a valuable contribution to resource-efficient neural network deployment on constrained hardware.