RNNPool: Efficient Non-linear Pooling for RAM Constrained Inference (2002.11921v2)

Abstract: Standard Convolutional Neural Networks (CNNs) designed for computer vision tasks tend to have large intermediate activation maps. These require large working memory and are thus unsuitable for deployment on resource-constrained devices typically used for inference on the edge. Aggressively downsampling the images via pooling or strided convolutions can address the problem but leads to a significant decrease in accuracy due to gross aggregation of the feature map by standard pooling operators. In this paper, we introduce RNNPool, a novel pooling operator based on Recurrent Neural Networks (RNNs), that efficiently aggregates features over large patches of an image and rapidly downsamples activation maps. Empirical evaluation indicates that an RNNPool layer can effectively replace multiple blocks in a variety of architectures such as MobileNets, DenseNet when applied to standard vision tasks like image classification and face detection. That is, RNNPool can significantly decrease computational complexity and peak memory usage for inference while retaining comparable accuracy. We use RNNPool with the standard S3FD architecture to construct a face detection method that achieves state-of-the-art MAP for tiny ARM Cortex-M4 class microcontrollers with under 256 KB of RAM. Code is released at https://github.com/Microsoft/EdgeML.

• The paper introduces an RNN-based pooling operator that efficiently aggregates features under RAM constraints, reducing memory and computation costs.
• It demonstrates improvements in architectures like MobileNetV2, achieving up to 10x lower memory usage and 25% reduced computational demands while maintaining accuracy.
• The approach enables effective CNN deployment on edge devices, enhancing real-world applicability for IoT and resource-constrained environments.

Efficient Non-linear Pooling for RAM Constrained Inference

The paper introduces a novel pooling operator, utilizing Recurrent Neural Networks (RNNs), designed to address memory constraints inherent in standard Convolutional Neural Networks (CNNs) during edge deployment. The authors contend that traditional pooling methods lead to significant accuracy degradation due to their simplistic aggregation, particularly when applied to large receptive fields in scenarios where resource constraints are paramount.

Core Contributions

The primary contribution is the introduction of the RNN-based pooling operator, termed as {, which efficiently aggregates features utilizing RNNs. This methodology exploits the sequential nature of RNNs to perform sophisticated feature aggregation, enabling rapid down-sampling without substantial accuracy losses.

Key theoretical insights examine the memory efficiency of CNNs, demonstrating that the proposed { operator can significantly down-sample input architectures such as MobileNets and DenseNets, resulting in reduced computational complexity and memory usage, while maintaining competitive accuracy.

Empirical Evaluations

An extensive empirical evaluation compared { with traditional pooling strategies across different standard architectures. For instance, the {+MobileNetV2 configuration notably reduced peak memory usage by up to 10x and computational requirements by 25% on image classification tasks, while maintaining equivalent accuracy. This evaluation exemplifies the practical benefits of {, aligning effectively with diverse architectures and showcasing improved efficiency.

Of particular note is the deployment of { within the S3FD architecture, achieving state-of-the-art results in face detection on ARM Cortex-M4 class microcontrollers, maintaining under 256 KB of RAM. The outcomes indicate that { offers a viable solution for deploying CNNs in resource-constrained environments, thus extending real-world applicability.

Implications and Future Directions

The introduction of { has significant implications for the broader deployment of CNNs on edge devices, wherein memory constraints are a critical limitation. The ability to maintain model accuracy while significantly reducing memory and computational load presents opportunities for developing more efficient IoT devices.

From a theoretical perspective, the paper advances the understanding of how RNNs can enhance CNN architectures, particularly in resource-constrained contexts, suggesting potential explorations into combining { with advanced architectural search techniques.

Future research may explore integrating { into neural architecture search to further optimize inference costs. The adaptability of { across various tasks extends an interesting avenue for future exploration, potentially impacting a wide range of real-time applications in edge AI and beyond.

Conclusion

Through RNN-based pooling, this paper provides a substantial advance in efficiently managing CNN inference on resource-constrained devices. While the practical deployment and efficiency gains are well-demonstrated, the adaptability and theoretical insights offer promising pathways for continuing developments in AI and edge computing.