DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework (1706.01215v3)

Abstract: Recent advances in deep learning motivate the use of deep neutral networks in sensing applications, but their excessive resource needs on constrained embedded devices remain an important impediment. A recently explored solution space lies in compressing (approximating or simplifying) deep neural networks in some manner before use on the device. We propose a new compression solution, called DeepIoT, that makes two key contributions in that space. First, unlike current solutions geared for compressing specific types of neural networks, DeepIoT presents a unified approach that compresses all commonly used deep learning structures for sensing applications, including fully-connected, convolutional, and recurrent neural networks, as well as their combinations. Second, unlike solutions that either sparsify weight matrices or assume linear structure within weight matrices, DeepIoT compresses neural network structures into smaller dense matrices by finding the minimum number of non-redundant hidden elements, such as filters and dimensions required by each layer, while keeping the performance of sensing applications the same. Importantly, it does so using an approach that obtains a global view of parameter redundancies, which is shown to produce superior compression. We conduct experiments with five different sensing-related tasks on Intel Edison devices. DeepIoT outperforms all compared baseline algorithms with respect to execution time and energy consumption by a significant margin. It reduces the size of deep neural networks by 90% to 98.9%. It is thus able to shorten execution time by 71.4% to 94.5%, and decrease energy consumption by 72.2% to 95.7%. These improvements are achieved without loss of accuracy. The results underscore the potential of DeepIoT for advancing the exploitation of deep neural networks on resource-constrained embedded devices.

• The paper presents DeepIoT, a novel compressor-critic framework for compressing various deep neural networks tailored for resource-constrained sensing systems.
• DeepIoT reduces neural network sizes by up to 98.9% and improves execution time and energy efficiency by over 94% without degrading sensing application accuracy.
• This compression approach enables complex deep learning tasks on low-end devices, expanding IoT capabilities and the applicability of AI in lightweight environments.
• The framework supports commonly used deep learning structures and compressed models can be deployed with existing libraries.
• DeepIoT's innovations include a unified compression methodology across architectures and a focus on condensing networks into denser formats by identifying parameter redundancies globally rather than relying on sparsifying or linear assumptions.

DeepIoT: Compressing Deep Neural Network Structures for Resource-Constrained Sensing Systems

The paper presents DeepIoT, a novel approach for compressing deep neural network structures tailored to sensing systems on embedded devices. It addresses the extensive memory, computational power, and energy demands of deploying deep learning models on resource-constrained platforms, by proposing a compressor-critic framework. This framework effectively compresses commonly used deep learning structures, including fully-connected, convolutional, and recurrent networks.

DeepIoT distinguishes itself with two primary innovations. Firstly, it offers a unified compression methodology that is applicable across various deep network architectures, unlike other techniques focused solely on specific network types. Secondly, instead of resorting to sparsifying weight matrices or adhering to linear assumptions, DeepIoT condenses networks into denser formats, targeting the minimal number of non-redundant elements, such as filters and dimensions, without sacrificing application performance. This is accomplished through identifying parameter redundancies with a global perspective, thereby creating superior compression results.

In practice, DeepIoT's compressed models utilize existing deep learning libraries compatible with embedded systems, requiring no further modifications post-compression. Empirical evaluations reveal significant enhancements in execution time and energy efficiency, reducing deep neural network sizes by up to 98.9%, with the execution time and energy consumption reduced by up to 94.5% and 95.7%, respectively. Notably, these benefits are realized without degrading the accuracy of the sensing applications.

The theoretical implications of DeepIoT center on effectively enabling complex sensing tasks on low-end devices, advancing IoT applications, and expanding the versatility and applicability of deep learning models in lightweight environments. The framework highlights the latent potential of deep neural networks in scenarios where hardware limitations previously restricted their use.

Looking forward, DeepIoT's compressor-critic framework offers a foundation for continued exploration of neural network compression. Future research may explore adaptive compression algorithms, enhanced collaborative frameworks that integrate cutting-edge reinforcement learning techniques, and broader application across diverse domains beyond sensing. Understanding the interplay of parameter redundancy and model accuracy remains pivotal as demand grows for efficient AI models ubiquitously integrated within the IoT landscape.

Ultimately, the compression strategies demonstrated by DeepIoT are poised to play a pivotal role in the sustainable deployment of deep learning systems across both consumer and industrial embedded platforms, catalyzing further advancements in IoT capabilities and resource-aware AI applications.