Distributed Deep Neural Networks over the Cloud, the Edge and End Devices (1709.01921v1)

Abstract: We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate inference of a deep neural network (DNN) in the cloud, a DDNN also allows fast and localized inference using shallow portions of the neural network at the edge and end devices. When supported by a scalable distributed computing hierarchy, a DDNN can scale up in neural network size and scale out in geographical span. Due to its distributed nature, DDNNs enhance sensor fusion, system fault tolerance and data privacy for DNN applications. In implementing a DDNN, we map sections of a DNN onto a distributed computing hierarchy. By jointly training these sections, we minimize communication and resource usage for devices and maximize usefulness of extracted features which are utilized in the cloud. The resulting system has built-in support for automatic sensor fusion and fault tolerance. As a proof of concept, we show a DDNN can exploit geographical diversity of sensors to improve object recognition accuracy and reduce communication cost. In our experiment, compared with the traditional method of offloading raw sensor data to be processed in the cloud, DDNN locally processes most sensor data on end devices while achieving high accuracy and is able to reduce the communication cost by a factor of over 20x.

• The paper presents a novel DDNN framework that maps DNN segments over cloud, edge, and end devices, enabling localized and efficient processing.
• The paper introduces a joint training strategy that cuts communication overhead by over 20 times while maintaining high overall accuracy.
• The paper demonstrates scalability and fault tolerance by using automatic sensor fusion and multi-layered inference validated on multi-camera datasets.

Distributed Deep Neural Networks Over the Cloud, the Edge, and End Devices

This paper introduces a novel framework for distributed deep neural networks (DDNNs) designed to operate over a distributed computing hierarchy encompassing the cloud, the edge, and end devices. The authors propose a methodology that allows deep neural networks (DNNs) to maintain high accuracy while minimizing communication and latency issues typically associated with cloud-based computations.

Core Contributions

1. DDNN Framework Implementation: The paper outlines the implementation of DDNNs by mapping sections of a DNN across a distributed hierarchy. This allows portions of the DNN to operate locally on end devices and progressively utilize more sophisticated computation at the edge and cloud levels.
2. Joint Training Methodology: By jointly training DNN sections, the approach minimizes communication overhead and resource usage, while optimizing the utility of the features extracted at various layers. This ensures that localized inferences at the edge are both fast and effective.
3. Automatic Sensor Fusion: Through aggregation schemes such as max pooling and concatenation, the framework supports automatic sensor fusion, improving system accuracy and robustness against individual sensor failures.

Experimental Evaluation

The empirical results focus on a multi-view multi-camera dataset to demonstrate DDNN performance. Significant findings include:

• Increased Efficiency: The DDNN achieves a reduction in communication costs by over 20 times compared to traditional cloud-only models, by exiting many samples locally.
• Accuracy: The proposed DDNN achieves high overall accuracy through a multi-layered approach that effectively exploits geographical sensor diversity.
• Scalability and Fault Tolerance: The solution exhibits strong scalability across devices, ensuring continued high performance even if some devices fail, thereby delivering inherent fault tolerance.

Implications and Future Directions

The development of DDNNs holds substantial implications for distributed systems and edge computing applications. By enabling efficient, localized processing, these networks can significantly enhance the deployment of AI applications in IoT environments, where bandwidth and latency are often critical constraints. Additionally, this architecture supports scalable and privacy-conscious AI services, leveraging local computation capabilities while minimizing data sharing.

Future Research Directions: The authors suggest exploring mixed-precision neural networks to enhance DDNN configurations further. Methods that balance computational efficiency with accuracy will be essential for deploying these systems in varied real-world scenarios. Additionally, expanding the framework to handle multiple input modalities and larger datasets could further validate the applicability of DDNNs across different domains, such as autonomous vehicles and smart city infrastructures.

The innovations presented in this paper present significant advancements in the efficient deployment of distributed machine learning models and provide a solid foundation for ongoing research in distributed AI systems.