Machine Learning at the Network Edge: A Survey (1908.00080v4)

Abstract: Resource-constrained IoT devices, such as sensors and actuators, have become ubiquitous in recent years. This has led to the generation of large quantities of data in real-time, which is an appealing target for AI systems. However, deploying machine learning models on such end-devices is nearly impossible. A typical solution involves offloading data to external computing systems (such as cloud servers) for further processing but this worsens latency, leads to increased communication costs, and adds to privacy concerns. To address this issue, efforts have been made to place additional computing devices at the edge of the network, i.e close to the IoT devices where the data is generated. Deploying machine learning systems on such edge computing devices alleviates the above issues by allowing computations to be performed close to the data sources. This survey describes major research efforts where machine learning systems have been deployed at the edge of computer networks, focusing on the operational aspects including compression techniques, tools, frameworks, and hardware used in successful applications of intelligent edge systems.

Machine Learning at the Network Edge: A Survey

Overview

The paper "Machine Learning at the Network Edge: A Survey" presents an extensive review of methodologies, technologies, and applications concerning the deployment of ML systems at the periphery of computer networks, commonly referred to as edge computing. Given the proliferation of resource-constrained Internet of Things (IoT) devices that generate substantial volumes of data, the thrust of this survey is on configuring machine learning models to operate efficiently in such decentralized environments. This decentralization aligns with efforts to mitigate the challenges associated with offloading data to far-off cloud servers, primarily concerning latency, privacy, and communication costs.

Key Technical Contributions

1. Architectural Innovations: The survey elucidates the evolution of lighter and faster architectures like MobileNets and ShuffleNet, which incorporate depthwise separable convolutions to optimize for the reduced computational capacity of edge-devices while marginally compromising on accuracy.
2. Distributed Training: An emerging practice in this domain involves deploying federated learning methodologies and distributed gradient descent techniques which allow model updates to occur on edge-devices, thus promoting user privacy and lowering the dependency on bandwidth-heavy cloud communication.
3. Model Compression: Given the necessity to fit ML models on devices with limited storage, methodologies such as quantization and pruning are explored. These techniques effectively reduce model size and computation requirements, ensuring deployment efficiency without notably sacrificing accuracy.
4. Distributed Inference: The paper discusses various systems for facilitating inference, both vertically (across an end-device to a cloud hierarchy) and horizontally (within the same network layer among devices), introducing promising approaches such as early exit of inference (EEoI) to balance the accuracy-latency trade-off.

Applications and Implications

The survey examines numerous applications, ranging from real-time video analytics for traffic management and surveillance, to edge-based speech recognition systems, highlighting the practical incentives of such decentralized intelligence. Particularly noteworthy is the use of edge-compute frameworks for autonomous vehicles, which require real-time processing of vast amounts of sensory data, something edge computing effectively supports by committing data processing close to the source.

Platforms and Hardware

Development frameworks such as TensorFlow Lite, CoreML, and Apache MXNet comprise a crucial part of the ecosystem, enabling the deployment of ML models tailored to mobile and edge environments. Complementary to these software solutions, the paper reviews specialized hardware like TPUs and FPGAs that support these computational demands. Devices like NVIDIA’s Jetson series and Google Coral's products exemplify the movement towards empowering edge-devices with adequate inferential capabilities while maintaining manageable power consumption profiles.

Challenges and Future Directions

Despite the advances, the survey outlines several impediments to the broader adoption of edge intelligence, such as the complexities inherent in handling heterogeneous data streams and the ethical imperative of ensuring fairness and explainability in ML systems designed for resource-constrained settings. Furthermore, the paper anticipates the integration of 5G networks and blockchain technologies, opening new avenues for efficiency and security enhancements in edge computing scenarios.

Conclusion

This survey highlights the evolving dynamics of deploying ML applications at the network edge, showcasing both the current landscape and prospective trajectories the research community might pursue. While it reflects on achievements within edge-centric ML ecosystems, it simultaneously challenges researchers to explore innovative solutions tailored to the variegated demands and constraints manifest in edge contexts, emphasizing data privacy, system robustness, and democratized AI access.