AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environments

Abstract: Deep learning models are increasingly deployed to edge devices for real-time applications. To ensure stable service quality across diverse edge environments, it is highly desirable to generate tailored model architectures for different conditions. However, conventional pre-deployment model generation approaches are not satisfactory due to the difficulty of handling the diversity of edge environments and the demand for edge information. In this paper, we propose to adapt the model architecture after deployment in the target environment, where the model quality can be precisely measured and private edge data can be retained. To achieve efficient and effective edge model generation, we introduce a pretraining-assisted on-cloud model elastification method and an edge-friendly on-device architecture search method. Model elastification generates a high-quality search space of model architectures with the guidance of a developer-specified oracle model. Each subnet in the space is a valid model with different environment affinity, and each device efficiently finds and maintains the most suitable subnet based on a series of edge-tailored optimizations. Extensive experiments on various edge devices demonstrate that our approach is able to achieve significantly better accuracy-latency tradeoffs (e.g. 46.74\% higher on average accuracy with a 60\% latency budget) than strong baselines with minimal overhead (13 GPU hours in the cloud and 2 minutes on the edge server).

• The paper presents AdaptiveNet, a novel framework that dynamically adapts deep learning models post-deployment to optimize performance in heterogeneous, resource-constrained edge environments.
• It utilizes on-cloud elastification to convert a pre-trained model into a versatile supernet and employs on-device, latency-guided search to select optimal subnets.
• Experimental results demonstrate significant accuracy improvements and efficient adaptation, achieving superior accuracy-latency trade-offs with minimal GPU hours and rapid on-device optimization.

AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environments

Introduction

The paper "AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environments" addresses the critical challenge of deploying deep learning models across heterogeneous edge devices. The diversity of edge environments, characterized by varying computational resources and data distributions, complicates the task of generating a single model that performs optimally across all scenarios. Traditional pre-deployment techniques, often reliant on centralized cloud processing for model generation, struggle to efficiently cope with this diversity.

AdaptiveNet proposes a novel solution by shifting the adaptation task to post-deployment, allowing the model to adapt dynamically to its specific edge environment. This approach not only enhances accuracy and resource utilization but also protects user privacy by eliminating the need for extensive edge data collection and processing in the cloud.

Methodology

AdaptiveNet combines on-cloud model elastification with on-device architecture search to handle the diversity and dynamic nature of edge environments effectively.

On-Cloud Elastification

The elastification process transforms a given pre-trained model into a "supernet," capable of adapting to varying environments through the selection of optimal subnet architectures. This involves:

1. Granularity-aware Graph Expansion: The process starts by identifying basic blocks in the model's computational graph, determining replaceable paths, and expanding the model into a supernet with multiple paths. This expansion includes merging blocks for efficiency and creating varied structures that suit different computational budgets.
2. Distillation-based Supernet Training: Incorporates a two-phase training strategy—branch-wise distillation followed by whole-model tuning. This method leverages knowledge from the original pre-trained model to ensure the quality of subnets generated from the supernet.

   Figure 1: The architecture overview of AdaptiveNet.

On-Device Adaptation

Once deployed, the supernet undergoes edge-specific optimization to select the most appropriate subnet using efficient search strategies and model evaluation techniques:

• Model-Guided Search: Utilizes a latency model to guide the search for subnets, ensuring they meet the latency constraints of the device while maximizing accuracy.
• Reuse-based Evaluation: Improves efficiency by reusing shared computation results across subnets, significantly reducing the time spent in evaluating potential models.

  Figure 2: Illustration of the branch-wise distillation phase.

Experimental Evaluation

Experimental results demonstrate that AdaptiveNet achieves superior accuracy-latency trade-offs across diverse tasks and devices:

• Classification, Detection, and Segmentation: Experiments conducted on tasks such as image classification, object detection, and semantic segmentation reveal that AdaptiveNet significantly outperforms baseline approaches, particularly at lower latency budgets.
• Scalability and Efficiency: The method exhibits minimal overhead, requiring only 13 GPU hours for the on-cloud stage and about 2 minutes for on-device adaptation. It effectively adapts models with substantial accuracy improvements compared to alternatives.

  Figure 3: The latency-accuracy tradeoffs of models generated by different techniques on the target devices.

Implications and Future Work

AdaptiveNet offers a paradigm shift in deploying neural networks to edge devices by facilitating real-time, privacy-preserving model adaptation. The approach holds significant promise for enhancing AI deployment in dynamic and resource-constrained environments. Future research may explore further reducing on-device adaptation latency and extending this framework to other modalities and applications beyond computer vision.

Conclusion

AdaptiveNet effectively addresses the limitations of pre-deployment model generation by leveraging post-deployment adaptation to improve accuracy and resource efficiency across heterogeneous and dynamic edge environments. The method’s integration of intelligent hardware-aware adaptation paves the way for broader adoption of AI applications in privacy-sensitive and resource-constrained edge scenarios.