HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments (2111.10635v4)

Abstract: Deep neural networks (DNNs) exploit many layers and a large number of parameters to achieve excellent performance. The training process of DNN models generally handles large-scale input data with many sparse features, which incurs high Input/Output (IO) cost, while some layers are compute-intensive. The training process generally exploits distributed computing resources to reduce training time. In addition, heterogeneous computing resources, e.g., CPUs, GPUs of multiple types, are available for the distributed training process. Thus, the scheduling of multiple layers to diverse computing resources is critical for the training process. To efficiently train a DNN model using the heterogeneous computing resources, we propose a distributed framework, i.e., Paddle-Heterogeneous Parameter Server (Paddle-HeterPS), composed of a distributed architecture and a Reinforcement Learning (RL)-based scheduling method. The advantages of Paddle-HeterPS are three-fold compared with existing frameworks. First, Paddle-HeterPS enables efficient training process of diverse workloads with heterogeneous computing resources. Second, Paddle-HeterPS exploits an RL-based method to efficiently schedule the workload of each layer to appropriate computing resources to minimize the cost while satisfying throughput constraints. Third, Paddle-HeterPS manages data storage and data communication among distributed computing resources. We carry out extensive experiments to show that Paddle-HeterPS significantly outperforms state-of-the-art approaches in terms of throughput (14.5 times higher) and monetary cost (312.3% smaller). The codes of the framework are publicly available at: https://github.com/PaddlePaddle/Paddle.

• The paper introduces a novel distributed framework that uses reinforcement learning to optimize resource scheduling across diverse computing environments.
• HeterPS’s RL scheduler dynamically allocates DNN layer workloads, achieving up to 14.5x throughput improvement and 312.3% cost reduction.
• The framework integrates distributed training, adaptive scheduling, and efficient data management, paving the way for scalable future innovations.

HeterPS: Distributed Deep Learning With Reinforcement Learning-Based Scheduling in Heterogeneous Environments

The paper by Liu et al. introduces HeterPS, an innovative framework designed to optimize the distributed training of deep neural networks (DNNs) across heterogeneous computing environments. The proposal aims to address the challenge of efficiently allocating computing resources, such as CPUs and GPUs, for diverse training workloads, in order to minimize costs while meeting throughput constraints.

The authors identify that the expansive nature of DNN models involves both compute-intensive and data-intensive tasks. These distinct characteristics necessitate strategic scheduling to leverage the benefits of various heterogeneous computing systems. HeterPS consists of a distributed architecture augmented by a Reinforcement Learning (RL)-based scheduling method, optimizing the training workflow across different hardware resources.

Key Contributions

1. Distributed Architecture: HeterPS enables the efficient training of diverse workloads with heterogeneous resources by utilizing a distributed system that incorporates CPUs, GPUs, and other AI processors. The framework pragmatically manages both data storage and communication among the distributed computing units.
2. RL-Based Scheduling: A unique feature of HeterPS is its employment of an RL-based scheduling algorithm. This approach schedules the workload of each layer to the appropriate computing resource type, optimizing the overall process to minimize costs and satisfy throughput demands.
3. Experimentation and Results: The authors conduct extensive experiments demonstrating that HeterPS significantly surpasses existing state-of-the-art methods. It achieves throughput improvement by up to 14.5 times and cost reduction by 312.3%.

Methodology

HeterPS is structured into three core modules:

• Distributed Training Module: This module manages model training using both parameter server and ring-allreduce architectures, facilitating parallelism and data management among heterogeneous resources.
• Scheduling Module: Utilizing an LSTM model, the RL strategy schedules layers to appropriate resources based on real executive feedback and a cost model. The paper indicates that this approach results in cost-effective provisioning plans that ensure resource allocation efficiency.
• Data Management Module: This module is responsible for efficiently handling data transfer and storage. It supports data caching and compression to optimize data flow and resource usage during model training.

Implications and Future Directions

The introduction of HeterPS signifies a scalable and adaptable method of addressing the complexities involved in distributed DNN training across heterogeneous environments. Its approach to leveraging RL for dynamic scheduling presents significant implications for the cost-efficiency of machine learning operations at scale, particularly in environments with access to varying computational resources.

Looking ahead, this research opens pathways to further integrate security and privacy solutions in distributed training frameworks, especially in scenarios involving decentralized data across multiple data centers. The potential extension of HeterPS into federated learning frameworks could capitalize on its versatile scheduling to maintain data confidentiality while achieving optimal training performance.

In summary, Liu et al. present a compelling architecture in HeterPS that not only optimizes resource usage but also lays foundational work for future advancements in adaptive and efficient deep learning training methodologies. The paper’s insights and empirical results contribute significantly to the ongoing discourse in distributed machine learning, offering practical solutions and a solid ground for future innovation in AI resource management.