Towards Energy Efficient Co-Scheduling in HPC

Abstract: Modern multi GPU HPC systems expose substantial computational capacity, yet inefficient GPU allocation often leads to wasted energy and underutilization. In practice, GPU applications exhibit heterogeneous and nonlinear scaling, making it inefficient to always use all available GPUs. We present EcoSched, an online scheduler that jointly optimizes GPU count selection and application coscheduling to improve workload level efficiency on multi GPU systems. EcoSched uses lightweight runtime profiling to estimate relative performance across GPU counts, applies a score based policy to balance energy efficiency and idle resources, and incorporates NUMA aware placement to mitigate interference. We implement EcoSched on heterogeneous CPU GPU platforms and evaluate it with diverse workloads on H100, A100, and V100 systems. EcoSched achieves up to 14.8% energy savings, 30.1% makespan improvement, and 40.4% EDP reduction over baseline schedulers, with modest performance overhead. These results show that jointly selecting GPU counts and coscheduling actions is essential for efficient multi GPU workload execution.

• The paper proposes EcoSched, a framework that jointly optimizes GPU-count allocation and job co-scheduling to reduce energy consumption with modest performance trade-offs.
• It leverages GPU DRAM utilization as a proxy for runtime, enabling lightweight online profiling and NUMA-aware partitioning to manage resource contention.
• Evaluation results show 14.8% energy savings, 30.1% makespan improvement, and significant EDP reduction compared to conventional sequential GPU allocation.

Energy-Efficient Co-Scheduling for Multi-GPU HPC Workloads: EcoSched

Motivation and Problem Formulation

Energy efficiency in HPC systems is an acute concern due to the scaling of multi-GPU compute nodes and the associated rise in operational costs. Conventional static resource allocations result in significant energy wastage: single-GPU jobs leave additional GPUs idle and even multi-GPU jobs exhibit heterogeneous scaling, often with minimal performance gains when additional GPUs are provisioned. The non-linear, workload-dependent scaling necessitates a joint optimization strategy for both GPU-count selection and application co-scheduling to minimize total energy consumption while controlling performance loss.

Figure 1: Application execution time across GPU counts, demonstrating heterogeneous and non-linear scaling trends.

Reducing GPU allocation can yield substantial energy benefits at modest performance costs, but true system efficiency is only realized when released GPUs are rapidly repurposed for concurrent execution. This coupled scheduling problem is challenging due to the vast action space, the unpredictability of online execution characteristics, and the risk of interference through shared memory and I/O resources.

Figure 2: Benchmarking energy and runtime trade-offs for selected applications on H100 with one GPU less than performance-optimal allocation.

EcoSched Architecture and Design

EcoSched is introduced as an online scheduler that integrates lightweight performance modeling and energy-aware decision-making. Scheduling proceeds in two phases within a bounded job window:

• Phase I (Performance Modeling): Online profiling exploits GPU DRAM utilization as a proxy for application scaling, avoiding exhaustive offline profiling while capturing relative scaling behaviors. GPU DRAM utilization exhibits strong correlation with runtime across architectures, enabling normalized performance and energy proxies that guide allocation decisions.

Figure 3: Correlation between GPU DRAM utilization and runtime across H100, A100, and V100.

• Phase II (Energy-Aware Co-Scheduling): Joint actions specifying GPU allocations and job selections are greedily scored to minimize a composite of normalized energy regret and idle-GPU fraction. NUMA-aware resource partitioning is deployed to mitigate resource contention, constraining co-located jobs to isolated NUMA domains for predictable performance and energy profiles.

  Figure 4: Schematic of EcoSched’s phased workflow—online modeling and energy-aware scheduling within a window.

Evaluation and Results

EcoSched is evaluated on diverse HPC and ML workloads across three NVIDIA platforms (H100, A100, V100) and benchmarked against sequential GPU allocation baselines, Marble (offline profiling-based), and an Oracle with perfect prior knowledge.

EcoSched achieves the following:

• 14.8% energy savings, 30.1% makespan improvement, and 40.4% EDP reduction on H100 relative to sequential GPU allocation.
• Outperforms Marble, which realizes only 4.2% energy savings and 11.5% makespan reduction.
• Approaches Oracle performance, attaining nearly all achievable efficiency on compute-bound V100 configurations.
• Energy and makespan improvements directly scale with the system’s slack and the non-linear scaling behavior of constituent applications.

Figure 5: Comparative analysis of scheduling strategies in energy savings, makespan, and EDP improvement across GPU platforms.

Mechanism and Scheduling Trade-Offs

EcoSched’s gains are enabled by selective GPU-count downsizing, targeting applications with early strong-scaling flattening. This reduces GPU-seconds and unlocks pack-friendly concurrent schedules at modest performance cost:

• Pot3d, resnet50, and gpt2 downsizing incurs only 10%, 5%, and 8% slowdowns, respectively, while increasing resource reuse.
• Concurrent job execution reshapes scheduling: EcoSched schedules pot3d with two GPUs, facilitating parallel execution and yielding a 30% makespan reduction and 17% energy decrease.

  Figure 6: Workflow visualization contrasting Marble’s serialization vs. EcoSched’s parallelization via downsizing.

  

  Figure 7: Normalized per-application energy breakdown with EcoSched, demonstrating energy savings concentrated in select applications.

Overhead and Practicality

EcoSched's online profiling imposes minimal execution overhead (sub-millisecond decision latency, <70kJ profiling energy per application). The energy cost is rapidly amortized through active and idle GPU savings. Application-level performance loss is controlled and is often an intentional trade-off for improved system-level efficiency.

Figure 8: Quantification of per-application runtime increases under EcoSched relative to solo performance-optimal execution.

Implications, Limitations, and Extensions

EcoSched demonstrates that coupled online scheduling decisions—joint GPU-count assignment and co-running sets—are indispensable for optimal energy utilization in static HPC environments. The reliance on lightweight signals (GPU DRAM utilization) and NUMA-aware isolation enables practical deployment without prohibitive profiling requirements.

The approach targets long-running simulation and ML training workloads where the initial profiling cost is trivial relative to execution duration. Integration as a local, window-based node scheduler allows for seamless adoption with global batch schedulers enforcing broader policy constraints.

Platform portability is limited by current reliance on NVIDIA telemetry, but the core algorithm is hardware-agnostic. EcoSched’s present focus on whole-GPU allocations could be expanded to finer-grained sharing modes (MIG, MPS), albeit at increased model complexity.

Conclusion

EcoSched presents a rigorous energy-aware co-scheduling framework for multi-GPU nodes under HPC batch constraints (2604.17640). By combining lightweight performance modeling, greedy energy-aware score-based action selection, and NUMA-aware placement, the scheduler achieves significant energy and throughput gains with bounded workflow overhead. The necessity of coupling GPU-count with job placement decisions is affirmed with empirical evidence, motivating adaptive, scalable scheduling logic in future HPC resource managers. Extensions to cluster-wide scheduling, multi-domain interference modeling, and portability across heterogeneous hardware platforms remain as open areas for further inquiry.