EcoShift: Performance-Aware Power Management for Power-Constrained Heterogeneous Systems

Abstract: Power-constrained HPC systems increasingly run heterogeneous CPU--GPU applications under strict cluster-wide power limits. Existing cluster-wide power management policies rely on fair-share or utilization heuristics and do not capture application-specific sensitivity to CPU and GPU power caps, leading to inefficient use of reclaimed power. We present EcoShift, a performance-aware cluster-wide power management framework. EcoShift combines online performance prediction with a dynamic-programming-based allocator to distribute reclaimed power across CPU--GPU applications for maximum average performance improvement. Through emulation-based evaluation on two heterogeneous Intel CPU and NVIDIA A100/H100 GPU platforms with diverse CPU--GPU workloads, EcoShift consistently outperforms state-of-the-art policies, achieving up to 6% average performance improvement while preserving the cluster-wide power constraint.

• The paper introduces EcoShift, a dynamic programming-based approach that optimizes power redistribution in heterogeneous HPC clusters.
• It employs online profiling and neural collaborative filtering to capture distinct CPU/GPU performance sensitivities, enabling up to 6% improvement.
• Evaluation shows near-optimal allocation (<3% gap with oracle) and fair performance distribution, highlighting its practical impact on large-scale systems.

EcoShift: Performance-Aware Power Management for Power-Constrained Heterogeneous Systems

Motivation and Background

The convergence of exascale computing and heterogeneous architectures, combining CPUs and GPUs, has foregrounded power management as a central operational constraint in HPC environments. Cluster-wide power capping is a practical necessity, yet uniform or simple utilization-based distribution of power frequently results in inefficiencies: heterogeneous workloads exhibit diverse and asymmetric sensitivity to CPU and GPU power limits. Empirical studies reveal that power reclamation opportunities—unused power allocation resulting from applications operating below their caps—are non-trivial. However, state-of-the-art policies such as fair-share and demand-proportionate allocation do not leverage the distinct marginal performance gains enabled by targeted allocation of reclaimed power.

The application-level performance response to incremental power—either in CPU or GPU domains—manifests as varied and often non-linear, encompassing regimes of strong sensitivity and regimes of diminishing returns. For instance, CPU-bound and GPU-bound applications benefit differently from increased resource budgets, and these effects are modulated by the current operating point. Uniform reallocation misses opportunities to deploy reclaimed power where it achieves maximal acceleration.

Figure 1: Heatmaps of normalized application performance on an Intel Xeon Platinum 8468 + NVIDIA H100 node under various CPU and GPU power caps; heterogeneous applications exhibit distinct and asymmetric sensitivity profiles.

EcoShift Architecture and Design

EcoShift proposes a cluster-level, performance-maximizing allocation mechanism that redistributes reclaimed power using online performance prediction and dynamic programming (DP)-based optimization. The workflow begins with a brief, online profiling phase to capture the unique power-performance surface of each running workload. Neural collaborative filtering (NCF) predicts unseen points on the application’s CPU/GPU cap matrix, thereby avoiding extended profiling or historical training data for all possibilities.

Figure 2: EcoShift workflow: concurrent online profiling, runtime-based performance prediction using NCF, and DP-based search for optimal cluster-wide allocation under constraints.

Given this per-application power-performance characterization, EcoShift frames the reclaimed-power assignment as a multiple-choice knapsack problem: for each application, enumerate possible CPU-GPU cap upgrades and their predicted runtime improvements, then assign the fixed additional power budget across the mix to maximize average relative improvement. The DP-based search leverages the monotonic, compressed 1D improvement-versus-budget curves that result from discarding dominated configurations for each application.

Figure 3: EcoShift’s DP search: each application’s feasible cap upgrades are represented as a performance-improvement curve indexed by extra power; DP finds the allocation maximizing total improvement under the power constraint.

Experimental Evaluation

Extensive emulation-based evaluation is conducted on two platforms (Intel Xeon Platinum with NVIDIA A100 or H100), using a comprehensive suite of 40 CPU-GPU heterogeneous applications spanning multiple sensitivity classes. EcoShift is benchmarked against two cluster-wide baselines, DPS and MixedAdaptive, as well as an unattainable Oracle (exhaustive brute-force search leveraging perfect information) in small-scale settings.

Performance results demonstrate:

• Average performance improvement with EcoShift consistently exceeds baselines, achieving up to 6% higher gains over state-of-the-art methods.
• The advantage is most pronounced when initial power caps are tight, and diminishes as absolute system capacity increases, aligning all policies in the regime where workloads are unsaturated.
• Application-level distributional analysis shows EcoShift’s improvement is not concentrated on outliers but raises the typical case across workloads.

  Figure 4: Average relative performance improvement for DPS, MixedAdaptive, and EcoShift under varying amounts of cluster-wide extra power, across simulated 100-node clusters (System 1, CPU 140 W/GPU 150 W initial caps).

  

  Figure 5: Sensitivity of performance improvement to initial cap settings for each policy, with fixed 7000 W cluster-wide extra power on System 1.

On both measured systems, the NCF-based predictor achieves mean accuracy between 93–95% across all evaluated applications and cap settings, supporting the DP allocator with reliable performance surfaces.

Robustness, Fairness, and Oracle Gap

EcoShift is evaluated not only for mean improvement but also for distributional fairness, using Jain’s index. Although performance-aware allocation may non-uniformly distribute reclaimed power (by design), the median fairness of EcoShift matches that of competing baselines; no systematic fairness collapse is observed.

Figure 6: Violin plots: Application-wise distribution of performance improvements (System 1, 100 nodes, initial CPU/GPU 140/150 W); EcoShift produces higher improvements across most workload types.

Figure 7: Jain's fairness index for the mixed workloads, showing comparable median fairness for EcoShift and baseline policies.

In cases amenable to brute-force search (10-application configurations), the gap between EcoShift’s DP allocation and the true Oracle is <3 percentage points in 90% of cases, indicating high solution optimality despite aggressive pruning of the search space.

Figure 8: Cumulative distribution function of the absolute improvement gap between EcoShift's DP allocation and the Oracle (across 100 random test cases).

Implications and Future Directions

EcoShift demonstrates that explicit modeling of heterogeneous power-performance surfaces, combined with tractable yet near-optimal dynamic programming allocation strategies, substantially increases the effective utilization of reclaimed power in large HPC clusters. The improvement—up to 6% on average—has significant operational impact at scale, potentially translating into millions of core-hours of accelerated computation annually in large centers.

The practical implications include:

• Reduced reliance on coarse heuristics; systems can deploy dynamic, performance-aware, application-specific power redistribution without requiring prohibitive offline profiling.
• The architecture is portable to any platform exposing appropriate telemetry and capping interfaces (e.g., RAPL, NVML, or future APU-class hardware).

Key future work includes integration with dynamic scheduling systems, evaluation in live concurrent deployments (addressing non-determinism, thermal transients, and power-capping enforcement nuances), and adaptation to tightly coupled heterogeneous packages such as AMD MI300A.

Conclusion

EcoShift presents a scalable, online approach for maximizing average performance in power-constrained heterogeneous clusters by redistributing reclaimed power via performance-aware, DP-based optimization. Demonstrated across multiple platforms and a diverse workload suite, the framework yields consistent, statistically significant average performance improvement—up to 6%—over previous methods while maintaining balanced per-application gains. The solution closes most of the gap with brute-force optimality yet achieves overheads suitable for real-time cluster operation. As heterogeneous architectures and strict power budgets proliferate, this approach provides a critical step toward responsive, intelligent, and efficient cluster power management.

Reference: "EcoShift: Performance-Aware Power Management for Power-Constrained Heterogeneous Systems" (2604.17635)