GridPilot: Grid-Responsive AI/HPC Control

• GridPilot is a real-time, grid-responsive control system that treats AI/HPC loads as controllable assets by using a three-tier predictive controller and safety island for sub-second adjustments.
• It integrates per-GPU PID loops, host-level AR(4) predictors, and hourly cluster planning to optimize power usage and respond deterministically to grid triggers.
• The system demonstrated an end-to-end trigger-to-target latency of 97.2 ms on a three-GPU testbed, outperforming Nordic FFR requirements and incorporating PUE-aware facility modeling.

GridPilot is a real-time, grid-responsive control system for AI supercomputers that treats multi-megawatt AI/HPC demand as a controllable electrical load rather than as a passive consumer. It was introduced as a three-tier predictive controller operating across milliseconds, seconds, and hours, augmented by a deterministic safety-island bypass for fast response. On a three-GPU NVIDIA V100 testbed, it achieved a measured end-to-end trigger-to-target response of 97.2 ms, which is 6.9x faster than the 700 ms requirement of Nordic Fast Frequency Reserve, and it incorporates an instantaneous Power Usage Effectiveness correction so that dispatched commitments remain robust at the facility meter rather than only at the IT-load level (Constantinescu et al., 25 May 2026).

1. Problem setting and operational target

GridPilot is situated in the context of two coupled trends: the rapid growth of AI/HPC electricity demand and the increasing share of variable wind and solar generation in modern power systems. The motivating problem is not generic energy efficiency, but the practical question of how quickly a software stack can translate a grid request into a measurable power change at the facility meter, where ancillary-service commitments are settled (Constantinescu et al., 25 May 2026).

The control target is shaped by contemporary frequency-response products. The paper distinguishes slower services such as Frequency Containment Reserve, with response in approximately 30 s, from faster products such as Nordic Fast Frequency Reserve, which requires full response in 700 ms after frequency drops below 49.7 Hz. It also notes emerging sub-second pilots in Croatia and Iberia. Within this setting, large AI/HPC data centers are treated as flexible loads that can adjust power within seconds, or faster, to absorb renewable variability and support frequency stability (Constantinescu et al., 25 May 2026).

The paper identifies three obstacles that make this difficult. First, prior work on GPU/CPU power control and cooling optimization generally does not measure end-to-end latency from a TSO-style trigger to actual power change. Second, grid commitments are defined at the facility meter, not at the GPU or server level, so static PUE assumptions can produce under-delivery at warm-water-cooled HPC sites. Third, the most valuable services are increasingly sub-second, which imposes hard determinism requirements on the entire software and hardware path (Constantinescu et al., 25 May 2026).

2. Multi-timescale architecture

GridPilot is organized as a hierarchical controller with three predictive tiers and a fast bypass path. The hierarchy separates inner-loop actuation from host-level coordination and cluster-level planning, while the bypass preserves deterministic response during the sub-second window in which grid-service compliance is decided (Constantinescu et al., 25 May 2026).

Component	Cadence	Function
Tier 1: per-GPU PID loop	200 Hz	Tracks per-GPU power target via NVML cap
Tier 2: per-host AR(4) coordinator	1 Hz	Chooses per-GPU caps within a host envelope
Tier 3: cluster operating-point selector	Hourly	Selects mean utilization and reserve band over 24 h
Safety island	Triggered event path	Bypasses higher tiers for deterministic fast response

Tier 1 operates on individual GPU boards with a 5 ms tick and uses NVIDIA power capping through NVML, exposed operationally as nvidia-smi -pl <W>. Its purpose is to make a commanded per-GPU target power $p^\star$ physically realizable with tight settling behavior. Tier 2 runs once per second on each host, predicts short-horizon utilization with an AR(4) model, and allocates per-GPU caps so that predicted host power matches the envelope received from the cluster layer. Tier 3 runs hourly over a 24 h look-ahead and selects an operating point defined by mean utilization $\mu$ and reserve band $\rho$, jointly optimizing frequency-response quality and carbon-free energy share while incorporating meter-level PUE effects (Constantinescu et al., 25 May 2026).

The safety island is the critical real-time mechanism. It is implemented as a small C program, under 400 lines of code, running as a real-time thread with SCHED_FIFO priority 80 and pinned to an isolated CPU core. It listens on a dedicated UDP socket for TSO trigger signals, reads the signal in approximately 1 ms, looks up precomputed per-GPU targets in under 50 $\mu$s, and writes new GPU power caps directly via NVML with approximately 5 ms worst-case latency. In the sub-second response window, it bypasses Tier 2 and Tier 3 entirely; those higher tiers continue running and reconcile the new operating point on their next scheduled ticks (Constantinescu et al., 25 May 2026).

3. Control formulation and facility-meter modeling

At Tier 1, each GPU is controlled by a discrete PID law with step size $\Delta t = 5\ \mathrm{ms}$. If $p_k$ is measured GPU power and $p^\star$ is the target, the error is $e_k = p^\star - p_k$, and the control input is

$$u_k = K_p e_k + K_i \sum_{i=0}^{k} e_i \Delta t + K_d \frac{e_k - e_{k-1}}{\Delta t},$$

with gains $(K_p, K_i, K_d) = (0.6, 0.05, 0.02)$. The implementation includes integral anti-windup with $\mu$0, saturation to the V100 SXM2 cap range $\mu$1, and a first-order thermal model with time constant $\mu$2. If predicted junction temperature exceeds 85 $\mu$3, the controller falls back to a 200 W cap for protection (Constantinescu et al., 25 May 2026).

Tier 2 predicts one-step-ahead host utilization at 1 Hz with an AR(4) model,

$\mu$4

where the coefficients are learned by Recursive Least Squares over a 30 s rolling window with forgetting factor $\mu$5. Using predicted utilization, host envelope, and calibrated per-GPU power models, Tier 2 chooses a vector of per-GPU targets $\mu$6 to keep predicted host power close to the host setpoint. Functionally, it acts as a short-horizon model-predictive coordinator, although the paper presents it as a compact host-level optimization rather than as a full MPC formalism (Constantinescu et al., 25 May 2026).

Tier 3 performs an hourly grid search over a discrete operating-point space. The admissible values are

$\mu$7

and the objective is

$\mu$8

where $\mu$9 is relative Fast Frequency Reserve provision quality at the facility meter and $\rho$0 is the carbon-free energy share. Scheduling decisions are driven by the composite signal

$\rho$1

with a dispatch rule that defers jobs when $\rho$2 exceeds its 24 h 66th percentile, the job has not consumed 70% of its maximum wait time, and the job is not short. The controller also applies an 80% power cap to running jobs under high $\rho$3, borrowing that factor from EcoFreq (Constantinescu et al., 25 May 2026).

A defining feature of GridPilot is that it models facility power rather than IT power alone. PUE is defined as

$\rho$4

Here $\rho$5, $\rho$6 with a floor at 20% of design, $\rho$7 with a floor at 15%, and $\rho$8 depends on an ambient-driven free-cooling fraction $\rho$9 that ramps linearly from 0 at 25 $\mu$0 wet-bulb to 1 at 12 $\mu$1. The model is calibrated to the Marconi100 design point with PUE $\mu$2 at full load, and the paper reports up to 30% Mean Absolute Percentage Error reduction versus static-PUE accounting for emissions estimation (Constantinescu et al., 25 May 2026).

4. Implementation and real-hardware testbed

All real-hardware results were produced on the ecocloud-exp06 node at EPFL EcoCloud. The node contains three NVIDIA Tesla V100 SXM2 GPUs with 32 GB memory and 300 W TDP each, a CPU with 36 physical and 72 logical cores, and 379 GiB of usable RAM. The software stack includes Ubuntu 24.04 LTS, NVIDIA driver 580.65.06, NVML 12.x, and CUDA 12.x. Telemetry is collected via NVML at 100 Hz for GPU power and via CPU plus RAPL telemetry at 10 Hz (Constantinescu et al., 25 May 2026).

The paper evaluates three workload archetypes chosen to span typical GPU power behaviors. matmul is a cuBLAS FP32 GEMM in a single stream and is compute-bound near TDP. inference is ResNet-50 FP16 with batch size 1 and is memory-bound with mean GPU power below 200 W. bursty is a synthetic workload with 4 s period and 50% duty cycle, alternating 2 s compute and 2 s idle, thereby generating strong power oscillations (Constantinescu et al., 25 May 2026).

The trigger-handling path is intentionally short. A UDP packet arrives on a dedicated control socket, the safety island parses it, maps it to precomputed per-GPU targets, applies new caps through NVML, and leaves Tier 1 to settle board power to the new operating point. The latency decomposition is written as

$\mu$3

with measured or assumed contributions of approximately 1 ms for trigger reception, under 50 $\mu$4s for decision, approximately 5 ms for NVML actuation, and approximately 90 ms for plant settling. The paper also provides a TLA$\mu$5 specification for the safety island and states that it terminates within four NVML update intervals, that is, within 20 ms, under POSIX SCHED_FIFO assumptions. By contrast, a Python-based implementation exhibits p99 latencies above 250 ms because of garbage collection, lazy imports, and operating-system scheduling effects (Constantinescu et al., 25 May 2026).

5. Experimental results

The calibration study searches 36 combinations of power caps and SM frequencies. It identifies a best-efficiency point around $\mu$6 and $\mu$7, with iterations-per-joule within $\mu$8 across workloads. The fitted GPU power model

$\mu$9

uses $\Delta t = 5\ \mathrm{ms}$0 and achieves approximately 3.45% MAE in leave-one-out cross-validation. When scaled to 980 nodes, the resulting facility-power envelope matches Marconi100’s published reference within +1.4% (Constantinescu et al., 25 May 2026).

Inner-loop step-response experiments show fast settling to within $\Delta t = 5\ \mathrm{ms}$1 of a new target after cap transitions such as 280 W to 200 W. The measured settling times are approximately 18 ms for matmul, 21 ms for inference, and 29 ms for bursty. These responses are reported as sharp transitions without overshoot beyond the $\Delta t = 5\ \mathrm{ms}$2 band (Constantinescu et al., 25 May 2026).

Closed-loop demand-following over a 30 s host-power trajectory yields tracking errors of 1.68% for inference, 2.12% for matmul, and 11.08% for bursty. The paper uses a 5% tracking-error band as a diagnostic threshold: below it, the inner GPU PID loop is sufficient; above it, Tier 2 is actively compensating for workload variability. The bursty workload exceeds that band, which the authors attribute to its deliberately aggressive oscillatory structure rather than to instability (Constantinescu et al., 25 May 2026).

The central real-time result is the end-to-end trigger-to-target latency. Across 90 trials, with 30 trials per workload and randomized inter-trial delays from 5 to 30 s, GridPilot achieves a measured end-to-end response of 97.2 ms, satisfying the Nordic FFR 700 ms budget with a 6.9x margin. This is the core empirical demonstration that the controller can translate a grid request into a real GPU-power response within a strict ancillary-service timescale (Constantinescu et al., 25 May 2026).

At facility scale, replay experiments across six representative European grids, from Sweden to Poland, show that the PUE-aware controller closes 2.5–5.8 percentage points of cooling-overhead drag at 50 MW IT power relative to a CI-only baseline. Gains are reported as larger on low-CI grids and smaller on high-CI grids. In a 100-host simulation on the German grid, the AR(4) predictor obtains MAE $\Delta t = 5\ \mathrm{ms}$3 on the utilization scale with p95 $\Delta t = 5\ \mathrm{ms}$4, FFR provision quality reaches 1.0 at a 20% reserve band, and the operating point shifts to 0.90 mean utilization during green-rich daytime windows and 0.40 overnight. The same simulation runs at more than 26,000x real time and yields net CO$\Delta t = 5\ \mathrm{ms}$5 savings of 21%, 20%, and 26% for Switzerland, Italy, and Germany, respectively (Constantinescu et al., 25 May 2026).

6. Position, interpretation, and limitations

GridPilot is positioned against several adjacent research lines rather than as a minor extension of any one of them. The paper distinguishes it from single-layer GPU DVFS and power-capping work such as MF-GPOEO, DRLCap, and Perseus; from steady-state multi-layer controllers such as DynamoLLM; from cooling and PUE models that are not coupled to frequency-response markets; and from cluster power managers such as PowerSched, EAR, GEOPM, and REGALE that operate under a fixed facility envelope. Its stated novelty is the composition of a millisecond GPU PID, a second-level host AR(4) predictor, an hourly cluster planner, a deterministic safety-island bypass, and explicit PUE-aware meter-level accounting within a single grid-service framework (Constantinescu et al., 25 May 2026).

The name may invite confusion with the older “pilot” literature in distributed computing. In the P* model, a pilot is a placeholder job or resource container submitted to a resource manager and later filled with Compute Units (Luckow et al., 2012). Pilot-Edge extends that abstraction across edge devices, cloud VMs, HPC systems, and serverless functions while preserving the separation between resource acquisition and workload placement (Luckow et al., 2021). GridPilot differs in subject matter: it is a controller for electrical-grid-responsive operation of AI/HPC facilities rather than a pilot-job framework for distributed resource management. This suggests that its “grid” refers to the power grid, not to grid middleware in the classical distributed-systems sense.

The paper is explicit about several limitations. Real-hardware measurements are confined to a testbed with three V100 GPUs; H100, H200, and MI300 platforms are identified as future targets. Fast Frequency Reserve triggers are synthetic rather than part of a formal TSO prequalification campaign. Rack-scale and full data-center cooling dynamics are modeled rather than directly measured. Broader job mixes, including multi-tenant cloud and interactive workloads, are left for future evaluation, as is tighter integration with production cluster managers such as PowerSched or EAR (Constantinescu et al., 25 May 2026).

The broader implication is that GridPilot serves as a proof of concept that MW-scale AI/HPC demand can be engineered as controllable, grid-responsive flexibility by design. A plausible implication is that ancillary-service participation by AI/HPC facilities will depend not only on server-level power control, but also on deterministic trigger handling and facility-meter correctness under dynamic PUE. Within that interpretation, GridPilot is best understood as an architectural template for coupling real-time power electronics constraints, workload scheduling, and cooling-aware accounting in a single control plane (Constantinescu et al., 25 May 2026).