ReGate: Fine-Grained NPU Power Gating

• ReGate is a hardware/software co-design framework that enables fine-grained power gating in NPUs by assigning component-specific management schemes.
• It employs both hardware-managed and compiler-directed strategies, optimizing systolic arrays, vector units, SRAM, HBM controllers, and inter-chip interconnects.
• Implemented on real hardware and simulators, ReGate achieves up to 32.8% energy reduction and minimal performance overhead while lowering operational CO2 emissions.

Searching arXiv for the specified paper and closely related NPU power-gating work. ReGate is a hardware/software co-design framework for enabling fine-grained power-gating in neural processing units (NPUs), introduced in “ReGate: Enabling Power Gating in Neural Processing Units” (Xue et al., 4 Aug 2025). It targets a specific inefficiency in modern NPU chips: static power dissipation accounts for 30%–72% of total energy consumption because contemporary NPUs largely lack effective power-management support. ReGate addresses this by selecting component-specific power-management schemes across systolic arrays (SAs), vector units (VUs), SRAM, high-bandwidth memory (HBM) controllers, and inter-chip interconnects (ICI), combining hardware-managed and software-managed mechanisms to reduce energy with negligible impact on AI workload performance (Xue et al., 4 Aug 2025).

1. Problem setting and design objective

ReGate begins from an empirical characterization of NPU energy use: across different generations of NPU chips, 30%–72% of energy consumption is attributed to static power dissipation (Xue et al., 4 Aug 2025). The framework’s central objective is therefore not to increase peak throughput, but to suppress leakage and idle power in domain-specific accelerators whose large functional blocks frequently remain inactive for substantial intervals.

The design premise is that NPUs differ fundamentally from generic processors. Conventional power-gating techniques for CPUs and GPUs do not transfer directly because NPUs contain large domain-specific units, exhibit long power-gate and wake-up latencies, and execute workloads through compiler-driven scheduling and statistically scheduled VLIW ISAs. ReGate therefore treats power management as an architectural specialization problem rather than as a direct reuse of generic idle-detection schemes (Xue et al., 4 Aug 2025). A plausible implication is that the framework’s novelty lies less in the existence of power-gating itself than in the mapping between NPU execution structure and the appropriate management domain for each component.

2. NPU-specific challenges and co-design principles

The framework identifies three constraints that shape NPU power management. First, large structures such as SAs show non-uniform utilization, including both temporal and spatial underutilization. Second, execution is often deterministic, which creates opportunities for predictive power control but also constrains the timing of wake-up events. Third, some components are software-managed by construction, especially scratchpad-style SRAM and statically scheduled vector execution, which means hardware alone cannot always determine when power transitions are safe or profitable (Xue et al., 4 Aug 2025).

ReGate’s response is to decide the “best-fit” scheme per component: hardware-managed power gating where idle intervals are long and readily detectable, and software-managed power gating where workload dependence or allocation semantics make compiler knowledge decisive. In the paper’s formulation, SAs, ICI, and HBM are hardware-managed by default unless overridden, while VUs and SRAM rely on compiler-inserted control for finer-grained savings (Xue et al., 4 Aug 2025). This partitioning is a defining feature of the framework. It rejects the misconception that a single global policy should govern all NPU subunits; instead, management granularity and control locus are treated as first-class architectural choices.

3. Component-specific power-gating mechanisms

ReGate analyzes power-gating opportunities across all major NPU components and assigns a distinct mechanism to each one (Xue et al., 4 Aug 2025).

Component	ReGate scheme	Basis for the scheme
Systolic arrays	Cycle-level PE-granular power gating	Deterministic dataflow execution
ICI and HBM controllers	Hardware-managed idle detection	Long idle intervals
Vector units	Software-managed gating via ISA support	Workload-dependent idle periods
SRAM	Software-managed segment-granularity gating	Software-managed allocation and usage

For systolic arrays, the key observation is that ML workload dataflows are highly deterministic and propagate diagonally. ReGate therefore powers processing elements on and off in synchrony with the inherent SA dataflow. Power-on and power-off signals propagate with the data path, allowing each PE to be active only when data arrives. Unused rows and columns, determined from zero-weights and operand shapes, are completely powered off. Because wake-up is overlapped with the dataflow, the paper reports negligible wake-up penalty for this cycle-level implementation (Xue et al., 4 Aug 2025).

For ICI and HBM controllers, the dominant pattern is long temporal idleness rather than cycle-scale variation. ICI is primarily active during collectives such as AllReduce, while HBM activity is bursty around DMA and operator boundaries. ReGate therefore uses lightweight hardware state machines that detect sufficiently long idle intervals and transition these units into power-gated or refresh states only when the interval exceeds the break-even time (BET) (Xue et al., 4 Aug 2025). This makes these subsystems suitable for autonomous hardware control without compiler intervention.

For vector units, hardware prediction is weaker because utilization depends strongly on workload and schedule. ReGate addresses this with compiler-inserted setpm instructions that power VUs down or up at precise points identified from the instruction schedule. The paper also includes a hardware best-effort idle timer as a fallback when software is unavailable, but the main design point is software-managed gating (Xue et al., 4 Aug 2025). For SRAM, the same logic extends further: because SRAM is a scratchpad rather than a cache, hardware cannot infer which regions are safe to disable. ReGate therefore gates SRAM at 4KB segment granularity under compiler control, using [OFF](https://www.emergentmind.com/topics/omni-modal-future-forecasting-off) when data can be discarded and SLEEP when retention is required (Xue et al., 4 Aug 2025).

4. ISA extensions, compiler control, and pipeline integration

A central mechanism in ReGate is the ISA extension for power management. The framework introduces a setpm instruction that applies to VUs, SAs, ICIs, and SRAM, with support for ON, OFF, AUTO, and, for SRAM, SLEEP modes (Xue et al., 4 Aug 2025). The instruction accepts a bitmap for multi-unit control or an address range for SRAM, allowing software to override or delegate to hardware policies at function or unit granularity.

Compiler analysis is therefore not ancillary but structurally integrated into the framework. Modern NPU compilers analyze static dataflow and the instruction schedule to place setpm instructions where long idleness exceeds BET and wake-up latency can be hidden. This is particularly important for VUs and SRAM, where software has better visibility into future inactivity than local hardware monitors do (Xue et al., 4 Aug 2025). The paper’s example timeline for VU software gating illustrates how explicit compiler coordination avoids otherwise mandatory wake-up stalls.

The NPU pipeline is also extended for correctness and hazard tracking. If an instruction targets a powered-off component, the hardware triggers wake-up and, if necessary, stalls dispatch until the component becomes ready (Xue et al., 4 Aug 2025). This mechanism clarifies an important point: power gating is not purely an offline optimization, but part of the live execution semantics of the processor. ReGate’s contribution is thus full-stack, spanning ISA, compiler, microarchitecture, and per-component control logic.

5. Quantitative results and efficiency envelope

ReGate was implemented on a production-grade NPU simulator and synthesized on real hardware, including a synthesized 7nm implementation for hardware-overhead characterization (Xue et al., 4 Aug 2025). The reported headline result is up to 32.8% energy reduction and 15.5% average total-energy reduction, with average performance overhead below 0.5% for ML workloads and less than 3.3% total NPU die area overhead (Xue et al., 4 Aug 2025).

The detailed overhead and timing results make the design space more explicit. The paper reports power-gating control logic area below 3.3% overall; SA PE-level control uses less than 0.001% area for global logic and 6.36% per PE, but because SAs occupy only a small fraction of die area, the total SA contribution is below 1%. VU and SRAM gating logic each incur less than 0.25% area overhead (Xue et al., 4 Aug 2025). Wake-up delays and BET values are also reported: SA PE gating has 1-cycle on/off delay and BET of 47 cycles; full SA gating has 10-cycle wake-up and BET of 469 cycles; VU gating has 2-cycle wake-up and BET of 32 cycles; SRAM segments require 4 cycles in SLEEP and 10 cycles in OFF; HBM and ICI have 60-cycle wake-up and BET around 400 cycles (Xue et al., 4 Aug 2025).

The comparative result against hardware-only approaches is especially significant. Hardware-only schemes incur up to 4.6% slowdown because wake-up stalls cannot be systematically hidden, whereas the co-designed version hides nearly all wake-up latency (Xue et al., 4 Aug 2025). ReGate is also reported to achieve within 0.4% of the mathematically ideal savings, defined as perfectly power-gating all idle periods with zero transition cost (Xue et al., 4 Aug 2025). In addition to energy, peak and average chip power are reduced by 15%–33%, and the paper reports up to 62.9% reduction in operational CO$_2$ emissions at datacenter scale (Xue et al., 4 Aug 2025).

6. Interpretation, scope, and relation to similarly named work

ReGate’s immediate significance is that it repositions static power as a first-order optimization target for NPUs rather than as background leakage. The framework shows that leakage can dominate total energy in modern accelerators and that much of this cost is recoverable without materially changing the programmer model, software compatibility, or application-level performance (Xue et al., 4 Aug 2025). This suggests a broader shift in accelerator design: as AI operators become increasingly deterministic and compiler-scheduled, power management can be elevated from a generic microarchitectural heuristic to a workload-aware control plane.

At the same time, the framework should not be mischaracterized as a universal, purely hardware technique. Its strongest results rely on hardware/software co-design, ISA support, compiler placement of setpm, and component-specific policies. Nor is its primary effect to accelerate kernel execution; the reported benefit is energy reduction with negligible slowdown, not speedup. These distinctions matter for interpreting its deployment envelope.

The name should also be distinguished from unrelated methods with similar spelling. “ReGATE: Learning Faster and Better with Fewer Tokens in MLLMs” (Li et al., 29 Jul 2025) concerns reference-guided adaptive token elision for multimodal LLM training rather than NPU power management. In contrast, ReGate in (Xue et al., 4 Aug 2025) is a chip-architecture framework for fine-grained power-gating across SAs, VUs, SRAM, HBM controllers, and ICI. The similarity in naming can obscure the fact that the two works occupy entirely different layers of the AI systems stack.

7. Limitations and broader implications

The framework’s reported success depends on structural properties of current NPU design: deterministic SA execution, statically analyzable schedules, and software-managed memories. Where these properties weaken, the balance between hardware-managed and software-managed policies may also shift. The paper does not present ReGate as a general-purpose replacement for CPU or GPU power-management techniques; rather, it derives its strength from fitting the specific execution model of NPUs (Xue et al., 4 Aug 2025).

Its broader implication is architectural. ReGate demonstrates that NPU sustainability is not only a question of arithmetic efficiency or algorithmic sparsity, but also of suppressing idle leakage in specialized units whose utilization is inherently irregular. Because the framework is validated through silicon synthesis, cycle-accurate simulation, and real application workloads, it establishes power gating as a practical systems primitive for production NPU chips rather than a purely analytical possibility (Xue et al., 4 Aug 2025). A plausible implication is that future NPU ISAs and compiler toolchains may treat energy-state transitions as routine schedulable events, much as instruction issue and memory placement are treated today.