CLEAR: Component-Level Energy Profiling

• The paper introduces CLEAR, a method that decomposes transformer inferences into reusable components for precise energy measurement.
• It employs repeated sampling to overcome the temporal mismatch between microsecond-scale execution and slower GPU energy sensors, achieving variance below 9.5% for key components.
• CLEAR reveals that attention blocks consume significantly more energy per FLOP than MLPs and LM heads, challenging traditional FLOP-based energy attribution.

Component-Level Energy Assessment via Repeated sampling (CLEAR) is a methodology for fine-grained empirical analysis of transformer inference energy that addresses the temporal mismatch between microsecond-scale component execution and millisecond-scale GPU energy sensors. Introduced in "Dissecting Transformers: A CLEAR Perspective towards Green AI" (Jain et al., 3 Oct 2025), CLEAR decomposes a transformer forward pass into reusable components, caches their input activations, and repeatedly re-executes each component until the runtime is long enough for robust NVML-based measurement. Applied to 15 models spanning four distinct architecture types, it reports component-wise energy variance below 9.5% while capturing more than 90% of total model energy as individual components, and it shows that Attention blocks consume significantly more energy per floating-point operation than MLPs and LM heads, so FLOPs alone fail to capture the true energy cost at a component level (Jain et al., 3 Oct 2025).

1. Problem formulation and motivation

CLEAR is motivated by the claim that transformer-based LLMs now spend most of their environmental budget on inference rather than training. Training is episodic, whereas inference runs continuously at planetary scale, so reducing per-query energy is critical for Green AI. Existing sustainability studies largely report model-level energy over entire runs or system-level aggregates over GPU, CPU, and memory, often via tools such as CodeCarbon, Carbontracker, or simulation. Such measurements support coarse comparisons, but they do not identify which internal components dominate energy, how energy splits within a transformer layer, or how architectural and precision choices alter the distribution of energy across components (Jain et al., 3 Oct 2025).

A central premise of CLEAR is that FLOPs are an inadequate proxy for component-level energy. The empirical model reported for total energy as a function of input length is

$E(L) \approx E_0 + k \cdot \mathrm{FLOPs}(L),$

where $E_0$ is a fixed, non-scaling overhead and $k$ is component-dependent. Under this decomposition, a naive redistribution of model-level energy in proportion to FLOP counts is systematically wrong at the component level, because it ignores both fixed overhead and variation in marginal energy per FLOP across components. The paper highlights Attention as the clearest counterexample: it has a substantially higher energy-per-FLOP ratio than MLPs and LM heads (Jain et al., 3 Oct 2025).

The methodological obstacle is temporal mismatch. Core transformer components such as attention, MLP sublayers, and normalization often execute in 10–100 $\mu$s, whereas NVIDIA NVML power sensors update at about 20–50 Hz, corresponding to one reading every 20–50 ms and often averaged over roughly 1 s. Two failure modes follow. In a complete miss, a component can execute entirely between sensor updates, yielding

$E_2 - E_1 = 0,$

despite true component energy $E_c > 0$. In a noisy partial overlap, a single-sensor-window estimate becomes

$E_2 - E_1 = E_c \pm \varepsilon,$

where $\varepsilon$ includes idle or background GPU power, sensor quantization, and unrelated activity. For microsecond-scale components, $E_c$ is small relative to both the window and the noise term, so the signal-to-noise ratio is poor. CLEAR is designed explicitly to overcome this constraint (Jain et al., 3 Oct 2025).

2. Methodological structure

CLEAR has two defining operations: decomposition of a forward pass into reusable components with cached inputs, and amplification of each component’s workload by repeated execution until the runtime reaches hundreds of milliseconds. The decomposition covers attention blocks, feed-forward or MLP blocks, normalization blocks such as LayerNorm and RMSNorm, the embedding layer, and the LM head; in some analyses it also uses grouped norms, denoted "Norm (All)," and residual-block energy, denoted "Block" (Jain et al., 3 Oct 2025).

To enable isolated replay, CLEAR inserts forward hooks at key points in the computation graph and records the corresponding input activations. If

$$\mathcal{A} = \{ \texttt{attn\_in}, \texttt{mlp\_in}, \texttt{lm\_head\_in}, \texttt{layer\_norm\_in}, \ldots \},$$

then a normal forward pass on input $E_0$0 populates an Activation Store

$E_0$1

This cache contains exactly what each component saw during the original inference. The model parameters can then be frozen, and any component $E_0$2 can be re-run multiple times on cached $E_0$3 without repeating the entire forward pass. The repeated executions therefore correspond to realistic workloads rather than synthetic microbenchmarks (Jain et al., 3 Oct 2025).

The amplification step executes a component $E_0$4 times back-to-back on its cached activation and measures cumulative energy across the resulting interval:

$E_0$5

With cumulative energy readings $E_0$6 and $E_0$7 from NVML, the measurement is modeled as

$E_0$8

The per-execution estimate is

$E_0$9

By choosing $k$0 large enough that the total runtime falls in the hundreds of milliseconds, CLEAR ensures that NVML observes multiple updates during the run, while the effective noise shrinks as $k$1. Multiple trials $k$2 then yield a trial-averaged estimate,

$k$3

The procedure includes short pauses between trials so that the sensor returns to a stable baseline (Jain et al., 3 Oct 2025).

The experimental implementation uses NVIDIA Ada-Lovelace workstation GPUs. The RTX 5000 Ada is described as having 12,800 CUDA cores, 400 Tensor Cores, 32 GB GDDR6, about 576 GB/s bandwidth, and about 250 W board power; the RTX 6000 Ada has 18,176 CUDA cores, 568 Tensor Cores, 48 GB GDDR6, about 960 GB/s bandwidth, and about 300 W board power. Power is read through NVML, and FLOP counts per component are obtained via PyTorch Profiler (Jain et al., 3 Oct 2025).

3. Precision, coverage, and validation

CLEAR has no external ground truth for component-level transformer energy, so validation is defined along two axes: consistency across trials and completeness of the component decomposition. Formally, the desiderata are low dispersion in $k$4 and approximate recovery of total measured model energy through summation of component estimates (Jain et al., 3 Oct 2025).

The reported precision is energy-dependent. For components consuming more than 5 mJ per execution, the average relative standard deviation is below 9.5% of the mean. For large components, such as block-level energy around 1 J, the relative standard deviation can be as low as 0.1%. At very low energies, near 1 mJ, the relative standard deviation can exceed 20%, reflecting proximity to sensor precision limits. On that basis, the analysis focuses on components above 5 mJ, where the measurements are described as statistically well behaved (Jain et al., 3 Oct 2025).

Completeness is assessed through capture metrics. At block level, "Captured (Block)" is the sum of attention, MLP, and normalization energies inside a transformer block, and "Block" is the energy measured by running the whole block with activation-store amplification. The corresponding ratio is

$k$5

At model level, "Captured (Model)" sums all measured components across blocks, embedding, LM head, and final norms, and "Model" is the amplified full forward-pass energy:

$k$6

For GPT-OSS-20B at 128 tokens in FP16, the reported per-block energy is 1157.197 mJ, while Captured(Block), defined as Attention + MLP + Norm, is 1161.541 mJ, giving 100.375% capture. At model level for the same setting, Model energy is 28,801.98 mJ and Captured(Model), defined as blocks + embedding + LM head + final norm, is 28,264.24 mJ, giving 98.133% capture. Across token settings in that table, %Capture(Model) ranges roughly 96.6–98.3%, and %Capture(Block) is typically about $k$7 (Jain et al., 3 Oct 2025).

Across all architectures and precisions, Captured(Model) is at least 90% of Model and is often 95–99%. The paper notes that ALBERT variants sometimes show lower block-level capture, plausibly because factorized embeddings and more pronounced idle energy are harder to attribute. Residual energy, on the order of about 2–10%, is attributed to tiny operations not explicitly hooked, sensor noise, and idle or background power (Jain et al., 3 Oct 2025).

4. Architectural scope and empirical energy breakdowns

CLEAR is applied to 15 models across four architecture types: encoder-only, decoder-only, encoder–decoder, and Mixture-of-Experts. The encoder-only set includes BERT-base, BERT-large, RoBERTa-base, RoBERTa-large, DistilBERT, DistilRoBERTa, ALBERT-base, and ALBERT-large. The decoder-only set includes Qwen-2.5-3B-Instruct, Llama-3.2-3B-Instruct, Gemma-3-4B-IT, Phi-4-Mini (4B), GPT-OSS-20B, and GPT-OSS-120B. Encoder–decoder models include BART and Flan-T5-style models from cited work. The MoE category includes OLMoE open Mixture-of-Experts LLMs (Jain et al., 3 Oct 2025).

Architecture type	Models	Component decomposition
Encoder-only	BERT, RoBERTa, DistilBERT, ALBERT variants	attention block, FFN/MLP, norms, CLS+LM head, embeddings
Decoder-only	Qwen, Llama, Gemma, Phi-4-Mini, GPT-OSS	self-attention, MLP, input norm, attention norm, final layer norm, embedding, LM head
Encoder–decoder	BART, Flan-T5-style models	encoder self-attention and FFN; decoder masked self-attention, cross-attention, FFN; norms; embeddings; LM head
MoE	OLMoE	router and experts within MoE FFN layers

In encoder-only models, attention blocks consume a large fraction of block energy, often more than 40%, while MLP or FFN energy is comparable or slightly larger depending on hidden size. Norms and embeddings are consistently below 5% each of total model energy. In large variants such as BERT-large, RoBERTa-large, and ALBERT-large, larger FFN dimensions increase the FFN energy share while reducing attention’s relative share, even if attention energy also rises in absolute terms. The paper cites BERT-large FP16 block averages of about 34–42 mJ per block, with attention around 16–21 mJ and FFN parts around 15–20+ mJ (Jain et al., 3 Oct 2025).

For decoder-only models, the dominant pattern in absolute Joules is different. In single-token generation without KV caching, which the paper adopts to keep FLOP accounting straightforward, MLPs dominate per-block energy. For Qwen-2.5-3B FP32, one block is reported as 113.71 mJ for MLP, 27.64 mJ for attention, and about 5.4 mJ combined for norms. For Llama-3.2-3B FP32, the corresponding per-block values are 127.24 mJ for MLP and 58.31 mJ for attention. For Gemma-3-4B FP32, they are 129.47 mJ for MLP and 42.82 mJ for attention. The LM head can be exceptionally expensive when vocabulary is large: Gemma-3-4B FP32 reports 1040.63 mJ for the LM head versus about 186.54 mJ per block, and Qwen-2.5-3B FP32 reports 459.66 mJ for the LM head. At the model level, the paper therefore states that MLPs and LM heads often dominate energy in decoder-only LLMs, while attention remains substantial but is not necessarily the largest consumer in absolute Joules (Jain et al., 3 Oct 2025).

A notable precision-related result concerns normalization. The paper reports that LayerNorm and RMSNorm consume more energy in FP16 than in FP32, because many frameworks cast to FP32 internally for numerical stability and then cast back, incurring extra data movement and type conversion. Thus overall energy may still be lower in FP16, while norms occupy a larger relative share of the energy budget (Jain et al., 3 Oct 2025).

5. Energy per FLOP and the status of attention

CLEAR defines two related ratios for a component $k$8 as a function of input length $k$9: the absolute energy per FLOP,

$\mu$0

and the marginal energy per additional FLOP across lengths,

$\mu$1

The reported plots for Qwen-2.5-3B and Gemma-3-4B show that $\mu$2 decreases with sequence length for all components, while $\mu$3 is nearly constant. This behavior is consistent with the decomposition $\mu$4: fixed overhead $\mu$5 is amortized as sequence length grows, while the approximately constant marginal ratio reveals a component-specific coefficient $\mu$6 (Jain et al., 3 Oct 2025).

Attention is the paper’s most important counterexample to FLOP-based attribution. Across all studied decoder-only models and input lengths, attention has the highest energy-per-FLOP ratio. MLP and LM head have noticeably lower J/FLOP, meaning that they realize more FLOPs per Joule. The paper therefore argues that FLOPs are not fungible: an attention FLOP costs more energy than an MLP FLOP, and a single global conversion factor from FLOPs to Joules systematically underestimates attention’s energy share while overestimating the share of MLPs and LM heads (Jain et al., 3 Oct 2025).

The qualitative explanation given is architectural and microarchitectural. MLP and LM head are dominated by dense matrix multiplications with regular dataflow, high arithmetic intensity, contiguous memory access, and efficient Tensor Core mapping. Attention, by contrast, includes query–key dot products, scaling and softmax, and more complex synchronization and data-dependent memory patterns; for autoregressive models the paper also points to key–value cache updates and accesses. The consequence is higher memory traffic, more cache misses, lower effective Tensor Core utilization, and larger bandwidth and synchronization overheads per unit compute. This suggests that the coefficient $\mu$7 is not merely an empirical fit parameter but a summary of hardware-utilization differences across computational primitives (Jain et al., 3 Oct 2025).

The broader implication is that component-level optimization targets cannot be derived reliably from FLOP counts alone. CLEAR’s empirical results support a two-term energy model with fixed overhead and component-specific marginal cost, rather than a one-factor FLOP accounting scheme (Jain et al., 3 Oct 2025).

6. Implications, limitations, and relation to adjacent sampling methodologies

CLEAR provides component-level energy baselines intended to support energy-efficient transformer design. The paper identifies attention-specific optimization as a primary target because attention has the highest energy-per-FLOP ratio. It lists sparse attention, low-rank or kernelized approximations, hybrid global–local structures, and GQA or MQA as architectural directions, and it further points to tailored GPU kernels and efficient KV-cache layouts as algorithm–hardware co-design opportunities. Even though MLPs are more energy-efficient per FLOP, they dominate absolute energy in many large decoders and encoders because FFN dimensions scale roughly four times hidden size and recur in every layer. The same logic applies to the LM head, whose cost is tightly coupled to vocabulary size; the paper explicitly notes that Gemma-3-4B spends about 1 J per LM head call, suggesting that vocabulary compression or factorized output layers could reduce energy substantially (Jain et al., 3 Oct 2025).

The reported $\mu$8 versus sequence-length behavior also informs deployment. Because fixed overheads are amortized at longer sequence lengths, the paper states that longer sequences yield lower J/FLOP for all components, and it suggests that larger batch sizes and longer contexts can be more energy-efficient per token when utilization is high and latency constraints permit. CLEAR-style profiling is proposed as a way to identify the operating point that balances latency and energy on a given hardware platform (Jain et al., 3 Oct 2025).

The limitations are explicit. All measurements depend on NVIDIA GPUs and NVML, so the sampling characteristics and component profiles may shift on other vendors or microarchitectures. NVML and PyTorch Profiler FLOP counts are not perfect ground truth, sensor noise and quantization remain inherent, and very small component energies approach the sensor’s precision limits. The present study addresses inference only; for decoder-only models it mainly considers single-token generation without KV caching, and extending the method to multi-token generation with full cache behavior is described as non-trivial. Even with high capture ratios, 2–10% of model energy remains unattributed. The current implementation also requires manual hook placement per model variant, so automated instrumentation is identified as an important future direction (Jain et al., 3 Oct 2025).

A plausible comparison is that CLEAR occupies, for GPU transformer inference, a methodological niche analogous to earlier repeated-sampling systems in other domains. ALEA addresses the same general problem—hardware power sensors are too slow to directly resolve short-lived software entities—by using statistical sampling to estimate execution time, mean power, and energy for basic blocks, reporting mean error rates between 1.4% and 3.5% with overhead capped at approximately 1% (Mukhanov et al., 2015). METRION, although it does not mention CLEAR by name, introduces a thread-level energy attribution model for CPU and DRAM that explicitly accounts for Simultaneous Multithreading, frequency scaling, multi-socket architectures, and Non-Uniform Memory Access, and it reports a Mean Absolute Percentage Error of 4.2% for CPU energy and 16.1% error for DRAM energy in the corresponding target workloads (Weigell et al., 7 Dec 2025). This suggests that CLEAR belongs to a broader class of empirical energy-attribution methods that compensate for sensor granularity through repeated sampling, but it is specialized to transformer components and to the internal structure of modern inference workloads.