Quantify the Sources of Energy Savings in Quantized Inference

Determine the relative contributions of reduced memory-access cost and lower-precision computation to the overall energy reductions achieved by quantized large language model inference.

Background

The thesis reports substantial energy savings from weight-only quantization but does not isolate the mechanisms responsible for those savings. Quantization simultaneously reduces the number of bytes transferred through the memory hierarchy and changes the arithmetic performed on GPU tensor cores.

Separating memory traffic, compute activity, and power consumption would clarify which components dominate the energy budget. This knowledge could guide the design of quantization methods and hardware architectures that target the most consequential sources of efficiency.