Formal rate-distortion analysis of NBE-based grid design

Derive a formal rate-distortion analysis establishing the relationship between Normalized Bin Entropy (NBE) and mean-squared quantization error for log-concave symmetric weight distributions, and determine when NBE-based grid selection is near-optimal.

Background

The paper introduces Normalized Bin Entropy as a grid-utilization metric and empirically observes that, for the Gaussian proxy at a 4-bit budget, the base maximizing NBE lies near the base minimizing mean-squared error. The authors provide an intuitive decomposition of the competing tail-clipping and in-range-rounding effects, but they do not prove the claimed relationship for the broader class of log-concave symmetric distributions. A formal rate-distortion treatment would establish the theoretical basis for using NBE to select quantization grids.

References

A formal rate-distortion analysis is left to future work.

— Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference  (2610.21257 - Wee et al., 15 Oct 2026) in Appendix, Section “Grid-Matching Theory,” subsection “Normalized Bin Entropy: Properties and Proofs,” Property 5