Approximation of target perplexity or KL loss in quantization

Develop quantization objectives or methods that directly approximate a target perplexity or KL loss rather than relying on the standard weighted mean-squared-error metric.

Background

The paper explains that weighted mean-squared error is widely used in model quantization because it is tractable, but it functions only as a proxy for output behavior. It cites prior work identifying the direct approximation of target perplexity or KL loss as an open research direction. This problem is relevant to softmax-layer quantization because the output distribution, rather than merely the intermediate linear-layer error, determines the model’s behavioral degradation.