Microscaling quantization for convolutional layers

Establish an effective method for combining microscaling quantization with convolutional-layer operations while avoiding the memory and recomputation limitations associated with quantizing im2col-expanded activation tensors.

Background

Microscaling quantization is efficient when quantization blocks align with the direction of dot-product computation, but convolutional layers use im2col transformations and involve tensors in multiple matrix multiplications with different orientations. Existing approaches either quantize full-precision tensors separately during forward and backward propagation or store quantized im2col-expanded activations for reuse, increasing memory movement and storage. The paper identifies the general integration of microscaling quantization with convolutional operations as unresolved; it subsequently proposes MicroQonv as a solution to this problem.