Gaussian Quantization Representation Learning

• Gaussian Quantization Representation Learning is a suite of techniques that impose or construct Gaussian or Gaussian-mixture structured representations to facilitate precise quantization aligned with analytical or empirical rate–distortion laws.
• These methods employ analytic transforms, mixture models, and adaptive quantizers to enable efficient compression in applications ranging from LLM weight compression to real-time image and 3D scene coding.
• The literature highlights a tension between matching Gaussian assumptions and reshaping distributions for hardware-friendly uniform quantization, influencing design trade-offs across diverse domains.

Taken together, the literature suggests that Gaussian quantization representation learning is a family of methods that design, learn, or impose representations whose coordinates, latent variables, or primitive attributes follow Gaussian or Gaussian-mixture structure before discretization, compression, or downstream inference. Across LLM weight compression, transform coding for Gaussian-mixture sources, vector-quantized generative models, spatial probability prediction, infrared data augmentation, and Gaussian splatting codecs, the recurring objective is to align the representation with a quantizer, codebook, or rate–distortion law that is analytically or empirically well matched to that structure (Vicentino, 30 Mar 2026, Park et al., 15 May 2026, Yan et al., 2024).

1. Scope and recurrent design patterns

The topic spans several distinct but related constructions. In one line of work, raw weights or features are Gaussianized by deterministic transforms and then quantized with Gaussian-optimal scalar codebooks. In another, the source itself is modeled as a Gaussian mixture, so representation learning reduces to identifying a discrete component label and a conditionally Gaussian residual. A third line embeds quantization into probabilistic latent-variable models, where codewords act as Gaussian means and discrete posteriors are regularized at the aggregate level. A fourth line uses explicit Gaussian primitives—in 2D or 3D splatting—as the representation, then quantizes their attributes for compression. A fifth, more contrarian line begins from the observation that neural weights are Gaussian-like but argues that uniform-like distributions are more favorable for hardware-friendly uniform quantizers, so the representation should be reshaped away from Gaussianity rather than matched to it (Yu et al., 2019).

Paradigm	Representative papers	Core mechanism
Gaussianized coordinates	PolarQuant	Normalize, rotate, Gaussian-match the quantizer
Gaussian-mixture transform coding	PrismQuant	Label + component-wise KLT + ECSQ
Probabilistic VQ with Gaussian means	GM-VQ	Codebook means + adaptive variances + ALBO
Distribution reshaping for low-bit CNNs	GDRQ, SYMOG	Reshape weight distributions toward quantization-friendly modes
Quantized Gaussian primitives	CompGS, RDO-Gaussian, GaussianImage++, GSVC	Quantize positions, covariances, colors, or SH attributes

This diversity matters because “Gaussian” does not denote a single commitment. In some papers it describes the source model itself; in others it describes the latent representation after a transform; elsewhere it refers to mixture components, quantization hyperparameters, or the primitive family used to represent a signal. A common misconception is that Gaussian quantization always assumes raw data are Gaussian. Several papers instead argue that Gaussian structure is often something to be constructed, approximated, or even avoided depending on the deployment regime (Vicentino, 30 Mar 2026, Li et al., 24 Jul 2025, Yu et al., 2019).

2. Source-aligned transforms and Gaussianized coordinates

A canonical example is PolarQuant, a post-training weight quantization method for LLMs. It operates in three stages: block-wise normalization to the unit hypersphere, Walsh–Hadamard rotation, and quantization with centroids matched to the Gaussian distribution. For a block $b_i \in \mathbb{R}^d$, the pipeline is

$$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$

with the modeling step $z_{i,j} \sim \mathcal{N}(0,1)$ used to justify a Lloyd–Max scalar quantizer for a standard normal source. The paper’s Proposition 1 states that for $\hat{b}$ uniform on the unit sphere, each coordinate of $\sqrt{d} H_d \hat{b}$ converges in distribution to $\mathcal{N}(0,1)$ as $d \to \infty$. Empirically, it reports that Hadamard rotation alone accounts for 98% of the quality improvement; on Qwen3.5-9B, absmax Q5 gives perplexity 6.90, while “+ Hadamard rotation only” gives 6.4010, and full PolarQuant Q5 gives 6.3909, versus 6.37 for FP16. As an INT4 preprocessing stage, PolarQuant Q5 dequantized and re-quantized by torchao INT4 yields perplexity 6.56 versus 6.68 for direct absmax INT4, at 43.1 tok/s and 6.5 GB VRAM (Vicentino, 30 Mar 2026).

The more general transform-coding version appears in PrismQuant, which considers an i.i.d. vector source drawn from a finite Gaussian mixture

$$p_X(x)=\sum_{c=1}^{K}\pi_c\,\mathcal{N}(x;\mu_c,R_c).$$

Conditioned on the component label $C=c$, each branch is Gaussian, so the representation becomes a discrete label plus a component-specific KLT coefficient vector. The central theoretical result is that the genie-aided conditional rate–distortion function is governed by a single global reverse-waterfilling level $\mu$ shared across all components and eigenmodes: $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$0 The paper proves the sandwich

$$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$1

so the mixture structure costs essentially only the amortized component-label entropy. The practical codec learns a GMM by EM, transmits the component label losslessly, applies the component-matched KLT, and uses entropy-constrained scalar quantization on active modes. This shifts representation learning from a single global covariance to a mixture of local Gaussian coordinate systems (Park et al., 15 May 2026).

A third theoretical route is lattice representation learning, which makes the Gaussian connection via dithered lattice quantization and its relationship to Gaussian VAEs. With a lattice $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$2, the quantizer is

$$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$3

The encoder computes $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$4, where $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$5 is uniform over the Voronoi cell. The paper’s key equivalence shows that a training-time continuous loss in terms of $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$6 is exactly tied to the inference-time discrete representation cost. It also shows that for good high-dimensional lattices the dither distribution approaches Gaussian, making lattice quantizers “good Gaussian quantizers.” This suggests that some Gaussian quantization schemes can be interpreted as continuous Gaussian proxy training followed by exact discrete lattice deployment (Lastras, 2020).

3. Quantization-aware latent and weight distribution learning

In neural parameter space, one influential construction is SYMOG, which treats low-bit quantization as the learning of a symmetric mixture of Gaussian modes aligned with fixed-point levels. For a layer $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$7, the regularizer is

$$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$8

so each weight is pulled toward its nearest quantization level. During training, the weight distribution evolves from an approximately unimodal Gaussian around zero to a symmetric mixture whose means coincide with fixed-point modes such as $$b_i \mapsto \hat{b}_i = \frac{b_i}{\|b_i\|_2} \mapsto \tilde{b}_i = H_d \hat{b}_i \mapsto z_i = \sqrt{d}\,\tilde{b}_i,$$9. The paper reports 2-bit results including 5.71% error on CIFAR-10 and 27.65% on CIFAR-100, and interprets the method as simultaneous learning of the task and the quantized representation rather than post hoc discretization (Enderich et al., 2020).

A probabilistically richer latent formulation is “Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior”, which defines

$z_{i,j} \sim \mathcal{N}(0,1)$0

with codebook vectors serving as Gaussian mixture means. The variational posterior factorizes as $z_{i,j} \sim \mathcal{N}(0,1)$1, where the categorical posterior is a softmax over Mahalanobis-like distances to codebook means, and the continuous posterior has adaptive variance tied to encoder–codebook distance. The paper introduces the Aggregated Categorical Posterior Evidence Lower Bound (ALBO), replacing the per-sample categorical entropy term with a KL on the aggregated posterior $z_{i,j} \sim \mathcal{N}(0,1)$2. This is intended to preserve sharp per-sample assignments for Gumbel–Softmax while still encouraging global codebook utilization. On CIFAR-10, GM-VQ + Entropy reports MSE $z_{i,j} \sim \mathcal{N}(0,1)$3 and perplexity 878.7; on CelebA it reports MSE $z_{i,j} \sim \mathcal{N}(0,1)$4 and perplexity 831.0 (Yan et al., 2024).

An important counterpoint is GDRQ, which starts from the observation that weights and activations are Gaussian-like but argues that low-bit uniform quantizers work better when the underlying distribution is made uniform-like. Its Scale-Clip rule sets the clipping threshold as

$z_{i,j} \sim \mathcal{N}(0,1)$5

and shows empirically that when $z_{i,j} \sim \mathcal{N}(0,1)$6, the weight histogram becomes nearly uniform. Group-based quantization then assigns different groups of filters different quantization parameters, with those scales later merged into BatchNorm so that there is no extra inference cost. This line of work directly contests any blanket claim that Gaussianity should always be preserved. Instead, it suggests that the right target distribution depends on the quantizer family and hardware substrate (Yu et al., 2019).

A related architectural view appears in “Finding Non-Uniform Quantization Schemes using Multi-Task Gaussian Processes,” which models layerwise bit allocation as Bayesian hyperparameter search under a Multi-Task Gaussian Processes prior, with tasks corresponding to different training budgets. Here Gaussian processes do not model source statistics directly; rather, they guide search over heterogeneous precision assignments. This suggests a broader interpretation of Gaussian quantization representation learning in which Gaussian priors are used not only for source coding but also for resource allocation over representational bottlenecks (Nascimento et al., 2020).

4. Gaussian mixtures as learned output spaces and distortion models

Some work uses Gaussian structure not to compress internal parameters but to replace dense outputs with compact probabilistic representations. In “Learning Gaussian Representation for Eye Fixation Prediction,” the output fixation map is modeled as a Gaussian mixture

$z_{i,j} \sim \mathcal{N}(0,1)$7

with one Gaussian per coarse spatial cell. The network predicts $z_{i,j} \sim \mathcal{N}(0,1)$8, transforms means relative to anchor centers, reconstructs a dense saliency map from the GMM, and trains with a correlation-coefficient loss. In a direct representation-capacity analysis, fitting a GMM with $z_{i,j} \sim \mathcal{N}(0,1)$9 to fixation points yields MSE $\hat{b}$0, KL $\hat{b}$1, CC $\hat{b}$2, SIM $\hat{b}$3, and NSS $\hat{b}$4, indicating that a small mixture can approximate the dense map closely (Song et al., 2024).

A very different use of Gaussianity appears in “Exploiting Gaussian Agnostic Representation Learning with Diffusion Priors for Enhanced Infrared Small Target Detection.” Here the Gaussian Group Squeezer is a non-uniform scalar quantizer applied only to background pixels, while target pixels are left intact. The number of intervals $\hat{b}$5 is sampled from a Gaussian distribution with mean 17 and variance 4, creating a family of quantized views of the same scene. A coarse reconstruction module and a latent diffusion model then map these quantized inputs back toward the real infrared distribution. On NUDT-SIRST, the paper reports that $\hat{b}$6 gives IoU $\hat{b}$7, $\hat{b}$8, and $\hat{b}$9; it also reports that Gaussian sampling of quantization parameters outperforms simple random sampling in both IoU and false alarms (Li et al., 24 Jul 2025).

In supervised decision problems, Regularized Classification-Aware Quantization studies Gaussian and Gaussian-mixture data under a fixed downstream linear classifier. The empirical objective is

$\sqrt{d} H_d \hat{b}$0

so $\sqrt{d} H_d \hat{b}$1-$\sqrt{d} H_d \hat{b}$2 classification loss is regularized by reconstruction error. The paper evaluates this on bivariate Gaussian data with covariance

$\sqrt{d} H_d \hat{b}$3

and on Gaussian mixtures with covariance $\sqrt{d} H_d \hat{b}$4. This setting makes explicit that Gaussian quantization representation learning need not minimize MSE alone; the quantized representation can instead be optimized for task loss regularized by geometry-preserving distortion (Severo et al., 2021).

5. Quantized Gaussian primitives for images, video, and 3D scenes

A major branch of the literature uses Gaussian primitives themselves as the representation and then compresses those primitives. In 3D radiance fields, CompGS quantizes selected 3D Gaussian Splatting attributes by K-means-based vector quantization while optimizing the scene, stores codebooks plus per-Gaussian indices, and further compresses indices via a run-length-like scheme. It reports 40 to 50x storage reduction and 2 to 3x rendering-time improvement with only a small image-quality drop (Navaneet et al., 2023). A related method, “Compact 3D Gaussian Representation for Radiance Field,” combines a learnable mask that removes Gaussians, a shared grid-based neural field for color, residual vector quantization for geometry, and entropy coding, reporting over $\sqrt{d} H_d \hat{b}$5 reduced storage while maintaining scene quality (Lee et al., 2023).

The rate–distortion formulation is made explicit in RDO-Gaussian, which treats compact 3D Gaussian learning as end-to-end RDO with dynamic pruning and entropy-constrained vector quantization. Its total loss combines Gaussian pruning, SH pruning, rate, vector-quantization distortion, and rendering terms, and the paper reports over 40x reduction in 3D Gaussian size together with superior rate–distortion performance over existing methods (Wang et al., 2024). GETA-3DGS pushes the same direction further with a 3DGS-aware quantization-aware dependency graph, render-aware saliency, and heterogeneous per-attribute mixed precision. It reports ~5x storage reduction over vanilla 3DGS, and its ablation states that a uniform 6-bit cap can cost up to $\sqrt{d} H_d \hat{b}$6 dB on view-dependent scenes relative to heterogeneous allocation, consistent with the paper’s reverse-water-filling analysis (Zhang et al., 3 May 2026).

In 2D image representation, GaussianImage models an image as a sum of 2D Gaussian splats,

$\sqrt{d} H_d \hat{b}$7

and couples this with vector quantization to form an image codec. It reports rendering speeds of 1500–2000 FPS and decoding around 2000 FPS, with rate–distortion performance comparable to COIN and COIN++ (Zhang et al., 2024). GaussianImage++ adds distortion-driven densification, content-aware Gaussian filters, and attribute-separated LSQ+ quantizers, using 12 bits for positions, 10 bits for covariances, and 6 bits for colors. It reports that GaussianImage++ outperforms GaussianImage and COIN in both representation and compression performance while maintaining real-time decoding (Li et al., 22 Dec 2025).

For video, GSVC represents frames with 2D Gaussian splats, uses temporal prediction from previous frames, Gaussian Splat Pruning, Gaussian Splat Augmentation, and dynamic key-frame selection, then quantizes and differentially codes Gaussian parameters. The paper reports rate–distortion trade-offs comparable to AV1 and VVC, with rendering around 1500 fps for 1920×1080 video (Wang et al., 21 Jan 2025). This extends Gaussian quantization representation learning from static images and scenes to temporally predicted, quantized Gaussian trajectories.

6. Conceptual tensions, limitations, and open directions

Several tensions organize the field. The first is Gaussian matching versus distribution reshaping. PolarQuant, PrismQuant, GM-VQ, and lattice methods attempt to build or exploit Gaussian or Gaussian-mixture coordinates because those admit strong coding arguments or stable probabilistic training objectives (Vicentino, 30 Mar 2026, Park et al., 15 May 2026, Yan et al., 2024, Lastras, 2020). GDRQ argues almost the opposite for low-bit uniform hardware: if the quantizer is fixed and uniform, then a uniform-like representation may be better than a Gaussian-like one (Yu et al., 2019). This suggests that the relevant notion of “optimality” is conditional on the deployed quantizer family, not on Gaussianity alone.

The second tension is analytic design versus learned adaptation. PolarQuant uses a fixed Walsh–Hadamard rotation and a Gaussian Lloyd–Max codebook learned once offline (Vicentino, 30 Mar 2026). PrismQuant learns a GMM by EM but keeps its transform coding analytic (Park et al., 15 May 2026). GM-VQ and GaussianImage++ instead learn quantizers or codebooks jointly with the representation itself (Yan et al., 2024, Li et al., 22 Dec 2025). In Gaussian splatting, CompGS and Compact 3DGS use relatively simple codebooks and masking, while RDO-Gaussian and GETA-3DGS introduce explicit rate–distortion optimization, entropy terms, or mixed-precision policies (Navaneet et al., 2023, Lee et al., 2023, Wang et al., 2024, Zhang et al., 3 May 2026).

A third issue is where the discrete bottleneck resides. In SYMOG it is the weight distribution; in GM-VQ it is the latent code index; in SalGMM it is the compact GMM parameterization of an output density; in GaussianImage++ or GSVC it is the primitive attribute stream (Enderich et al., 2020, Yan et al., 2024, Song et al., 2024, Li et al., 22 Dec 2025, Wang et al., 21 Jan 2025). This suggests that Gaussian quantization representation learning is less a single algorithmic template than a design principle: move the bottleneck to a space where Gaussian assumptions or Gaussian primitives are structurally advantageous.

The literature also identifies unresolved limitations. PolarQuant notes that the Gaussian assumption may not hold equally well for all layers or very small block sizes, and that it does not exploit inter-block correlations (Vicentino, 30 Mar 2026). PrismQuant’s asymptotic rate–distortion theory still depends on learning accurate mixture structure and on transmitting component labels (Park et al., 15 May 2026). Compact 3DGS and CompGS leave substantial memory in non-quantized attributes such as positions and opacities, and note that codebooks are largely scene-specific (Lee et al., 2023, Navaneet et al., 2023). RDO-Gaussian observes that separate models are still trained for different rates (Wang et al., 2024). GaussianImage++ keeps the decoder simple but does not introduce a learned entropy model, so its bitstream remains less sophisticated than high-end neural codecs (Li et al., 22 Dec 2025). A plausible implication is that future work will combine explicit Gaussian structure, attribute- or component-aware rate control, and stronger entropy models rather than choosing only one of these axes.

Finally, the field increasingly treats Gaussian structure as a representation prior rather than merely a statistical description. Gaussianization by rotation, mixture-aware local transforms, adaptive Gaussian variances in VQ, Gaussian-parameterized augmentation, and quantized Gaussian primitives all instantiate the same broader strategy: select a representation in which quantization is easy to analyze, easy to optimize, or easy to deploy. That shared strategy is what makes the topic coherent across otherwise distant areas such as LLM compression, probabilistic tokenization, saliency prediction, infrared augmentation, and real-time splatting codecs (Vicentino, 30 Mar 2026, Yan et al., 2024, Song et al., 2024, Li et al., 24 Jul 2025, Wang et al., 21 Jan 2025).