Leech Lattice Quantization

• Leech lattice-based quantization is a technique that exploits the 24-dimensional lattice's unique geometric structure and symmetry to enable non-parametric, lookup-free quantization.
• It delivers state-of-the-art rate-distortion performance in visual representation tasks by using a fixed spherical codebook, eliminating the need for auxiliary entropy losses.
• The approach also offers insights into quantum geometric models of space-time, unifying concepts in information theory, machine learning, and mathematical physics.

Leech lattice-based quantization exploits the unique structural, packing, and symmetry properties of the 24-dimensional Leech lattice to define non-parametric, lookup-free quantization schemes. This approach underpins both state-of-the-art discrete representation learning for visual data and certain quantum geometric models of space-time, establishing Λ₂₄ as a unifying object across information theory, machine learning, and mathematical physics.

1. Lattice Coding and Non-Parametric Quantization

A real, full-rank $d$-dimensional lattice $\Lambda\subset\mathbb{R}^d$ is defined by $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$ where $G\in\mathbb{R}^{d\times d}$ is the generator matrix. The associated vector quantizer maps $x\in\mathbb{R}^d$ to its nearest lattice point via

$$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$

Existing non-parametric, lookup-free quantizers are special cases of lattice quantization:

• Lookup-Free Quantization (LFQ): $\Lambda = \{z\in\{\pm 1\}^d\}$ (i.e., $G=I$), producing codebooks of all sign vectors.
• Binary Spherical Quantization (BSQ): LFQ vectors are projected onto the sphere: $$\Lambda_{\mathrm{BSQ}} = \{z/\sqrt{d} : z\in\{\pm1\}^d\}$$.
• Finite Scalar Quantization (FSQ): Uses scalar rounding within bounded intervals, i.e., $\Lambda = (\frac{1}{2}L\tanh(x))$ rounded to $\Lambda\subset\mathbb{R}^d$0 with $\Lambda\subset\mathbb{R}^d$1.

BSQ and LFQ suffer from non-uniform Voronoi cell volumes under non-ideal input distributions, prompting the introduction of entropy regularization terms to manage both codebook balance and quantization robustness:

$\Lambda\subset\mathbb{R}^d$2

where $\Lambda\subset\mathbb{R}^d$3 denotes the quantization probability vector.

2. Structure and Properties of the Leech Lattice $\Lambda\subset\mathbb{R}^d$4

The Leech lattice $\Lambda\subset\mathbb{R}^d$5 is the unique 24-dimensional even unimodular lattice without roots, constructed as:

$\Lambda\subset\mathbb{R}^d$6

A generator matrix is specified in reference (Zhao et al., 16 Dec 2025), Appendix A.

Key quantitative invariants:

• Shortest nonzero vectors: $\Lambda\subset\mathbb{R}^d$7 ($\Lambda\subset\mathbb{R}^d$8 of length $\Lambda\subset\mathbb{R}^d$9)
• Kissing number: $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$0 (number of shortest vectors)
• Optimal packing density: $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$1
• Minimal normalized second moment: $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$2

The distribution of shortest vectors enables the definition of a highly uniform 196,560-point spherical code. Normalizing these to unit $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$3 norm gives

$\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$4

3. Spherical Leech Quantization (Λ₂₄-SQ) Methodology

Spherical Leech Quantization (Λ₂₄-SQ) leverages the above spherical code for quantizing $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$5 dimensional features in a non-parametric fashion:

• Given $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$6, normalize: $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$7
• Quantize: $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$8
• Output quantized unit-norm code: $\Lambda = \{ \lambda = Gz \mid z\in\mathbb{Z}^d \}$9

Λ₂₄-SQ quantization dispenses with all auxiliary losses (entropy, commitment) required in BSQ/LFQ, relying solely on the fixed spherical Leech codebook. Training proceeds with:

$G\in\mathbb{R}^{d\times d}$0

optionally augmented by perceptual (LPIPS) or adversarial (GAN) losses. The codebook $G\in\mathbb{R}^{d\times d}$1 is fixed and excluded from gradient updates, resulting in zero parameter overhead.

In implementation, brute-force search over $G\in\mathbb{R}^{d\times d}$2 can be accelerated via approximate nearest neighbor (ANN) algorithms (e.g., FAISS), and symmetries of $G\in\mathbb{R}^{d\times d}$3 can be exploited to reduce computational cost (Zhao et al., 16 Dec 2025).

4. Rate-Distortion Trade-Off and Empirical Efficiency

For Λ₂₄-SQ, the coding rate per vector is $G\in\mathbb{R}^{d\times d}$4 bits. The mean squared quantization distortion, defined as $G\in\mathbb{R}^{d\times d}$5, benefits from the Leech lattice's minimal normalized second moment.

Empirical results using ViT-AE backbones on ImageNet indicate:

• At $G\in\mathbb{R}^{d\times d}$6 bits, BSQ achieves rFID = 1.14; Λ₂₄-SQ improves by $G\in\mathbb{R}^{d\times d}$710–20% with rFID = 0.83, PSNR increase of $G\in\mathbb{R}^{d\times d}$8 dB, SSIM $G\in\mathbb{R}^{d\times d}$9, and LPIPS decrease of $x\in\mathbb{R}^d$0.
• Among all tested spherical lattices (e.g., $x\in\mathbb{R}^d$1), the Λ₂₄-SQ curve consistently lies on the rate-distortion lower bound envelope, confirming optimal Voronoi dispersiveness and packing efficiency (Zhao et al., 16 Dec 2025).

5. Applications in Deep Visual Tokenization and Beyond

5.1 Compression Autoencoders

Λ₂₄-SQ is deployed as the quantization bottleneck for ViT-based autoencoders:

• Encoder: ViT-B/16 to extract 24-dim patch features
• Bottleneck: Λ₂₄-SQ quantization per feature
• Decoder: transformer or conv-upsampling to reconstruct pixels

On ImageNet/COCO, for a fixed codebook capacity:

• BSQ ($x\in\mathbb{R}^d$2 bits): PSNR = 25.08, SSIM = 0.7662, LPIPS = 0.0744, rFID = 1.14
• Λ₂₄-SQ: PSNR = 26.00, SSIM = 0.8008, LPIPS = 0.0632, rFID = 0.83

For Kodak tiles at $x\in\mathbb{R}^d$3, MS-SSIM increases from $x\in\mathbb{R}^d$4 (BSQ) to $x\in\mathbb{R}^d$5 (Λ₂₄-SQ).

5.2 Autoregressive Image Generation

Λ₂₄-SQ is integrated into large-scale bitwise AR models such as Infinity-CC, achieving significant gains:

• 1B parameter model: BSQ+∞CC yields gFID = 2.18; Λ₂₄-SQ+∞CC achieves gFID = 1.82
• Scaling to 2.8B parameters: Λ₂₄-SQ reaches validation gFID = 1.82, approaching the “oracle” value of 1.78.

High codebook size ($x\in\mathbb{R}^d$6) allows for either monolithic ($x\in\mathbb{R}^d$7-way) or factorized ($x\in\mathbb{R}^d$8-way) softmax heads, with architectural and optimization tricks (cut-cross-entropy, Z-loss, distributed orthonormalization) further boosting generation quality.

Systematic ablations confirm that lattices with higher minimal distance ($x\in\mathbb{R}^d$9) yield lower distortion for fixed codebook size. Vector field (VF) alignment and larger AR models further extend performance (Zhao et al., 16 Dec 2025).

6. Broader Implications: Geometric and Physical Interpretation

While Λ₂₄-SQ is primarily an engineering tool in visual representation, the Leech lattice's mathematical universality also enables its appearance in models of quantized geometry:

• In certain quantum generalizations of Einstein spaces, Λ₂₄ (complexified and embedded in Lorentzian signature) supplies the “code” space for matrix nonlinear Schrödinger equations (Chapline, 2015). The associated automorphism groups (e.g., Conway’s group, Suzuki sporadic group, Mathieu group $$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$0) enable a lifting from discrete lattice symmetries to continuous gauge symmetry algebras ($$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$1).
• The distribution of minimal vectors suggests deep analogies between error correction, information quantization, and the emergence of gauge structure in physical models.

A plausible implication is that the Leech lattice provides an optimal “platform” not only for rate-distortion trade-offs in machine learning, but also for the encoding of fundamental physical degrees of freedom in certain geometric quantization frameworks (Chapline, 2015).

7. Summary Table: Λ₂₄-SQ Versus Prior Quantizers

Method	Codebook Size	Spherical?	Entropy Loss?	PSNR (ImageNet)	rFID	Bitrate (approx.)
BSQ	$$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$2	Yes	Yes	25.08	1.14	$$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$3 bits
Λ₂₄-SQ	$$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$4	Yes	No	26.00	0.83	$$Q_\Lambda(x) = \operatorname{argmin}_{\lambda\in\Lambda} \|x - \lambda\|_2.$$5 bits

Λ₂₄-SQ achieves superior reconstruction metrics and tighter rate-distortion envelopes while eliminating the need for auxiliary loss heuristics. The fixed, symmetric codebook scales efficiently to compression and tokenization tasks requiring exceptionally large discrete vocabularies (Zhao et al., 16 Dec 2025).