1.58-bit Quantization Techniques in Deep Learning

• 1.58-bit quantization is a technique that maps neural network weights to {-1, 0, +1} using scaling and rounding, achieving an effective bit-width of approximately 1.585 bits.
• It leverages quantization-aware training and post-training quantization with STE and optimizers like AdamW to maintain near full-precision accuracy across architectures.
• Practical implementations in transformers, CNNs, and TTS demonstrate significant storage reductions and computational efficiency, making it ideal for resource-limited applications.

A 1.58-bit quantization technique refers to weight quantization schemes in which each parameter is ternarized to one of three discrete levels—commonly $-1$, $0$, or $+1$—achieving an effective model bit-width of $\log_2 3 \approx 1.585$ bits per weight. Such ultra-low-bitwidth quantization dramatically reduces model size, memory bandwidth, and compute requirements, while preserving model accuracy near full-precision levels across a broad range of deep learning architectures, including transformers, CNNs, GNNs, and specialized models for text, vision, and speech domains.

1. Formal Definition and Quantization Functions

The canonical 1.58-bit quantization maps each floating-point weight $w$ to the set $\{-1,0,+1\}$ via a scale factor and a rounding/clipping operation. The most basic formulation is:

$$q(w) = \mathrm{clip}(\mathrm{round}(w / \gamma), -1, +1)$$

where the scaling factor $\gamma$ is typically the layerwise mean or median of the absolute weight values:

$\gamma = \frac{1}{N} \sum_{i=1}^N |w_i|$

with $N$ the number of weights in the layer. Extensions may use robustified statistics (e.g., channelwise or blockwise means/medians) or learned, tensor-specific scales. In practice, weights are stored as small signed integers and a single floating-point scale factor per layer or block.

This ternary quantizer yields three possible values per weight, whose empirical Shannon entropy under optimized training is approximately $0$0 bits per weight (Ma et al., 2024, Nielsen et al., 2024, Nielsen et al., 2024).

Post-training quantization and quantization-aware training (QAT) both employ this mapping, but QAT additionally uses STE-based gradient flows to update the underlying shadow weights.

2. Training Schemes and Optimization Algorithms

Quantization-aware training: Most 1.58-bit methods maintain full-precision "shadow" weights for optimization. The forward pass uses quantized weights, and the backward pass applies the straight-through estimator (STE): $0$1 if $0$2 in the quantization range, and zero otherwise (Ma et al., 2024, Nielsen et al., 2024, Nielsen et al., 2024). Optimizers are typically AdamW with careful tuning of learning rates ($0$3 to $0$4 for small models, higher for LLMs) and $0$5 or mild $0$6 regularization.

Post-training quantization (PTQ): PTQ methods such as AdaRound, BRECQ, and OBC adapt scale and zero-point per layer/channel to minimize reconstruction loss between quantized and original layer outputs. These can be adapted directly to a ternary scheme by using $0$7 quantization levels, per-channel scaling, and by potentially learning rounding parameters for each weight (Zhang et al., 17 Dec 2025).

Advanced schemes: ParetoQ introduces a Stretched Elastic Quantizer (SEQ) with a learned scale, supporting unified experimentation from binary to 4-bit settings and analytical scaling-law studies (Liu et al., 4 Feb 2025). HESTIA adopts a differentiable softmax-based quantizer, annealed according to a Hessian-guided schedule, to preserve smooth gradients early in quantized training and harden assignments later (Wang et al., 28 Jan 2026).

3. Practical Implementations and Model Architectures

The 1.58-bit paradigm is realized across a wide spectrum of architectures:

• Transformer-based LLMs: BitNet b1.58 and its derivatives apply ternary quantization to all major weight matrices (attention QKV, MLP, output heads) with RMSNorm and STE, reaching or exceeding the accuracy of 16-bit LLaMA and Mistral models at equivalent scale (Ma et al., 2024, Nielsen et al., 2024).
• CNNs & MLPs: Encoder-only, encoder-decoder, and MLP-based models for classification match or outperform 16/32-bit counterparts when scaling width for expressiveness (Nielsen et al., 2024).
• Text-to-Speech (TTS) and Vision Transformers: BitTTS applies 1.58-bit QAT combined with a weight-indexing scheme that packs five ternary weights into a single byte, yielding $0$8 MB models with minimal MOS loss (RTF and synthesis quality close to full precision), while FLUX achieves 7.7$0$9 model storage reductions on T2I pipelines via post-training ternarization and kernel fusion (Kawamura et al., 4 Jun 2025, Yang et al., 2024).
• KV-cache and VideoLLMs: 1.58-bit quantization of KV caches (value) combined with per-channel assignments and semantic token protection enables up to $+1$0 compression of inference memory with negligible performance drop (Tao et al., 20 Mar 2025).

Table: Core Quantization Functions

Method	Quantization Function	Scale Type
BitNet b1.58	$+1$1	$+1$2 = layerwise mean($+1$3)
SDQ-LLM	$+1$4–$+1$5 ternary $+1$6 at OSR$+1$7d per column	None (pre-processing: Hadamard)
ParetoQ	SEQ: $+1$8 with learned $+1$9, see main text	Learnable per-tensor
PTQ (AdaRound)	$\log_2 3 \approx 1.585$0	Per-channel min/max
HESTIA	$\log_2 3 \approx 1.585$1	Hessian-guided, per-tensor

4. Empirical Results and Comparative Performance

Extensive evaluations demonstrate that 1.58-bit quantization (QAT-trained from scratch or via fine-tuning) maintains near-parity with full-precision baselines across multiple tasks and scales:

• LLM Perplexity and Accuracy: For LLaMA and OLMo architectures with up to $\log_2 3 \approx 1.585$2B parameters, BitNet b1.58 achieves validation perplexity within $\log_2 3 \approx 1.585$3 of FP16, and on some tasks, even surpasses FP16 accuracy (Ma et al., 2024, Nielsen et al., 2024, Liu et al., 4 Feb 2025).
• Text & Vision Tasks: On CIFAR-10/100 and standard NLP benchmarks, ternary (b1.58) models achieve $\log_2 3 \approx 1.585$4 of full-precision accuracy. For text-to-image (FLUX), a $\log_2 3 \approx 1.585$5 inference memory reduction is realized with only a $\log_2 3 \approx 1.585$6–$\log_2 3 \approx 1.585$7 point metric drop (Yang et al., 2024).
• Ablation and Scaling Studies: Doubling hidden size in small LMs or vision models compensates for ternary capacity loss at minimal overhead (Nielsen et al., 2024). For encoder–decoder models, b1.58 sometimes outperforms full-precision with no capacity increase (Nielsen et al., 2024).

Best practices further include merging knowledge distillation or layer-wise normalization (extra RMSNorm), and gradual quantization schedules ($\log_2 3 \approx 1.585$8-schedules) to stabilize convergence (Steinmetz et al., 12 May 2025).

5. Hardware Realizations and Computational Advantages

1.58-bit quantization enables highly efficient hardware implementations:

• Representation: $\log_2 3 \approx 1.585$9 states per weight allow $w$0 bits of entropy per parameter. Weight packing (e.g., grouping $w$1 ternaries into a byte) and entropy coding can reach this theoretical minimum (Kawamura et al., 4 Jun 2025).
• Accelerators: The BitROM CiROM architecture stores two ternary weights per transistor, achieves $w$2 TOPS/W (65 nm), and $w$3 kB/mm$w$4 density. The computation pipeline eliminates multiply units in favor of conditional add/sub and zero-skipping accumulators (Zhang et al., 10 Sep 2025, Ma et al., 2024).
• Kernels: Custom GPU/ASIC kernels realize $w$5 storage and $w$6 RAM reductions (FLUX), while dedicated ternary matmul logic delivers $w$7–$w$8 theoretical speedups over GEMMs (Yang et al., 2024, Nielsen et al., 2024).
• Inference Efficiency: Multiplication becomes sign-tested addition/subtraction (for $w$9), supporting aggressive pipelining and bit-packed storage.

6. Extensions, Limitations, and Applications

• Mixed-Precision and Hybrid Schemes: 1.58-bit backbones can be coupled to low-rank FP16 correction (Hybrid Gated Flow) for recovery of up to $\{-1,0,+1\}$0 of the quality gap at minimal overhead (total $\{-1,0,+1\}$1 bits/weight) (Pizzo, 5 Feb 2026).
• Scaling Laws: Dedicated analysis in ParetoQ and BitNet b1.58 confirm a new scaling law in the ternary regime; information capacity $\{-1,0,+1\}$2 yields an accuracy–size Pareto frontier often superior to 2- and 4-bit baselines (Ma et al., 2024, Liu et al., 4 Feb 2025).
• QAT Transitions: Continual pre-training with early 16-to-1.58 bit transitions outperforms training from scratch at 1.58 bits on LLM benchmarks (Nielsen et al., 17 Feb 2025).
• Limitations: In small LMs, hidden sizes must be inflated ($\{-1,0,+1\}$3) to achieve comparable PPL; for vision, fine-grained textures can degrade in ultra-low bits. Some architectural choices (RMSNorm, bias-free linears) enhance stability (Nielsen et al., 2024, Steinmetz et al., 12 May 2025, Yang et al., 2024).
• Applications: Edge LLMs, on-device TTS, real-time text-to-image, privacy-preserving DNNs, and resource-limited inference are among the principal domains benefiting from 1.58-bit quantization (Kawamura et al., 4 Jun 2025, Yang et al., 2024, Zhang et al., 17 Dec 2025).

7. Privacy and Security Implications

Aggressive 1.58-bit PTQ reduces vulnerability to membership inference attacks by up to an order of magnitude relative to FP16, indicating possible benefits for privacy-by-design (Zhang et al., 17 Dec 2025). Adjusting the final or input layer to higher bit-widths restores accuracy with partial retention of privacy gains, allowing fine-grained control along the privacy–utility spectrum.

---

For a comprehensive set of empirical benchmarks, detailed algorithms, and scaling law investigations, see (Ma et al., 2024, Nielsen et al., 2024, Steinmetz et al., 12 May 2025, Kawamura et al., 4 Jun 2025, Zhang et al., 10 Sep 2025, Tao et al., 20 Mar 2025, Yang et al., 2024, Xia et al., 27 Sep 2025, Pizzo, 5 Feb 2026, Wang et al., 28 Jan 2026, Nielsen et al., 2024, Zhang et al., 17 Dec 2025, Liu et al., 4 Feb 2025), and (Nielsen et al., 17 Feb 2025).