Ultra-Low Precision Models

• Ultra-low precision models are neural networks that reduce parameter precision to 1-3 bits, enabling significant energy, memory, and speed improvements.
• They employ various quantization techniques including uniform, non-uniform, and reduced floating-point formats to maintain near full-precision accuracy.
• Advanced methods such as post-training and quantization-aware training, combined with hardware-software co-design, achieve competitive performance across vision, language, and edge applications.

Ultra-low precision models are neural networks whose weights, activations, and, in some cases, gradients are quantized or represented using extremely reduced numerical precision—typically fewer than 4 bits per value. This regime encompasses 1–3 bit quantization, tailored floating-point formats (e.g., FP8), discrete binary/ternary parameterizations, or unconventional representations such as posits. These techniques drastically reduce memory, energy, and computational costs, making them critical for on-device learning, edge inference, billion-parameter LLM deployment under tight budgets, and real-time closed-loop adaptation. This article develops the theory and practice of ultra-low precision models, elaborates methodologies spanning post-training and quantization-aware training, details empirical advances, and discusses their impact on software and hardware implementation.

1. Numerical Foundations and Quantization Strategies

Ultra-low precision models exploit the observation that modern DNNs are over-parameterized with significant representational redundancy. Quantization methods reduce 16/32-bit floating-point parameters to 2, 3, or even 1 bit by discretizing the range of possible values or restructuring parameter representations.

Quantization Operators. Uniform quantization to $b$ bits maps real values $w$ or $x$ into $2^b$ levels: $$Q(w) = \operatorname{clip}\left(\mathrm{round}\left( \frac{w}{S}\right) + z,\, 0,\, 2^b-1\right)$$ where $S$ is a scaling parameter and $z$ is the zero-point (Park et al., 2022). Dequantized values reconstruct as $\hat{w} = S\cdot(Q(w)-z)$. For lower bits, specialized schemes—sign quantization (binary), thresholding at multiple regime levels (ternary), or weighted non-uniform mappings—are essential to minimize information loss.

Reduced Floating-Point and Alternative Formats. Ultra-low-precision operation can be achieved not just by fixed-point quantization but by using reduced floating-point formats that balance dynamic range and precision (e.g., FP16, FP8, or posits) (Tagliavini et al., 2017, Langroudi et al., 2019). In hardware, mixed-precision units able to operate simultaneously on several subword lanes enable efficient bulk computation.

Advanced Quantization Schemes.

• Power-of-Two Integer Quantization: Maps each value to the nearest signed power-of-two, allowing all multiplications to be replaced by bit-shifts, integer additions, and XORs (Liu et al., 2023).
• Group/Channel-wise and Saliency-aware Assignment: Recent work partitions model parameters into groups or channels and assigns bits according to second-order loss sensitivity (e.g., Hessian) or saliency metrics (Bhatnagar et al., 28 Sep 2025, Shen et al., 2019, Wang et al., 23 Sep 2025).
• Tensor Decomposition: Factorizes weight matrices into low-rank tensor-train structures, dramatically reducing learning parameters and facilitating low-precision training (Zhang et al., 2021).

2. Methodologies: Post-Training and Quantization-Aware Training

There are two dominant classes of quantization methodology:

Post-Training Quantization (PTQ)

PTQ quantizes a pre-trained (and optionally fine-tuned) model without further training or with limited (blockwise, adapter-based, or partial) retraining on a small calibration set. Main PTQ techniques include:

• Uniform and Layerwise Quantization: Assigns all or selected layers a uniform bit-width; sometimes enhanced by entropy/saliency evaluation to apply ultra-low bits where tolerated (Bhatnagar et al., 28 Sep 2025).
• Mixed-Precision Allocation: Group- or channel-wise strategies optimize the assignment of precision, allocating more bits to sensitive groups/layers as measured by loss curvature (Hessian) or output entropy (Shen et al., 2019, Wang et al., 23 Sep 2025).
• Distributional Alignment Loss: PTQ often only matches the mean-square error between reference and quantized activations. Recent advances introduce a sliced-Wasserstein loss to enforce high-order (distributional) output matching, improving quantized model fidelity at 2–3 bits (Cao et al., 11 Jan 2026).
• Saliency-aware or Hybrid Quantization: Segments weights into outlier (more critical, higher bits) and inlier groups (less critical, 1–2 bits), optimizing quantization error trade-offs (Wang et al., 23 Sep 2025).
• Token Pruning in Multimodal and Vision-LLMs: Selective pruning of tokens post-quantization can remove up to 99% of visual tokens while maintaining accuracy (Wang et al., 23 Sep 2025).

Quantization-Aware Training (QAT)

QAT simulates quantization during training (or fine-tuning), allowing gradients to adapt the model's parameters for resilience against quantization-induced noise.

• Straight-Through Estimator (STE): Non-differentiable quantization is surrogated with an STE, which allows gradient flow through the quantizer (Zhong et al., 2022, Zhang et al., 2021).
• Teacher Intervention and Knowledge Distillation: Layerwise intervention plugs in full-precision teacher activations during ultra-low-precision QAT for transformers, mitigating error accumulation and facilitating convergence (Kim et al., 2023).
• Precision Highway: Selectively allows an end-to-end high-precision path (e.g., over skip-connections or recurrent states), drastically suppressing error accumulation without resorting to costly global high-precision (Park et al., 2018).
• Dynamic, Data-Driven Quantization Boundaries: Adaptive trainable bounds (dual, learnable clipping) and gating functions enable models to track sample-wise activation distribution asymmetry, essential for tasks like super-resolution (Zhong et al., 2022).

3. Empirical Results and State-of-the-Art Model Performance

Ultra-low-bit models have achieved performance surprisingly close to full-precision baselines over a wide array of vision, language, and multimodal tasks:

Model/Task	Bitwidth	Accuracy Degradation	Memory Reduction	Reference
ResNet-50/ImageNet QAT + precision highway	2b/2b	–2.45% Top-1	~8x	(Park et al., 2018)
BERT/SST-2, MNLI, CoNLL (mixed Hessian PTQ)	2–8b	<1% (except SQuAD: –1.9)	13x (weights only)	(Shen et al., 2019)
LLaVA-1.5, Qwen-2.5-VL (LUQ)	<4b	≤6% on VQA	31–40% over 4-bit	(Bhatnagar et al., 28 Sep 2025)
DS-CNN/Keyword Spotting on MCU	FP16	None	>2× speed; 0.81 MAC/clk	(Nadalini et al., 2023)
LoRA fine-tuning (LowRA)	1.15–2b	<0.2 BLEU/ROUGE loss	30–50% of 4-bit PTQ	(Zhou et al., 12 Feb 2025)
Vision-Language (Bi-VLM)	1–2b	+4–45% SOTA improvement	93% reduction	(Wang et al., 23 Sep 2025)

Notably, advanced techniques such as teacher intervention and layerwise quantization can maintain within 1–2% accuracy of full-precision models for most tasks, pushing the Pareto frontier of bits–accuracy trade-offs (Zhou et al., 12 Feb 2025, Bhatnagar et al., 28 Sep 2025, Park et al., 2018).

4. Hardware and Algorithm Co-Design

Ultra-low precision model research is tightly integrated with hardware-aware design. State-of-the-art results depend on alignment between model quantizer, memory layout, and instruction set.

• SIMD and LUT-based Kernels: DeepGEMM demonstrates that 2-bit x 2-bit convolution primitives using register-resident lookup tables can exceed the speed of even hand-tuned INT8 AVX2 kernels, achieving up to 1.74× speedup on x86 CPUs (Ganji et al., 2023).
• RISC-V Vectorized FP Units: On battery-powered MCUs, 16-bit SIMD floating-point matrix multiplications enable sub-20ms training steps for full backpropagation with corresponding >2× speed-ups relative to FP32 (Nadalini et al., 2023).
• Transprecision Units: Custom FPUs supporting 8/16/32-bit with lane scalability achieve 30% energy reduction and 12% lower runtime while ensuring any model variable is allocated the minimum safe bitwidth (Tagliavini et al., 2017).
• BinaryConnect and Multiplication-Free Training: By quantizing all MAC operands to signed powers-of-two, MF-MAC architectures eliminate all multi-bit multiplies in both forward and backward passes. On ResNet-50, INT4+XOR MF-MACs achieve 95.8% energy savings with <1% accuracy loss (Liu et al., 2023).
• FPGA Acceleration of Tensorized Nets: Fully on-FPGA, rank-adaptive tensorized models trained in 4-bit fixed-point consume just 1/292 the memory and 1/123 the energy of equivalent CPU implementations (Zhang et al., 2021).

5. Specializations: Multimodal, Vision, Language, and Edge-TinyML

The challenges and benefits of ultra-low precision manifest differently across application areas.

Multimodal LLMs and Vision-LLMs:

• MLLMs and VLMs exhibit higher-entropy activations for image tokens compared to text (Bhatnagar et al., 28 Sep 2025). LUQ and Bi-VLM apply layerwise entropy or magnitude-based saliency to assign ultra-low bits, with selective use of mixed calibration sets to mitigate the more variable multimodal distributions (Bhatnagar et al., 28 Sep 2025, Wang et al., 23 Sep 2025).

Edge and TinyML:

• On-device continual adaptation for tiny MCUs is feasible in real time with vectorized FP16 (Nadalini et al., 2023).
• Dynamic, dual-trainable bounded quantizers and lightweight gates enable 2–3 bit operation for super-resolution and other low-level tasks without catastrophic loss, outperforming static quantizer baselines by >1 dB PSNR (Zhong et al., 2022).

LLMs and Adapter Compression:

• Large transformer LMs (BERT, LLaMA-2) and LoRA adapters can be quantized to 2 bits per parameter with sophisticated mixed-precision assignment and block/group granularity (Zhou et al., 12 Feb 2025, Mirzaei et al., 30 Oct 2025, Shen et al., 2019).
• Distribution-aware regularization (sliced Wasserstein, saliency-based) and per-channel threshold learning further recover the loss gap introduced by crude low-bit quantization (Cao et al., 11 Jan 2026, Cao et al., 14 Apr 2025).

6. Open Problems and Future Directions

• Sub-2 bit quantization robustness: While 2–3 bits are now feasible, 1-bit universal quantization for both weights and activations remains brittle except in heavily structured or hybrid models.
• Dynamic and hybrid bit allocations: Automated selection of per-layer or per-block bitwidth, joint with hardware resource constraints, is an emerging area for mixed-precision optimizers (Park et al., 2022).
• Advanced outlier/exception handling: Hybrid storage formats selectively assign higher precision to critical weight subsets (e.g., saliency/entropy outliers) (Wang et al., 23 Sep 2025).
• Next-generation QAT strategies: Teacher-Intervention, cross-modal calibration, and explicit loss-surface flattening will increasingly be required for quantizing models at scale with minimal data and compute (Kim et al., 2023, Bhatnagar et al., 28 Sep 2025).
• Integration with new hardware: Hardware evolution toward wider on-die LUTs, low-bit MACs with fused adder/XOR/shift, and precision-scalable FPUs is expected to further lower the cost-performance curve of ultra-low-bit models (Tagliavini et al., 2017, Ganji et al., 2023).

Ultra-low precision models now regularly achieve accuracy within a few percent of floating-point baselines across diverse domains by combining discriminatory bit-allocation, adaptive quantizer design, and algorithm–hardware codesign. This paradigm is unlocking on-the-fly adaptation, real-time inference, and mass deployment of large neural architectures in ultra-constrained devices, while opening outstanding new frontiers in the fundamental understanding and application of discrete, low-precision representational mathematics at scale.