TernaryLM: Efficient Ternary Language Model
- Ternary Language Models are transformer architectures that constrain dominant weight matrices to the ternary set {-1, 0, +1} with floating-point scaling to enhance deployment efficiency.
- Native quantization-aware training and advanced post-training methods enable TernaryLMs to closely match full-precision performance while significantly reducing memory and computation requirements.
- Optimized hardware-software co-design, including specialized packing and kernel operations, translates the 1.58-bit structure into real-world speed and energy efficiency gains.
A ternary LLM is a LLM whose dominant weight matrices are constrained, during training, inference, or both, to the ternary alphabet , usually together with floating-point scaling terms. In current literature, the term covers both a general model class—often called TriLM or ternary LLM—and a specific 132M-parameter decoder-only system named TernaryLM (Nargund et al., 7 Feb 2026). The central motivation is deployment efficiency: ternary weights reduce storage and bandwidth pressure relative to FP16 or FP32, and they can replace general multiply-accumulate patterns with sign-controlled add, subtract, and zero-skip operations. At the same time, the literature shows that “ternary” rarely means fully ternary end to end: embeddings, output projections, activations, optimizer states, and modality-specific interfaces are often kept at higher precision (Nargund et al., 7 Feb 2026, Kaushal et al., 2024, Chen et al., 2024).
1. Historical emergence and scope
Ternary language modeling emerged from the broader low-bit quantization program in neural networks, but its modern form is tied to transformer LLMs and the inference-time memory wall. Early work on extreme quantization for generative transformers showed that encoder–decoder text generation could survive ternary and even binary compression under carefully designed weight and activation quantizers. In particular, a ternary BART-base reached an R1 score of $41$ on CNN/DailyMail, “merely 3.9 points behind the full model while being 16x more efficient,” and obtained BLEU $21.7$ on WMT16 En–Ro against a full-precision mBART score of $26.8$ (Liu et al., 2023).
The term later broadened from task-specific quantized generators to pretrained low-bit language-model families. Spectra introduced an open suite spanning FloatLMs, QuantLMs, and TriLMs from $99$M to $3.9$B parameters trained on $300$B tokens, and reported that above roughly the billion-parameter regime, TriLMs outperform QuantLMs and FloatLMs for a given bit budget; its $3.9$B TriLM was reported to match the FloatLM $3.9$B across benchmark families while using fewer bits than the FloatLM $830$M (Kaushal et al., 2024). The 2026 system explicitly named TernaryLM then shifted the discussion toward native training, presenting a $41$0M-parameter decoder-only transformer trained from scratch with ternary weights in $41$1 and adaptive layer-wise scaling (Nargund et al., 7 Feb 2026).
The literature also distinguishes core ternary LMs from adjacent uses of ternary structure. LoTA-QAF uses ternary low-rank adapters for quantized LLM fine-tuning rather than a ternary base model (Chen et al., 24 May 2025). TODO uses ternary preference labels—win, lose, tie—for alignment, not ternary weights or activations (Guo et al., 2024). This distinction is important because “ternary” in current LLM research may refer to parameterization, adaptation, deployment encoding, or preference modeling rather than a single architectural category.
2. Parameterization and architectural patterns
Across papers, the irreducible ternary object is the codebook $41$2, often described as “1.58-bit” because $41$3. In practice, however, ternary LMs are usually implemented as scaled ternary projections rather than raw integer-only networks. A common formulation, used in Spectra-style TriLMs, computes a matrix scale from the absolute mean of the latent full-precision weights,
$41$4
then forms a ternary proxy by clipping and rounding $41$5 to $41$6, and finally reconstructs the forward weight as $41$7 (Kaushal et al., 2024). Spectra 1.1 keeps the same basic structure and describes TriLMs as decoder-only transformers with ternary-weighted linear layers, shared floating-point scales, RMSNorm, RoPE, and SwiGLU (Vaidhya et al., 28 Jun 2025).
The specific 2026 TernaryLM model uses a different native-training parameterization: $41$8 with
$41$9
where $21.7$0 is a learnable per-layer positive scaling factor (Nargund et al., 7 Feb 2026). This places the ternary decision boundary under layer-adaptive statistics rather than a fixed threshold.
Post-training ternary conversion methods often modify the reconstruction rather than the codebook. TernaryLLM introduces Dual Learnable Ternarization, keeping $21.7$1 but reconstructing each weight as
$21.7$2
so that nonzero means and asymmetric outliers in LLM weight groups can be absorbed by a learnable shift $21.7$3 and scale $21.7$4 (Chen et al., 2024). PTQTP goes further and approximates each pretrained weight matrix by a sum of two ternary trit-planes,
$21.7$5
so its deployed representation is ternary at the plane level rather than as a single ternary code per parameter (Xiao et al., 21 Sep 2025).
A persistent architectural regularity is partial rather than universal ternarization. The 132M TernaryLM quantizes “all linear projection matrices” but keeps the embedding layer and output projection in FP32 because quantized embeddings caused severe degradation (Nargund et al., 7 Feb 2026). The modular-addition interpretability study likewise could not optimize binarized or ternarized embed/unembed layers and therefore left them full precision (Li, 2024). Spectra keeps embeddings and LM head in half precision (Kaushal et al., 2024). In multimodal systems, the boundary is even sharper: LLaVaOLMoBitNet1B uses a ternary text backbone but retains a full-precision CLIP ViT-L/14 encoder and full-precision projector (Sundaram et al., 2024). A common misconception is therefore that a ternary LM is usually an all-ternary network; current systems generally ternarize the dominant transformer linear projections, not every parameter path.
3. Training paradigms
Three training regimes dominate the literature: native quantization-aware pretraining, post-training ternarization of pretrained models, and direct low-precision optimization without persistent master weights.
Native quantization-aware pretraining is represented by Spectra and TernaryLM. Spectra trains FloatLMs and TriLMs on the same $21.7$6B-token SlimPajama subset and reports that TriLMs require a distinctive schedule: a peak learning-rate drop at the halfway point and removal of weight decay at two-thirds of training, which produces a sharp loss drop and then accelerated convergence (Kaushal et al., 2024). The 2026 TernaryLM trains a $21.7$7M decoder-only transformer on TinyStories with AdamW, cosine scheduling, gradient clipping, label smoothing, and straight-through estimators, reporting validation perplexity $21.7$8, MRPC $21.7$9 F1, and stable training across TinyStories, WritingPrompts, and Shakespeare (Nargund et al., 7 Feb 2026).
Spectra 1.1 extends native ternary pretraining into the scaling-law regime and reports that TriLM loss is more sensitive to data than to parameter count. It fits a Chinchilla-style law with exponents $26.8$0 for model size and $26.8$1 for dataset size, in contrast to a FloatLM fit with exponents $26.8$2 and $26.8$3, respectively (Vaidhya et al., 28 Jun 2025). The paper’s stated implication is that ternary LMs benefit more from extra tokens than from parameter scaling, which motivates training Spectra-1.1 on up to $26.8$4 trillion tokens.
Post-training ternarization methods begin from a pretrained floating-point model and optimize the ternary approximation itself. TernaryLLM operates in the W1.58A16 regime and combines Dual Learnable Ternarization with Outlier-Friendly Feature Knowledge Distillation, using cosine-similarity feature matching to preserve semantic geometry under extreme weight compression (Chen et al., 2024). PTQTP uses an alternating procedure over two ternary trit-planes and row/group scaling coefficients, with monotonic non-increase of reconstruction error as its central algorithmic property (Xiao et al., 21 Sep 2025). TWLA pushes PTQ further toward end-to-end low-bit inference by targeting W1.58A4 through asymmetric ternary quantization, orthogonal weight shaping, and inter-layer activation bit allocation (Zhao et al., 11 Jun 2026).
A third line asks whether ternary models can be trained without keeping latent full-precision master weights. Direct Quantized Training with Stochastic Rounding proposes exactly that: weights are kept quantized throughout training and updated directly by stochastic rounding instead of STE over persistent shadow weights (Zhao et al., 2024). It reports that “training with only low-precision weights is feasible even when they are constrained to ternary values,” but its reproduced $26.8$5-bit DQT model remains much weaker than BitNet-style QAT, with WikiText-2 loss $26.8$6 and perplexity $26.8$7 against reproduced BitNet b1.58 loss $26.8$8 and perplexity $26.8$9 (Zhao et al., 2024). The result establishes feasibility rather than parity.
4. Empirical performance and scaling behavior
The empirical literature supports three broad conclusions. First, ternary language modeling is viable on real language tasks rather than only on toy problems. Second, native ternary pretraining generally outperforms post-training quantization below 4 bits when the comparison is made at fixed bit budget. Third, ternary weight models often preserve accuracy far better than binary ones, especially for generation and reasoning.
Representative reported results span early encoder–decoder models, decoder-only native-training systems, and PTQ/QAT conversions:
| System | Setting | Reported result |
|---|---|---|
| Ternary BART (Liu et al., 2023) | CNN/DailyMail summarization | R1 $99$0, $99$1 more efficient |
| Spectra TriLM (Kaushal et al., 2024) | $99$2B ternary pretrained LM | Matches FloatLM $99$3B across benchmark families; $99$4B bits |
| TernaryLM (Nargund et al., 7 Feb 2026) | $99$5M native decoder-only model | TinyStories val PPL $99$6; MRPC $99$7 F1 |
| TernaryLLM (Chen et al., 2024) | LLaMA-3-8B, W1.58A16 | C4 PPL $99$8; zero-shot average $99$9 |
| PTQTP (Xiao et al., 21 Sep 2025) | Qwen3-14B ternary PTQ | Math-500 $3.9$0; GSM8K $3.9$1 |
Spectra’s bit-budget results are among the strongest arguments for ternary pretraining. It reports that above roughly the billion-parameter scale, TriLMs consistently outperform FloatLMs and post-training QuantLMs for a given bit size, and that a $3.9$2B TriLM can be bit-wise smaller than a FloatLM $3.9$3M while remaining close to the FloatLM $3.9$4B on commonsense and knowledge tasks (Kaushal et al., 2024). Its later Spectra-1.1 suite, trained on $3.9$5T tokens, reports MMLU averages $3.9$6, $3.9$7, and $3.9$8 for its 1B, 2B, and 3B models, with the largest model approaching LLaMA-1 7B’s MMLU average $3.9$9 while remaining ternary-weighted (Vaidhya et al., 28 Jun 2025).
QAT and PTQ conversion papers show that pretrained mainstream LLMs can also be pushed into the ternary regime, though with a clearer quality-efficiency tradeoff. On LLaMA-3-8B, TernaryLLM reports WikiText2 $300$0, C4 $300$1, PTB $300$2, and zero-shot average $300$3, outperforming the DB-LLM W2A16 baseline by $300$4 perplexity points on C4 and $300$5 points in average zero-shot accuracy (Chen et al., 2024). PTQTP is particularly notable because it shows that post-training ternary decomposition can preserve reasoning far better than binary PTQ: on Qwen3-14B it reports Math-500 $300$6 and GSM8K $300$7, while several ultra-low-bit binary PTQ baselines collapse to $300$8 on Math-500 in the paper’s table (Xiao et al., 21 Sep 2025).
The collective record therefore supports a narrow but important claim: ternary LMs are not generally lossless replacements for full-precision models, but they are no longer confined to catastrophic degradation or toy benchmarks. The strongest results appear either when the model is pretrained natively under ternary constraints or when PTQ uses a richer ternary reconstruction than a single symmetric threshold.
5. Inference systems and hardware co-design
The deployment literature treats ternary modeling as a systems problem as much as a numerical one. Weight compression alone does not guarantee speed; practical gains require packing schemes, mixed-precision kernels, and hardware paths that exploit the small ternary alphabet.
On CPUs, Bitnet.cpp builds a ternary mixed-precision GEMM stack for BitNet-style inference and reports up to a $300$9 speed increase over full-precision baselines and up to $3.9$0 over low-bit baselines (Wang et al., 17 Feb 2025). Its key technical point is that ternary inference should be treated as an element-wise $3.9$1 problem rather than a generic bit-plane problem, enabling TL and I2_S kernels with lossless or near-lossless execution relative to the reference quantized model. Spectra 1.1 extends this systems direction with 2-bit and 1.6-bit packing schemes for ternary weights and the GPU kernel TriRun, which accelerates end-to-end inference by up to $3.9$2 versus floating-point baselines (Vaidhya et al., 28 Jun 2025).
On FPGA and edge hardware, specialized accelerator papers make the hardware consequences explicit. TeLLMe is presented as the first ternary LLM accelerator for low-power FPGAs supporting both prefill and decoding with 1.58-bit weights and 8-bit activations, and reports peak decode throughput $3.9$3 tokens/s under roughly $3.9$4W, time-to-first-token $3.9$5s for 64-token prompts, and $3.9$6s for 128-token prompts (Qiao et al., 22 Apr 2025). TerEffic pushes farther toward high-throughput deployment with a 1.6-bit packing scheme and FPGA-native ternary matmul, reporting $3.9$7 tokens/s and $3.9$8 tokens/s/W for a $3.9$9M-parameter fully on-chip configuration, and $3.9$0 tokens/s at $3.9$1W for a larger $3.9$2B HBM-assisted configuration (Yin et al., 23 Feb 2025).
A separate hardware concern is reliability. ReTern studies BitNet b1.58 $3.9$3M and $3.9$4B on ternary compute-in-memory accelerators subject to stuck-at faults, and reports about $3.9$5 perplexity reduction on Wikitext under faults together with hardware overheads below $3.9$6 energy, below $3.9$7 latency, and below $3.9$8 area (Malhotra et al., 1 Jun 2025). The paper’s significance is that it shows ternary deployment is not only about compression and arithmetic; mapping, fault tolerance, and memory-array behavior can materially affect language quality.
A recurring theme across these systems papers is that ternary weights have the greatest practical value when they are co-designed with storage layout. Packing five ternary values into one byte, grouping ternary weights into lookup-table indices, or converting ternary matrix multiplication into mux/select and add/subtract operations are all attempts to convert information-theoretic 1.58-bit structure into real bandwidth and energy gains. Without such co-design, ternary models can remain memory-efficient but latency-neutral.
6. Interpretability, boundaries, and adjacent extensions
One frequently stated hope is that ternary weights might make transformers intrinsically easier to reverse engineer because the parameter space is discrete. The mechanistic evidence so far does not support that conclusion. A dedicated interpretability study on binary and ternary one-layer transformers trained on modular addition found that low-bit models learned Fourier-organized, “clock”-like circuits similar to full-precision models, rather than a qualitatively simpler symbolic algorithm; the authors explicitly concluded that their experiments provided no evidence that ternary transformers are inherently more interpretable and noted that binary models were in some respects noisier under the same analysis tools (Li, 2024). This result is limited to a toy task, but it directly contradicts the notion that parameter discreteness alone implies circuit simplicity.
Another boundary concerns the meaning of “ternary” itself. Many influential systems are ternary in the dominant LLM core but not in every parameter path. The 132M TernaryLM leaves embeddings and output projection in FP32 (Nargund et al., 7 Feb 2026). LLaVaOLMoBitNet1B, described as the first ternary multimodal LLM, is ternary in the OLMoBitNet1B text backbone but retains a full-precision CLIP ViT-L/14 encoder and a full-precision two-layer GELU projector; its reported multimodal results are POPE $3.9$9, VQAv2 $830$0, and TextVQA $830$1 (Sundaram et al., 2024). The practical lesson is that current ternary LMs are often ternary-dominant rather than universally ternary.
Several adjacent developments broaden the ecosystem without changing this basic definition. LoTA-QAF shows that ternary low-rank adapters can be merged losslessly into quantized base weights during fine-tuning, outperforming 16-bit LoRA by up to $830$2 on MMLU recovery in one reported setting while remaining a quantized-LLM adaptation method rather than a ternary base model (Chen et al., 24 May 2025). TWLA targets W1.58A4 by combining ternary weights with 4-bit activations in a retraining-free PTQ pipeline, addressing one of the major open problems left by weight-only ternary methods: activation outliers (Zhao et al., 11 Jun 2026). By contrast, TODO belongs to alignment rather than model compression, since its “ternary” structure is the preference relation $830$3, not the network parameterization (Guo et al., 2024).
Taken together, the field now supports a relatively clear characterization. A ternary LLM is best understood as a transformer LLM whose dominant linear transforms operate through ternary weight codes, usually with scaling terms and often with partial full-precision exceptions. Native pretraining, post-training ternarization, and adapter-based methods all exist, but the strongest evidence for competitiveness comes from large-scale native QAT suites such as Spectra and carefully engineered PTQ/QAT systems such as TernaryLLM, PTQTP, and TWLA. The main unresolved questions are whether fully ternarized interfaces can be trained without severe degradation, whether low-bit activations can be standardized without QAT-scale cost, and how much of the promised efficiency depends on custom kernels or specialized hardware rather than on ternary weights alone.
