---
title: 'Nacrith: Neural Lossless Compression'
url: https://www.emergentmind.com/topics/nacrith
type: topic
---

# Nacrith: Neural Lossless Compression

Searching arXiv for the Nacrith paper and closely related neural lossless compression work.
Nacrith is a lossless compression system for natural language that also extends to arbitrary binary data through a hybrid container format. It combines a pretrained 135M-parameter transformer language model, SmolLM2-135M, with lightweight online statistical components and a 32-bit arithmetic coder, with the central objective of converting accurate next-symbol prediction into short code lengths [2602.19626]. In the reported formulation, Nacrith is distinguished by a CDF precision upgrade from $2^{16}$ to $2^{24}$, a token-level N-gram model, an adaptive log-space bias head, confidence-based LLM skipping, an NC06 hybrid text/binary format, a llama.cpp inference backend, parallel multi-GPU compression across up to 8 workers, and native KV cache sliding-window support [2602.19626]. The system is presented as a neural lossless compressor whose main novelty lies not only in the use of an LLM with arithmetic coding, but in the joint treatment of predictive quality, probability discretization, and implementation architecture [2602.19626].

## 1. Problem setting and system definition

Nacrith is positioned as a neural lossless compressor that addresses several limitations attributed to prior LLM-based compressors: many prior systems use very large models, some require fine-tuning, some use coarse probability quantization, and most handle only text rather than arbitrary binary files [2602.19626]. Its core claim is that if the next symbol can be predicted extremely well, arithmetic coding can convert those predictions into very short code lengths [2602.19626].

At a high level, Nacrith takes input, tokenizes text into subword tokens, predicts the next-token distribution from context, converts that predicted distribution into a high-precision discrete CDF, and encodes each actual token with a 32-bit arithmetic coder [2602.19626]. Decompression mirrors the same sequence exactly: the decoder reconstructs the same predictive distribution from the already-decoded prefix, uses the arithmetic decoder to recover the next token, updates all online models identically, and repeats [2602.19626]. The paper identifies this requirement as **predictive symmetry**, namely that encoder and decoder must compute exactly the same probability distribution at each step in order to guarantee perfect lossless recovery [2602.19626].

The architecture combines seven named elements: a pretrained base transformer, SmolLM2-135M; an online token-level N-gram model; an online adaptive log-space bias head correcting document-specific LLM errors; a lightweight context mixer; a confidence-based skip mechanism; a high-precision CDF quantization scheme using $2^{24}$ total mass; and an NC06 hybrid format for binary files [2602.19626]. The paper’s ablations further suggest that the largest individual gain comes from better CDF precision rather than from changing the language model [2602.19626]. This suggests that, in this formulation of neural lossless compression, coding precision is treated as a first-order algorithmic variable rather than a low-level implementation detail.

## 2. End-to-end compression and decompression pipeline

For text input $s$, Nacrith tokenizes $s$ into BPE tokens $(t_1,\dots,t_n)$, initializes the LLM KV cache, N-gram model, mixer, and adaptive head, and then processes token positions sequentially [2602.19626]. At each position, it computes an N-gram prediction $P_{\text{ng}}$; if the N-gram is very confident, it skips the LLM and uses $P_{\text{ng}}$ directly; otherwise it runs the transformer to obtain $P_{\text{llm}}$, adjusts that distribution with the adaptive head, converts the selected distribution into an integer CDF with total mass $2^{24}$, arithmetic-codes the true token, and updates the online models with the observed token [2602.19626].

The paper gives the compression pseudocode explicitly:

```text
Algorithm 1. Nacrith compression pipeline
Require: text string s
1: Tokenize s -> (t1, ... , tn)
2: Initialize: LLM KV-cache, N-gram, mixer, adaptive head
3: for i = 1 to n do
4: Png + Ngram.predict(t1, ... , ti-1)
5: if H (Png) < T then
6: p + Png {skip LLM}
7: else
8: Pllm + LLM(t1, ... ,ti-1)
9: p + AdaptiveHead.adjust(pllm)
10: end if
11: cdf - probs_to_cdf(p, 2^24)
12: ArithEncoder.encode(cdf, ti)
13: Update: N-gram, Mixer, AdaptiveHead with ti
14: end for
15: return ArithEncoder.finish()
```

Decompression repeats the same steps in the same order, except that the next token is obtained from the arithmetic decoder rather than from the original source sequence [2602.19626]. Once the token is decoded, it is appended to the token sequence, the N-gram and adaptive head are updated, the LLM KV cache advances, and the process repeats [2602.19626]. Because the updates are deterministic and symmetric, decompression is lossless [2602.19626].

Nacrith is not text-only. Its NC06 format segments an arbitrary byte stream into alternating text-like and binary chunks using explicit heuristics: printable ASCII bytes $32\text{–}126$ plus tab/LF/CR are considered text-like; short text runs $<64$ bytes are demoted to binary; small binary gaps $\le 8$ bytes between text runs are bridged; and small binary chunks $<64$ bytes adjacent to text are absorbed into text regions [2602.19626]. Text chunks are routed through the neural token-based pipeline, whereas binary chunks are concatenated into a blob and compressed with LZMA if large ($\ge 4$ KB), otherwise gzip, or stored raw if neither helps [2602.19626]. The paper states that this hybrid path extends neural compression to arbitrary binary files and describes it as, to the authors’ knowledge, a first among LLM-based compressors [2602.19626].

## 3. Core model components and probabilistic machinery

The backbone model is SmolLM2-135M, described as a 30-layer causal transformer with 135 million parameters, BPE vocabulary size $V=49{,}152$, and context window 2,048 tokens [2602.19626]. The next-token distribution is written as

$$
P_{\text{llm}(t_{n+1}\mid t_1,\dots,t_n)=\operatorname{softmax}(Wh_n+b)
\tag{1}
$$

where $h_n$ is the hidden state at position $n$ [2602.19626]. Nacrith uses this model in pretrained-only form, with no fine-tuning, and runs it in FP32 to preserve deterministic probabilities for lossless decoding [2602.19626]. The weights are stored in GGUF F32 format, about 500 MB [2602.19626].

The token-level N-gram model is an interpolated online model of orders 1–4 [2602.19626]. Its unigram component uses Laplace smoothing,

$$
P_{\text{uni}(t)=\frac{c(t)+1}{N+V}
\tag{6}
$$

and higher-order estimates interpolate with lower-order estimates according to

$$
\lambda_k=\frac{n_k}{n_k+\varepsilon} \quad (\varepsilon=5)
$$

and

$$
P_k=\lambda_k\cdot P_k(\cdot\mid c_k) + (1-\lambda_k)\cdot P_{k-1}.
\tag{7}
$$

This model is document-specific and is described as adapting quickly to repeated names, formatting, and local phrase recurrence [2602.19626]. The paper further reports implementation optimizations: deterministic 64-bit rolling hashes for context keys instead of Python tuples, continuation dictionaries capped at 64 entries, and counts stored in preallocated int32 NumPy arrays, reducing memory from about 3.6 GB per worker to about 128 MB per worker [2602.19626].

The context mixer combines multiple predictors linearly:

$$
P_{\text{mix}=\sum_{j=1}^{M} w_j p^{(j)}, \qquad \sum_j w_j = 1.
\tag{8}
$$

The paper explicitly prefers linear rather than geometric mixing because linear mixing preserves confidence from the dominant predictor [2602.19626]. Weights are updated online by

$$
\log w_j \mathrel{+}= \eta \cdot \log p^{(j)}(t_i)
\tag{9}
$$

followed by renormalization [2602.19626]. Initial weights are $w_{\text{llm}}=0.85$, with the remaining 0.15 split among secondary models, and there is a 100-token warmup during which only the LLM is used [2602.19626]. The paper also states that in practice the mixer converges quickly to $w_{\text{llm}}\approx 1$, so the mixer has almost no effect on final compression and the N-gram’s real value is primarily in skipping LLM calls on highly predictable tokens [2602.19626].

The adaptive log-space bias head maintains a bias vector $b\in\mathbb{R}^V$, initialized to zero, and adjusts LLM probabilities multiplicatively in log space:

$$
\tilde{p}(t) = \frac{p_{\text{llm}(t)e^{b_t}}{\sum_{t'} p_{\text{llm}(t') e^{b_{t'}}}
= \operatorname{softmax}(\log p_{\text{llm}} + b).
\tag{10}
$$

After observing the true token $t^*$, one gradient descent step is performed on cross-entropy loss,

$$
b_t \mathrel{-}= \alpha \frac{\partial L}{\partial b_t} = \alpha \left(\tilde{p}(t)-\mathbf{1}[t=t^*]\right)
\tag{11}
$$

with learning rate $\alpha=0.001$ [2602.19626]. The paper states that these computations use float64 to ensure bit-exact reproducibility under identical software/hardware conditions [2602.19626]. Functionally, the mechanism is intended to correct systematic per-document LLM errors without fine-tuning the base model [2602.19626].

## 4. Coding precision, arithmetic coding, and systems engineering

A central argument of the paper is that coarse CDF quantization is a major bottleneck in large-vocabulary neural coding [2602.19626]. With vocabulary size $V=49{,}152$ and total CDF mass $T=2^{16}=65{,}536$, the mandatory one-count floor for each token consumes

$$
\frac{V\cdot \text{MIN\_PROB}}{T} = \frac{49{,}152}{65{,}536} \approx 75\%.
\tag{2}
$$

The paper approximates the associated quantization penalty for a peaked distribution as

$$
\Delta H \approx \log_2 4 = 2 \text{ bits/token}.
\tag{3}
$$

Nacrith instead uses $T=2^{24}=16{,}777{,}216$, for which the floor fraction becomes

$$
\frac{V}{T} = \frac{49{,}152}{16{,}777{,}216} \approx 0.29\%.
\tag{4}
$$

The discrete counts are allocated as

$$
c_i = \max\left(1,\; p_i \cdot (T - V)\right)
\tag{5}
$$

with residual mass added to the most probable token [2602.19626]. The paper says this reduces floor overhead to about 0.004 bits/token and that the ablation attributes about 0.5 bpb improvement to this CDF-24 change alone [2602.19626].

Nacrith uses a 32-bit arithmetic coder in the style of Witten et al. [2602.19626]. The coder maintains an interval $[\text{low},\text{high}] \subset [0,2^{32})$, or equivalently range $R=\text{high}-\text{low}+1$, and narrows that interval according to the token’s CDF interval [2602.19626]. The paper does not spell out the range-coder equations, but it explicitly describes the process as narrowing the interval by each symbol’s CDF interval and using binary search in the same CDF during decoding [2602.19626]. It also notes that using $2^{24}$ total mass is safe with a 32-bit coder because the minimum symbol width under floor count 1 remains above representability limits [2602.19626].

The confidence-based LLM skip criterion uses Shannon entropy of the N-gram distribution, with threshold $T=1.5$ bits [2602.19626]. When

$$
H(P_{\text{ng}}) < T,
$$

Nacrith skips the transformer and uses

$$
p = P_{\text{ng}}.
\tag{12}
$$

The reported skip rate reaches 30–70% on highly compressible text, and the ablation identifies N-gram plus skip as the second-largest source of compression gain after CDF-24 [2602.19626]. This indicates that the N-gram’s main utility is selective compute avoidance on easy tokens rather than blended prediction on difficult ones [2602.19626].

Several system-level mechanisms are presented as necessary for practical deployment on consumer GPUs. The paper reports a switch from PyTorch to llama.cpp via `llama-cpp-python`, with roughly 7x faster single-token incremental decode on the same hardware because computation stays in C/C++, Python dispatch overhead is avoided, and only one Python-to-C boundary crossing occurs per forward call [2602.19626]. The model remains in GGUF F32, the full 2048-token KV cache is allocated, logits are transferred GPU-to-CPU each step as approximately 196 KB, and softmax is computed on CPU [2602.19626].

For contexts exceeding the 2048-token window, Nacrith drops $C=512$ tokens from the left using native llama.cpp cache manipulation, rather than clearing and rebuilding the full cache [2602.19626]. The paper reports a naive slide cost of about 693 ms on GTX 1050 Ti, a native slide cost of about 19 ms, and a speedup of about 37x [2602.19626]. The amortized token cost is expressed as

$$
T_{\text{tok} \approx \frac{T_{\text{slide} + C\cdot T_{\text{incr}}}{C} \approx T_{\text{incr} + \frac{T_{\text{slide}}{C}
\tag{13}
$$

and, with $T_{\text{slide}}\approx T_{\text{incr}}$, the paper states that the overhead is about $1+\frac{1}{C}\approx 1.002\times$, effectively negligible [2602.19626].

## 5. File formats, hardware profile, and reproducibility-oriented implementation

Two file formats are described. NC05 is the text-only format and stores a 4-byte magic `NC05`, a 1-byte feature-flags field for N-gram, adaptive head, and confidence skip, a 2-byte temperature, and a 2-byte chunk count [2602.19626]. This is followed by a per-chunk table of 12 bytes each—token count, bit count, and stream length—and then the concatenated arithmetic-coded streams [2602.19626]. NC06 extends this to binary files by adding a version byte, a structured entry table describing text/binary chunk layout, a binary section, and parallelized text streams [2602.19626]. The metadata is intended to ensure exact reconstruction of which path each chunk used during compression [2602.19626].

The system includes a dual tokenizer architecture: llama.cpp is used for inference, while Hugging Face tokenization and detokenization are retained because llama.cpp’s built-in detokenizer drops content for 47 whitespace/repeat tokens in the SmolLM2 vocabulary [2602.19626]. Token IDs are stated to be identical across the two paths [2602.19626]. A fallback path is also specified: PyTorch plus CUDA Graphs on GPU, or dynamic KV cache on CPU, if llama.cpp is unavailable [2602.19626].

Nacrith can split a text into up to 8 chunks and compress them concurrently, typically on newline boundaries [2602.19626]. Each worker has its own llama.cpp model instance, N-gram model, mixer, adaptive head, and KV cache, thereby avoiding synchronization overhead [2602.19626]. Worker count is auto-selected from free VRAM according to

$$
N = \min\left(8,\; 1+\frac{\text{VRAM}_{\text{free}} - R - F}{E}\right)
\tag{14}
$$

where $F\approx 1169$ MB is the first-instance cost, $E\approx 660$ MB is each additional instance, and $R=512$ MB is reserved for the OS [2602.19626]. On a 4 GB GTX 1050 Ti, the paper states that this supports up to 3 workers [2602.19626].

The reported deployment profile is deliberately modest. All experiments were run on an NVIDIA GeForce GTX 1050 Ti with 4 GB VRAM and CUDA capability 6.1, with about 1.2 GB VRAM per worker and a model file of about 500 MB [2602.19626]. This consumer-grade hardware target is integral to the paper’s positioning of Nacrith as a practical system rather than a purely scaling-based demonstration [2602.19626].

## 6. Empirical performance, ablations, and interpretation

The main performance metric is bits per byte (bpb), defined as

$$
\text{bpb} = \frac{8C}{B},
$$

for compressed size $C$ bytes and original size $B$ bytes [2602.19626]. The paper also reports compression ratio as $\frac{C}{B}$ [2602.19626].

On `alice29.txt` from the Canterbury Corpus (152,089 bytes), Nacrith achieves 17,458 bytes, 11.5%, and 0.918 bpb [2602.19626]. The comparison values reported are gzip -9 at 54,191 B and 2.851 bpb, zstd -19 at 49,215 B and 2.589 bpb, xz -9 at 48,500 B and 2.551 bpb, Brotli -q11 at 46,487 B and 2.445 bpb, bzip2 -9 at 43,202 B and 2.273 bpb, PAQ8px -8L at 32,857 B and 1.728 bpb, CMIX v21 at 31,076 B and 1.635 bpb, ts_zip at approximately 21,703 B and approximately 1.142 bpb, and Nacrith at 17,458 B and 0.918 bpb [2602.19626]. The paper states that this is 3.1x better than gzip, 2.5x better than bzip2, 44% better than CMIX v21, and 20% better than ts_zip [2602.19626].

On `enwik8` (100 MB), Nacrith achieves 0.9389 bpb, 11,737,280 bytes, and 11.74% of original size [2602.19626]. The reported comparisons are gzip -9 at 2.916 bpb, xz -9 at 1.989 bpb, bzip2 -9 at 2.321 bpb, Brotli -q11 at 2.059 bpb, zstd -19 at 2.156 bpb, CMIX v21 at 1.17 bpb, PAQ8px at approximately 1.27 bpb, NNCP v3 at approximately 1.19 bpb, FineZip at 1.024 bpb, ts_zip at approximately 1.11 bpb, and Nacrith at 0.9389 bpb [2602.19626]. The paper states that Nacrith beats ts_zip by 15%, FineZip by 8%, and gzip by about 3.1x on this benchmark [2602.19626].

An out-of-distribution evaluation on a 333,794-byte UK government report published after SmolLM2’s training cutoff is used to address memorization concerns [2602.19626]. The reported results are gzip -9 at 2.189 bpb, zstd -19 at 1.910 bpb, xz -9 at 1.739 bpb, bzip2 -9 at 1.661 bpb, Brotli -q11 at 1.646 bpb, CMIX v21 at 1.148 bpb, ts_zip at 0.964 bpb, FineZip using the same SmolLM2-135M at 0.977 bpb, and Nacrith at 0.723 bpb [2602.19626]. Because both FineZip and Nacrith use the same underlying SmolLM2-135M model in this comparison, the paper attributes the gain to Nacrith’s architecture and coding details rather than model scale or pretraining-data memorization [2602.19626].

The paper also compares Nacrith against low-order byte-level Shannon entropy references on `alice29.txt`, reporting $H_0=4.568$ bpb, $H_1=3.419$ bpb, and $H_2=2.485$ bpb, while Nacrith attains 0.918 bpb [2602.19626]. The interpretation given is not that Shannon’s limit is violated, but that low-order byte-level entropy estimates are weak references because they ignore long-range token-level structure [2602.19626]. This is an important clarification because the result is explicitly framed as demonstrating richer modeling rather than overturning information-theoretic bounds [2602.19626].

The ablation study is especially central to the system’s interpretation. On enwik8 first 1 MB / alice29, the reported sequence is A0: LLM + AE at 1.817 / 1.861 bpb; A1: + CDF-24 at 1.300 / 1.358 with gain $-0.517$ / $-0.503$; A2: + Adaptive head at 1.285 / 1.341 with gain $-0.015$ / $-0.017$; A3: + N-gram + Skip at 0.897 / 0.940 with gain $-0.388$ / $-0.401$; and A4: Full system at 0.896 / 0.939 with gain $-0.001$ / $-0.001$ [2602.19626]. The paper’s stated takeaways are that CDF-24 is the largest single gain, N-gram plus skip is second, the adaptive head helps modestly, and the final mixer contributes almost nothing beyond that [2602.19626]. This strongly constrains the mechanistic interpretation of Nacrith’s improvements: most of the benefit comes from precise discretization and selective local prediction rather than from mixture complexity.

## 7. Conceptual significance, comparisons, and limitations

Nacrith is presented as evidence that a relatively small pretrained token-level transformer can outperform a range of classical compressors on English text when embedded in a carefully engineered probabilistic coding system [2602.19626]. The conceptual claim is that classical compressors operate primarily on bytes and exploit local repetitions or adaptive contexts, whereas a pretrained transformer can exploit syntax, semantics, topic, discourse, and world knowledge, and arithmetic coding can convert those predictive gains into compressed size reductions [2602.19626]. The paper further emphasizes that the result is obtained without fine-tuning, distinguishing Nacrith from systems such as FineZip in the comparison discussed by the authors [2602.19626].

The paper gives a direct conceptual comparison to ts_zip and FineZip. Relative to ts_zip, Nacrith is said to have similar model scale but to add CDF-24 precision, adaptive bias, N-gram plus confidence skip, binary support, parallel chunk compression, and llama.cpp optimization [2602.19626]. Relative to FineZip, the paper states that FineZip uses a much larger model and fine-tuning, whereas Nacrith uses a much smaller 135M pretrained-only model yet still reports better compression in the presented experiments [2602.19626]. This suggests that, within the scope of the reported results, the system is intended as an argument for the importance of probability discretization and online adaptation alongside model quality.

The reported throughput remains substantially below that of classical compressors. Single-worker throughput on GTX 1050 Ti is about 50–70 tokens/s at the start and settles to about 20–30 tokens/s once the KV cache is full; with 3 workers, aggregate throughput is about 60–90 tokens/s [2602.19626]. The paper explicitly admits that this is still far slower than classical compressors [2602.19626]. It also notes several limitations: dependence on GPU inference for practicality, the requirement that the approximately 500 MB model be available at both endpoints, the 2048-token context-window limit, likely language- and domain-dependence, and the possibility that deterministic lossless reconstruction may depend on tightly controlled numerical and software environments [2602.19626].

A broader implication is that neural lossless compression need not be framed solely as a scaling problem. The Nacrith paper argues, by architecture and ablation rather than by rhetoric alone, that probability quality, probability discretization, and systems design all materially affect final compression performance [2602.19626]. A plausible implication is that future work in this area may derive large gains from precision engineering, online correction, and inference-path optimization even when the backbone language model remains comparatively small.

Source: https://www.emergentmind.com/topics/nacrith