Nacrith: Neural Lossless Compression

• Nacrith is a neural lossless compression system for text and binary data that integrates a 135M transformer with online statistical models and adaptive bias.
• It converts accurate next-symbol predictions into short code lengths using high-precision CDF (2^24) arithmetic coding combined with a token-level N-gram model for efficient LLM skipping.
• Empirical results demonstrate that Nacrith outperforms classical compressors, highlighting the significance of probability discretization and systems engineering in compression performance.

Searching arXiv for the Nacrith paper and closely related neural lossless compression work. Nacrith is a lossless compression system for natural language that also extends to arbitrary binary data through a hybrid container format. It combines a pretrained 135M-parameter transformer LLM, SmolLM2-135M, with lightweight online statistical components and a 32-bit arithmetic coder, with the central objective of converting accurate next-symbol prediction into short code lengths (Tacconelli, 23 Feb 2026). In the reported formulation, Nacrith is distinguished by a CDF precision upgrade from $2^{16}$ to $2^{24}$, a token-level N-gram model, an adaptive log-space bias head, confidence-based LLM skipping, an NC06 hybrid text/binary format, a llama.cpp inference backend, parallel multi-GPU compression across up to 8 workers, and native KV cache sliding-window support (Tacconelli, 23 Feb 2026). The system is presented as a neural lossless compressor whose main novelty lies not only in the use of an LLM with arithmetic coding, but in the joint treatment of predictive quality, probability discretization, and implementation architecture (Tacconelli, 23 Feb 2026).

1. Problem setting and system definition

Nacrith is positioned as a neural lossless compressor that addresses several limitations attributed to prior LLM-based compressors: many prior systems use very large models, some require fine-tuning, some use coarse probability quantization, and most handle only text rather than arbitrary binary files (Tacconelli, 23 Feb 2026). Its core claim is that if the next symbol can be predicted extremely well, arithmetic coding can convert those predictions into very short code lengths (Tacconelli, 23 Feb 2026).

At a high level, Nacrith takes input, tokenizes text into subword tokens, predicts the next-token distribution from context, converts that predicted distribution into a high-precision discrete CDF, and encodes each actual token with a 32-bit arithmetic coder (Tacconelli, 23 Feb 2026). Decompression mirrors the same sequence exactly: the decoder reconstructs the same predictive distribution from the already-decoded prefix, uses the arithmetic decoder to recover the next token, updates all online models identically, and repeats (Tacconelli, 23 Feb 2026). The paper identifies this requirement as predictive symmetry, namely that encoder and decoder must compute exactly the same probability distribution at each step in order to guarantee perfect lossless recovery (Tacconelli, 23 Feb 2026).

The architecture combines seven named elements: a pretrained base transformer, SmolLM2-135M; an online token-level N-gram model; an online adaptive log-space bias head correcting document-specific LLM errors; a lightweight context mixer; a confidence-based skip mechanism; a high-precision CDF quantization scheme using $2^{24}$ total mass; and an NC06 hybrid format for binary files (Tacconelli, 23 Feb 2026). The paper’s ablations further suggest that the largest individual gain comes from better CDF precision rather than from changing the LLM (Tacconelli, 23 Feb 2026). This suggests that, in this formulation of neural lossless compression, coding precision is treated as a first-order algorithmic variable rather than a low-level implementation detail.

2. End-to-end compression and decompression pipeline

For text input $s$, Nacrith tokenizes $s$ into BPE tokens $(t_1,\dots,t_n)$, initializes the LLM KV cache, N-gram model, mixer, and adaptive head, and then processes token positions sequentially (Tacconelli, 23 Feb 2026). At each position, it computes an N-gram prediction $P_{\text{ng}}$; if the N-gram is very confident, it skips the LLM and uses $P_{\text{ng}}$ directly; otherwise it runs the transformer to obtain $P_{\text{LLM}}$, adjusts that distribution with the adaptive head, converts the selected distribution into an integer CDF with total mass $2^{24}$, arithmetic-codes the true token, and updates the online models with the observed token (Tacconelli, 23 Feb 2026).

The paper gives the compression pseudocode explicitly:

$P_{\text{ng}}$7

Decompression repeats the same steps in the same order, except that the next token is obtained from the arithmetic decoder rather than from the original source sequence (Tacconelli, 23 Feb 2026). Once the token is decoded, it is appended to the token sequence, the N-gram and adaptive head are updated, the LLM KV cache advances, and the process repeats (Tacconelli, 23 Feb 2026). Because the updates are deterministic and symmetric, decompression is lossless (Tacconelli, 23 Feb 2026).

Nacrith is not text-only. Its NC06 format segments an arbitrary byte stream into alternating text-like and binary chunks using explicit heuristics: printable ASCII bytes $2^{24}$0 plus tab/LF/CR are considered text-like; short text runs $2^{24}$1 bytes are demoted to binary; small binary gaps $2^{24}$2 bytes between text runs are bridged; and small binary chunks $2^{24}$3 bytes adjacent to text are absorbed into text regions (Tacconelli, 23 Feb 2026). Text chunks are routed through the neural token-based pipeline, whereas binary chunks are concatenated into a blob and compressed with LZMA if large ($2^{24}$4 KB), otherwise gzip, or stored raw if neither helps (Tacconelli, 23 Feb 2026). The paper states that this hybrid path extends neural compression to arbitrary binary files and describes it as, to the authors’ knowledge, a first among LLM-based compressors (Tacconelli, 23 Feb 2026).

3. Core model components and probabilistic machinery

The backbone model is SmolLM2-135M, described as a 30-layer causal transformer with 135 million parameters, BPE vocabulary size $2^{24}$5, and context window 2,048 tokens (Tacconelli, 23 Feb 2026). The next-token distribution is written as

$2^{24}$6

where $2^{24}$7 is the hidden state at position $2^{24}$8 (Tacconelli, 23 Feb 2026). Nacrith uses this model in pretrained-only form, with no fine-tuning, and runs it in FP32 to preserve deterministic probabilities for lossless decoding (Tacconelli, 23 Feb 2026). The weights are stored in GGUF F32 format, about 500 MB (Tacconelli, 23 Feb 2026).

The token-level N-gram model is an interpolated online model of orders 1–4 (Tacconelli, 23 Feb 2026). Its unigram component uses Laplace smoothing,

$2^{24}$9

and higher-order estimates interpolate with lower-order estimates according to

$2^{24}$0

and

$2^{24}$1

This model is document-specific and is described as adapting quickly to repeated names, formatting, and local phrase recurrence (Tacconelli, 23 Feb 2026). The paper further reports implementation optimizations: deterministic 64-bit rolling hashes for context keys instead of Python tuples, continuation dictionaries capped at 64 entries, and counts stored in preallocated int32 NumPy arrays, reducing memory from about 3.6 GB per worker to about 128 MB per worker (Tacconelli, 23 Feb 2026).

The context mixer combines multiple predictors linearly:

$2^{24}$2

The paper explicitly prefers linear rather than geometric mixing because linear mixing preserves confidence from the dominant predictor (Tacconelli, 23 Feb 2026). Weights are updated online by

$2^{24}$3

followed by renormalization (Tacconelli, 23 Feb 2026). Initial weights are $2^{24}$4, with the remaining 0.15 split among secondary models, and there is a 100-token warmup during which only the LLM is used (Tacconelli, 23 Feb 2026). The paper also states that in practice the mixer converges quickly to $2^{24}$5, so the mixer has almost no effect on final compression and the N-gram’s real value is primarily in skipping LLM calls on highly predictable tokens (Tacconelli, 23 Feb 2026).

The adaptive log-space bias head maintains a bias vector $2^{24}$6, initialized to zero, and adjusts LLM probabilities multiplicatively in log space:

$2^{24}$7

After observing the true token $2^{24}$8, one gradient descent step is performed on cross-entropy loss,

$2^{24}$9

with learning rate $s$0 (Tacconelli, 23 Feb 2026). The paper states that these computations use float64 to ensure bit-exact reproducibility under identical software/hardware conditions (Tacconelli, 23 Feb 2026). Functionally, the mechanism is intended to correct systematic per-document LLM errors without fine-tuning the base model (Tacconelli, 23 Feb 2026).

4. Coding precision, arithmetic coding, and systems engineering

A central argument of the paper is that coarse CDF quantization is a major bottleneck in large-vocabulary neural coding (Tacconelli, 23 Feb 2026). With vocabulary size $s$1 and total CDF mass $s$2, the mandatory one-count floor for each token consumes

$s$3

The paper approximates the associated quantization penalty for a peaked distribution as

$s$4

Nacrith instead uses $s$5, for which the floor fraction becomes

$s$6

The discrete counts are allocated as

$s$7

with residual mass added to the most probable token (Tacconelli, 23 Feb 2026). The paper says this reduces floor overhead to about 0.004 bits/token and that the ablation attributes about 0.5 bpb improvement to this CDF-24 change alone (Tacconelli, 23 Feb 2026).

Nacrith uses a 32-bit arithmetic coder in the style of Witten et al. (Tacconelli, 23 Feb 2026). The coder maintains an interval $s$8, or equivalently range $s$9, and narrows that interval according to the token’s CDF interval (Tacconelli, 23 Feb 2026). The paper does not spell out the range-coder equations, but it explicitly describes the process as narrowing the interval by each symbol’s CDF interval and using binary search in the same CDF during decoding (Tacconelli, 23 Feb 2026). It also notes that using $s$0 total mass is safe with a 32-bit coder because the minimum symbol width under floor count 1 remains above representability limits (Tacconelli, 23 Feb 2026).

The confidence-based LLM skip criterion uses Shannon entropy of the N-gram distribution, with threshold $s$1 bits (Tacconelli, 23 Feb 2026). When

$s$2

Nacrith skips the transformer and uses

$s$3

The reported skip rate reaches 30–70% on highly compressible text, and the ablation identifies N-gram plus skip as the second-largest source of compression gain after CDF-24 (Tacconelli, 23 Feb 2026). This indicates that the N-gram’s main utility is selective compute avoidance on easy tokens rather than blended prediction on difficult ones (Tacconelli, 23 Feb 2026).

Several system-level mechanisms are presented as necessary for practical deployment on consumer GPUs. The paper reports a switch from PyTorch to llama.cpp via llama-cpp-python, with roughly 7x faster single-token incremental decode on the same hardware because computation stays in C/C++, Python dispatch overhead is avoided, and only one Python-to-C boundary crossing occurs per forward call (Tacconelli, 23 Feb 2026). The model remains in GGUF F32, the full 2048-token KV cache is allocated, logits are transferred GPU-to-CPU each step as approximately 196 KB, and softmax is computed on CPU (Tacconelli, 23 Feb 2026).

For contexts exceeding the 2048-token window, Nacrith drops $s$4 tokens from the left using native llama.cpp cache manipulation, rather than clearing and rebuilding the full cache (Tacconelli, 23 Feb 2026). The paper reports a naive slide cost of about 693 ms on GTX 1050 Ti, a native slide cost of about 19 ms, and a speedup of about 37x (Tacconelli, 23 Feb 2026). The amortized token cost is expressed as

$s$5

and, with $s$6, the paper states that the overhead is about $s$7, effectively negligible (Tacconelli, 23 Feb 2026).

5. File formats, hardware profile, and reproducibility-oriented implementation

Two file formats are described. NC05 is the text-only format and stores a 4-byte magic NC05, a 1-byte feature-flags field for N-gram, adaptive head, and confidence skip, a 2-byte temperature, and a 2-byte chunk count (Tacconelli, 23 Feb 2026). This is followed by a per-chunk table of 12 bytes each—token count, bit count, and stream length—and then the concatenated arithmetic-coded streams (Tacconelli, 23 Feb 2026). NC06 extends this to binary files by adding a version byte, a structured entry table describing text/binary chunk layout, a binary section, and parallelized text streams (Tacconelli, 23 Feb 2026). The metadata is intended to ensure exact reconstruction of which path each chunk used during compression (Tacconelli, 23 Feb 2026).

The system includes a dual tokenizer architecture: llama.cpp is used for inference, while Hugging Face tokenization and detokenization are retained because llama.cpp’s built-in detokenizer drops content for 47 whitespace/repeat tokens in the SmolLM2 vocabulary (Tacconelli, 23 Feb 2026). Token IDs are stated to be identical across the two paths (Tacconelli, 23 Feb 2026). A fallback path is also specified: PyTorch plus CUDA Graphs on GPU, or dynamic KV cache on CPU, if llama.cpp is unavailable (Tacconelli, 23 Feb 2026).

Nacrith can split a text into up to 8 chunks and compress them concurrently, typically on newline boundaries (Tacconelli, 23 Feb 2026). Each worker has its own llama.cpp model instance, N-gram model, mixer, adaptive head, and KV cache, thereby avoiding synchronization overhead (Tacconelli, 23 Feb 2026). Worker count is auto-selected from free VRAM according to

$s$8

where $s$9 MB is the first-instance cost, $(t_1,\dots,t_n)$0 MB is each additional instance, and $(t_1,\dots,t_n)$1 MB is reserved for the OS (Tacconelli, 23 Feb 2026). On a 4 GB GTX 1050 Ti, the paper states that this supports up to 3 workers (Tacconelli, 23 Feb 2026).

The reported deployment profile is deliberately modest. All experiments were run on an NVIDIA GeForce GTX 1050 Ti with 4 GB VRAM and CUDA capability 6.1, with about 1.2 GB VRAM per worker and a model file of about 500 MB (Tacconelli, 23 Feb 2026). This consumer-grade hardware target is integral to the paper’s positioning of Nacrith as a practical system rather than a purely scaling-based demonstration (Tacconelli, 23 Feb 2026).

6. Empirical performance, ablations, and interpretation

The main performance metric is bits per byte (bpb), defined as

$(t_1,\dots,t_n)$2

for compressed size $(t_1,\dots,t_n)$3 bytes and original size $(t_1,\dots,t_n)$4 bytes (Tacconelli, 23 Feb 2026). The paper also reports compression ratio as $(t_1,\dots,t_n)$5 (Tacconelli, 23 Feb 2026).

On alice29.txt from the Canterbury Corpus (152,089 bytes), Nacrith achieves 17,458 bytes, 11.5%, and 0.918 bpb (Tacconelli, 23 Feb 2026). The comparison values reported are gzip -9 at 54,191 B and 2.851 bpb, zstd -19 at 49,215 B and 2.589 bpb, xz -9 at 48,500 B and 2.551 bpb, Brotli -q11 at 46,487 B and 2.445 bpb, bzip2 -9 at 43,202 B and 2.273 bpb, PAQ8px -8L at 32,857 B and 1.728 bpb, CMIX v21 at 31,076 B and 1.635 bpb, ts_zip at approximately 21,703 B and approximately 1.142 bpb, and Nacrith at 17,458 B and 0.918 bpb (Tacconelli, 23 Feb 2026). The paper states that this is 3.1x better than gzip, 2.5x better than bzip2, 44% better than CMIX v21, and 20% better than ts_zip (Tacconelli, 23 Feb 2026).

On enwik8 (100 MB), Nacrith achieves 0.9389 bpb, 11,737,280 bytes, and 11.74% of original size (Tacconelli, 23 Feb 2026). The reported comparisons are gzip -9 at 2.916 bpb, xz -9 at 1.989 bpb, bzip2 -9 at 2.321 bpb, Brotli -q11 at 2.059 bpb, zstd -19 at 2.156 bpb, CMIX v21 at 1.17 bpb, PAQ8px at approximately 1.27 bpb, NNCP v3 at approximately 1.19 bpb, FineZip at 1.024 bpb, ts_zip at approximately 1.11 bpb, and Nacrith at 0.9389 bpb (Tacconelli, 23 Feb 2026). The paper states that Nacrith beats ts_zip by 15%, FineZip by 8%, and gzip by about 3.1x on this benchmark (Tacconelli, 23 Feb 2026).

An out-of-distribution evaluation on a 333,794-byte UK government report published after SmolLM2’s training cutoff is used to address memorization concerns (Tacconelli, 23 Feb 2026). The reported results are gzip -9 at 2.189 bpb, zstd -19 at 1.910 bpb, xz -9 at 1.739 bpb, bzip2 -9 at 1.661 bpb, Brotli -q11 at 1.646 bpb, CMIX v21 at 1.148 bpb, ts_zip at 0.964 bpb, FineZip using the same SmolLM2-135M at 0.977 bpb, and Nacrith at 0.723 bpb (Tacconelli, 23 Feb 2026). Because both FineZip and Nacrith use the same underlying SmolLM2-135M model in this comparison, the paper attributes the gain to Nacrith’s architecture and coding details rather than model scale or pretraining-data memorization (Tacconelli, 23 Feb 2026).

The paper also compares Nacrith against low-order byte-level Shannon entropy references on alice29.txt, reporting $(t_1,\dots,t_n)$6 bpb, $(t_1,\dots,t_n)$7 bpb, and $(t_1,\dots,t_n)$8 bpb, while Nacrith attains 0.918 bpb (Tacconelli, 23 Feb 2026). The interpretation given is not that Shannon’s limit is violated, but that low-order byte-level entropy estimates are weak references because they ignore long-range token-level structure (Tacconelli, 23 Feb 2026). This is an important clarification because the result is explicitly framed as demonstrating richer modeling rather than overturning information-theoretic bounds (Tacconelli, 23 Feb 2026).

The ablation study is especially central to the system’s interpretation. On enwik8 first 1 MB / alice29, the reported sequence is A0: LLM + AE at 1.817 / 1.861 bpb; A1: + CDF-24 at 1.300 / 1.358 with gain $(t_1,\dots,t_n)$9 / $P_{\text{ng}}$0; A2: + Adaptive head at 1.285 / 1.341 with gain $P_{\text{ng}}$1 / $P_{\text{ng}}$2; A3: + N-gram + Skip at 0.897 / 0.940 with gain $P_{\text{ng}}$3 / $P_{\text{ng}}$4; and A4: Full system at 0.896 / 0.939 with gain $P_{\text{ng}}$5 / $P_{\text{ng}}$6 (Tacconelli, 23 Feb 2026). The paper’s stated takeaways are that CDF-24 is the largest single gain, N-gram plus skip is second, the adaptive head helps modestly, and the final mixer contributes almost nothing beyond that (Tacconelli, 23 Feb 2026). This strongly constrains the mechanistic interpretation of Nacrith’s improvements: most of the benefit comes from precise discretization and selective local prediction rather than from mixture complexity.

7. Conceptual significance, comparisons, and limitations

Nacrith is presented as evidence that a relatively small pretrained token-level transformer can outperform a range of classical compressors on English text when embedded in a carefully engineered probabilistic coding system (Tacconelli, 23 Feb 2026). The conceptual claim is that classical compressors operate primarily on bytes and exploit local repetitions or adaptive contexts, whereas a pretrained transformer can exploit syntax, semantics, topic, discourse, and world knowledge, and arithmetic coding can convert those predictive gains into compressed size reductions (Tacconelli, 23 Feb 2026). The paper further emphasizes that the result is obtained without fine-tuning, distinguishing Nacrith from systems such as FineZip in the comparison discussed by the authors (Tacconelli, 23 Feb 2026).

The paper gives a direct conceptual comparison to ts_zip and FineZip. Relative to ts_zip, Nacrith is said to have similar model scale but to add CDF-24 precision, adaptive bias, N-gram plus confidence skip, binary support, parallel chunk compression, and llama.cpp optimization (Tacconelli, 23 Feb 2026). Relative to FineZip, the paper states that FineZip uses a much larger model and fine-tuning, whereas Nacrith uses a much smaller 135M pretrained-only model yet still reports better compression in the presented experiments (Tacconelli, 23 Feb 2026). This suggests that, within the scope of the reported results, the system is intended as an argument for the importance of probability discretization and online adaptation alongside model quality.

The reported throughput remains substantially below that of classical compressors. Single-worker throughput on GTX 1050 Ti is about 50–70 tokens/s at the start and settles to about 20–30 tokens/s once the KV cache is full; with 3 workers, aggregate throughput is about 60–90 tokens/s (Tacconelli, 23 Feb 2026). The paper explicitly admits that this is still far slower than classical compressors (Tacconelli, 23 Feb 2026). It also notes several limitations: dependence on GPU inference for practicality, the requirement that the approximately 500 MB model be available at both endpoints, the 2048-token context-window limit, likely language- and domain-dependence, and the possibility that deterministic lossless reconstruction may depend on tightly controlled numerical and software environments (Tacconelli, 23 Feb 2026).

A broader implication is that neural lossless compression need not be framed solely as a scaling problem. The Nacrith paper argues, by architecture and ablation rather than by rhetoric alone, that probability quality, probability discretization, and systems design all materially affect final compression performance (Tacconelli, 23 Feb 2026). A plausible implication is that future work in this area may derive large gains from precision engineering, online correction, and inference-path optimization even when the backbone LLM remains comparatively small.