MiniTruePrefixes: Prefix-Level Factuality Control

• MiniTruePrefixes is a prefix-level Natural Language Inference model that detects factual inconsistencies during autoregressive generation by evaluating incomplete text prefixes.
• It integrates a specialized LLaMA-based entailment classifier into controlled decoding, improving factual accuracy by 5–14 F1 points over baseline models.
• The model employs a two-stage training process with both true and synthetic hallucination data, enabling fast, memory-efficient, and token-level factuality control.

Searching arXiv for the MiniTruePrefixes paper and closely related factuality/prefix-NLI work. arxiv_search(^^^^1^^^^="^^^^0 max_results=10) Searching arXiv for PrefixNLI and related prefix-level entailment work. arxiv_search: {"^^^^1^^^^":"MiniTruePrefixes PrefixNLI factual inconsistencies as soon as they arise", "max_results": 10} to=arxiv_search {"^^^^1^^^^":"MiniTruePrefixes PrefixNLI factual inconsistencies as soon as they arise", "max_results": 10} MiniTruePrefixes is a prefix-level Natural Language Inference model introduced for detecting factual inconsistencies during autoregressive generation as soon as they appear, rather than after a full sentence or summary has been completed. It is presented together with the PrefixNLI task, which extends entailment detection from complete hypotheses to arbitrary text prefixes, and is used as a controlled decoding component for factuality in abstractive summarization. The model is a lightweight LLaMA-based entailment classifier trained specifically on prefix hypotheses, and the reported result is that it outperforms comparable baseline NLI models by 5–14 F1 points in prefix-level entailment while enabling decoding-time factuality control with substantially lower runtime than lookahead approaches (Harary et al., 3 Nov 2025).

1. Definition and problem setting

MiniTruePrefixes is designed for the mismatch between sentence-level factuality models and token-level text generation. Standard NLI-based factuality control typically evaluates complete hypotheses, but autoregressive generators make decisions over evolving prefixes $y_{1:t}$. The paper argues that this mismatch affects both training-time and inference-time control: reward-based approaches often assign reward only at EOS, and controlled decoding methods based on lookahead score speculative full completions rather than the actual current prefix (Harary et al., 3 Nov 2025).

The associated task, PrefixNLI, is defined as binary entailment over a premise $x$ and an arbitrary text prefix $y_{1:t}$. The task definition is stated as follows: a text prefix is entailed by a premise if there may be a sensible completion of the prefix to a complete text that would be entailed by the premise. Prefixes may therefore be incomplete, ungrammatical, or still salvageable; non-entailment is assigned only when the prefix already contains unsupported information.

This framing separates prefix-aware factuality from conventional sentence-level verification. A plausible implication is that factuality control becomes more tightly aligned with the actual search process of autoregressive decoding, because the judgment is applied to the object the generator is currently constructing rather than to a later speculative continuation.

2. PrefixNLI formulation

The paper formulates PrefixNLI as a binary classification problem using a TrueTeacher-style prompting scheme. The model input is:

Premise: {document} Hypothesis: {summary prefix}.

The model outputs token "1" if the prefix is entailed and "0" otherwise. Entailment is predicted when

$P_{\text{entail}(y_{1:t}\mid x)} > 0.5$

where the paper notes that this probability is technically the probability of the class-label token "1" (Harary et al., 3 Nov 2025).

A central contribution is the rule for deriving prefix labels from hallucination annotations. For a non-entailed hypothesis with a detected hallucinated span $s$, prefixes ending before the start of $s$ are labeled entailed, prefixes ending at or after the end of $s$ are labeled non-entailed, and prefixes partially overlapping $s$ are excluded. This construction turns sentence-level factuality resources into prefix-level supervision without requiring annotation of every partial hypothesis.

The significance of this formulation is methodological. It preserves the binary entailment structure of standard NLI while adapting it to partial hypotheses, allowing the model to be deployed directly inside decoding.

3. Data resources and prefix-label construction

The paper constructs both evaluation and training datasets specifically for prefix entailment. Two evaluation benchmarks are derived from existing factuality resources.

RAGTruthPrefixes is built from RAGTruth and uses human word-level hallucination span annotations directly. Each sentence in each summary is converted into prefix examples. The reported scale is 194,283 faithful prefixes, 16,395 unfaithful prefixes, 880 unique documents, average hallucination span length 11.8 tokens, and average prefix length 14.7 tokens (Harary et al., 3 Nov 2025).

SummEditsPrefixes is built from SummEdits. For unfaithful summaries, the unsupported span is identified by taking the longest common prefix and suffix between the faithful seed summary and the LLM-modified summary, with the differing middle span treated as hallucinated. The benchmark is restricted to the News domain and to summaries with up to a single LLM modification. To avoid length bias, examples are stratified by prefix position $t$ so each prefix length has equal numbers of faithful and unfaithful examples. The reported scale is 4,455 faithful prefixes, 4,455 unfaithful prefixes, 25 unique documents, average hallucination span length 4.2 tokens, and average prefix length 27.6 tokens.

Training uses two complementary prefix-level sources. The first is TrueTeacher-derived prefix data, where GPT-4 identifies the first hallucinated span in unfaithful summaries. The second is a synthetic hallucination dataset in which GPT-4 generates short summaries with subtle, believable hallucinations, targeting minor numeric, date, or detail misrepresentation, emotional or symbolic interpretation unsupported by the text, and inaccurate generalization formed by combining facts. The appendix observation that first-pass hallucination identification missed subtle cases motivated the addition of synthetic hallucination injection.

These resources are notable because they operationalize prefix factuality as a supervised learning problem rather than as a purely heuristic decoding signal.

4. Model architecture and training procedure

MiniTruePrefixes is described as a lightweight LLaMA-based entailment classifier rather than a new architecture. The backbone is LLaMA-3.2-Instruct 1B, and the setup follows the entailment classification style of TrueTeacher. The paper also emphasizes a practical advantage of the decoder-only architecture: prefix caching can be exploited when scoring prefixes during decoding (Harary et al., 3 Nov 2025).

Training proceeds in two stages. In the first stage, the model MiniTrue is trained as a sentence-level entailment model on TrueTeacher and ANLI using cross-entropy loss, with learning rate $2 \times 10^{-4}$, batch size 32, 3 epochs, and LoRA. In the second stage, MiniTruePrefixes is further fine-tuned on the prefix-level datasets, again with cross-entropy loss, learning rate $x$0, batch size 32, 3 epochs, LoRA, and maximum input length 2048 tokens. To preserve prior entailment knowledge, all layers except the final one were frozen. The selected checkpoint was the one with the highest F1 on the unfaithful class on the dev set.

The architecture and training choices position the model as a compact factuality controller rather than as a large generator. This suggests that factuality supervision can be concentrated in a specialized verifier whose computational role is different from that of the base summarizer.

5. Intrinsic performance on prefix entailment

The main intrinsic evaluation reports micro-averaged F1 for the unfaithful class and compares MiniTruePrefixes with MiniTrue and MiniCheck (Flan-T5).

Model	SummEditsPrefixes	RAGTruthPrefixes
MiniCheck (Flan-T5)	72.9 ± 1.0	33.3 ± 0.5
MiniTrue	69.8 ± 1.2	27.5 ± 0.5
MiniTruePrefixes	78.1 ± 1.0	47.6 ± 0.6

The reported gains are +5.2 F1 and +14.3 F1 over MiniCheck, and +8.3 F1 and +20.1 F1 over MiniTrue, on SummEditsPrefixes and RAGTruthPrefixes respectively (Harary et al., 3 Nov 2025). The appendix also reports faithful-class F1 of 81.4 / 95.0 for MiniTruePrefixes on SummEditsPrefixes and RAGTruthPrefixes, compared with 78.7 / 91.0 for MiniTrue and 76.8 / 89.7 for MiniCheck.

The prefix-length analysis is especially important. In the 0–32% prefix-length bin, MiniTruePrefixes achieves 27.0 F1, whereas MiniTrue achieves 4.9. The paper notes that the improvement persists even for full sentences. This is the clearest evidence that the model is not merely a stronger sentence-level verifier; it is specifically better on incomplete hypotheses.

6. Controlled decoding with prefix entailment

MiniTruePrefixes is integrated into controlled decoding by scoring candidate next tokens before they are committed. At each step, the generator proposes top-$x$1 candidate next tokens. For each candidate token $x$2, the system forms the extended prefix $x$3 and computes

$x$4

If the entailment score is below the rectification threshold $x$5, the token logit is modified as

$x$6

The paper explains that this is the scaled log-odds of entailment; it is negative when $x$7, so it suppresses unfaithful continuations, while tokens with $x$8 are left unchanged (Harary et al., 3 Nov 2025).

The reported hyperparameters are beam size $x$9, top-$y_{1:t}$0, rectification threshold $y_{1:t}$1, scaling factor $y_{1:t}$2, and candidate caps per beam of 20 for LLaMA models, 10 for OLMo-1B, and 7 for OLMo-7B. The paper contrasts this method with lookahead decoding, which first generates a full speculative continuation and then scores it. Prefix scoring avoids that speculative completion and is therefore described as less noisy, more faithful to the actual decoding state, and much faster.

This decoding rule converts prefix entailment from a post hoc verifier into an online control signal. A plausible implication is that the method occupies an intermediate position between reranking and token-level reward shaping.

7. Summarization results, runtime, and limitations

For LLaMA generators on XSum and CNN/DM, the paper reports improved faithfulness under both MiniCheck and GPT-4.1 evaluation. Under the MiniCheck metric, LLaMA-3.2-1B improves from 66.7 / 71.7 to 74.8 / 79.2, LLaMA-3.2-3B improves from 76.2 / 80.1 to 82.4 / 85.8, and LLaMA-3.2-8B improves from 81.5 / 87.9 to 87.0 / 90.8 on XSum / CNN-DM respectively (Harary et al., 3 Nov 2025). GPT-4 faithfulness also improves, with examples including 2.53 → 3.13 on XSum for the 1B model and 3.63 → 3.84 on XSum for the 8B model.

Runtime is a major part of the claim. Lookahead decoding is reported at 114.1s and 158.9s on 1B XSum and CNN respectively, whereas MiniTruePrefixes-guided decoding runs at 4.84s / 5.67s for 1B, 5.43s / 7.18s for 3B, and 7.23s / 9.20s for 8B. The paper states that lookahead is about 25.8× slower than the prefix-based method. It also reports that ROUGE-L changes are small, MAUVE is largely preserved, and that on XSum ROUGE-L slightly decreases, which the authors attribute to reference summaries often being unfaithful.

The abstract states that when guided by MiniTruePrefixes, LLaMA-3.2-3B-Instruct matches the faithfulness and runtime of the 8B model from the same model family, while using only half the memory (Harary et al., 3 Nov 2025). The detailed comparison is more qualified: on XSum, 3B with MiniTruePrefixes reaches 82.4 MiniCheck versus 81.5 for vanilla 8B, with runtime 5.43s versus 5.30s; on CNN/DM, 3B with MiniTruePrefixes reaches 85.8 versus 87.9 for vanilla 8B. This suggests that the strongest version of the claim is dataset-dependent, but the broader reported outcome is that the 3B model closes much of the factuality gap to 8B while remaining more memory-efficient.

The stated limitations are also explicit. The method requires logit access, so it cannot be applied to closed-source APIs that do not expose token logits. It incurs moderate inference overhead because entailment is computed at each decoding step. Training data are English-only, and experiments focus on news summarization. Manual error analysis attributes many failures to generic NLI difficulties such as surface heuristics, reasoning gaps, implicit inference, paraphrase mismatch, and world knowledge, with only a minority due to underdetermined prefixes. These limitations place MiniTruePrefixes within a broader research program: factuality control that is prefix-aware, decoder-compatible, and potentially extensible to token-level reward shaping in future training.