---
title: Token Probability Deviation in LM Research
url: https://www.emergentmind.com/topics/token-probability-deviation-tbd
type: topic
---

# Token Probability Deviation in LM Research

Token Probability Deviation (TBD) is a non-unified term in recent language-model research. It is defined explicitly in reasoning-distillation auditing as a score that measures how far generated token probabilities fall below a high reference probability, with lower scores for questions likely seen during distillation [2510.04850]. In adjacent literatures, closely related constructs quantify deviation from a reference policy at the token level, deviation between correct and distractor option probabilities, deviation from an evaluator model’s preferred token, or deviation between model-assigned and theoretically expected probabilities [2506.14574][2501.06468][2405.19648][2511.00620]. Taken together, these uses indicate that TBD is presently better understood as a family of token-distribution diagnostics than as a single canonical metric.

## 1. Formalization in distillation data detection

The most explicit definition of TBD appears in work on detecting whether a question was used in reasoning distillation [2510.04850]. For a question \( q \), a distilled model generates a response sequence \((y_1, y_2, \ldots, y_M)\), and at each step assigns probability
\[
p_\theta(y_i \mid y_{<i}, q).
\]
TBD measures how far each generated token falls below a high reference value \( \tau \), typically close to \(1\), through the token-level deviation
\[
d_i(q; \tau) = \max\Big(0,\, \tau - p_\theta(y_i \mid y_{<i}, q)\Big).
\]
The sample-level score is then
\[
\mathcal{S}(q,\theta) = \frac{1}{E} \sum_{i=1}^{M} d_i(q;\tau)^{\alpha},
\]
where \( \alpha > 0 \) controls the emphasis on larger deviations and
\[
E = \sum_{i=1}^{M} \mathbf{1}(p_\theta(y_i \mid y_{<i}, q) < \tau)
\]
counts outlier tokens. Classification is threshold-based:
\[
G(q; \mathcal{M}) =
\begin{cases}
1 & \text{if } \mathcal{S}(q, \mathcal{M}) < \lambda \quad (\text{member}) \\
0 & \text{otherwise} \quad (\text{non-member}).
\end{cases}
\]
The practical hyperparameter choices reported are \( M = 300 \), \( \tau = 1 \), and \( \alpha = 0.6 \) [2510.04850].

The underlying empirical observation is that distilled models tend to generate near-deterministic tokens for seen questions, while producing more low-probability tokens for unseen questions. In this setting, lower TBD scores indicate likely members and higher scores indicate non-members. On the S1 dataset, TBD achieved an AUC of \(0.918\) and a TPR@1% FPR of \(0.470\); the best baseline on the same table reached \(0.847\) AUC and \(0.160\) TPR@1% FPR [2510.04850]. The paper further reports that input-token-based baselines perform near random in the question-only regime, whereas simple generated Min-K\% or perplexity improve over that baseline family but still lag behind TBD.

A notable design choice is the restriction to the first \( M \) generated tokens and to outlier tokens with probability below \( \tau \). The reported ablations state that truncation, token-deviation focus, and deviation exponentiation each improve performance [2510.04850]. This formulation makes TBD a black-box, question-only detector whose signal arises entirely from output-token confidence structure rather than from access to the hidden distillation corpus.

## 2. Decoding, expected values, and output-space analysis

A separate line of work argues that standard decoding discards useful information in the full next-token distribution [2406.10267]. On Likert-scale scoring tasks with options \(A=1, B=2, \ldots, E=5\), the paper contrasts greedy decoding,
\[
s_{max} = F(\arg\max_{t \in \{A, B, C, D, E\}} p(t)),
\]
with the expected score
\[
E(s) = p(A)\cdot 1 + p(B)\cdot 2 + p(C)\cdot 3 + p(D)\cdot 4 + p(E)\cdot 5.
\]
The reported issue is that at usual temperatures (\(T \leq 1\)) the model is spuriously overconfident, so \(E(s) \approx s_{max}\). The proposed remedy is to scale logits with large temperature, specifically \(T=10\), to increase entropy and spread probability mass across more options [2406.10267].

On the SummEval dataset of 1600 annotated summaries, this high-entropy expected-value decoding yields large gains in correlation with human judgment. Reported examples include Mistral 7B relevance increasing from \(6.4\%\) to \(28.4\%\), Mixtral 8x7B relevance increasing from \(46\%\) to \(56\%\), and SOLAR 10.7B relevance increasing from \(19\%\) to \(43\%\); quantized models are described as only marginally worse than their full-precision counterparts [2406.10267]. The same study notes that part of the gain seems related to positional bias and that the expected-value method is more robust to answer-order effects than greedy decoding.

The paper also introduces a probability-based tree sampling algorithm that systematically enumerates highly probable completions using a top-\(P\) cumulative probability threshold \((\hat{p})\), a loglikelihood threshold \((\alpha)\), and designated stop tokens [2406.10267]. In creative paraphrasing experiments with Mixtral Instruct at \(T=2\) and top-\(p=0.9\), only \(1\)-\(2\) paraphrases were produced with substantial probability, while the remaining tail typically devolved into irrelevant or rambling outputs. The paper links this output collapse to what it calls a potential source for Token Probability Deviation, in the sense that empirical token distributions can deviate from more natural expectations of diversity [2406.10267].

## 3. Token-level alignment and preference optimization

In supervised fine-tuning, token probability is also used as a proxy for semantic importance. ProFit reports a strong correlation between a token’s predicted probability and its semantic importance, with core tokens assigned consistently higher predicted probabilities and trivial tokens concentrated in the low-probability tail; the paper reports a hypothesis test with \( p = 1 \times 10^{-6} \) [2601.09195]. Its binary masking operator is
\[
\mathcal{M}_t = \mathbb{I}\left[ \mathrm{sg}(\pi_{\theta}(y_t^* \mid x, y^*_{<t})) > \tau \right],
\]
and the modified objective is
\[
\mathcal{L}_{\mathrm{ProFit}}(\theta) = \mathbb{E}_{\mathcal{D}} \left[ - \frac{1}{T} \sum_{t=1}^{T} \mathcal{M}_t \log \pi_{\theta}(y_t^* \mid x, y^*_{<t}) \right].
\]
The paper’s interpretation is that low-probability tokens induce disproportionately large gradients and can overshadow learning of core semantics. On Qwen3-4B-Base, ProFit reports \(52.33\%\) average accuracy versus \(41.39\%\) for standard SFT, and on Llama-3.1-8B it reports a \(+9.97\%\) gain over vanilla SFT [2601.09195].

Within preference optimization, TGDPO makes token probability deviation explicit as deviation from a reference policy [2506.14574]. For token \( y_t \) at position \( t \), the base deviation is
\[
\log \frac{\pi(y_t \mid [x, y_{<t}])}{\pi_{ref}(y_t \mid [x, y_{<t}])}.
\]
TGDPO assigns token-dependent weights derived from token-level reward guidance, using
\[
f_w([x, y_{<t}], y_t) = 1 + \alpha r([x, y_{<t}], y_t), \qquad
f_l([x, y_{<t}], y_t) = 1 - \alpha r([x, y_{<t}], y_t),
\]
and the full loss in Equation (20) applies these weights to the tokenwise log-ratios for winning and losing responses. The paper states that this is the mathematical quantification of token probability deviation in its framework. It reports win-rate gains of up to \(7.5\) points on MT-Bench, \(6.2\) points on AlpacaEval 2, and \(4.3\) points on Arena-Hard [2506.14574].

TokenRatio extends the same token-level concern to pairwise preference learning by introducing Token-level Bregman Preference Optimization (TBPO) [2605.12288]. The paper argues that sequence-level DPO leaves per-prefix optimality implicit and that large TBD means the policy is not locally optimal at each decision point. TBPO instead posits a token-level Bradley–Terry preference model and derives a Bregman-divergence density-ratio matching objective. Its two practical variants are TBPO-Q, which learns a lightweight state baseline, and TBPO-A, which removes the baseline through advantage normalization. The paper reports improved alignment quality, improved training stability, and increased output diversity relative to strong sequence-level and token-level baselines [2605.12288]. This suggests that, in alignment settings, TBD functions as a credit-assignment problem over tokenwise policy shifts.

## 4. Retrieval, hallucination detection, and evaluation

In retrieval-augmented multiple-choice QA, first-token probability is used as a confidence score that guides dynamic context adjustment [2501.06468]. For options \( \mathcal{O} = \{o_1, o_2, \ldots, o_n\} \), the model computes
\[
P(\text{token}=o_i \mid P(Q, K)) = \mathrm{softmax}(l_{o_i}),
\]
normalizes across valid options to obtain \( \mathbf{p} = [p_1,\ldots,p_n] \), and uses
\[
c = \max_i p_i
\]
as the confidence score. If \( c < \theta \), the system adjusts chunk number \(k\) and chunk window size \(w\); alternatively, a best-probability search selects \( h^* = \arg\max_{h \in \mathcal{H}} c_h \). The paper frames the gap between correct-option and distractor probabilities as what may more generally be studied as Token Probability Deviation in RAG [2501.06468]. Reported results include \(80\%+\) accuracy on filtered questions at \( \theta = 0.5 \), about 250 answered questions above that threshold, and up to \(78.4\%\) accuracy with combined strategies, a \(26.8\%\) increase over the baseline [2501.06468].

For hallucination detection, one supervised approach reduces the problem to four scalar features computed by an evaluator model under teacher forcing [2405.19648]. The four features are minimum token probability,
\[
\mathrm{mtp} = \min_{1 \leq i \leq n} P_{LLM_E}(t_i),
\]
average token probability,
\[
\mathrm{avgtp} = \frac{1}{n} \sum_{i=1}^{n} P_{LLM_E}(t_i),
\]
maximum LLM probability deviation,
\[
\mathrm{Mpd} = \max_{1 \leq i \leq n} \left[ P_{LLM_E}(v^*) - P_{LLM_E}(t_i) \right],
\]
and minimum LLM probability spread,
\[
\mathrm{mps} = \min_{1 \leq i \leq n} \big( P_{LLM_{E_i}}(v^*) - P_{LLM_{E_i}}(v^{-}) \big).
\]
The paper states directly that Maximum LLM Probability Deviation implements the notion of token probability deviation by measuring how much less probable the generated token was, according to the evaluator, than the evaluator’s own most probable token [2405.19648]. Using only these four features, logistic regression and a simple neural network surpass state-of-the-art results on multiple tasks across HaluEval, remain competitive on HELM, and underperform on the True-False benchmark, illustrating both the usefulness and the limits of shallow token-probability features.

SPAD pushes hallucination analysis toward mechanistic attribution by decomposing each token probability into seven sources: Query, RAG, Past, Current Token, FFN, Final LayerNorm, and Initial Embedding [2512.07515]. The final probability for token \(y\) is exactly decomposed as
\[
P_{\text{final}}(y)
=
P_{\text{initial}}(y)
+
\Delta P_{\text{LN}}
+
\sum_{l=1}^{L}
\left(
\Delta P_{\text{ffn}^{(l)}} +
\sum_{S \in \{Q,R,P,S\}} \Delta P^{(l)}_S
\right).
\]
SPAD then aggregates token-level attribution vectors by POS tags and treats deviations from normal source-POS patterns as hallucination signals. The paper gives the example that nouns should normally rely on RAG, whereas high FFN or Final LayerNorm attribution on NOUNS is suspicious [2512.07515]. Reported results include \(F1 = 0.7912\), \(AUC = 0.8685\) on RAGTruth with Llama2-13B, and \(F1 = 0.7907\), \(AUC = 0.7848\) on Dolly with Llama2-13B, outperforming baselines such as ReDeEP [2512.07515].

Bias evaluation provides another deviation-based use of token probabilities. In work comparing token probability and textual entailment as bias metrics, the term TBD is not explicitly formalized, but the raw deviation
\[
\Delta_{TP} = \mathrm{score}(S) - \mathrm{score}(\tilde{S})
\]
is described as central to fine-grained analysis [2510.07662]. The same paper reports very low correlation between token-probability and NLI bias metrics, with maximum \(R^2 = 0.328\), and concludes that neither token probability nor NLI is uniformly superior [2510.07662]. Here, deviation captures stereotype preference at the sentence-pair level rather than membership, calibration, or alignment.

## 5. Related probability-distance phenomena

Several recent studies analyze token-probability deviations without using TBD as a single named metric. In controlled probabilistic scenarios, GPT-4.1 and DeepSeek-Chat achieve \(100\%\) validity on prompts such as coin flips and die rolls, yet their token-level probabilities and entropies diverge systematically from theoretical distributions [2511.00620]. For a six-sided die, the theoretical probability is \(0.167\); the paper reports \(0.96\) for GPT-4.1 and \(0.593\) for DeepSeek-Chat under specified prompts, with entropy errors greater than \(30\%\) in all cases and close to \(100\%\) for the coin flip scenario [2511.00620]. The paper’s central claim is that certainty inferred from logits does not imply alignment with the theoretical probability distribution of the task.

Prompt framing alone can also induce measurable probability divergence. In a production–perception study, the same poem tokens are re-scored under production-oriented and perception-oriented prompts, and token probability distance is measured by
\[
\overline{|p_a - p_b|} = \frac{1}{n} \sum_{i=1}^{n} |p_{a,i} - p_{b,i}|.
\]
Across five open-weight models, production–perception distances consistently exceed production–production distances, with an overall average ratio of approximately \(1.8\) [2607.11703]. For Llama-3.1-8B, the reported mean absolute distances are \(0.034\) for production–perception and \(0.010\) for production–production, and the temporal analysis models decay as \( y = a e^{-bx} + c \) [2607.11703]. This constitutes a prompt-conditioned probability deviation over the same generated text rather than a deviation tied to truth, preference, or membership.

At the representation level, token probability information appears to be encoded directly in model parameters. Work on output embeddings reports an approximate common log-linear encoding of output token probabilities,
\[
-\log \alpha_{w,\mathcal{D},\theta} \approx A_\mathcal{D} \cdot E_w^{(o)} + B_\mathcal{D},
\]
with adjusted \(R^2 \approx 0.89\) for GPT2 and \(R^2 \approx 0.88\) for GPT-J, and with more than \(30\%\) of output-embedding dimensions deletable without significant movement in output distribution and sequence generation [2406.01468]. This suggests that many TBD-like observables may have a sparse geometric substrate in the output embedding space.

Token-probability structure has also been lifted into spatial detectors. CodeVision converts line-by-line code log probabilities into a matrix \( M \in \mathbb{R}^{n \times m} \), where \( M_{ij} \) is the log probability of the \(j\)-th token in the \(i\)-th line, and then treats this matrix as a 2D token probability map for ViT or ResNet classification [2501.03288]. In that work, token probability deviation is not a single scalar; rather, the detector learns spatial patterns of confidence, indentation, and local irregularity from the map itself.

## 6. Methodological cautions and standardization issues

Recent work on decoder monitoring argues that not every token-probability observable is diagnostically meaningful [2607.11317]. The paper shows that the centered token log-probability increment
\[
D_t = \log p(w_t) + H_t
\]
is a mean-zero martingale under the model’s own sampling law because
\[
\mathbb{E}[D_t \mid \mathcal{F}_{t-1}] = -H_t + H_t = 0.
\]
As a result, it measures sampling self-consistency rather than trajectory health and is nearly silent during confident repetition, where both \( \log p(w_t) \) and entropy are close to zero. The proposed replacement is a degeneration-aware alarm score,
\[
a_t = \min\left(1,\, w_{\mathrm{rep}} \cdot r_t + w_{\mathrm{ent}} \cdot u_t\right),
\]
with \( w_{\mathrm{rep}}=0.7 \) and \( w_{\mathrm{ent}}=0.3 \), combined with an e-process-inspired detector. On GSM8K with DeepSeek-R1-Distill-Qwen-1.5B in FP16 and INT4, calibration turns a monitor that fires on \(93\)-\(95\%\) of generations into a more selective detector, with \( \phi \approx 0.3 \), precision \( \approx 0.6 \) against a \(0.38\) base rate, and a positive but statistically inconclusive INT4 accuracy change from \(63\%\) to \(69\%\) at a \(28\%\) token-budget cost [2607.11317].

This caution matters for TBD because the literature uses “deviation” in several incompatible ways. In the distillation-detection setting, deviation is distance below a high-certainty reference [2510.04850]. In alignment, it is deviation from a reference policy weighted by token-level reward [2506.14574]. In retrieval and bias measurement, it can be the gap between option probabilities or paired sentence scores [2501.06468][2510.07662]. In hallucination detection, it may be disagreement with an evaluator model or anomalous source attribution for a POS category [2405.19648][2512.07515]. In probabilistic calibration studies, it is the divergence between model confidence and known theoretical probabilities [2511.00620].

A plausible implication is that TBD has become a useful cross-cutting lens for analyzing token distributions, but not yet a standardized object with fixed semantics, aggregation rules, or null hypotheses. The current literature supports at least four distinct functions: auditing training-data exposure, improving decoding and scoring by using the full token distribution, shaping token-level optimization during post-training, and detecting reliability failures such as hallucinations or degenerate generation [2510.04850][2406.10267][2601.09195][2607.11317]. Further standardization would likely require explicit separation between uncertainty, calibration, memorization, preference shift, and structural attribution, since current papers show that these phenomena can all be expressed as “token probability deviation” while remaining technically non-equivalent.

Source: https://www.emergentmind.com/topics/token-probability-deviation-tbd