No-Token Training: Adaptive Token Optimization

• No-token training is a family of methods that dynamically reassigns which tokens are predicted, updated, or labeled instead of following a fixed sequential order.
• It leverages techniques such as mutual information–guided reordering, partial-token reinforcement learning, and training-free controls to reduce computational load and improve model performance.
• These approaches demonstrate practical gains in arithmetic, language generation, and vision tasks by treating tokens as adaptive units of information and control.

"No-token training" (Editor's term) denotes a family of methods that relax the default assumption that every token must be processed, supervised, or updated in a uniform full-sequence, left-to-right manner. In the recent literature, this family includes mutual-information-driven target-token reordering during language-model training, policy-gradient estimation from only a selected subset of generated tokens, streaming safeguards that avoid token-level labels, and training-free procedures that exploit latent token-level capabilities already present in frozen models. The common principle is not the elimination of tokens, but the reassignment of which tokens are predicted first, which tokens receive gradient updates, and which token-level signals must be explicitly labeled or learned (Yang et al., 31 Oct 2025, Sang et al., 20 Feb 2026, Fang et al., 10 Feb 2026).

1. Scope, baselines, and problem formulation

The baseline that most directly motivates this area is standard next-token prediction (NTP), in which the model predicts token $x_{t+1}$ from the left context $x_1,\dots,x_t$ with loss

$$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$

This objective fixes both the supervision granularity and the prediction order. Recent work argues that this rigid choice can be suboptimal when token informativeness is not aligned with position, when reinforcement learning over long chain-of-thought trajectories makes every generated token a training cost, or when streaming safety is desired but token-level labels are unavailable (Yang et al., 31 Oct 2025, Sang et al., 20 Feb 2026, Fang et al., 10 Feb 2026).

A useful organizing distinction is between three regimes. First, some methods still train the model, but alter the token order or the subset of tokens that contribute gradients. Second, some methods avoid any additional training and instead exploit latent token-level structure at inference time. Third, some theoretical work studies how training itself changes token dynamics across layers, rather than assuming token behavior is fixed a priori. This suggests that “no-token training” is best interpreted as a token-budget perspective on optimization rather than a literal absence of tokenized computation.

Regime	Core departure	Representative work
Language-model training	Predict information-rich tokens rather than always the next token	(Yang et al., 31 Oct 2025)
RL fine-tuning	Update using only a selected subset of generated tokens	(Sang et al., 20 Feb 2026)
Streaming safety / inference control	Avoid token-level labels or weight updates	(Fang et al., 10 Feb 2026, Han et al., 16 May 2025, Goel et al., 18 Mar 2026)

2. Mutual-information-guided training beyond next-token prediction

"Training LLMs Beyond Next Token Prediction -- Filling the Mutual Information Gap" formalizes a departure from NTP by prioritizing information-rich tokens during training (Yang et al., 31 Oct 2025). The central claim is that the position predicted at each time step in NTP might not be the one with maximal mutual information with the source, so training may emphasize easy or redundant tokens rather than the most informative ones.

Given source tokens $S$ and a target token $t$, the mutual information is defined as

$$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$

For a target sequence of length $L$, the token selection rule is

$t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$

after which the selected token is appended to the source and removed from the remaining target set. The resulting training order is therefore adaptive and determined by informativeness, not by position. For open-vocabulary text generation, MI estimation is approximated with a bigram classifier trained with logistic regression: $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$

Empirically, the method is evaluated on arithmetic, multi-label classification, and natural-language generation. On arithmetic tasks, Max(MI($S;t$)) achieves substantially higher fixed-iteration accuracy than both Plain and Reverse ordering; for 2-digit multiplication it reaches $x_1,\dots,x_t$0 versus $x_1,\dots,x_t$1 for Plain and $x_1,\dots,x_t$2 for Reverse, while the average over the reported arithmetic tasks is $x_1,\dots,x_t$3 for Max(MI($x_1,\dots,x_t$4)) versus $x_1,\dots,x_t$5 for Plain and $x_1,\dots,x_t$6 for Reverse. On WikiText-2, Max(MI($x_1,\dots,x_t$7)) lowers perplexity by an average improvement of approximately $x_1,\dots,x_t$8 compared to plain and TF-IDF ordering. On XSUM summarization, ROUGE-1 is reported as $x_1,\dots,x_t$9 for MI-based order versus $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$0 for Plain. The paper also emphasizes that the optimal order is task-dependent and that there is no single universal best order (Yang et al., 31 Oct 2025).

The broader significance is conceptual as much as empirical. The method keeps cross-entropy at each position, but makes the order of positions dynamically determined by MI. In that sense, it shifts the training problem from “how should the model predict?” to “which token should be predicted now?” The paper describes this as closing or narrowing the “mutual information gap” (Yang et al., 31 Oct 2025).

3. Partial-token reinforcement learning

"Not all tokens are needed(NAT): token efficient reinforcement learning" moves the token-budget question from supervised pretraining to RL fine-tuning (Sang et al., 20 Feb 2026). Its premise is that backpropagation over every generated token creates a hidden tax on long chain-of-thought trajectories, even when rollout generation itself is optimized.

NAT introduces an unbiased partial-token policy-gradient estimator via Horvitz-Thompson reweighting. Let $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$1 indicate whether token $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$2 in trajectory $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$3 is selected, and let $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$4 be its inclusion probability. Given token-level loss $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$5, the Horvitz-Thompson estimator of the mean loss per sequence is

$$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$6

with

$$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$7

The paper instantiates this framework with Uniform Random Sampling (URS), which includes each token independently with fixed probability $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$8, and Random Prefix Cutting (RPC), which keeps a random prefix of length $$\mathcal{L}(\theta) = - \sum_{t=1}^{T} \log P_{\theta}(x_t \mid x_1, \dots, x_{t-1}).$$9 and reweights by the resulting prefix-dependent inclusion probabilities (Sang et al., 20 Feb 2026).

The distinction between URS and RPC is operationally important. URS is unbiased and reduces backward-pass compute and gradient memory almost linearly with $S$0, but does not reduce forward-pass compute or activation memory because causal Transformers still require all prior activations. RPC is also unbiased, but additionally saves both forward and backward compute and memory because only a prefix is processed. Deterministic truncation is used as a contrast case: it saves compute, but introduces systematic bias because suffix tokens are ignored (Sang et al., 20 Feb 2026).

Across mathematical reasoning benchmarks, NAT matches full-token GRPO performance while using as few as $S$1 of tokens. In the reported Qwen3-8B experiments, RPC saves $S$2 peak GPU memory and $S$3 forward and backward RL training time; the tabled example gives $S$4 GB $S$5 GB and $S$6 s $S$7 s. The selected token ratio for RPC is reported at approximately $S$8–$S$9, slightly above the design target of $t$0. The paper further notes that URS increases variance through gradient norm inflation by a factor $t$1, whereas RPC is more stable because the masked tokens are positively correlated through contiguous prefixes (Sang et al., 20 Feb 2026).

Within the no-token-training viewpoint, NAT is notable because it preserves the learning signal of full-sequence RL while explicitly refusing to backpropagate through every token. The token budget becomes a first-class optimization primitive rather than an incidental byproduct of sequence length.

4. Training-free token-level control without new supervision

A parallel research line asks whether token-level behavior can be steered or extracted without any additional training at all. In this regime, the model remains frozen, and the intervention targets attention weights, probing tokens, or latent features rather than parameters.

"ZeroTuning: Unlocking the Initial Token's Power to Enhance LLMs Without Training" identifies the semantically empty initial token as a task-agnostic attention sink and uses it as a control point (Han et al., 16 May 2025). If the attention weights at a decoding step are $t$2, ZeroTuning rescales the initial-token weight by $t$3: $t$4 This preserves the relative ratios among non-initial tokens while sharpening them when $t$5 and flattening them when $t$6. The method profiles heads on a small calibration set and applies head-specific adjustments. Reported gains include $t$7 on classification, $t$8 on QA tasks, and an MT-Bench increase for Llama-3.1-8B from $t$9 to $$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$0. The paper also reports robustness to few-shot settings, long contexts, quantization, decoding strategies, and prompt variations (Han et al., 16 May 2025).

"Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing" shows that frozen LLMs exhibit latent multi-token prediction capabilities even though they were trained solely for next-token generation (Goel et al., 18 Mar 2026). The method appends on-the-fly mask tokens $$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$1 after the prompt, where a strong initialization is the mean-of-prompt embedding,

$$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$2

and can update them during generation via

$$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$3

The model samples Top-$$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$4 candidates from mask-token logits to construct a speculative token tree, prunes redundant paths, and verifies candidates in parallel against the base model’s own next-token distribution, yielding lossless generation. Across SpecBench, the method increases acceptance length by approximately $$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$5 on LLaMA3 and $$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$6–$$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$7 on Qwen3, with throughput gains of up to $$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$8–$$\mathrm{MI}(S; t) = \sum_{s \in V} \sum_{t' \in \tilde V} P(s, t') \log \frac{P(s, t')}{P(s)P(t')}.$$9. The paper also reports that pruning gives up to $L$0 higher acceptance and that GPU-friendly static attention-mask construction yields a further $L$1–$L$2 throughput gain (Goel et al., 18 Mar 2026).

"NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels" targets streaming safety (Fang et al., 10 Feb 2026). It uses pretrained Sparse Autoencoders (SAEs) on LLM hidden states to expose sparse, interpretable features, then selects unsafe features using only sequence-level labels from a calibration dataset. With max-pooled sample-level feature activations, the discrimination score for feature $L$3 is

$L$4

and the online risk score at token $L$5 is

$L$6

Generation is interrupted when $L$7 exceeds a threshold. Reported results give average F1 $L$8 for prompt classification and $L$9 for response classification, exceeding both supervised streaming and post-hoc safeguards in the reported comparisons. The framework is also described as robust across models, SAE variants, and risk scenarios, with mid-to-late layers performing best (Fang et al., 10 Feb 2026).

Taken together, these methods show that the absence of new token-level training does not imply the absence of token-level structure. Instead, latent token-level capabilities can often be read out or modulated directly from a pretrained model.

5. Token-efficient computation beyond decoder-only LLMs

Related work in vision models extends the token-budget perspective to compute reduction rather than language-model supervision. Although these methods are not training procedures for LLMs, they are relevant because they treat tokens as compressible computational objects rather than fixed units that must all be retained.

"Token Transforming: A Unified and Training-Free Token Compression Framework for Vision Transformer Acceleration" recasts token reduction as a matrix transformation

$t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$0

with $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$1, $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$2, and $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$3 (Zeng et al., 6 Jun 2025). In this view, pruning corresponds to special diagonal $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$4, merging to block-diagonal $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$5, and the proposed Token Transforming framework allows fully general many-to-many mappings. Token informativeness is defined by the attention received,

$t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$6

after which similarities and assignment normalization determine the transformation weights. The paper reports reducing $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$7 FLOPs and accelerating DeiT-S by $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$8 with marginal $t^* = \arg\max_{t_i \in T} \mathrm{MI}(S; t_i),$9 accuracy drop, and elsewhere gives up to $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$0 FLOPs reduction from $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$1 to $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$2 GFLOPs. The framework is extended to segmentation, object detection, depth estimation, and LLM generation, all in a training-free manner (Zeng et al., 6 Jun 2025).

"Training-free Token Reduction for Vision Mamba" addresses a different architecture in which attention-based importance scores are unavailable and token order matters (Ma et al., 18 Jul 2025). The key observation is that the Mamba timescale parameter

$$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$3

naturally serves as a structure-aware importance score. With aggregated and averaged importance

$$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$4

MTR sorts tokens by importance, partitions them into Keep, Target, and Source sets, merges each Source token into the most similar Target token, and then restores original sequence order. The framework is training-free, plug-and-play, and parameter-free apart from the chosen compression ratio. Reported results include approximately $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$5 FLOPs reduction on Vim-B with only a $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$6 drop in ImageNet performance without retraining, and a $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$7 drop on VideoMamba-B at the same reduction ratio (Ma et al., 18 Jul 2025).

A plausible implication is that token efficiency is becoming architecture-specific. In ViTs, the dominant issue is information loss under exclusive pruning or merging; in Mamba, the dominant issue is order sensitivity and the absence of attention. This difference cautions against treating token reduction as a single generic recipe.

6. Training-aware theories of token dynamics

Theoretical work on token behavior in Transformers adds a different dimension to the topic: instead of choosing which tokens to supervise, it studies how training reshapes token distributions across depth. "Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers" analyzes a noisy mean-field Transformer in which only a parameter-linear FFN is trained under $$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$8 regularization (Isobe et al., 8 May 2026).

Without FFN training, attention drives token distributions toward clustering through an entropy-regularized interaction energy

$$P(s_i, t_j) = \sigma\!\left(\mathbf{w}_2\big(\mathbf{w}_1[\mathbf{e}_{s_i};\mathbf{e}_{t_j}] + b_1\big) + b_2\right).$$9

Training is formulated as the optimal-control problem

$S;t$0

The main result is a training-induced phase structure with three stages: initial clustering, a turnpike plateau near the clustering well, and terminal escape near the final layers. The paper proves an exponential turnpike estimate,

$S;t$1

and characterizes the energy barrier for clustering escape by

$S;t$2

The theoretical significance is that training and inference cannot always be separated cleanly. Existing mean-field accounts often explain clustering under prescribed parameters; this work shows that learned FFN control can overcome attention-induced clustering near the output when the terminal loss demands it. The paper explicitly relates this to final-layer jumps or shifts in token representations. Within the broader no-token-training discussion, this suggests that token selection, token clustering, and token efficiency are not only algorithmic design choices but also emergent consequences of the interaction between training objectives and layerwise dynamics (Isobe et al., 8 May 2026).

7. Limitations, misconceptions, and open directions

A common misconception is that no-token training means training without tokens. The surveyed literature does not support that interpretation. Instead, it studies settings in which token order is rearranged, only a subset of tokens contributes gradients, token-level labels are avoided, or frozen models expose usable token-level signals without retraining. This suggests that the operative concept is selective token use, not token elimination.

Several limitations recur across the papers. Mutual-information-based token ordering is computationally more expensive because MI calculation is an added preprocessing step; for text generation it therefore relies on approximations such as a bigram classifier trained with logistic regression. The same work notes that MI-based ordering is not always optimal, especially for tasks with strict positional dependencies, and that large instruction-tuned models are sometimes less sensitive to sequence reordering (Yang et al., 31 Oct 2025). NAT preserves unbiasedness, but URS does not reduce forward-pass compute and can become unstable as $S;t$3 decreases because gradient norm inflation scales as $S;t$4; deterministic truncation is faster but biased (Sang et al., 20 Feb 2026). Training-free MTP is lossless only after verification, and in the worst case is no better than standard autoregressive decoding if speculative proposals are not accepted; its dynamic tree expansion is also not fully optimized for dynamic masking or kernels on GPUs (Goel et al., 18 Mar 2026). NExT-Guard avoids token-level supervision, but depends on pretrained SAEs from publicly available base LLMs and achieves the strongest discrimination in mid-to-late layers rather than uniformly across the network (Fang et al., 10 Feb 2026). In vision token reduction, directly applying existing ViT token reduction to Vision Mamba causes significant performance degradation because Mamba lacks attention mechanisms and is sensitive to token order (Ma et al., 18 Jul 2025).

The open directions identified in the papers are correspondingly diverse. They include more efficient MI approximations, extensions to encoder-decoder and non-autoregressive models, and multi-task settings for MI-based reordering; information-aware token importance scheduling beyond URS and RPC in token-efficient RL; further mechanistic analysis of why mask tokens align with future-token states; and training-aware mean-field theories that treat training and inference dynamics together rather than separately (Yang et al., 31 Oct 2025, Sang et al., 20 Feb 2026, Isobe et al., 8 May 2026).

As a research program, no-token training therefore names not a single algorithmic family but a shift in emphasis: tokens are increasingly treated as adaptive units of information, supervision, computation, and control. The technical question is no longer simply how to model token sequences, but which token-level operations are actually necessary for a given objective.