Length Value Model (LenVM): Token Control

• Length Value Model (LenVM) is a framework that treats remaining token count as a value estimation problem for autoregressive generation.
• It integrates methods like EGTP, PLP, and transformer augmentations to enable fine-grained, internal length control during inference.
• Empirical results demonstrate improved token-budgeted reasoning, better exact length matching, and efficient, interpretable control of sequence length.

Length Value Model (LenVM) denotes a set of methods for modeling, predicting, or controlling the number of tokens that an autoregressive model will emit. In its canonical formulation, LenVM treats remaining generation length as a token-level value estimation problem: each generated token receives a constant negative reward, and the model learns a bounded discounted return that is monotonic in the remaining generation horizon (Zhang et al., 29 Apr 2026). Closely related work uses the same label, or an adjacent formulation, for two other mechanisms: lightweight hidden-state-based length forecasting via Entropy-Guided Token Pooling (EGTP) and Progressive Length Prediction (PLP) (Xie et al., 12 Feb 2026), and transformer augmentations that inject a learned length vector into attention and feed-forward sublayers while scaling a small set of length-sensitive units to reproduce internal length-control phenomena observed in LLMs (Moon et al., 27 Jul 2025). Across these usages, LenVM is associated with fine-grained length control, token-budgeted inference, prompt-boundary forecasting, and mechanistic analysis of internal length representations.

1. Terminological scope and problem setting

Across the cited literature, LenVM is unified by a common objective: given a prompt or a partial decoding state, estimate or steer how many tokens remain until generation terminates. This objective matters because generation length directly influences inference cost and reasoning performance, while existing methods are characterized as operating primarily at the coarse-grained sequence level rather than token-by-token (Zhang et al., 29 Apr 2026).

Formulation	Core mechanism	Reported role
Token-level LenVM	Value estimation of remaining horizon	Exact matching, budget control, RL-compatible length signal
EGTP + PLP formulation	Hidden-state reuse with entropy-guided pooling and step-wise prediction	Static scheduling and dynamic remaining-length prediction
Length-vector injection formulation	Learned length embedding injected into attention and FFN, plus top-k unit scaling	Reproducing and manipulating internal length representations

A common misconception is that length control is necessarily an external decoding heuristic or a single pre-decode predictor. The literature here instead locates usable length information inside the model’s own hidden states, attention dynamics, and token-level state transitions. This suggests that LenVM is best understood as a family of internal length-signal models rather than a single fixed architecture.

2. Token-level value estimation of remaining length

The central formulation defines token-level length modeling as a value estimation problem over autoregressive prefixes. For a trajectory of length $L$, each non-terminal step $t=0,\dots,L-1$ is assigned a constant negative reward

$r_t = -(1-\gamma),$

with $r_L=0$ at EOS. The discounted return from step $t$ is

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$

so $G_t \in (-1,0)$ and is strictly monotonic in the remaining length $L-t$. The return satisfies the Bellman recursion

$G_t = r_t + \gamma G_{t+1},$

and under a fixed rollout policy $\pi$, the value function is

$t=0,\dots,L-1$0

For a realized return, inversion back to raw token count is given by

$t=0,\dots,L-1$1

This formulation compresses raw horizons into a bounded interval while preserving order, which is why it is described as a monotone proxy for remaining generation length (Zhang et al., 29 Apr 2026).

Training uses dense Monte Carlo supervision generated from sampled prompt-completion trajectories. For each token position, the target is

$t=0,\dots,L-1$2

paired with the base model’s hidden state $t=0,\dots,L-1$3. The LenVM head is attached to the final transformer layer as an MLP plus sigmoid:

$t=0,\dots,L-1$4

The training loss is the token-wise mean squared error

$t=0,\dots,L-1$5

The resulting supervision is described as annotation-free, dense, unbiased, and scalable, and although a GAE-style variant is possible, full-return regression with $t=0,\dots,L-1$6 is reported to work best (Zhang et al., 29 Apr 2026).

3. Inference-time control, reranking, and RL interfaces

LenVM supports both hard constraints and soft control at inference time. For hard constraints such as the LIFEBench Equal To, At Most, and At Least settings, decoding evaluates candidate next tokens through the hypothetical next-state value $t=0,\dots,L-1$7. Exact matching to a target length $t=0,\dots,L-1$8 selects the token minimizing

$t=0,\dots,L-1$9

whereas At Most selects the token maximizing $r_t = -(1-\gamma),$0, since values closer to $r_t = -(1-\gamma),$1 imply a shorter continuation, and At Least selects the token minimizing $r_t = -(1-\gamma),$2, since more negative values imply a longer continuation (Zhang et al., 29 Apr 2026).

Soft continuous control is implemented through exponential tilting of the next-token distribution. The reweighted distribution solves

$r_t = -(1-\gamma),$3

with closed form

$r_t = -(1-\gamma),$4

Here $r_t = -(1-\gamma),$5 recovers the original model, while $r_t = -(1-\gamma),$6 increasingly biases decoding toward tokens with low, negative value estimates and hence shorter completions. Varying $r_t = -(1-\gamma),$7 traces a smooth Pareto frontier between upstream task quality and expected length.

The same value signal also exposes an RL-compatible interface. Reported uses include an additive objective

$r_t = -(1-\gamma),$8

where $r_t = -(1-\gamma),$9, and potential-based shaping with $r_L=0$0 as the shaping potential. The stated applications include inference-time length control for black-box or white-box LLMs and VLMs without retraining the base weights, token-budgeted reasoning, scheduler or load-balancer support through prompt-boundary length prediction, and dynamic meta-control of compute versus performance (Zhang et al., 29 Apr 2026).

4. Empirical behavior, scaling, and value-space interpretability

On LIFEBench exact length matching, applying LenVM to Qwen2.5-7B-Instruct improves the Equal-To length score from approximately $r_L=0$1 to approximately $r_L=0$2, while reducing Equal-To deviation from approximately $r_L=0$3 to approximately $r_L=0$4; the reported target lengths span $r_L=0$5 to $r_L=0$6 tokens, and the method is stated to outperform GPT-4o, GPT-5.4, Claude, Gemini, and related closed-source systems on exact matching (Zhang et al., 29 Apr 2026). On performance-efficiency trade-offs, LenVM with exponential tilting substantially improves fixed-budget reasoning: on GSM8K, with a Qwen2.5-3B base model and a LenVM(1.5B), an average budget of approximately $r_L=0$7 tokens yields Pass@1 of approximately $r_L=0$8, compared with $r_L=0$9 for a hard token-budget baseline. Similar large improvements are reported on MATH500 and MathVista. For prompt-boundary forecasting, mean relative error improves with scale, ranging from approximately $t$0–$t$1 at $t$2B to approximately $t$3 on math, $t$4 on code, and $t$5 on instruction-following at $t$6B. Validation loss is reported to decrease consistently with increasing model size, number of prompts, and number of completions per prompt.

The learned value function also provides an interpretable token-level diagnostic. The one-step TD residual is defined as

$t$7

A positive residual, $t$8, indicates that the new token increased the estimated remaining horizon, while a negative residual indicates that the token pushed generation toward termination. Tokens frequently associated with large positive residuals include “think,” “ah,” “let,” “consider,” “hmm,” and “try,” which are described as positive markers often associated with new sub-reasoning or pivots. Tokens associated with large negative residuals include “therefore,” “clearly,” “perfect,” the check emoji, the party emoji, line breaks, and answer-closing punctuation. This does not imply that specific lexical items deterministically set length; rather, it shows that the value signal exposes local state changes that correlate with longer or shorter reasoning regimes (Zhang et al., 29 Apr 2026).

5. Internal length representations and transformer-level control

Independent mechanistic work shows that output sequence length is encoded within internal LLM representations and that multi-head attention mechanisms are critical in determining output sequence length (Moon et al., 27 Jul 2025). The reported evidence indicates that length information can be adjusted in a disentangled manner: by scaling specific hidden units within the model, it is possible to control output sequence length without losing the informativeness of the generated text, which is taken to indicate that length information is partially disentangled from semantic information. Additional observations include the fact that some hidden units become increasingly active as prompts become more length-specific, reflecting the model’s internal awareness of that attribute.

A constructed LenVM-style realization of these findings begins from a desired output length $t$9, converts the scalar into a $$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$0-dimensional length vector $$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$1, and injects that vector into both attention and feed-forward sublayers. The scalar is first mapped to a sinusoidal or learned scalar embedding $$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$2 with components

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$3

then projected by a two-layer MLP,

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$4

Within attention, the usual scaled dot-product computation is augmented by length-conditional bias terms

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$5

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$6

and

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$7

The feed-forward sublayer replaces the first linear layer with

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$8

so that the length vector directly biases the intermediate representation.

Fine-grained control is added through a gating mechanism over hidden dimensions. Two gate sets are trained,

$$G_t = \sum_{k=t}^{L} r_k\gamma^{k-t} = -(1-\gamma^{L-t}),$$9

and generation-time scaling is applied to the top-$G_t \in (-1,0)$0 length-sensitive units. The empirical findings reported for this construction are highly specific: the second transformer layer’s attention outputs retain nearly all of the length signal, with $G_t \in (-1,0)$1 in a linear regression of hidden state to token index; only a handful of hidden dimensions, $G_t \in (-1,0)$2, dominate length encoding, and individual dimensions can exceed $G_t \in (-1,0)$3; scaling those top-$G_t \in (-1,0)$4 dimensions by $G_t \in (-1,0)$5 shifts generated length by $G_t \in (-1,0)$6 without reducing Rouge-L by more than $G_t \in (-1,0)$7–$G_t \in (-1,0)$8 points; and human judges report that $G_t \in (-1,0)$9 produces more concise but still informative summaries, whereas $L-t$0 produces longer, more detailed text with only a mild informativity penalty. The specialized gates are additionally reported to remain stable under QLoRA fine-tuning and 4-bit or 8-bit quantization, and to generalize from summarization to machine translation and story generation. These results directly counter the misconception that length control must be achieved by blunt truncation or external penalties alone (Moon et al., 27 Jul 2025).

6. Hidden-state forecasting variants: EGTP, PLP, and serving efficiency

A related line of work reuses the main model’s hidden states for efficient length prediction and describes the combined framework as LenVM in its detailed exposition (Xie et al., 12 Feb 2026). Its static component, Entropy-Guided Token Pooling, computes the token entropy

$L-t$1

converts those entropies into normalized weights

$L-t$2

and pools final-layer token representations into

$L-t$3

A two-headed prediction head then emits a length-bin distribution $L-t$4 and a continuous forecast

$L-t$5

Training uses soft label distributions over bins,

$L-t$6

for the bin $L-t$7 containing the true length, and optimizes the joint loss

$L-t$8

The dynamic component, Progressive Length Prediction, repeats the same light prediction head at every decoding step using a step-wise input

$L-t$9

to estimate the remaining length, with training objective

$G_t = r_t + \gamma G_{t+1},$0

The combined objective is written as

$G_t = r_t + \gamma G_{t+1},$1

This formulation is explicitly motivated by length-aware batching and scheduling with almost zero extra cost. On Qwen2.5-7B, EGTP is reported to run in $G_t = r_t + \gamma G_{t+1},$2 ms on an RTX 4090, compared with $G_t = r_t + \gamma G_{t+1},$3–$G_t = r_t + \gamma G_{t+1},$4 ms for baselines that require a separate DistilBERT or OPT forward pass; PLP adds another $G_t = r_t + \gamma G_{t+1},$5 ms per decode step; and memory overhead is approximately $G_t = r_t + \gamma G_{t+1},$6 MB, compared with $G_t = r_t + \gamma G_{t+1},$7–$G_t = r_t + \gamma G_{t+1},$8 MB for auxiliary predictors. On the ForeLen benchmark, EGTP achieves a $G_t = r_t + \gamma G_{t+1},$9 average MAE reduction over TRAIL and $\pi$0 over SSJF-Reg; on Qwen2.5 3B, mean MAE drops from $\pi$1 to $\pi$2 tokens, and on Llama3.2 3B from $\pi$3 to $\pi$4. In end-to-end vLLM serving with a shortest-job-first scheduler, padding ratio is reduced from $\pi$5 to $\pi$6, job completion time from $\pi$7 ms to $\pi$8 ms for long sequences, and throughput increases by over $\pi$9. When PLP is added in RL sampling workloads, remaining-length MAE drops from $t=0,\dots,L-1$00 to $t=0,\dots,L-1$01, a further $t=0,\dots,L-1$02 improvement. The reported limitations include handling extremely out-of-distribution contexts beyond $t=0,\dots,L-1$03k tokens and further reducing per-step overhead in very high-throughput RL pipelines (Xie et al., 12 Feb 2026).