Spontaneous Scale-Dependent Verbosity

• The paper demonstrates that overelaboration in larger models leads to lower accuracy on inverse-scaling tasks, with small models outperforming by up to 28.4 percentage points.
• Spontaneous scale-dependent verbosity is defined as a behavior where increased model size produces longer, sometimes error-prone outputs under neutral prompting conditions.
• Controlled brevity constraints improve performance by reducing extraneous tokens, unveiling the latent capability of larger models without sacrificing useful reasoning.

Searching arXiv for the cited papers on spontaneous scale-dependent verbosity and verbose CoT. Spontaneous scale-dependent verbosity denotes a scale-linked generation pattern in LLMs in which larger models, under neutral prompting, tend to produce longer and more elaborate responses than smaller models, and this additional elaboration can either degrade performance through overreasoning or yield only modest gains unless the extra tokens carry useful reasoning and validation content (Hakim, 11 Mar 2026). In the literature, the term is used in two closely related but distinct senses. One sense identifies a harmful failure mode under standard benchmark prompting, where verbosity emerges without explicit chain-of-thought instruction and can reverse apparent capability rankings across model sizes (Hakim, 11 Mar 2026). The other sense treats verbosity as an experimental variable in chain-of-thought studies and asks whether longer traces help merely by being longer; the available evidence argues against a pure length-based account and attributes any benefit primarily to semantic reasoning content, especially checking and validation steps (Wang et al., 29 Jun 2026).

1. Definition and conceptual scope

In the benchmark-evaluation setting, spontaneous scale-dependent verbosity refers to a pattern where larger models produce outputs that are too verbose for the task, often with implicit reasoning that becomes error-prone (Hakim, 11 Mar 2026). The phenomenon is termed “spontaneous” because it appears under intentionally bare prompts rather than under explicit chain-of-thought elicitation. The canonical example is a control prompt of the form Problem: {problem_text}\n\nSolution: for GSM8K, or simple question/answer templates for multiple-choice tasks; under these conditions, any extra elaboration is treated as an intrinsic, scale-dependent generation tendency rather than a prompt artifact (Hakim, 11 Mar 2026).

A distinct but related usage appears in work on verbose chain-of-thought. There, the question is whether longer reasoning traces help because they contain useful intermediate reasoning or because extra tokens provide more serial computation before answer commitment (Wang et al., 29 Jun 2026). The relevant contrast is between a semantic-trace hypothesis, according to which chain-of-thought helps because the trace explicitly contains useful intermediate reasoning, and a forward-pass-computation hypothesis, according to which extra tokens help because they buy additional serial computation (Wang et al., 29 Jun 2026). The evidence favors a content-driven view with a possible secondary role for prose quality, rather than a pure length/computation account (Wang et al., 29 Jun 2026).

Taken together, these uses identify a common theme: response length is not an autonomous proxy for capability. A plausible implication is that verbosity is best understood as a task- and scale-sensitive behavioral tendency whose effect depends on the semantic function of the added text rather than on token count alone.

2. Emergence under standard prompting

The strongest formulation of the harmful-verbosity account comes from the study of inverse scaling under universal prompts. That work evaluates 31 models spanning 0.5B to 405B parameters on 1,485 problems from GSM8K, BoolQ, ARC-Easy, CommonsenseQA, and MMLU-STEM, using greedy decoding only with do_sample=False, top-p disabled, top-k disabled, repetition penalty = 1.0, and max output tokens = 512 (Hakim, 11 Mar 2026). Models are grouped as small when $N \le 10$B and large when $N \ge 70$B (Hakim, 11 Mar 2026).

Within this setup, the paper identifies 115 inverse-scaling problems, or 7.7% of all evaluated items, on which smaller models outperform larger ones (Hakim, 11 Mar 2026). On those problems, the reported average advantage is 28.4 percentage points in favor of small models, with Cohen’s $d = 1.34$, and the aggregate comparison gives 66.1% accuracy for small models versus 41.5% for large models, a 24.6-point degradation (Hakim, 11 Mar 2026). The paper also reports a continuous scale–performance relation on inverse problems with Pearson $r = -0.388$, $p = 0.0035$, indicating that performance tends to worsen as parameter count increases across the full spectrum (Hakim, 11 Mar 2026).

The proposed mechanism is overelaboration. Large models on inverse-scaling problems produce 202 tokens on average, compared with 127 tokens for small models, while producing slightly fewer explicit reasoning steps: 9.1 steps for large models versus 10.5 steps for small models (Hakim, 11 Mar 2026). This distinction is central. The phenomenon is not described as “more reasoning steps” in a narrow explicit sense; instead, it is characterized as more expansive implicit reasoning, explanation, qualification, and elaboration (Hakim, 11 Mar 2026). The paper further reports that response length is negatively correlated with large-model accuracy on inverse-scaling problems, with $r = -0.43$ (Hakim, 11 Mar 2026).

This suggests that the critical variable is not simply whether a model reasons, but whether its default style becomes overprocessed relative to task requirements. The paper explicitly frames the issue as a prompt-sensitivity problem rather than an intrinsic inability, arguing that larger models’ competence is being masked by universal prompting that does not match the model’s scale (Hakim, 11 Mar 2026).

3. Operationalization and empirical signatures

The literature operationalizes the phenomenon through response length, accuracy, problem-level inverse-scaling gaps, and controlled comparisons of concise versus verbose traces. In the benchmark study, response length is measured as the token count of the generated output, excluding prompt tokens, summarized by model category through the average response length $\bar{L}$, and length diversity in contamination analysis is quantified with the coefficient of variation $\mathrm{CV}_i = \sigma_{L_i}/\mu_{L_i}$ (Hakim, 11 Mar 2026). Accuracy is defined as

$$\text{Acc}_m = \frac{1}{N} \sum_{i=1}^{N} \mathbb{1}[\hat{y}_{m,i} = y_i],$$

and the inverse-scaling gap for a problem is

$$\Delta_i = \text{Acc}_{\text{small},i} - \text{Acc}_{\text{large},i},$$

so that positive $N \ge 70$0 indicates superior small-model performance (Hakim, 11 Mar 2026).

In the chain-of-thought study, the principal effect size is instead

$N \ge 70$1

This quantity is used in two experimental legs: an in-distribution same-model comparison between shorter and longer natural generations on the same question, and a controlled intervention in which concise and verbose traces are rewritten while holding the underlying computation fixed (Wang et al., 29 Jun 2026).

For the in-distribution leg, the crucial criterion is that paired traces must follow the same reasoning plan. On GSM8K, the paper applies a pre-filter based on a computation-graph signature, specifically a Weisfeiler–Lehman hash over the arithmetic operation graph, matching traces that perform the same operations in the same dependency structure regardless of numeric literals (Wang et al., 29 Jun 2026). A local LLM judge, E3, then confirms same-plan equivalence on number-redacted traces; on MATH-500, E3 alone is used (Wang et al., 29 Jun 2026). Pairing additionally requires a generated-token ratio $N \ge 70$2 (Wang et al., 29 Jun 2026).

For the controlled intervention, semantic equivalence is defined through directed acyclic graph (DAG) equivalence. Two arithmetic traces are DAG-equivalent if they contain the same computational nodes, in the same dependency order, with identical intermediate values, regardless of whether they are expressed concisely or verbosely (Wang et al., 29 Jun 2026). Validation is carried out with two validators: E2, an algorithmic, DAG-informed validator for arithmetic benchmarks only, calibrated via logistic regression on six features, and E3, a local LLM judge that returns one of {equivalent, not_equivalent, ambiguous} with confidence on number-redacted traces (Wang et al., 29 Jun 2026). Statistical procedures include question-clustered bootstrap in Leg 1, paired bootstrap in Leg 2, 1,000-sample or 10,000-sample bootstrap confidence intervals depending on the leg, stratification by source × anchor correctness × validator, and cross-checks with a frontier judge (Claude Opus 4.8) and a non-Qwen judge (Yi-1.5-9B) for robustness (Wang et al., 29 Jun 2026).

4. Causal interventions: brevity constraints and controlled verbosity

The causal evidence for harmful spontaneous scale-dependent verbosity comes from explicit brevity constraints imposed on inverse-scaling problems (Hakim, 11 Mar 2026). The intervention is run on all 115 inverse-scaling problems using seven models: 3 small models—Qwen2.5-0.5B, Llama-3.2-3B, Gemma-2-2B—and 4 large models—Llama-3.1-70B, Llama-3.1-405B, Qwen2.5-32B, DeepSeek-67B (Hakim, 11 Mar 2026). Three conditions are compared: Control, Brief, and Direct (Hakim, 11 Mar 2026). The Brief condition specifies task-specific limits: for GSM8K, under 50 words; for BoolQ, 10 words or less; and for multiple-choice tasks, just the letter plus one sentence (Hakim, 11 Mar 2026). The paper emphasizes that outputs are not hard-truncated; the brevity reduction is an empirical consequence of instruction-following (Hakim, 11 Mar 2026).

Under control prompts, large models underperform small models by 44.2 percentage points, with 40.2% accuracy for large models and 84.4% for small models (Hakim, 11 Mar 2026). Under brief prompts, the gap shrinks by 67% to 14.8 points: large models improve by +26.3 points, while small models decline only −3.1 points (Hakim, 11 Mar 2026). Under direct prompts, the gap narrows further to 7.8 points, although both groups lose some accuracy relative to the brief condition, implying that some reasoning remains useful even when excessive reasoning is harmful (Hakim, 11 Mar 2026). The paper reports paired $N \ge 70$3 with $N \ge 70$4 (Hakim, 11 Mar 2026).

These behavioral changes are accompanied by large length reductions. For large models, median output length falls from 197 tokens to 78 tokens under brief prompts, a 60.4% reduction, and to 57 tokens under direct prompting, a 71.1% reduction (Hakim, 11 Mar 2026). Small models reduce length by only about 15%, whereas large models reduce it by about 60%, which the paper interprets as evidence that verbosity is scale-dependent (Hakim, 11 Mar 2026).

The controlled-intervention evidence on verbose chain-of-thought yields a different but complementary result. When semantic content is held fixed via DAG equivalence, verbose traces do improve accuracy, but the effects are modest: 25 of 32 benchmark-target cells are positive under at least one validator, with typical effect sizes of 1–4 percentage points under E3, while E2 estimates are generally 3–4× larger than E3 (Wang et al., 29 Jun 2026). The interpretation is therefore not that verbosity is useless, but that whatever benefit exists depends on the quality and structure of the prose rather than on token count alone (Wang et al., 29 Jun 2026).

5. Dataset patterns, reversals, and boundary conditions

The benchmark evidence shows that spontaneous scale-dependent verbosity is not uniformly distributed across tasks. Among the 115 inverse-scaling problems, dataset-specific prevalence is reported as BoolQ: 34/300 = 11.3%, CommonsenseQA: 29/300 = 9.7%, ARC-Easy: 28/300 = 9.3%, GSM8K: 13/300 = 4.3%, and MMLU-STEM: 11/285 = 3.9% (Hakim, 11 Mar 2026). The paper also reports dataset-specific “optimal scales” ranging from 0.5B for BoolQ and MMLU-STEM to 3.0B for GSM8K (Hakim, 11 Mar 2026). This indicates that the best-performing model scale is not constant across benchmarks.

The most prominent finding is that brevity constraints can reverse performance hierarchies. On GSM8K, the control gap is +13.1 pp favoring small models, but the brief gap becomes −7.7 pp favoring large models (Hakim, 11 Mar 2026). On MMLU-STEM, the control gap is +27.3 pp, and the brief gap becomes −15.9 pp (Hakim, 11 Mar 2026). Strong but incomplete reductions are also reported for ARC-Easy, where the gap is reduced by 73.8% from 71.4 pp to 18.8 pp, and for CommonsenseQA, where the reduction is 61.5%, from 56.0 pp to 21.6 pp (Hakim, 11 Mar 2026). BoolQ is the explicit exception: its gap slightly increases from 23.5 pp to 24.3 pp, and the paper interprets this as evidence that brevity is not universally beneficial because some elaboration can be functional on passage-based tasks (Hakim, 11 Mar 2026).

The chain-of-thought study also emphasizes boundary conditions. In the in-distribution same-plan leg, for independently trained reasoners, once the reasoning plan is fixed, extra tokens do essentially nothing: pooled MATH-500 gives $N \ge 70$5 with confidence interval $N \ge 70$6, which the paper describes as essentially zero (Wang et al., 29 Jun 2026). Two exceptions are reported: DeepSeek-R1-Distill released weights, which show a small positive effect on MATH-500 of +0.019 with confidence interval $N \ge 70$7 and +0.046 on GSM8K in one arm; and non-reasoners on GSM8K, such as mistral-7B at +0.079 and qwen3-0.6B-nothink at +0.046 (Wang et al., 29 Jun 2026). The authors regard these cases as idiosyncratic rather than general across independently trained reasoners (Wang et al., 29 Jun 2026).

A further boundary condition comes from maximum numerical redaction, where all numbers are replaced by <NUM>. Under this intervention, the verbose-helps effect is amplified by a median factor of $N \ge 70$8 across four arithmetic benchmarks, 30 of 32 cells preserve sign, and 28 of 32 remain confidence-interval significant (Wang et al., 29 Jun 2026). The paper provides examples including SVAMP × Qwen3-4B self, moving from near-zero to +0.226, and OLMo on MultiArith shared, increasing from +0.169 → +0.547 (Wang et al., 29 Jun 2026). The same study also reports that on StrategyQA, verbose chain-of-thought can hurt for some models, underscoring that the effect is not universal (Wang et al., 29 Jun 2026).

6. Mechanisms and interpretive disputes

Two competing explanations structure the current interpretation of the phenomenon. The first is a length-based or forward-pass-computation account, according to which longer generations help because they postpone answer commitment and provide more serial computation (Wang et al., 29 Jun 2026). The second is a content-based account, according to which extra tokens help only insofar as they encode useful reasoning operations, intermediate values, or verification procedures (Wang et al., 29 Jun 2026).

The evidence weighs against a pure length-based interpretation. In the same-plan in-distribution analysis, fixing the reasoning plan largely eliminates any benefit of length for independently trained reasoners (Wang et al., 29 Jun 2026). The study further reports that a blind analysis of surplus tokens shows that pure verbosity—described as “elaborate/restate”—never robustly distinguishes winners from losers, whereas validation/checking content, specifically “verify/check” and “plug-in/test,” does (Wang et al., 29 Jun 2026). In the oracle-trace experiment, padding L1 with fluent non-reasoning filler leaves accuracy near L1 and does not approach L4 performance, while a length-matched semantic condition remains significantly better (Wang et al., 29 Jun 2026). The paper therefore concludes that token count alone is insufficient (Wang et al., 29 Jun 2026).

The benchmark study makes a parallel argument in a different register. There, larger models are not described as lacking capability; rather, their capability is obscured by a generation style that is too elaborate under universal prompts (Hakim, 11 Mar 2026). The authors speculate that RLHF-style training may encourage length and “helpfulness,” and that larger models may be more likely to internalize and enact those signals, but they do not claim to have fully identified the root cause (Hakim, 11 Mar 2026). Their key conceptual point is that larger models may produce responses that appear more careful or complete yet are overprocessed for some benchmark items (Hakim, 11 Mar 2026).

The two papers are therefore compatible. One shows that unprompted verbosity in larger models can be causally harmful under standard evaluation (Hakim, 11 Mar 2026). The other shows that, when verbosity is isolated from semantic content, its benefit is usually negligible or modest, and disappears when extra text is merely filler (Wang et al., 29 Jun 2026). This suggests that the decisive factor is not “more tokens” in the abstract but whether the additional text performs task-relevant reasoning or checking.

7. Implications for evaluation and deployment

The evaluation consequences are explicit. Standard benchmark protocols can be systematically misleading for a subset of problems because universal prompts may favor smaller models on some items and mask larger models’ capabilities (Hakim, 11 Mar 2026). The benchmark study identifies four implications: benchmark scores under universal prompts can underestimate frontier models, especially on the 7.7% of inverse-scaling problems; problem-level analysis matters because aggregate accuracy hides failures and reversals; brevity-constrained evaluation can reveal latent capability, especially on math and science reasoning benchmarks; and removing non-discriminative items could reduce evaluation cost by about 28% while preserving discriminative power (Hakim, 11 Mar 2026).

For deployment, the practical recommendation is scale-aware prompt engineering and problem-aware routing rather than a single prompt format for all models (Hakim, 11 Mar 2026). The proposed operational policy is to use smaller models where they suffice and are cheaper, apply brevity constraints selectively for large models on tasks prone to overelaboration, and route inputs based on problem type and prompt sensitivity (Hakim, 11 Mar 2026). Because the interventions improve both accuracy and efficiency / computational cost, the issue is treated not only as an evaluation artifact but also as a deployment concern (Hakim, 11 Mar 2026).

The chain-of-thought results refine this recommendation by showing that shortening outputs indiscriminately is not equivalent to improving reasoning. Some reasoning is useful, but the benefit depends on whether the added text contributes facts, operations, intermediate values, and validation/checking steps (Wang et al., 29 Jun 2026). A plausible implication is that optimal prompting should not simply minimize length; it should suppress overelaboration while preserving semantically functional reasoning structure.

In this sense, spontaneous scale-dependent verbosity names a specific failure mode of scale-sensitive generation behavior, while the broader empirical literature distinguishes harmful overelaboration from useful explicit reasoning. Larger models may appear worse under neutral prompts because they generate responses that are too expansive for certain tasks (Hakim, 11 Mar 2026), yet longer traces do not help reliably unless they encode substantive reasoning or checking content (Wang et al., 29 Jun 2026). The resulting picture is not that brevity is always superior, nor that verbosity is intrinsically helpful, but that the performance effect of length is mediated by task fit, model scale, and the semantic work performed by the added tokens.