Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

Abstract: Inference efficiency is typically pursued by shrinking the model: distillation, pruning, quantization, and sparse routing each lower per-token cost while treating token count as fixed. But output length has been inflating, and it is precisely the component the standard toolkit leaves untouched. Here, we argue that brevity is the missing inference-efficiency lever, and that pretraining data curation is a practical way to pull it: a model trained on concise, correct data learns to answer in fewer tokens; i.e. it has a lower Cost-of-Pass. We apply our VLM curation pipeline to the MAmmoTH-VL single-image subset, and compare models trained on our curated data, the standard MAmmoTH-VL data, and external open-weight frontier VLMs. On a controlled 20-evaluation set and 14 VLMs at 1B-4B activated parameters, we hold output length fixed with a per-model regression, separating brevity from quality, and price models in FLOPs per correct answer. Curation buys a 35x Cost-of-Pass advantage over the most verbose 4B comparator (Qwen3.5-4B) within $\sim$1 pp of accuracy (0.41 vs 14.58 TFLOPs per correct answer; 0.691 vs 0.704 mean accuracy). Curation also buys a +17.55-percentage-point matched-length accuracy gain over the uncurated baseline that grows with model scale (from +16.7 pp at 1B to +21.2 pp at 4B). This brevity improvement concedes no quality: generic verbosity buys no accuracy at any capability or scale, and the window where reasoning-structured verbosity still earns its tokens shrinks from 4 of 8 capability groups at 2B to 1 of 8 at 4B. Per example, the concise model even reaches correct answers the verbose reasoning model misses, marking reasoning as a distinct curation target rather than something brevity gives up. Inference efficiency in this regime is a tokens-per-correct problem, and brevity is the lever that targets it directly.

• The paper demonstrates that inducing concision via data curation reduces inference cost by up to 35× with minimal quality loss.
• The paper employs a rigorous methodology that curates training data, achieving dramatic token reduction (47 vs. 1823 tokens per correct answer) across 20 tasks.
• The paper establishes that optimizing output brevity with cost-of-pass metrics offers Pareto improvements over traditional per-token efficiency methods.

Brevity as an Untapped Lever for Vision-LLM Inference Efficiency

Motivation and Context

The paper "Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation" (2606.25432) rigorously interrogates the dominant paradigm of inference efficiency in VLMs, dissecting the consequences of output length inflation and positioning data curation as a direct mechanism for controlling verbosity. The standard inference toolkit—spanning distillation, pruning, quantization, and routing—implicitly treats output length as fixed, optimizing only the per-token compute cost. However, recent empirical trends show that both per-query output length and total token consumption are inflating rapidly, particularly in reasoning models. This inflation undermines cost reductions achieved via traditional methods, resulting in escalating inference bills across deployed AI pipelines.

The paper posits that output brevity is an underexploited axis of inference efficiency. Importantly, it reframes inference cost in terms of Cost-of-Pass: the compute spent per correct answer, which directly captures both output length and accuracy. By targeting brevity via pretraining data curation, rather than decode-time length penalties or sampling controls, the model internalizes concision and achieves Pareto improvements in cost and quality.

Methodology and Experimental Design

The authors employ a controlled experimental setup: they curate the MAmmoTH-VL single-image subset with rigorous target concision and correctness, then train DatologyAI models on this curated mixture. A suite of 20 diverse evaluation tasks is compiled, spanning VQA, OCR, captioning, spatial reasoning, counting, and mathematical inference, and evaluated across 14 open-weight VLMs (1B–4B activated parameters). DatologyAI is compared with both its baseline (Mammoth, uncurated data) and frontier external models (Qwen, InternVL, Isaac).

To isolate the effects of brevity from quality, they use per-model regression analysis that controls for output length, ensuring accuracy comparisons are not confounded by easier-to-score concise responses. Cost-of-Pass is calculated in FLOPs per correct answer, factoring in both parameter count and output length. This hardware-independent metric is corroborated with OckBench's tokens-per-correct (OckScore) and dollar cost estimates under realistic roofline scenarios.

Figure 1: Brevity is directly enacted via data curation; curated VLMs emit dramatically fewer tokens per response and achieve far lower inference cost per correct answer than verbose comparators, with no quality loss.

Results

Cost-of-Pass and Inference Efficiency

Curated models cluster at the lowest end of the Cost-of-Pass frontier. Datology 4B achieves a correct answer for 0.41 TFLOPs versus Qwen3.5-4B's 14.58, within ~1 percentage point of accuracy—a 35× inference cost advantage at comparable quality. At fixed scale, curated models are both cheaper per correct and more accurate than their uncurated baselines (e.g., Datology vs Mammoth). The gap in cost is entirely explained by tokens per correct answer; per-token FLOPs are identical across models at matched parameters.

OckScore corroborates this: Datology 4B yields ~47 tokens per correct answer, compared to 1,823 for Qwen3.5-4B—a ~40× gap. These ratios transfer directly to dollar costs under bandwidth-constrained settings.

Figure 2: OckScore demonstrates that curated models occupy the bottom-left (low-cost, low-error) corner, dramatically outperforming verbose models in tokens-per-correct and error rate.

Quality at Matched Length

At the same output length and on the same benchmark, curated models are correct +17.6 percentage points more often than their uncurated pair (AME). The effect grows with scale: +16.7 pp at 1B, +15.6 pp at 2B, +21.2 pp at 4B. The accuracy advantage is not a metric artifact; length-controlled regression confirms it. Captioning precision and recall also favor curated models, and reduction in hallucination rates is attributed to brevity rather than improved per-token grounding.

Figure 3: The accuracy gap is not an artifact of output length; at matched scale, model identity—not length—is the decisive factor for correctness.

Competitiveness Against Frontier Models

Against the frontier-comparable pool, curated models hold the Pareto front for Cost-of-Pass against six of seven comparators. In length-controlled accuracy, they trail Qwen models by a modest 4–7 pp but cost 2–4× less per correct answer. Crucially, Qwen3.5-4B admits no concise operating point; its accuracy comes bundled with chain-of-thought verbosity, dictating a high token and cost bill.

Figure 4: Length-accuracy splines for all frontier comparators show that curated models achieve competitive accuracy at much lower output length and inference cost.

Capability-Level Dissociation

Generic verbosity (non-reasoning Mammoth) provides zero accuracy benefit at any capability or scale. Reasoning-structured verbosity (chain-of-thought Qwen3.5) offers an advantage in a shrinking window: 4 of 8 capability groups at 2B, only 1 at 4B. Per-example contingency analysis reveals that concise and verbose models reach correct answers on different samples, reinforcing brevity and reasoning as distinct curation targets.

Figure 5: Curated models collapse response length for concise tasks; reasoning models persist in verbosity regardless of task type, precluding concise operation.

Practical Implications and Theoretical Consequences

The results decisively establish brevity via curation as a dominant lever for inference efficiency. At scale, curating for brevity yields strict Pareto improvements: shorter outputs, higher matched-length accuracy, and substantial (>30×) reductions in inference cost. Quantization, distillation, and MoE routing attack only the per-token cost; brevity addresses tokens-per-correct. These levers compound multiplicatively, enabling competitive correctness from smaller, cheaper models.

Production use cases must price per-correct cost, not per-token cost, as output length inflation causes inference bills to grow even as per-token cost falls. The amortized inference perspective highlights the economic rationale: brevity incurs a one-off cost during training, propagated across all generations thereafter.

In practical terms, continual deployment usage can identify recurring verbosity patterns, targeting them for subsequent curation, progressively tightening cost and quality loops—an approach amenable to continual learning and production-aligned optimization.

Limitations and Directions for Future Work

While brevity is found to generally dominate, reasoning remains a distinct capability. The paper argues that curating for both concision and reasoning is the next natural build; the models studied here do not explicitly optimize reasoning-structured concision. There is also scope for cross-modal generalization (e.g., image-grounded precision metrics could further refine quality measurement) and for investigations into more adversarial refusal scenarios.

Conclusion

Curation-driven brevity in VLMs yields dramatic reductions in inference cost without sacrificing accuracy. The effect scales with model size, outstripping traditional per-token efficiency methods. Verbose outputs—whether structurally justified via reasoning or generically inflated—rarely earn their computational cost. The implication is clear: optimizing output length by architectural or decode-time means is suboptimal; data curation internalizes concision and achieves efficiency at scale. For future VLMs, combining curation-induced brevity with robust reasoning capability appears essential. Continual curation will transform inference efficiency from a per-token to a per-correct axis, aligning model economics with practical deployment realities.