Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
117 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Importance of Generation Order in Language Modeling (1808.07910v1)

Published 23 Aug 2018 in cs.LG, cs.CL, and stat.ML

Abstract: Neural LLMs are a critical component of state-of-the-art systems for machine translation, summarization, audio transcription, and other tasks. These LLMs are almost universally autoregressive in nature, generating sentences one token at a time from left to right. This paper studies the influence of token generation order on model quality via a novel two-pass LLM that produces partially-filled sentence "templates" and then fills in missing tokens. We compare various strategies for structuring these two passes and observe a surprisingly large variation in model quality. We find the most effective strategy generates function words in the first pass followed by content words in the second. We believe these experimental results justify a more extensive investigation of generation order for neural LLMs.

Citations (30)

Summary

We haven't generated a summary for this paper yet.