Lost in the Middle: Positional Bias in LLMs

• Lost in the Middle is a positional bias in long-context language models where mid-sequence information triggers lower performance compared to content at the beginning or end.
• Empirical studies reveal a U-shaped accuracy curve, with performance dips sometimes falling below closed-book baselines when answer-critical details reside in the middle.
• Mitigation strategies like attention calibration, revised positional encoding, and query-aware prompts offer practical ways to rebalance context utilization and boost model accuracy.

“Lost in the Middle” denotes a positional bias in long-context LLMs: when the same relevant information is placed at different positions in a long input, model performance is often highest when that information appears at the beginning or the end of the context and lowest when it appears in the middle. The term was systematized by Liu et al. in “Lost in the Middle: How LLMs Use Long Contexts” (Liu et al., 2023), and subsequent work has extended it from multi-document question answering and key–value retrieval to retrieval-augmented generation, long-document summarization, multi-hop reasoning, graph tasks, and multimodal knowledge-based visual question answering (Baker et al., 2024).

1. Origin and canonical formulation

The original formulation asked a narrow but foundational question: if a model can accept 8K, 16K, or 100K tokens, does it actually use information uniformly across that context? Liu et al. answered by designing tasks in which the content is fixed and only the position of the relevant information changes, thereby isolating positional sensitivity from content differences (Liu et al., 2023).

In the multi-document QA setup, the prompt contains a sequence of passages

$x = (d_1, d_2, \dots, d_k),$

with exactly one answer-bearing document $d_{i^\star}$ and $k-1$ distractors. Performance is evaluated as a function of the relevant document’s relative position

$p = \frac{i^\star}{k-1},$

where $p=0$ denotes the beginning and $p=1$ the end (Liu et al., 2023). In the key–value retrieval setup, the input is a serialized JSON object with random UUID key–value pairs plus a query key, and the model must output the associated value; because the keys and values are random 128-bit UUIDs, the task isolates exact retrieval rather than semantic interpretation (Liu et al., 2023).

The resulting empirical signature was a U-shaped curve: strong performance at the start, strong performance again at the end, and a pronounced trough in the middle. The original paper described this as a combination of primacy bias and recency bias, and showed that the assumption “a larger context window implies roughly uniform access across positions” is often false in practical long-context settings such as long documents, multi-document RAG, long code files, and chat histories (Liu et al., 2023).

2. Controlled evidence and empirical signature

The canonical evidence comes from position-controlled and length-controlled evaluations. In NaturalQuestions-Open, Liu et al. varied the number of documents $k\in\{10,20,30\}$ while holding the answer-bearing document fixed and reordering only its position. They also compared closed-book and oracle conditions. For GPT-3.5-Turbo in the 20-document setting, closed-book accuracy was 56.1% and oracle accuracy was 88.3%; with 20 documents, accuracy was 75.8% when the answer was at position 1, 53.8% at index 9, and 63.2% at the last position. In the 30-document setting for GPT-3.5-Turbo (16K), accuracy was 73.4% at index 0, 50.5% at index 9, 50.9% at index 14, and 63.7% at index 29 (Liu et al., 2023).

A notable consequence is that adding the correct passage can make performance worse than closed-book behavior if the passage is buried in the middle: in the 20-document setting above, the middle-position score of roughly 54% was below the 56.1% closed-book baseline (Liu et al., 2023). The same paper also showed that extended-context variants did not inherently fix the problem: GPT-3.5-Turbo and GPT-3.5-Turbo(16K) had almost identical accuracy-vs-position curves for prompt lengths both models could fit, indicating that enlarging the context window mainly allowed more tokens to fit rather than teaching better use of middle positions (Liu et al., 2023).

The synthetic key–value retrieval task established that the phenomenon is not restricted to semantic QA. Claude-1.3 and Claude-1.3(100K) were nearly perfect across all positions and lengths tested, but GPT-3.5-Turbo, GPT-3.5-Turbo(16K), and MPT-30B-Instruct again showed U-shaped curves, with noticeable drops at middle positions, especially at longer contexts of 140 or 300 key–value pairs (Liu et al., 2023). Query-aware contextualization—placing the query both before and after the data—nearly solved the key–value task for all models, with GPT-3.5-Turbo(16K) reaching 100% accuracy at 300 pairs, but it made only minor differences on realistic multi-document QA, where the U-shaped curve persisted (Liu et al., 2023).

3. Architectural interpretations and theoretical accounts

Several later works reframed the phenomenon as an interaction between architecture, training regime, and inference-time attention. Liu et al. already found that encoder–decoder models were more position-robust within their training-time encoder length: for Flan-UL2 at contexts of at most 2048 tokens, the difference between best and worst answer positions was only about 1.9 percentage points, but once pushed beyond training-time lengths, Flan-UL2 and Flan-T5-XXL also developed the characteristic U-shape (Liu et al., 2023). They further observed that instruction tuning improved absolute accuracy but did not eliminate primacy/recency bias, and that larger Llama-2 variants exhibited clearer U-shaped behavior than smaller ones (Liu et al., 2023).

One influential mechanistic proposal linked lost-in-the-middle to an intrinsic U-shaped attention bias. In “Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization,” the observed document-level attention in Vicuna-7B-16k was modeled as

$$\mathrm{Attn}(d, k) = \mathrm{rel}(d) + \mathrm{bias}(k) + \epsilon,$$

where $\mathrm{rel}(d)$ is document relevance and $\mathrm{bias}(k)$ is a positional term that remains U-shaped even after document shuffling. The same paper reported that in 74% of error cases, the document with highest model attention was also the document whose content was most reflected in the answer, supporting the claim that positional attention bias propagates into generation behavior (Hsieh et al., 2024).

Other accounts emphasized training distribution and architectural baselines. “Lost-in-the-Middle in Long Contexts Better via Plug-and-Play Positional Encoding” argued that RoPE’s long-term decay and non-uniform head behavior contribute materially to the problem, motivating inference-time head-specific rescaling (Zhang et al., 2024). “Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMs” proposed that primacy and recency arise from mixed long-term and short-term retrieval demands during training: free-recall-style objectives induced primacy, running-span-style objectives induced recency, and mixed demands produced a U-shaped serial-position curve, with attention sinks implicated in the primacy component (Salvatore et al., 11 Oct 2025). Two 2026 theory papers made stronger claims: “Kinetic theory for Transformers and the lost-in-the-middle phenomenon” proved a quantitative mean-field limit for a causal self-attention toy model and derived a U-shaped token retrieval profile with primacy, recency, and a unique interior minimum under an explicit smallness condition (Duerinckx et al., 9 May 2026), while “Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias” argued that the U-shape is already present at initialization in causal decoders with residual connections, with a logarithmic Primacy Tail, an $d_{i^\star}$0 Recency Delta, and a factorial interior dead zone of order $d_{i^\star}$1 (Chowdhury, 10 Mar 2026).

4. Generalizations beyond single-span retrieval

Later work showed that the phenomenon is not confined to single-hop retrieval with one relevant span. In “Lost in the Middle, and In-Between,” the setting is multi-hop QA over 20 documents, and performance degrades not only when evidence lies near the middle of the overall context but also when multiple evidence documents are separated by distractors. For HotpotQA, 2WikiMultihopQA, and MuSiQue, accuracy was generally higher when evidence documents were adjacent than when they were separated, leading the authors to distinguish the original edge-vs-middle effect from an additional in-between effect (Baker et al., 2024).

A related but distinct extension appears in graph reasoning. “Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks” defined the distance between two occurrences of a common node as the number of tokens separating them in the serialized graph prompt and modeled joint-task performance as

$d_{i^\star}$2

where $d_{i^\star}$3 captures lost-in-the-middle and $d_{i^\star}$4 captures an independent distance penalty. The paper reported that model accuracy can decline by up to 6x as the distance between node connections increases, and argued that lost-in-distance and lost-in-the-middle occur independently (Firooz et al., 2024).

The same positional issue was also observed in long-input, long-output generation. “Lost-in-the-Middle in Long-Text Generation” introduced LongInOutBench, in which each sample contains three full arXiv papers and the model must generate a long integrated summary. Position sensitivity is measured through Single-Context Questions tied to paper 1, paper 2, or paper 3, and standard baselines showed substantially lower scores for the 2nd paper, indicating that middle-document content was underrepresented in the generated summaries (Zhang et al., 10 Mar 2025). In long-document summarization, “Markov-Enhanced Clustering for Long Document Summarization” adopted the same terminology for book-length inputs of about 108k tokens on BookSum and proposed chunking, embedding, K-means++, cluster summarization, a Markov chain over clusters, and a most probable Hamiltonian path as a way to avoid giving an LLM a single monolithic sequence in which the middle is easily under-used (Amari et al., 22 Jun 2025).

The most striking modality shift came from multimodal KB-VQA. “Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering” tested three open-source 7B/8B VLM readers and found that the classical U-shape flipped to primacy: at $d_{i^\star}$5, gold-at-first beat gold-at-last by 16 to 26 points on every reader-by-benchmark cell, an effect the paper termed Lost at the End (Liu et al., 15 Jun 2026).

5. Mitigation strategies

Mitigation work has targeted prompting, reranking, positional encoding, task design, and attention calibration. The earliest practical lesson from Liu et al. was straightforward: fewer, higher-quality documents often outperform larger, noisier contexts; critical material should be placed at the start or end; and in key–value-like retrieval, query-aware prompts can nearly remove the position problem, even though they do not remove it on realistic QA (Liu et al., 2023).

A prominent inference-time mitigation is attention calibration. “Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization” estimated a document’s relevance by subtracting the attention paid to a fixed dummy document at the same position,

$d_{i^\star}$6

then used the calibrated scores to rescale token-level attention in the last 16 decoder layers. On NaturalQuestions with the gold document in the middle, calibrated attention improved Recall@3 from 0.21 to 0.68 at $d_{i^\star}$7, and in QA it yielded improvements of up to roughly 15 percentage points at the most difficult middle positions (Hsieh et al., 2024).

Two positional-encoding interventions pursued the same goal from a RoPE perspective. “Found in the Middle: How LLMs Use Long Contexts Better via Plug-and-Play Positional Encoding” introduced Ms-PoE, a head-specific rescaling of RoPE indices chosen according to a position-awareness score, and reported average accuracy gains of up to 3.8 on ZeroSCROLLS without fine-tuning (Zhang et al., 2024). “An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding” introduced CREAM, which combines continuity-preserving and relativity-preserving index sampling with a truncated Gaussian over middle positions during fine-tuning. CREAM trains at the original 4K context length yet extends Llama-2-7B to much longer target lengths, and on the Liu et al. key–value benchmark it outperformed PoSE at every position, with particularly large gains in the middle (Wu et al., 2024).

Another line of work uses decompositional training. “Never Lost in the Middle” proposed Attention Strengthening Multi-doc QA (ASM QA), which decomposes multi-document QA into Question Repetition, Index Prediction, and Answer Summarization. With shuffled evidence positions and hard negatives, the resulting Ziya-Reader achieved a 13.7% absolute gain in shuffled settings and 21.5% in a passage retrieval task, while attention visualizations showed robust focus on repeated answer spans across positions (He et al., 2023). In long-text generation, RAL-Writer addressed middle loss by retrieving and restating relevant middle chunks at the tail of each step-specific prompt, using an importance score

$d_{i^\star}$8

where $d_{i^\star}$9 is embedding relevance and $k-1$0 is a U-shaped position score; this substantially reduced the accuracy drop on questions about the second paper in three-paper synthesis prompts (Zhang et al., 10 Mar 2025).

Not all fixes transfer across settings. In multimodal KB-VQA, retrieval-side interventions such as MMR, oracle reranking, and rank-based reordering did not yield separable improvement on a frozen reader, and the gold-at-first vs gold-at-last gap remained intact, implying that some regimes require reader-side rather than retriever-side intervention (Liu et al., 15 Jun 2026).

6. Reassessment, controversies, and current understanding

Although the original U-shaped curve remains the canonical definition, later work showed that its strength and exact form are regime-dependent. “Positional Biases Shift as Inputs Approach Context Window Limits” introduced a relative-length view

$k-1$1

and defined LiM intensity as

$k-1$2

Across six models and four tasks, LiM was strongest when the input used up to about 50% of the context window, peaking around 0.25 for reasoning and 0.38 for retrieval; beyond about 0.5, primacy weakened, recency remained, and the pattern became better described as a distance-from-end bias rather than a symmetric U-shape (Veseli et al., 10 Aug 2025).

A reproducibility study in RAG reached a related conclusion from another angle. “Lost in the Evidence? Reproducing Document Position and Context Size Effects in RAG” argued that topic sampling is a major source of variance and showed that small topic budgets can flip the apparent best document ordering. Using calibrated topic sets, the paper found that classic lost-in-the-middle curves were not reliably reproduced in oracle position-sweep settings with contemporary LLaMA-3.1 8B and Mistral-NeMo 12B, yet in realistic multi-hop HotpotQA-style RAG, context size and ordering still interacted strongly with retrieval quality, and placing stronger evidence toward the end could improve performance (Gabín et al., 26 May 2026).

A further refinement came from GM-Extract. “What Works for ‘Lost-in-the-Middle’ in LLMs?” argued that the classic U-shaped curve was not consistently observed on an automotive control-variable benchmark, and introduced separate Document Metric and Variable Extraction Metric to disentangle spatial retrieval from semantic retrieval. The study found that performance can change significantly simply by altering how the data is represented in the context window and that spatial failures and semantic failures need not coincide (Gupte et al., 17 Nov 2025).

Taken together, these later results do not negate the original finding. They show that positional degradation is robust, but the exact geometry—U-shaped, end-biased, primacy-dominated, or distance-sensitive—depends on relative context usage, architecture, modality, prompt structure, and whether the task demands retrieval of a single fact or coordinated use of several dispersed pieces of evidence. This suggests that “lost in the middle” now denotes both the original U-shaped phenomenon and a broader class of position-sensitive long-context failures (Veseli et al., 10 Aug 2025).