---
title: 'Lost in the Middle: Positional Bias in LLMs'
url: https://www.emergentmind.com/topics/lost-in-the-middle
type: topic
---

# Lost in the Middle: Positional Bias in LLMs

“Lost in the Middle” denotes a positional bias in long-context language models: when the same relevant information is placed at different positions in a long input, model performance is often highest when that information appears at the beginning or the end of the context and lowest when it appears in the middle. The term was systematized by Liu et al. in “Lost in the Middle: How Language Models Use Long Contexts” [2307.03172], and subsequent work has extended it from multi-document question answering and key–value retrieval to retrieval-augmented generation, long-document summarization, multi-hop reasoning, graph tasks, and multimodal knowledge-based visual question answering [2412.10079].

## 1. Origin and canonical formulation

The original formulation asked a narrow but foundational question: if a model can *accept* 8K, 16K, or 100K tokens, does it actually *use* information uniformly across that context? Liu et al. answered by designing tasks in which the content is fixed and only the **position** of the relevant information changes, thereby isolating positional sensitivity from content differences [2307.03172].

In the multi-document QA setup, the prompt contains a sequence of passages
$$
x = (d_1, d_2, \dots, d_k),
$$
with exactly one answer-bearing document $d_{i^\star}$ and $k-1$ distractors. Performance is evaluated as a function of the relevant document’s relative position
$$
p = \frac{i^\star}{k-1},
$$
where $p=0$ denotes the beginning and $p=1$ the end [2307.03172]. In the key–value retrieval setup, the input is a serialized JSON object with random UUID key–value pairs plus a query key, and the model must output the associated value; because the keys and values are random 128-bit UUIDs, the task isolates exact retrieval rather than semantic interpretation [2307.03172].

The resulting empirical signature was a **U-shaped curve**: strong performance at the start, strong performance again at the end, and a pronounced trough in the middle. The original paper described this as a combination of **primacy bias** and **recency bias**, and showed that the assumption “a larger context window implies roughly uniform access across positions” is often false in practical long-context settings such as long documents, multi-document RAG, long code files, and chat histories [2307.03172].

## 2. Controlled evidence and empirical signature

The canonical evidence comes from position-controlled and length-controlled evaluations. In NaturalQuestions-Open, Liu et al. varied the number of documents $k\in\{10,20,30\}$ while holding the answer-bearing document fixed and reordering only its position. They also compared **closed-book** and **oracle** conditions. For GPT-3.5-Turbo in the 20-document setting, closed-book accuracy was **56.1%** and oracle accuracy was **88.3%**; with 20 documents, accuracy was **75.8%** when the answer was at position 1, **53.8%** at index 9, and **63.2%** at the last position. In the 30-document setting for GPT-3.5-Turbo (16K), accuracy was **73.4%** at index 0, **50.5%** at index 9, **50.9%** at index 14, and **63.7%** at index 29 [2307.03172].

A notable consequence is that adding the correct passage can make performance *worse* than closed-book behavior if the passage is buried in the middle: in the 20-document setting above, the middle-position score of roughly **54%** was below the **56.1%** closed-book baseline [2307.03172]. The same paper also showed that extended-context variants did not inherently fix the problem: GPT-3.5-Turbo and GPT-3.5-Turbo(16K) had almost identical accuracy-vs-position curves for prompt lengths both models could fit, indicating that enlarging the context window mainly allowed more tokens to fit rather than teaching better use of middle positions [2307.03172].

The synthetic key–value retrieval task established that the phenomenon is not restricted to semantic QA. Claude-1.3 and Claude-1.3(100K) were nearly perfect across all positions and lengths tested, but GPT-3.5-Turbo, GPT-3.5-Turbo(16K), and MPT-30B-Instruct again showed U-shaped curves, with noticeable drops at middle positions, especially at longer contexts of **140** or **300** key–value pairs [2307.03172]. Query-aware contextualization—placing the query both before and after the data—nearly solved the key–value task for all models, with GPT-3.5-Turbo(16K) reaching **100%** accuracy at **300** pairs, but it made only minor differences on realistic multi-document QA, where the U-shaped curve persisted [2307.03172].

## 3. Architectural interpretations and theoretical accounts

Several later works reframed the phenomenon as an interaction between architecture, training regime, and inference-time attention. Liu et al. already found that encoder–decoder models were more position-robust **within** their training-time encoder length: for Flan-UL2 at contexts of at most **2048** tokens, the difference between best and worst answer positions was only about **1.9 percentage points**, but once pushed beyond training-time lengths, Flan-UL2 and Flan-T5-XXL also developed the characteristic U-shape [2307.03172]. They further observed that instruction tuning improved absolute accuracy but did not eliminate primacy/recency bias, and that larger Llama-2 variants exhibited clearer U-shaped behavior than smaller ones [2307.03172].

One influential mechanistic proposal linked lost-in-the-middle to an **intrinsic U-shaped attention bias**. In “Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization,” the observed document-level attention in Vicuna-7B-16k was modeled as
$$
\mathrm{Attn}(d, k) = \mathrm{rel}(d) + \mathrm{bias}(k) + \epsilon,
$$
where $\mathrm{rel}(d)$ is document relevance and $\mathrm{bias}(k)$ is a positional term that remains U-shaped even after document shuffling. The same paper reported that in **74%** of error cases, the document with highest model attention was also the document whose content was most reflected in the answer, supporting the claim that positional attention bias propagates into generation behavior [2406.16008].

Other accounts emphasized training distribution and architectural baselines. “Lost-in-the-Middle in Long Contexts Better via Plug-and-Play Positional Encoding” argued that RoPE’s long-term decay and non-uniform head behavior contribute materially to the problem, motivating inference-time head-specific rescaling [2403.04797]. “Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMs” proposed that primacy and recency arise from mixed **long-term** and **short-term** retrieval demands during training: free-recall-style objectives induced primacy, running-span-style objectives induced recency, and mixed demands produced a U-shaped serial-position curve, with attention sinks implicated in the primacy component [2510.10276]. Two 2026 theory papers made stronger claims: “Kinetic theory for Transformers and the lost-in-the-middle phenomenon” proved a quantitative mean-field limit for a causal self-attention toy model and derived a U-shaped token retrieval profile with primacy, recency, and a unique interior minimum under an explicit smallness condition [2605.09213], while “Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias” argued that the U-shape is already present at initialization in causal decoders with residual connections, with a logarithmic **Primacy Tail**, an $\mathcal{O}(1)$ **Recency Delta**, and a factorial interior dead zone of order $\mathcal{O}(1/(H-1)!)$ [2603.10123].

## 4. Generalizations beyond single-span retrieval

Later work showed that the phenomenon is not confined to single-hop retrieval with one relevant span. In “Lost in the Middle, and In-Between,” the setting is multi-hop QA over **20 documents**, and performance degrades not only when evidence lies near the middle of the overall context but also when multiple evidence documents are separated by distractors. For HotpotQA, 2WikiMultihopQA, and MuSiQue, accuracy was generally higher when evidence documents were adjacent than when they were separated, leading the authors to distinguish the original edge-vs-middle effect from an additional **in-between** effect [2412.10079].

A related but distinct extension appears in graph reasoning. “Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks” defined the distance between two occurrences of a common node as the number of tokens separating them in the serialized graph prompt and modeled joint-task performance as
$$
F(p_1,p_2)=\gamma\,G(p_1)\,G(p_2)\,H(d),
$$
where $G(p)$ captures lost-in-the-middle and $H(d)$ captures an independent distance penalty. The paper reported that model accuracy can decline by up to **6x** as the distance between node connections increases, and argued that lost-in-distance and lost-in-the-middle occur independently [2410.01985].

The same positional issue was also observed in long-input, long-output generation. “Lost-in-the-Middle in Long-Text Generation” introduced LongInOutBench, in which each sample contains **three** full arXiv papers and the model must generate a long integrated summary. Position sensitivity is measured through Single-Context Questions tied to paper 1, paper 2, or paper 3, and standard baselines showed substantially lower scores for the **2nd** paper, indicating that middle-document content was underrepresented in the generated summaries [2503.06868]. In long-document summarization, “Markov-Enhanced Clustering for Long Document Summarization” adopted the same terminology for book-length inputs of about **108k** tokens on BookSum and proposed chunking, embedding, K-means++, cluster summarization, a Markov chain over clusters, and a most probable Hamiltonian path as a way to avoid giving an LLM a single monolithic sequence in which the middle is easily under-used [2506.18036].

The most striking modality shift came from multimodal KB-VQA. “Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering” tested three open-source 7B/8B VLM readers and found that the classical U-shape flipped to **primacy**: at $k=10$, gold-at-first beat gold-at-last by **16 to 26 points** on every reader-by-benchmark cell, an effect the paper termed **Lost at the End** [2606.16494].

## 5. Mitigation strategies

Mitigation work has targeted prompting, reranking, positional encoding, task design, and attention calibration. The earliest practical lesson from Liu et al. was straightforward: fewer, higher-quality documents often outperform larger, noisier contexts; critical material should be placed at the start or end; and in key–value-like retrieval, query-aware prompts can nearly remove the position problem, even though they do not remove it on realistic QA [2307.03172].

A prominent inference-time mitigation is **attention calibration**. “Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization” estimated a document’s relevance by subtracting the attention paid to a fixed dummy document at the same position,
$$
\mathrm{CalAttn}(d,k)=\mathrm{Attn}(d,k)-\mathrm{Attn}(d_0,k),
$$
then used the calibrated scores to rescale token-level attention in the last 16 decoder layers. On NaturalQuestions with the gold document in the middle, calibrated attention improved Recall@3 from **0.21** to **0.68** at $K=20$, and in QA it yielded improvements of up to roughly **15 percentage points** at the most difficult middle positions [2406.16008].

Two positional-encoding interventions pursued the same goal from a RoPE perspective. “Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding” introduced **Ms-PoE**, a head-specific rescaling of RoPE indices chosen according to a position-awareness score, and reported average accuracy gains of up to **3.8** on ZeroSCROLLS without fine-tuning [2403.04797]. “An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding” introduced **CREAM**, which combines continuity-preserving and relativity-preserving index sampling with a truncated Gaussian over middle positions during fine-tuning. CREAM trains at the original **4K** context length yet extends Llama-2-7B to much longer target lengths, and on the Liu et al. key–value benchmark it outperformed PoSE at every position, with particularly large gains in the middle [2406.07138].

Another line of work uses **decompositional training**. “Never Lost in the Middle” proposed Attention Strengthening Multi-doc QA (ASM QA), which decomposes multi-document QA into **Question Repetition**, **Index Prediction**, and **Answer Summarization**. With shuffled evidence positions and hard negatives, the resulting Ziya-Reader achieved a **13.7% absolute gain in shuffled settings** and **21.5%** in a passage retrieval task, while attention visualizations showed robust focus on repeated answer spans across positions [2311.09198]. In long-text generation, RAL-Writer addressed middle loss by retrieving and restating relevant middle chunks at the tail of each step-specific prompt, using an importance score
$$
I(i)=R(i)-P(i),
$$
where $R(i)$ is embedding relevance and $P(i)$ is a U-shaped position score; this substantially reduced the accuracy drop on questions about the second paper in three-paper synthesis prompts [2503.06868].

Not all fixes transfer across settings. In multimodal KB-VQA, retrieval-side interventions such as **MMR**, oracle reranking, and rank-based reordering did not yield separable improvement on a frozen reader, and the gold-at-first vs gold-at-last gap remained intact, implying that some regimes require reader-side rather than retriever-side intervention [2606.16494].

## 6. Reassessment, controversies, and current understanding

Although the original U-shaped curve remains the canonical definition, later work showed that its strength and exact form are regime-dependent. “Positional Biases Shift as Inputs Approach Context Window Limits” introduced a relative-length view
$$
L_{\text{rel}}=\frac{L_{\text{input}}}{L_{\text{max}}},
$$
and defined LiM intensity as
$$
LiMi=\begin{cases}
(acc^F-acc^M)+(acc^L-acc^M), & \text{if } acc^F>acc^M \text{ and } acc^L>acc^M,\\
0, & \text{otherwise.}
\end{cases}
$$
Across six models and four tasks, LiM was strongest when the input used up to about **50%** of the context window, peaking around **0.25** for reasoning and **0.38** for retrieval; beyond about **0.5**, primacy weakened, recency remained, and the pattern became better described as a **distance-from-end bias** rather than a symmetric U-shape [2508.07479].

A reproducibility study in RAG reached a related conclusion from another angle. “Lost in the Evidence? Reproducing Document Position and Context Size Effects in RAG” argued that topic sampling is a major source of variance and showed that small topic budgets can flip the apparent best document ordering. Using calibrated topic sets, the paper found that classic lost-in-the-middle curves were not reliably reproduced in oracle position-sweep settings with contemporary LLaMA-3.1 8B and Mistral-NeMo 12B, yet in realistic multi-hop HotpotQA-style RAG, context size and ordering still interacted strongly with retrieval quality, and placing stronger evidence toward the **end** could improve performance [2605.27105].

A further refinement came from GM-Extract. “What Works for ‘Lost-in-the-Middle’ in LLMs?” argued that the classic U-shaped curve was not consistently observed on an automotive control-variable benchmark, and introduced separate **Document Metric** and **Variable Extraction Metric** to disentangle spatial retrieval from semantic retrieval. The study found that performance can change significantly simply by altering how the data is represented in the context window and that spatial failures and semantic failures need not coincide [2511.13900].

Taken together, these later results do not negate the original finding. They show that positional degradation is robust, but the exact geometry—U-shaped, end-biased, primacy-dominated, or distance-sensitive—depends on relative context usage, architecture, modality, prompt structure, and whether the task demands retrieval of a single fact or coordinated use of several dispersed pieces of evidence. This suggests that “lost in the middle” now denotes both the original U-shaped phenomenon and a broader class of position-sensitive long-context failures [2508.07479].

Source: https://www.emergentmind.com/topics/lost-in-the-middle