Lost in the Middle (LiM) Phenomenon

• Lost in the Middle (LiM) is a U-shaped positional performance profile where context items in the middle yield lower recall compared to those at the beginning or end.
• Empirical studies using multi-document QA and synthetic retrieval show that models like GPT-3.5-Turbo experience significant accuracy drops in the middle section of long sequences.
• Mitigation strategies such as attention recalibration, modified positional encoding, and training-time reshaping are proposed to improve uniform evidence retrieval.

Searching arXiv for recent and foundational papers on "Lost in the Middle" and related mitigation/mechanistic work. arxiv_search(query="Lost in the Middle LLMs long context positional bias", max_results=10, sort_by="relevance") Lost in the Middle (LiM) denotes a U-shaped positional performance profile in long-context use: items near the beginning (primacy) and end (recency) of a sequence are retrieved most accurately, while items in the middle are recalled worst. In the canonical long-context formulation, LiM was identified by moving the answer-bearing span through otherwise fixed prompts and observing substantial drops in task accuracy when the relevant evidence occupied middle positions rather than boundary positions (Liu et al., 2023). Subsequent work has broadened LiM from a single benchmark effect into a broader family of position-dependent utilization phenomena whose shape depends on retrieval demands, causal masking, positional encoding, attention dynamics, context-window utilization, task structure, and modality (Salvatore et al., 11 Oct 2025).

1. Canonical empirical profile

The original LiM studies examined tasks that isolate positional dependence while keeping semantic content fixed. In multi-document question answering, one passage contains the answer and the remaining passages are distractors; in synthetic key-value retrieval, the queried key-value pair is placed at varying positions in a long serialized object. The central observation is that decoder-only LLMs often do not use long contexts uniformly: they perform best when relevant information is placed near the beginning or near the end, and significantly worse when it is placed in the middle (Liu et al., 2023).

Representative numbers make the pattern concrete. In multi-document QA with $k=20$, GPT-3.5-Turbo achieved $75.8\%$ when the answer-bearing passage was at the start, $53.8\%$ when it was at a middle index, and $63.2\%$ when it was at the end; with $k=30$, GPT-3.5-Turbo (16K) moved from $73.4\%$ at the start to $50.5\%$ in the middle and $63.7\%$ at the end (Liu et al., 2023). The same paper showed that longer advertised context windows did not, by themselves, guarantee robust within-window utilization: extended-context variants often produced curves nearly superimposed on their shorter-window counterparts within shared ranges (Liu et al., 2023).

The same positional asymmetry has been documented beyond short-answer retrieval. In long-input/long-output generation, LiM appears as under-attention to semantically important middle segments during comprehension and corresponding omission of middle-position facts in long-form output. In that regime, the phenomenon is not chiefly measured by a single exact-match score, but by content omission, weaker cross-document synthesis, and reduced consistency for questions whose evidence lies in the middle of the input (Zhang et al., 10 Mar 2025).

2. Formalizations and measurement

The simplest formalization treats downstream performance as a function of evidence position. The original long-context framing can be expressed as an accuracy curve $A(i)$ over source position $i$, with a high-left, low-middle, high-right profile when LiM is present (Liu et al., 2023). Later work emphasized that absolute context length alone is insufficient for cross-model comparison and introduced relative input length

$75.8\%$0

to normalize positional effects by a model’s context window (Veseli et al., 10 Aug 2025).

Using this normalization, position-sensitive behavior can be decomposed into primacy, recency, and LiM intensity:

$75.8\%$1

$75.8\%$2

where $75.8\%$3, $75.8\%$4, and $75.8\%$5 denote first, middle, and last positions (Veseli et al., 10 Aug 2025). This formulation makes explicit that LiM is present only when both boundaries outperform the middle.

A distinct line of work imported metrics from cognitive psychology to characterize positional behavior more granularly. In controlled memory-style tasks, the serial position curve (SPC) measures recall probability by input position, the probability of first recall (PFR) tracks where retrieval starts, and conditional response probability (CRP) measures transition probabilities by lag (Salvatore et al., 11 Oct 2025). These metrics separate overall accuracy from retrieval strategy. For example, a model may remain near ceiling in total recall while its PFR becomes bimodal, revealing an underlying begin-or-end strategy even when the overt U-shape is weak (Salvatore et al., 11 Oct 2025).

Another formalization models observed attention as a sum of relevance and positional bias:

$75.8\%$6

and then estimates a bias-corrected relevance term by subtracting attention to a dummy document placed at the same position (Hsieh et al., 2024). This reframes LiM not as an inability to identify relevance, but as a relevance signal distorted by systematic positional preference.

3. Mechanistic accounts

Several non-exclusive mechanistic accounts appear in the literature. One influential interpretation treats LiM as an adaptation to mixed information-retrieval demands rather than a simple defect. In controlled training-from-scratch experiments, free recall (FR), which imposes a uniform long-term memory demand, induced primacy; running span (RS), which imposes a recent-priority short-term demand, induced recency; and joint FR+RS training produced the characteristic U-shape (Salvatore et al., 11 Oct 2025). The paper summarizes this with a conceptual mixture model,

$75.8\%$7

where $75.8\%$8 is uniform and $75.8\%$9 is end-weighted (Salvatore et al., 11 Oct 2025).

That account is explicitly architectural. Primacy appeared in autoregressive architectures and disappeared in a bidirectional encoder-decoder trained on the same FR objective: an autoregressive seq2seq RNN showed strong primacy, whereas T5 showed a flat SPC and uniform PFR (Salvatore et al., 11 Oct 2025). The same study identified attention sinks as functionally important for long-range retrieval under uniform demand: targeted dropout of first-token sink heads at $53.8\%$0 markedly reduced primacy and overall FR/FR+RS performance, while leaving RS unaffected (Salvatore et al., 11 Oct 2025).

A second family of explanations emphasizes intrinsic positional attention bias. In retrieval-augmented QA, average self-attention over documents forms a U-shaped curve even when document order is randomly shuffled, and a simple additive decomposition into relevance plus positional bias yields strong rank-correlation behavior after calibration (Hsieh et al., 2024). A related account attributes LiM partly to Rotary Position Embeddings (RoPE): as relative distances grow, rotated query-key components become increasingly misaligned, introducing a practical long-distance decay that compounds causal sinks and suppresses middle tokens (Zhang et al., 2024).

Theoretical work has pushed the explanation deeper into architecture. One exact theory models initialized causal decoders as iterated powers of the Cesàro matrix and derives a closed-form influence density with a logarithmic “Primacy Tail,” an $53.8\%$1 “Recency Delta,” and a factorially suppressed middle region of order $53.8\%$2, where $53.8\%$3 is depth (Chowdhury, 10 Mar 2026). A separate kinetic-theory treatment of causal self-attention proves a U-shaped token-retrieval profile with primacy, recency, and a unique interior minimum under an explicit smallness condition in a continuous-time toy model with ALiBi-like recency bias (Duerinckx et al., 9 May 2026). These results do not collapse the empirical literature into one cause; rather, they indicate that causal routing alone can generate a U-shaped baseline even before task-specific learning.

4. Regime dependence and variants

Later work showed that LiM is not a single invariant curve. When analyzed as a function of relative input length, LiM is strongest when inputs occupy up to $53.8\%$4 of a model’s context window. Beyond that, primacy weakens while recency remains relatively stable, eliminating the classic U-shape and replacing it with a distance-based bias favoring information nearer the end of the input (Veseli et al., 10 Aug 2025). In that analysis, LiM intensity peaks around $53.8\%$5 for reasoning and $53.8\%$6 for retrieval (Veseli et al., 10 Aug 2025).

The phenomenon also changes shape across modalities. In multimodal retrieval-augmented KB-VQA, a controlled gold-position protocol found that the text-style U-shape flips to primacy: gold-at-first beat gold-at-last by $53.8\%$7 to $53.8\%$8 points across all six reader-by-benchmark cells at $53.8\%$9, yielding a monotonic “first $63.2\%$0 middle $63.2\%$1 last” pattern that the authors call “Lost at the End” (Liu et al., 15 Jun 2026). Retrieval-side fixes such as MMR, oracle reranking, and rank-based reordering did not eliminate that gap on a frozen reader (Liu et al., 15 Jun 2026).

Multi-hop settings introduce an additional positional variable: the distance between jointly relevant pieces of evidence. In long-context multi-hop QA, performance is lower not only when evidence is placed in the middle, but also when required documents are separated by distractors rather than made adjacent (Baker et al., 2024). In graph tasks, this was formalized as “Lost-in-Distance,” with a multiplicative model

$63.2\%$2

that separates absolute-position effects $63.2\%$3 from proximity effects $63.2\%$4; accuracy can decline by up to $63.2\%$5 as contextual distance increases, independent of graph encoding and model size (Firooz et al., 2024).

At the same time, several studies qualify the universality of the original narrative. A reproducibility study on contemporary RAG systems reported comparatively flat synthetic position sweeps under its protocol and argued that small topic samples can mask or exaggerate ordering effects; it recommended calibrated topic budgets of $63.2\%$6 for HotpotQA and $63.2\%$7 for NQ to stabilize $63.2\%$8 trends (Gabín et al., 26 May 2026). In a single-needle factoid QA study, Gemini 2.5 Flash answered all $63.2\%$9 questions correctly at every feasible length from approximately $k=30$0 to $k=30$1 of its token limit, with no detected position effect for that task (McKinnon, 8 Nov 2025). These results do not negate LiM; they delimit the regimes in which it is strongest.

5. Mitigation strategies

Mitigation approaches fall into architectural calibration, training-time reshaping, and inference-time reorganization. A direct attention-level intervention, “found-in-the-middle,” estimates positional bias with a dummy document and rescales document attention so that calibrated relevance rather than position determines allocation. On the most challenging mid-sequence settings, this yielded $k=30$2–$k=30$3 percentage-point improvements in QA accuracy, and on NQ document ranking it raised Recall@3 from $k=30$4 to $k=30$5 at $k=30$6 and from $k=30$7 to $k=30$8 at $k=30$9 (Hsieh et al., 2024).

A second line modifies positional encoding. Multi-scale Positional Encoding (Ms-PoE) rescales RoPE position indices with head-specific ratios, preserving position-aware heads while more aggressively rescaling position-unaware heads. It introduced no new parameters or extra attention matrices and achieved an average gain of up to $73.4\%$0 on ZeroSCROLLS, with large improvements such as Vicuna-7B on Qasper from $73.4\%$1 to $73.4\%$2 and on BookSumSort from $73.4\%$3 to $73.4\%$4 (Zhang et al., 2024). CREAM, a training-efficient long-context extension recipe, combines continuity–relativity indexing with truncated-Gaussian middle sampling; on LongChat-Lines with Linear interpolation it reached an average $73.4\%$5 versus PoSE’s $73.4\%$6, and on LiM retrieval CREAM-YaRN outperformed PoSE-YaRN by over $73.4\%$7 on average (Wu et al., 2024).

Training-time behavioral reshaping can target LiM more directly. Position-agnostic decompositional training via Attention Strengthening Multi-doc QA (ASM QA) forces a model to repeat the question, predict evidence indices, and then answer. In shuffled Multi-doc QA, Ziya-Reader’s drop was only $73.4\%$8 points, whereas ChatGLM2-6B-32k dropped by up to $73.4\%$9 points; the paper reports a $50.5\%$0 absolute gain in shuffled settings and $50.5\%$1 in passage retrieval (He et al., 2023).

Inference-time restructuring is especially prominent in long-form generation. RAL-Writer uses a plan-then-write pipeline with position-aware retrieval and restatement: chunks are scored by semantic relevance and a middle-favoring penalty, then restated and appended near the prompt tail to exploit recency bias. On LongInOutBench with Qwen2.5-14B-Instruct, RAL-Writer improved consistency over AgentWrite by $50.5\%$2 overall and by $50.5\%$3 at $50.5\%$4k words, while specifically reducing the middle-paper dip in single-context consistency (Zhang et al., 10 Mar 2025). A different restructuring strategy, Markov-Enhanced Clustering, compresses long documents into cluster summaries and orders them with a Markov-chain Hamiltonian path; on BookSum, it improved ROUGE-1/2 and coherence relative to both LLM-Full and unordered Cluster-Sum, which is consistent with mitigating middle omission indirectly through structured compression and ordering (Amari et al., 22 Jun 2025).

Not all interventions are universally beneficial. In multimodal KB-VQA, retrieval-side diversification and simple reordering did not remove slot-0 primacy on a frozen reader (Liu et al., 15 Jun 2026). In GM-Extract, white-box methods such as Ms-PoE and Hidden State Scaling helped Vicuna on some metrics but often degraded LLaMA-2-7B, while PEFT improved an instruction-tuned base yet frequently harmed a non-instruction-tuned base (Gupte et al., 17 Nov 2025). The mitigation literature therefore treats LiM less as a single bug with a single fix than as a family of position-sensitive behaviors that interact strongly with model family, formatting, and task.

6. Interpretation, controversies, and outlook

A central controversy concerns whether LiM should be interpreted as an information-loss defect. One argument, grounded in controlled memory paradigms, is that LiM is a resource-rational adaptation to mixed long-term and short-term retrieval demands under autoregressive constraints and attention-sink dynamics (Salvatore et al., 11 Oct 2025). A different line of work argues that the U-shape is already an architectural baseline at initialization, before training or RoPE can contribute, because causal masking and residual structure themselves generate primacy and recency (Chowdhury, 10 Mar 2026). These accounts are not mutually exclusive: one locates a baseline geometry, the other explains how training demands amplify or reshape it.

Another misconception is that LiM is always literally “lost in the middle.” The empirical record is more conditional. As inputs approach context-window limits, primacy can collapse and a distance-from-end bias can replace the U-shape (Veseli et al., 10 Aug 2025). In multimodal KB-VQA the curve can become almost purely primacy-dominant (Liu et al., 15 Jun 2026). In multi-hop and graph reasoning, performance depends not only on edge distance but also on the separation between jointly relevant facts (Baker et al., 2024). In some contemporary simple factoid settings, no measurable LiM appears at all (McKinnon, 8 Nov 2025).

A plausible implication is that LiM should be treated as a regime-dependent property of long-context utilization rather than as a universal scalar weakness. The strongest evaluations therefore emphasize position-stratified accuracy curves, relative-length control, conditional retrieval-versus-reasoning analysis, and metrics that distinguish semantic extraction from spatial provenance (Veseli et al., 10 Aug 2025). For deployed RAG and multimodal systems, this shifts attention from raw recall@$50.5\%$5 toward position-aware utility and end-to-end ordering sensitivity (Liu et al., 15 Jun 2026). For model design, it suggests that robust mid-context access may require not merely larger windows, but explicit control over retrieval-demand mixtures, positional calibration, and the structural pathways through which evidence reaches generation.