Position Curse: Positional Failures in Imaging and LLMs

• Position Curse is a phenomenon where information linked to position becomes inaccessible or degraded under standard measurement protocols despite being inherently present.
• In optical imaging, it appears as Rayleigh’s curse, with Fisher information scaling quadratically with separation until alternative measurement bases recover constant sensitivity.
• In large language models, the curse manifests as systematic failures in reverse and relative indexing, indicating a misalignment between token order and retrieval efficiency.

“Position Curse” denotes a family of technically distinct but structurally related phenomena in which information tied to position, separation, offset, or order becomes inaccessible or severely degraded under a particular observation, representation, or training protocol. In optical imaging, the phrase is used as a reformulation of Rayleigh’s curse: under direct image-plane intensity detection, the Fisher information for the separation of two incoherent point sources vanishes quadratically as the separation tends to zero, even though suitable coherent measurements retain constant nonzero information (Rehacek et al., 2016). In LLMs, “The Position Curse” refers to systematic failure on position-based retrieval, especially backward indexing from the end of a list or relative indexing from another item, despite near-saturated counting and strong long-context “needle-in-a-haystack” retrieval (Zhang et al., 8 May 2026). Closely related work on the Reversal Curse shows that autoregressive models trained on facts of the form “A is B” often fail to answer the reverse query “B is A,” exposing an order-dependent failure of factual generalization (Berglund et al., 2023). This suggests a common pattern: the difficulty is frequently measurement-dependent, representation-dependent, or query-direction-dependent rather than an unavoidable property of the underlying signal.

1. Scope and recurring structure

In the cited literature, the term and its close analogues appear in several settings. The recurring structure is that a system retains the relevant information in principle, yet a standard protocol fails to access it efficiently.

Domain	Positional object	Reported failure
Diffraction-limited imaging	Separation $\mathfrak{s}$ of two incoherent point sources	Direct imaging gives Fisher information $\propto \mathfrak{s}^2$ as $\mathfrak{s}\to 0$
LLM sequence reasoning	Item position in an ordered sequence $S=[s_1,\ldots,s_L]$	Backward retrieval substantially lags forward retrieval
Autoregressive factual recall	Order of entities in “A is B”	Reverse-direction retrieval is not learned automatically
Mean-field shallow-network optimization	Position of parameter mass in Wasserstein space	Population risk decay is dimension-limited by explicit lower bounds

The optical and LLM usages are the most direct. In optics, the positional quantity is geometric separation in the image plane; in LLM evaluation, it is discrete list position or relative offset. In both cases, the relevant signal remains present, but the default readout is poorly aligned with it (Rehacek et al., 2016, Zhang et al., 8 May 2026).

A broader interpretation also appears in optimization theory. There, the “position” is the evolving parameter distribution $\pi^t$ in $\mathcal{P}_2(\mathbb{R}^{d+2})$, and the curse is that moving this distribution to a region representing a good approximation becomes increasingly costly in high dimension (Na et al., 7 Feb 2025). A plausible implication is that “Position Curse” is best treated as a cross-domain label for failures induced by an ill-suited coordinate system, basis, or directional query.

2. Optical imaging: Rayleigh’s curse as a position-estimation failure

In “Dispelling Rayleigh’s Curse,” the classical two-source resolution problem is formulated as estimation of the separation parameter $\mathfrak{s}$ for two equally bright incoherent point sources at

$X_\pm = \pm \frac{\mathfrak{s}}{2}.$

For a linear, shift-invariant, diffraction-limited imaging system with amplitude point-spread function $\Psi(x)$ and intensity PSF $I(x)=|\Psi(x)|^2$, direct imaging yields the image-plane probability density

$\propto \mathfrak{s}^2$0

The corresponding classical Fisher information is

$\propto \mathfrak{s}^2$1

and, for small separations,

$\propto \mathfrak{s}^2$2

Hence $\propto \mathfrak{s}^2$3, while the Cramér–Rao bound diverges as $\propto \mathfrak{s}^2$4. In this formulation, the position curse is not a loss of photons but a collapse of statistical sensitivity: the symmetric intensity profile becomes first-order insensitive to infinitesimal changes in separation (Rehacek et al., 2016).

The same work shows that the collapse is not fundamental. If one replaces image-plane intensity detection by coherent measurements with definite parity and suitable phase choice, the Fisher information becomes

$\propto \mathfrak{s}^2$5

which is independent of $\propto \mathfrak{s}^2$6 and coincides with the quantum Fisher information. A measurement basis built from

$\propto \mathfrak{s}^2$7

where $\propto \mathfrak{s}^2$8 are real orthogonal polynomials with respect to $\propto \mathfrak{s}^2$9, yields image-plane modes that are orthogonalized derivatives of the PSF. In the small-separation regime, the first derivative mode captures essentially all available Fisher information. The result is that the curse is a property of the standard measurement basis $\mathfrak{s}\to 0$0, not of the underlying optical state (Rehacek et al., 2016).

“Tempering Rayleigh’s curse with PSF shaping” gives a weaker but experimentally simpler mitigation while retaining direct intensity detection. For two incoherent sources with

$\mathfrak{s}\to 0$1

standard smooth symmetric PSFs again give

$\mathfrak{s}\to 0$2

However, if the effective PSF near the origin is parabolic,

$\mathfrak{s}\to 0$3

then the Fisher information scales linearly,

$\mathfrak{s}\to 0$4

The paper shows that any well-behaved symmetric PSF can be converted into this class with a nonabsorbing signum phase filter in a $\mathfrak{s}\to 0$5 system, implementing a Hilbert transform of the field. For a Gaussian PSF, direct imaging gives

$\mathfrak{s}\to 0$6

whereas the signum-shaped PSF gives

$\mathfrak{s}\to 0$7

Experimentally, variance improvement factors up to about $\mathfrak{s}\to 0$8 were observed compared with direct imaging at the same photon budget (Paur et al., 2018).

3. Position-based retrieval in LLMs

“The Position Curse: LLMs Struggle to Locate the Last Few Items in a List” studies position-based retrieval over an ordered sequence

$\mathfrak{s}\to 0$9

The paper defines four index operators: $S=[s_1,\ldots,s_L]$0 with $S=[s_1,\ldots,s_L]$1. It then evaluates two complementary query types: position$S=[s_1,\ldots,s_L]$2item and item$S=[s_1,\ldots,s_L]$3position. Each position is specified as a forward or backward offset from an anchor, either an endpoint of the list or another item in the list. Across both open-source and frontier closed-source models, backward retrieval substantially lags forward retrieval, and the failure is especially severe for backward queries $S=[s_1,\ldots,s_L]$4 and relative indexing $S=[s_1,\ldots,s_L]$5 (Zhang et al., 8 May 2026).

The striking feature is that counting is almost intact while indexing is not. For sequence lengths up to $S=[s_1,\ldots,s_L]$6, Qwen3.5 models achieve 78%, 96%, and 100% counting accuracy at 2B, 4B, and 9B respectively. Yet at $S=[s_1,\ldots,s_L]$7, letters, endpoint anchors, forward endpoint retrieval $S=[s_1,\ldots,s_L]$8 ranges from 22.9% for Qwen3.5-2B to 83.9% for Qwen3.5-9B, while backward endpoint retrieval $S=[s_1,\ldots,s_L]$9 ranges only from 5.8% to 16.1%. Relative item$\pi^t$0position retrieval at $\pi^t$1 reaches only 15.2% accuracy for Qwen3.5-9B. Confusion heatmaps for backward tasks collapse into vertical stripes, indicating positional collapse onto a few canonical answers rather than random noise (Zhang et al., 8 May 2026).

The paper emphasizes that this failure persists even in extremely short contexts. Claude Opus 4.6 misidentifies the second-to-last letter 27% of the time on four-letter prompts of the form “What is the second-to-last letter?” For a two-line Python if-elif branch, Claude Opus 4.6 misidentifies the second-to-last line most of the time, despite the snippet being trivial. This is presented as a paradox relative to long-context content retrieval: models can find a single relevant fact buried among hundreds of thousands of irrelevant tokens, yet often fail to retrieve the last few items in a short list (Zhang et al., 8 May 2026).

Mitigation by post-training is partial. PosBench, a position-focused supervised fine-tuning mixture, contains $\pi^t$2 synthetic retrieval examples, $\pi^t$3 code-retrieval examples, and $\pi^t$4 adapted real-data retrieval examples, with the sampling biased toward backward and relative addressing. LoRA fine-tuning with rank $\pi^t$5 and $\pi^t$6 improves both forward and backward retrieval. On the held-out code benchmark PyIndex, Qwen3.5-4B improves from 32.2% overall accuracy to 70.8% after LoRA; backward indexing improves from 17% to 71%, expression indexing from 22% to 68%, chained indexing from 23% to 72%, and nested indexing from 6% to 44%. The paper nevertheless states that absolute performance remains far from saturated (Zhang et al., 8 May 2026).

4. Order-sensitive factual retrieval: the Reversal Curse

A closely related phenomenon is the Reversal Curse in autoregressive factual generalization. The core claim is that if a model is trained on a sentence of the form “A is B,” it will not automatically generalize to the reverse direction “B is A.” In the paper’s notation, if $\pi^t$7 is a name and $\pi^t$8 is a description, then when a model is trained only on “$\pi^t$9 is $\mathcal{P}_2(\mathbb{R}^{d+2})$0,” the probability of generating $\mathcal{P}_2(\mathbb{R}^{d+2})$1 from $\mathcal{P}_2(\mathbb{R}^{d+2})$2 is not higher than for a random alternative name $\mathcal{P}_2(\mathbb{R}^{d+2})$3: $\mathcal{P}_2(\mathbb{R}^{d+2})$4 The failure is thus directional and tied to the order in which the factual association was observed (Berglund et al., 2023).

Controlled finetuning experiments on synthetic celebrity facts make the asymmetry explicit. For GPT-3-175B, exact-match accuracy on held-out prompts is $\mathcal{P}_2(\mathbb{R}^{d+2})$5 in the same direction for NameToDescription facts and $\mathcal{P}_2(\mathbb{R}^{d+2})$6 in the same direction for DescriptionToName facts, but reverse-direction accuracy collapses to $\mathcal{P}_2(\mathbb{R}^{d+2})$7 and $\mathcal{P}_2(\mathbb{R}^{d+2})$8 respectively. Log-probability comparisons show no significant difference between the correct reversed answer and a random name in any of 12 finetuning seeds. The phenomenon is robust across GPT-3 sizes, LLaMA-1 models, multiple hyperparameters, and heavy paraphrase augmentation (Berglund et al., 2023).

Real-world evaluation on celebrity parent–child relations shows an analogous asymmetry. GPT-4 correctly answers child$\mathcal{P}_2(\mathbb{R}^{d+2})$9parent questions 79% of the time, compared with 33% for the reverse parent$\mathfrak{s}$0child direction. The paper interprets this as evidence that the model learns directional conditional patterns from pretraining corpora rather than a symmetric relational representation. It also notes an important boundary condition: if “A is B” appears in-context, models can deduce the reverse relationship. The curse is therefore about parametric knowledge, not about all forms of in-context reasoning (Berglund et al., 2023).

In relation to the Position Curse, the Reversal Curse is not a literal indexing problem but an order-based one. The commonality is that a representation sufficient for one directional query is not automatically usable for the inverse query. This suggests that positional and relational symmetry is not reliably internalized by standard autoregressive next-token training.

5. Positional geometry in optimization and high-dimensional learning

In “Curse of Dimensionality in Neural Network Optimization,” the relevant “position” is the parameter distribution $\mathfrak{s}$1 of a two-layer mean-field network,

$\mathfrak{s}$2

evolving under 2-Wasserstein gradient flow. For target functions $\mathfrak{s}$3 with $\mathfrak{s}$4, the paper proves lower bounds on the decay of population risk. With globally Lipschitz activation,

$\mathfrak{s}$5

so the population risk cannot decay faster than $\mathfrak{s}$6. For locally Lipschitz activations with local Lipschitz constants $\mathfrak{s}$7,

$\mathfrak{s}$8

so the rate cannot exceed $\mathfrak{s}$9. The paper interprets this as a curse of dimensionality in optimization time: even for smooth targets, moving parameter mass in Wasserstein space to the region encoding a good approximation becomes fundamentally slow in high dimension (Na et al., 7 Feb 2025).

Other recent work shows that such curses can often be weakened by changing the representation class rather than the data alone. In unsupervised feature selection, feature-wise discriminability

$X_\pm = \pm \frac{\mathfrak{s}}{2}.$0

induces a feature-wise intrinsic dimensionality

$X_\pm = \pm \frac{\mathfrak{s}}{2}.$1

Features with low $X_\pm = \pm \frac{\mathfrak{s}}{2}.$2 are interpreted as more resilient to concentration and therefore more useful for learning procedures (Stubbemann et al., 2023).

In structured density estimation, the analogous structural quantity is graph resilience $X_\pm = \pm \frac{\mathfrak{s}}{2}.$3, defined through the shortest disintegration of an undirected graph. For $X_\pm = \pm \frac{\mathfrak{s}}{2}.$4-Lipschitz densities Markov with respect to $X_\pm = \pm \frac{\mathfrak{s}}{2}.$5, the effective exponent in sample complexity depends on $X_\pm = \pm \frac{\mathfrak{s}}{2}.$6 rather than the ambient dimension $X_\pm = \pm \frac{\mathfrak{s}}{2}.$7; the rate becomes $X_\pm = \pm \frac{\mathfrak{s}}{2}.$8 instead of the unstructured $X_\pm = \pm \frac{\mathfrak{s}}{2}.$9 regime (Vandermeulen et al., 2024). In kernel methods, Isolation Kernel is reported as the only kernel in the cited comparison with a provable distinguishability guarantee independent of dimension and distribution,

$\Psi(x)$0

contrasting with the indistinguishability of metric-based Lipschitz continuous kernels in high dimension (Ting et al., 2021).

Taken together, these results support a broader interpretation: position-sensitive failures are frequently not absolute impossibility theorems. They arise when the chosen feature map, measurement basis, or optimization geometry does not expose the relevant positional degrees of freedom efficiently.

6. Mitigation strategies and unresolved questions

The dominant mitigation strategy in optics is basis redesign. Rehacek et al. show that projections onto orthogonalized derivatives of the PSF retain a constant Fisher information $\Psi(x)$1 for all separations, including $\Psi(x)$2, while direct intensity detection does not (Rehacek et al., 2016). Paúr et al. show that a signum phase filter can reshape any well-behaved symmetric PSF into one whose near-origin intensity is parabolic, improving the small-separation Fisher scaling from quadratic to linear under ordinary intensity detection (Paur et al., 2018). The common lesson is that the curse is measurement-dependent.

In LLMs, the current evidence points to partial but incomplete remediation. Position-focused post-training with PosBench and LoRA improves both synthetic list indexing and held-out code indexing, yet backward and nested indexing remain far from ceiling. The paper therefore argues that position-based retrieval should become a key capability for future pretraining objectives and model design, especially for coding agents operating over large codebases (Zhang et al., 8 May 2026). The mechanistic hypothesis advanced there is that counting and forward indexing can be supported by forward ordinal information in the residual stream, whereas backward retrieval requires position subtraction, which is difficult under RoPE-style architectures because positional information primarily affects attention keys and queries rather than values.

For order-dependent factual generalization, simple augmentation does not suffice. The Reversal Curse is reported as robust across model sizes and model families and not alleviated by data augmentation. The open questions posed there concern whether non-autoregressive architectures behave similarly, how pervasive the effect is across relation types, and whether training objectives can be modified to encode symmetry through explicit regularizers, contrastive objectives, or structured relational memory (Berglund et al., 2023).

Across these domains, a shared conclusion emerges. Position-related failure modes are often strongest under default protocols that privilege one basis, one traversal direction, or one geometry. They are frequently weakened when the protocol is changed to one that directly represents derivatives, backward offsets, graph structure, or density-adaptive neighborhoods. This suggests that “Position Curse” is less a single theorem than a recurrent diagnostic for representational misalignment between a task’s latent positional structure and the machinery used to extract it.