Is Position Bias in Dense Retrievers Built In-or Learned from Data?

Abstract: Dense retrievers exhibit positional bias, favoring documents whose query-relevant information appears near the beginning and degrading retrieval performance when the information appears later. While prior work on positional bias in dense retrievers has largely focused on architectural explanations, we study how the positional distribution of evidence in training data affects retrieval-level bias direction. To test this, we construct synthetic position-targeted training sets in which query-relevant evidence appears at the beginning, middle, or end of documents, and fine-tune eight architecturally diverse pretrained models under position-skewed and balanced training distributions. At the ranking level, we observe a strong directional pattern across the examined models: skewed training distributions favor evidence at the corresponding positions. Position-balanced training reduces positional sensitivity by 57--87\% on position-aware benchmarks, with competitive mean retrieval performance in our controlled setting. Representation-level analyses further suggest that fine-tuning often reshapes learned positional preferences, although pre-existing architectural or pretraining-specific tendencies persist in some models. These results identify training-position distribution as a major controllable factor in retrieval-level position bias and suggest balanced data curation as a practical mitigation strategy.

• The paper finds that dense retrievers' position bias is learned from the positional distribution in training data rather than being an inherent model property.
• Experiments across eight architectures show that position-balanced training reduces sensitivity by 57–87%, ensuring more consistent retrieval performance.
• The study highlights that balanced evidence positioning maintains competitive mean performance while mitigating retrieval vulnerabilities caused by skewed training data.

Summary of "Is Position Bias in Dense Retrievers Built In-or Learned from Data?" (2605.26578)

Dense retrievers (DRs) in information retrieval (IR) are widely used for open-domain QA and retrieval-augmented generation, but consistently favor documents where query-relevant content is positioned toward the beginning. This study rigorously investigates whether this position bias is an intrinsic property of model architecture and pretraining, or is primarily learned from the positional distribution of training data. Leveraging position-controlled synthetic datasets, eight heterogeneous DR architectures were fine-tuned under systematically varied position distributions. The results provide strong evidence that retrieval-level position bias aligns with the training data—bias is not built in, but predominantly learned.

Figure 1: Position-wise nDCG@10 across training configurations for SQuAD-PosQ and FineWeb-PosQ, illustrating the impact of training skew on retrieval bias.

---

Experimental Setup

Eight DRs, encompassing both encoder (e.g., BERT-base, ModernBERT-base/large, Longformer-base) and decoder (e.g., GPT-2-medium, BLOOM-560M, TinyLlama-NoPE, Qwen3-0.6B) architectures, varied positional encoding method and document pooling strategies. Models were fine-tuned on synthetically generated datasets derived from Wikipedia, where relevant information was explicitly located at the beginning, middle, or end of documents. Each architecture was trained on four configurations: begin-skewed (100% begin), mid-skewed (100% middle), end-skewed (100% end), and position-balanced (uniform) distributions.

Position-control in data was achieved via a three-stage protocol: corpus binning, persona-conditioned query generation, and multi-reranker segment verification. Fine-tuning was performed using InfoNCE loss with chunk-aware negatives and held hyperparameters.

---

Main Results

Training Data Directs Retrieval Position Bias

Across all eight models, retrieval performance was maximized where the model had seen relevant evidence during training; begin-trained models favored early evidence, mid-trained models favored evidence in the middle, and end-trained favored later evidence. This directional effect held irrespective of architecture, positional encoding, or pooling strategy.

Position-Balanced Training Substantially Reduces Positional Sensitivity

Position sensitivity, measured by the Position Sensitivity Index (PSI), dropped by 57–87% in the uniform training setting relative to the most skewed configuration (see Table 1 in the paper). Uniformly trained models produced flat nDCG@10 curves across all document positions, signifying robustness to physical evidence location.

Retrieval Robustness Does Not Sacrifice Mean Performance

Balanced models maintain competitive or superior mean nDCG@10 on position-aware benchmarks. While skewed models excel at their trained position, they experience notable performance drops elsewhere, creating instability for retrieval-augmented downstream applications.

Benchmark-Specific Biases May Favor Skewed Priors

Evaluation on BEIR datasets revealed that early-skewed training can outperform balanced models if most annotated evidence is concentrated near the beginning of documents (as in HotpotQA, FEVER). However, this does not indicate evidence-location robustness, but rather exploitation of dataset skew.

Figure 2: Position-wise nDCG@10 for ModernBERT-base under four pooling strategies, consistently showing that fine-tuning position distribution determines retrieval bias.

---

Representation-Level Analyses

Query-Document Embedding Similarity Reflects Positional Preferences

Cosine similarity analyses moving evidence location across a document demonstrate that the embedding space peaks for positions aligned with training data, mirroring ranking-level findings.

Fine-Tuning Reshapes Document Representations

Document embeddings, even before query input, show increased similarity to segments emphasized during training post-fine-tuning, confirming that training alters positional preferences at the representation level.

Figure 3: Segment-wise cosine similarity heatmap for all eight models, showing fine-tuning-induced positional reshaping of document embeddings.

---

Implications, Limitations, and Future Directions

Practical Implications

Position bias is a major vulnerability for DRs—skewed training diminishes robustness, especially for retrieval-augmented systems (RAG) in real-world or adversarial settings. Data curation is a practical mitigation: balanced evidence position during training yields stable retrieval and minimizes spurious position-based shortcuts. Benchmark design must account for evidence position or risk misleading efficacy conclusions.

Theoretical Implications

Architectural factors alone do not fix position bias direction; empirical evidence indicates that position bias is primarily a learned shortcut. Fine-tuning can reverse or override weak pretraining tendencies, supporting theories that inductive biases are subordinate to training distribution for this phenomenon.

Future Directions

• Extension to multilingual and domain-specific retrieval under position-controlled settings.
• Human validation and analysis of downstream robustness in retrieval-augmented generation.
• Investigating interplay with other shortcut biases (e.g., length, literalness).
• Design of benchmarks accounting for and measuring retrieval robustness to evidence location.

  Figure 4: Position-wise nDCG@10 on PosIR domains, confirming that position-skewed training leads to matching directional bias.

  

  Figure 5: Evidence start-position distributions across BEIR datasets, highlighting skew toward beginning and its consequences for model evaluation.

---

Conclusion

This paper establishes that position bias in dense retrievers is predominantly learned from the positional distribution of training data, not an immutable artifact of architecture or pretraining. Fine-tuning under skewed distributions induces matching retrieval bias across diverse DR architectures. Position-balanced training robustly mitigates position sensitivity without sacrificing retrieval efficacy. The findings underscore the criticality of training data curation and bespoke evaluation design for building position-robust retrieval systems and reliable retrieval-augmented AI applications.