Source Bias in Information Systems

• Source bias is systematic distortion arising from the differential visibility, selection, and interpretation of sources in various domains.
• It spans multiple fields—such as search, media, weak supervision, and scientific inference—highlighting diverse operational definitions and measurement techniques.
• Practical studies address source bias through techniques for evaluating source diversity, bias-aware analysis, and debiasing methods to improve fairness and accuracy.

Source bias is a cross-domain term for systematic distortion attributable to the origin, selection, weighting, or interpretation of sources rather than to the target phenomenon alone. In search and recommendation, it often denotes a “systematic prevalence of specific information sources” or a ranking preference for one source type over another; in media studies it includes selection of events, outlets, and reused materials; in weak supervision and clinical machine learning it refers to group-dependent bias in labeling sources or observation processes; and in scientific inference it can denote source-linked distortions such as source-lens clustering, source-model mismatch, or source-localization miscalibration (Urman et al., 2021, Shin et al., 2023, Chang et al., 2022, Valageas, 2013, Gair et al., 2015). The literature therefore does not present a single universal taxonomy. A common thread is that bias arises because some sources are made more visible, more trusted, more reusable, or more measurable than others, with downstream effects on fairness, calibration, and inference.

1. Conceptual scope

Across the literature, source bias is operationalized at several distinct but related levels. Some work studies source prevalence in ranked outputs, some studies source selection in editorial or retrieval pipelines, some studies source-conditioned judgment, and some studies source-generated measurement error. This suggests that source bias is best treated as a family of mechanisms rather than a single scalar property.

Domain	Operational meaning of source bias	Representative papers
Search and retrieval	Prevalence, concentration, or suppression of source domains in ranked outputs	(Urman et al., 2021, Singh et al., 26 Aug 2025)
News and media	Selection of events, outlets, reused material, or provenance relations	(Galeazzi et al., 2023, Bourgeois et al., 2019, Zhukova et al., 4 Aug 2025)
Datasets and weak labels	Bias in dataset content, testing processes, or labeling sources	(Pagliai et al., 2024, Chang et al., 2022, Shin et al., 2023)
Human evaluation and review	Judgments distorted by source labels or source cues	(Nahar et al., 28 May 2026, Alebachew et al., 25 Apr 2025)
Scientific and engineering inference	Source-linked estimator distortion, clustering bias, or model mismatch	(Valageas, 2013, Gair et al., 2015, Lahtinen, 2024)

A recurrent methodological lesson is that source bias often remains latent when analysis focuses only on final outputs. The relevant distortion may enter earlier, through which domains are surfaced, which events are covered, which tests are ordered, which labeling functions are trusted, or which source cues evaluators perceive.

2. Media, search, and recommendation systems

In information access systems, source bias is frequently studied as a problem of visibility allocation. A comparative audit of video search engines defined the target phenomenon as domain-level source diversity bias, measured as the mean number of distinct domains in top-10 results and as domain frequencies at top-1 and top-10 ranks. Using 62 queries, 31 in English and their Russian translations, the study found that English queries produced more diverse outputs than Russian queries across all five engines; YouTube dominated top results on nearly all engines; and Google, while more diverse in raw domain count, returned no sampled top-10 results from Vimeo, Dailymotion, or Rutube, unlike the other engines. The authors treated that pattern as suggestive, rather than causal proof, of possible own-content bias related to YouTube (Urman et al., 2021).

News-source bias has also been formalized as a distinction between narrative bias and selection bias. In a six-year dataset on the Italian vaccine debate containing 682 sources, about 353,530 vaccine-related pieces of content, and 95,230,911 user interactions, narrative bias was modeled through anti-vax, neutral, and pro-vax framing, whereas selection bias was modeled through source propensities to report adverse, neutral, or positive events. The paper found that third-party source evaluations tracked the narrative-bias dimension much more closely than the selection-bias dimension, and that more extreme narrative or selection positions were associated with higher engagement. It also reported common audiences among outlets with similar ideological positions, indicating that source bias structures both production and consumption (Galeazzi et al., 2023).

A related line of work treats media bias as event selection behavior. Using GDELT 2.0 source-event matrices and Bayesian Personalized Ranking with matrix factorization, one study modeled source bias as a latent preference over events and reported AUC greater than 90% across five one-week periods, outperforming popularity and k-NN baselines. The learned source representations clustered by geography, ownership, syndication, and medium. On the mitigation side, Maximum Marginal Relevance over latent source embeddings increased the events-to-articles ratio from 0.41 to 0.60 for top-25 sources and from 0.22 to 0.44 for top-100, while reducing GINI from 0.79 to 0.74 and from 0.78 to 0.68, respectively (Bourgeois et al., 2019).

Source bias has recently been extended to mixed ecosystems of human-generated content and AI-generated content. In sequential recommendation, the relevant source distinction is between HGC and AIGC. The paper defined a source-preference metric

$$\text{Relative }\Delta = \frac{Metric_\text{HGC} - Metric_\text{AIGC}}{(Metric_\text{HGC} + Metric_\text{AIGC}) / 2} \times 100\%,$$

with negative values indicating AIGC preference. Across BERT4Rec, SASRec, GRU4Rec, and LRURec on Amazon Health, Beauty, and Sports, Relative $\Delta$ was usually strongly negative. In a feedback loop with a position-based click model,

$$P(E=1 \mid \text{rank}(i)=k)=\left(\frac{1}{k}\right)^\eta,$$

AIGC preference intensified as AIGC entered histories and retraining data, producing a “source echo chamber.” A debiasing objective based on L1 rewrite-invariance on both item and user sides stabilized source balance in the loop (Zhou et al., 2024).

3. Source-aware analysis tools and provenance models

Several systems focus not on measuring source bias as a single score, but on making source-conditioned framing or provenance legible. A bias-aware news reader operationalized bias by word choice and labeling (WCL) through target-dependent sentiment classification. Its five-task workflow—article gathering, preprocessing, target concept analysis, frame identification, and visualization—uses Stanford CoreNLP, semantic-concept resolution, and a news-adapted BERT model to classify whether a target is portrayed positively or negatively in a sentence. The system’s overview histograms encode concept frequency and aggregated sentiment, while the article view highlights local mentions. The paper is explicit that this is only a high-level effect of WCL bias, not a complete account of media bias (Hamborg et al., 2021).

An earlier bias-aware news recommendation system combined source-level ideological placement with article-level lexical bias scoring. Source position came from a Pew Research readership spectrum spanning liberal to conservative outlets, and article bias was computed as the proportion of sentences containing non-NPOV lexicon terms from prior Wikipedia-based work. Topic matching used LDA topic distributions with cosine distance, and the interface placed source ideology on the x-axis and article bias score on the y-axis. The implementation used five sources—New Yorker, New York Times, BBC, Fox News, and Breitbart—and reported a preliminary user study with 5 users whose rankings correlated qualitatively with the system’s bias scores (Patankar et al., 2018).

Provenance-centric work has framed source bias as a problem of recovering information flow across articles. The COSS proposal treats bias by commission, omission, and source selection as a joint objective. Its pipeline comprises candidate retrieval from sources such as LexisNexis, CommonCrawl, MediaCloud, or GDELT; paragraph- or sentence-level text alignment for paraphrased reuse; polarity relabeling; construction of a directed text-reuse graph; and pattern analysis. Source selection bias is then studied via which sources and polarities are reused, which paragraphs remain unsourced, and which source material is omitted. The paper is conceptual rather than fully evaluated, but it makes explicit that source bias can be modeled as provenance rather than only as explicit citation (Zhukova et al., 4 Aug 2025).

Agentic retrieval systems introduce a more operational source-selection problem. A multi-agent Bias Mitigation Agent represented each candidate document as

$d_i = (\chi, \rho, \beta, \gamma),$

where $\chi$ is text content, $\rho$ relevance, $\beta$ bias confidence, and $\gamma \in \{0,1\}$ a binary bias label. In zero-shot mode, the selector chose

$$\alpha = \arg\max_{c_i \in \mathcal{C}'} \rho_i, \qquad \mathcal{C}'=\{c_i \in \mathcal{C} \mid \gamma_i=0 \land \beta_i \ge 0.7\}.$$

On MBIC and BABE-derived news corpora with 112 queries, GPT-4o-mini in zero-shot mode reduced output bias rate from 49.11% to 8.93%, a relative reduction of 81.82% compared to naïve relevance-only retrieval (Singh et al., 26 Aug 2025).

4. Datasets, labels, and weak supervision

A major strand of work locates source bias in the data and labeling pipeline itself. A multilingual audit using the bipol metric defined dataset-level bias through a classifier-based prevalence term and a lexicon-based sensitive-term imbalance term. The study introduced translated MAB-based datasets for Italian, Dutch, and German, each derived from an English source and together contributing almost 6 million labeled samples across five languages. Using mT5 and mBERT, it reported mT5 validation macro-F1 values from 0.768 to 0.787 across languages and found measurable bias in all 10 evaluated datasets, including English GLUE/SuperGLUE-related corpora such as CoLA, QNLI, MRPC, MNLI, CB, and ReCoRD. The paper further stated that many datasets showed overall male bias, while also noting that translated corpora may miss culture-specific bias patterns (Pagliai et al., 2024).

Clinical machine learning exposes a related form of source bias through disparate censorship and undertesting. When labels are derived from clinician-ordered tests and untested patients are assigned negative labels, the observed label becomes

$\tilde y = y \cdot t.$

The paper defined disparate censorship by unequal testing rates across groups and undertesting by unequal testing at equal risk. Its theory showed that performance gaps arise when higher-risk groups are undertested or when testing boundaries are not aligned with true subgroup-specific risk boundaries. In MIMIC-IV, Black patients were less likely than White patients to receive several common tests, including CBC (73.71% vs 68.20%), BMP (71.26% vs 63.72%), ABG (13.75% vs 10.42%), and blood cultures (15.20% vs 13.01%), making source bias in labels a property of the observation process rather than of the learner alone (Chang et al., 2022).

Weak supervision formalizes source bias at the level of labeling functions. Starting from a standard label model

$\Delta$0

the paper proposed a source-bias model in which source accuracy decays with distance from LF-specific centers under group transformations:

$\Delta$1

It then proved that even when a fair fully supervised model is possible, the corresponding weakly supervised dataset can become arbitrarily unfair because some labeling sources degrade to random guessing on one group. Its Source Bias Mitigation procedure, based on groupwise source-accuracy estimation and optimal-transport-style correction, was reported to improve accuracy on weak supervision baselines by as much as 32% while reducing demographic parity gap by 82.5% (Shin et al., 2023).

5. Human and organizational decision processes

In organizational decision systems, source bias often appears as a property of the human decision rule that later propagates into data or automation. In online micro-lending, a structural econometric model separated preference-based bias from belief-based bias in gendered approval decisions. The estimated parameters

$\Delta$2

indicated that both bias types favored female applicants. Counterfactual simulations showed that removing both biases increased expected company profit from 151046.8 to 160190.4 and reduced the gender gap in true positive rates from 9.69% to 2.48%. Training XGBoost on data generated under these counterfactuals suggested that machine learning could partially mitigate, but not erase, upstream human source bias (Hu et al., 2022).

Software engineering work has proposed analogous source-bias detection in code review. The relevant source is the perceived code author rather than the code artifact itself. A controlled experimental design was proposed in which pull requests are randomly labeled as authored by different contributor groups, while review behavior is captured through the Tobii Pro Fusion eye tracker, cursor movement, keyboard logs, and comments. The proposal uses the spotlight model of attention and sequence models including Markov Models, RNNs, and CRFs to detect potentially biased review interactions. The work is explicitly a proposal rather than a completed empirical study, but it frames source bias as prejudice-induced distortion of ostensibly technical judgments (Alebachew et al., 25 Apr 2025).

A more direct test of source-conditioned evaluation used logical fallacy judgments under five authorship conditions: Human, AI, Human+AI, AI+Human, and no disclosure. In an online study with $\Delta$3, the human Condition × Fallacy interaction was $\Delta$4, $\Delta$5, $\Delta$6. The fallacy penalty dropped from 1.47 in the control condition to 0.42 in the Human condition and 0.49 in the Human+AI condition, while trust and overall evaluation were also highest for human-linked labels. GPT-5.2, Gemini 2.5 Flash, and Claude Sonnet 4.5 remained comparatively stable across source labels, though model-specific strictness differed (Nahar et al., 28 May 2026).

A related methodological usage concerns sources of bias rooted in shared delivery systems rather than in information origin. In pragmatic clinical trials, negative spillover arises when intervention and control groups share scarce resources and the intervention changes utilization. The paper identified four necessary conditions: shared resources, scarcity, intervention-driven resource use, and resource-sensitive outcomes. It argued that such spillover can harm controls and overestimate treatment effects, and that it is distinct from contamination and can occur even in double-blind studies (Mann, 2023).

6. Scientific and engineering inference

In scientific inference, source bias often denotes estimator distortion tied to source geometry or source-model mismatch. In gravitational-wave population inference, using an incorrect waveform model produces systematic parameter shifts that may appear acceptable for single events but become visible over populations as a P-P plot sag below the diagonal. Under the exact likelihood, calibration satisfies

$\Delta$7

With approximate waveform models, the P-P curve sags; with a Gaussian-process-marginalized likelihood, the sag is reduced, and when waveform errors follow the GP assumptions exactly, calibration is restored (Gair et al., 2015).

Weak-lensing cosmology provides another source-linked estimator bias. There, source galaxies are clustered with the matter density field that lenses them, producing source-lens clustering bias. For two-point statistics, the paper found that this bias is typically several orders of magnitude below the weak-lensing signal, except when correlating a very low-redshift galaxy with a higher-redshift galaxy, where it can reach 10% of the signal for shear. For three-point statistics, source-lens clustering bias is typically of order 10% of the signal as soon as the source redshifts are not identical, and intrinsic-alignment bias is also typically about 10%. Both therefore matter for better than 10% accuracy in three-point estimators (Valageas, 2013).

Electromagnetic source localization treats source bias as mislocalization of the reconstruction maximum. In a discretized Poisson-type inverse problem with

$\Delta$8

the paper showed that lead-field scale, geometry, and regularization can shift maxima toward the measurement boundary. Standardization rescales the estimate by the resolution diagonal,

$\Delta$9

thereby removing unfair advantages of larger or more sensitive columns. Within the paper’s Gaussian and single-source framework, this can achieve perfect localization in idealized noiseless settings, but the resulting high-value region is wider than for unstandardized BMNE, and robustness is best near sensors rather than in far-field regimes (Lahtinen, 2024).

Taken together, these literatures suggest that source bias is best understood as a structured distortion introduced by where information comes from, which sources are selected, how source-specific signals are observed, or how source-conditioned estimators are constructed. The practical implication is that bias analysis cannot be confined to final predictions or final rankings alone. It must also interrogate source visibility, source provenance, source-generated labels, source cues in human judgment, and source-specific calibration.