Epistemic Blinding: An Inference-Time Protocol for Auditing Prior Contamination in LLM-Assisted Analysis

Abstract: This paper presents epistemic blinding in the context of an agentic system that uses LLMs to reason across multiple biological datasets for drug target prioritization. During development, it became apparent that LLM outputs silently blend data-driven inference with memorized priors about named entities - and the blend is invisible: there is no way to determine, from a single output, how much came from the data on the page and how much came from the model's training memory. Epistemic blinding is a simple inference-time protocol that replaces entity identifiers with anonymous codes before prompting, then compares outputs against an unblinded control. The protocol does not make LLM reasoning deterministic, but it restores one critical axis of auditability: measuring how much of an output came from the supplied data versus the model's parametric knowledge. The complete target identification system is described - including LLM-guided evolutionary optimization of scoring functions and blinded agentic reasoning for target rationalization - with demonstration that both stages operate without access to entity identity. In oncology drug target prioritization across four cancer types, blinding changes 16% of top-20 predictions while preserving identical recovery of validated targets. The contamination problem is shown to generalize beyond biology: in S&P 500 equity screening, brand-recognition bias reshapes 30-40% of top-20 rankings across five random seeds. To lower the barrier to adoption, the protocol is released as an open-source tool and as a Claude Code skill that enables one-command epistemic blinding within agentic workflows. The claim is not that blinded analysis produces better results, but that without blinding, there is no way to know to what degree the agent is adhering to the analytical process the researcher designed.

• The paper introduces epistemic blinding, a protocol that anonymizes named entities to isolate data-driven inferences from pre-trained priors.
• It implements a lightweight, model-agnostic string replacement and matched A/B prompt system, demonstrated in oncology and financial analyses.
• Empirical results reveal significant shifts in candidate rankings under blinding, underscoring the impact of prior contamination on LLM outputs.

Epistemic Blinding: Auditing Prior Contamination in LLM-Assisted Data Analysis

Introduction

LLMs inherently blend data-driven reasoning with extensive parametric priors learned during pretraining, a phenomenon acutely problematic when analyzing named entities with uneven training set representation. In the context of agentic LLM systems for biological and financial data analysis, this blending leads to “prior contamination”—outputs become amalgams of explicit input features and potentially untraceable background knowledge. This paper formalizes epistemic blinding, an inference-time protocol for distinguishing between data-driven inferences and prior-driven outputs by anonymizing entity identifiers before LLM engagement and systematically auditing resultant predictions (2604.06013).

Protocol Description and Implementation

Epistemic blinding replaces all named entity identifiers in input datasets (such as gene symbols, stock tickers) with deterministic, anonymous codes prior to model prompting. It enforces consistency across datasets, handles potential subtle feature leaks via normalization or binning, and preserves all contextually relevant data columns required for reasoning—excluding only identifiers and highly correlated features liable to reveal true identities. The protocol is both lightweight and model-agnostic, implementable as a straightforward string replacement followed by matched A/B prompt sessions.

Figure 1: Traditional unblinded workflows promote entities familiar from literature, whereas epistemic blinding enforces ranking strictly on quantitative feature signal, surfacing novel candidates.

A supporting open-source tool and Claude Code skill integration facilitate seamless adoption, automatically identifying entity columns and generating matched prompts for immediate auditability.

Application to Oncology Drug Target Identification

An LLM-driven pipeline was constructed for multi-modal, multi-dataset integration, scoring $\sim$20,000 human genes per disease across eight purely quantitative, publication-independent features, including somatic mutation data, GWAS associations, foundation model protein/transcriptomic embeddings, and genetic constraint metrics. Entity identifiers were excluded from all optimization and reasoning steps.

Scoring function discovery leveraged an LLM-based evolutionary optimization system (ShinkaEvolve) operating exclusively on blinded numerical features. Remarkably, the model independently converged on an evidence-first, hierarchical scoring paradigm, capturing both direct and network-proximal targets, with an average strict hit rate of $10/18$ diseases yielding at least one approved target in the top 20. At no stage did the model observe entity names, demonstrating the sufficiency of quantitative signal for effective optimization.

Quantifying LLM Prior Contamination

Matched blinded and unblinded LLM reasoning sessions were conducted for four oncology indications, with systematic analysis of top-20 gene rankings and justifications. Across all disease settings, the recovery of validated targets remained identical, but an average of 16% of top-20 nominations changed ranks under blinding.

In every instance, the directionality of rank shifts was consistent: unblinded models promoted well-known, literature-salient genes (e.g., KRAS, PTEN) and demoted obscure genes with strong quantitative features (e.g., DPP8, KCNH7).

Figure 2: In IDH-wildtype glioblastoma, unblinded LLMs systematically promote famous genes (orange lines) and demote data-strong, obscure candidates (blue lines), as shown by rank shifts between blinded and unblinded conditions.

This pattern is not stochastic but arises predictably with decreasing feature signal clarity—strong, unambiguous quantitation yields close agreement between conditions, while ambiguous feature landscapes amplify parametric knowledge injection into outputs.

Most explicitly, model explanations in the blinded condition cited only supplied features; unblinded justifications incorporated parametric knowledge absent from input data (e.g., references to “proven therapeutic tractability via covalent RAS inhibitors” for KRAS). Such leakage was not an artifact but the dominant mechanism differentiating entity ranking.

Generalization Beyond Oncology

To demonstrate domain-agnostic applicability, epistemic blinding was deployed in S&P 500 equity screening. All input financial metrics were normalized to minimize structural identifiability, and tickers alone were blinded. Across five seeds, a mean 35% of top-20 recommendations changed under blinding, with systematic promotion of brand-name companies and demotion of quantitatively strong, obscure tickers. This eliminated the hypothesis that such contamination is unique to biological analysis; any scenario with unevenly represented entities in the LLM pretraining corpus exhibits the same prior-driven bias.

Figure 3: S&P 500 analysis demonstrates that systematic, non-random biases in recommendations originate from parametric brand familiarity when ticker symbols are visible.

Theoretical and Practical Implications

Epistemic blinding clarifies one axis of model auditability absent from conventional LLM deployments. While it does not universally improve prediction “accuracy,” it enables researchers to measure the contribution of supplied data versus parametric memorization in their analytical pipelines—a critical distinction for hypothesis-driven workflows and unbiased candidate discovery. The protocol is especially consequential when:

• Supplied data contains sufficient signal for inference,
• Entities are unevenly represented in training data,
• Downstream decisions depend on LLM outputs.

Scenarios intentionally leveraging LLM knowledge (literature review, hypothesis suggestion) do not benefit from blinding, but whenever claims must be traceable to provided evidence, epistemic blinding is warranted.

Limitations

The protocol, as presented, was evaluated only with Claude (Anthropic); effects may differ slightly with alternative architectures. Only binary (fully blinded vs. unblinded) comparisons were considered—partial blinding strategies remain unexplored. Stochastic LLM output variance and the lack of ground truth for novel discoveries are additional caveats.

Future Directions

Epistemic blinding’s utility invites expansion to other domains where named entities are pivotal—clinical trial matching, legal document analysis, scientific paper triage, or candidate selection. Granular control and partial blinding methods can further refine the partitioning of feature and parametric influences, and model-specific behaviors merit further inspection. As LLMs permeate every stage of analytical pipelines, standardized reporting of blinded analysis results should be prioritized to ensure methodological transparency and reproducibility.

Conclusion

Epistemic blinding provides a tractable, model-agnostic mechanism for quantifying and mitigating prior contamination in LLM-assisted, data-driven workflows involving named entities. It should become routine in analytical contexts where attribution to supplied data is essential. The codebase is available at https://github.com/mcuccarese/epistemic-blinding, facilitating immediate adoption with minimal operational overhead.

Citation: "Epistemic Blinding: An Inference-Time Protocol for Auditing Prior Contamination in LLM-Assisted Analysis" (2604.06013).