Detection of Data Poisoning in Large-Scale LLM Pipelines

Develop scalable techniques for preventing data poisoning, verifying the trustworthiness of training data, and detecting poisoned samples that resemble legitimate data in large-scale LLM pipelines.

Background

Data and model poisoning can persist through training datasets, model weights, embedding repositories, and retrieval corpora, allowing adversarial behavior to influence later inferences. The paper notes that the size and diversity of contemporary training datasets make it impractical to verify every source manually or with simple validation procedures. Poisoned examples may be deliberately crafted to appear legitimate, limiting the effectiveness of conventional anomaly-detection methods.

References

Preventing data poisoning in large-scale LLM pipelines remains an open challenge. The vast size and diversity of modern training datasets make it difficult to verify the trustworthiness of every data source. Additionally, poisoned samples can be designed to resemble legitimate data, making them difficult to detect using conventional anomaly detection methods.

— Shifting from Injection to Interaction: Rethinking Web Security in the Age of LLMs and Beyond  (2609.03999 - Singh et al., 3 Sep 2026) in Section 5, subsection LLM_04: Data and Model Poisoning, Open Research Challenges box