Wikipedia in the Era of LLMs: Evolution and Risks (2503.02879v1)

Abstract: In this paper, we present a thorough analysis of the impact of LLMs on Wikipedia, examining the evolution of Wikipedia through existing data and using simulations to explore potential risks. We begin by analyzing page views and article content to study Wikipedia's recent changes and assess the impact of LLMs. Subsequently, we evaluate how LLMs affect various NLP tasks related to Wikipedia, including machine translation and retrieval-augmented generation (RAG). Our findings and simulation results reveal that Wikipedia articles have been influenced by LLMs, with an impact of approximately 1%-2% in certain categories. If the machine translation benchmark based on Wikipedia is influenced by LLMs, the scores of the models may become inflated, and the comparative results among models might shift as well. Moreover, the effectiveness of RAG might decrease if the knowledge base becomes polluted by LLM-generated content. While LLMs have not yet fully changed Wikipedia's language and knowledge structures, we believe that our empirical findings signal the need for careful consideration of potential future risks.

• The paper quantifies LLM-induced changes on Wikipedia, showing a measurable content shift and altered linguistic styles evident since 2020.
• It documents specific linguistic changes like reduced auxiliary verbs and increased lexical diversity and sentence complexity aligning with LLM writing patterns.
• The study demonstrates that LLM-modified content can negatively impact downstream NLP tasks, inflating machine translation scores and degrading retrieval-augmented generation performance.

This paper presents a comprehensive empirical investigation into how LLMs are directly and indirectly impacting Wikipedia and downstream NLP tasks.

• It quantifies a measurable LLM-induced content shift on Wikipedia, with simulation results showing an estimated impact of approximately 1%–2% in word frequency changes across several categories.
• It documents systematic changes in linguistic style—such as reduced auxiliary and “to be” verb usage, enhanced lexical diversity, and increased sentence complexity—that align with LLM preferences and are evident in trends from 2020 to 2025.
• It demonstrates that LLM-modified content can inflate machine translation evaluation scores and degrade retrieval-augmented generation performance, sometimes reversing model rankings and introducing accuracy errors in information retrieval tasks.