Intentional inclusion of copyrighted bestsellers in LLM training data

Ascertain whether companies developing large language models intentionally include copyrighted bestselling books in their training datasets, distinguishing deliberate curation from incidental inclusion via large-scale web scraping to clarify the provenance of model exposure to such works.

Background

The EchoTrace benchmark includes 35 full-length books spanning public domain works, copyrighted bestsellers, and recent non-training titles. This composition is designed to evaluate verbatim memorization in LLMs across texts likely and unlikely to be present in training corpora.

Within this setup, the authors explicitly note uncertainty regarding whether companies intentionally incorporate copyrighted bestsellers into training data. Resolving this uncertainty is important for interpreting extraction results and for understanding whether observed reproductions arise from deliberate inclusion or from incidental exposure through web-scraped content.