Extracting books from production language models (2601.02671v1)

Abstract: Many unresolved legal questions over LLMs and copyright center on memorization: whether specific training data have been encoded in the model's weights during training, and whether those memorized data can be extracted in the model's outputs. While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We investigate this question using a two-phase procedure: (1) an initial probe to test for extraction feasibility, which sometimes uses a Best-of-N (BoN) jailbreak, followed by (2) iterative continuation prompts to attempt to extract the book. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction success with a score computed from a block-based approximation of longest common substring (nv-recall). With different per-LLM experimental configurations, we were able to extract varying amounts of text. For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone), while it was necessary for Claude 3.7 Sonnet and GPT-4.1. In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%). GPT-4.1 requires significantly more BoN attempts (e.g., 20X), and eventually refuses to continue (e.g., nv-recall=4.0%). Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production LLMs.

• The paper demonstrates that production LLMs can extract nearly complete books, with up to 95.8% memorization observed using adversarial techniques.
• The study introduces a two-phase methodology that leverages ground-truth seeds and blockwise verification to measure near-verbatim text output.
• The findings challenge assumptions on AI safeguards by highlighting technical and legal risks related to copyright infringement and fair use defenses.

Extracting Books from Production LLMs: Technical Overview and Implications

Introduction and Motivation

The paper "Extracting books from production LLMs" (2601.02671) provides an empirical and methodological investigation into the ability of production LLMs—specifically Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3—to memorize and subsequently regurgitate substantial amounts of in-copyright material included in their training data. The study problematizes legal defenses (e.g., fair use and transformativeness) commonly leveraged by vendors, highlighting the disconnect between assumed model behaviors and observed outcomes. The authors systematically test whether system- and model-level safeguards in production LLMs suffice to prevent extraction of verbatim books, and develop a rigorous quantitative methodology for measuring near-verbatim output at scale.

Extraction Procedure: Two-Phase Methodology

Extraction is framed as a two-phase process:

• Phase 1: Test the model's ability to complete a short, ground-truth-sourced seed (typically the book's first sentence(s)) using an explicit instruction. For Gemini 2.5 Pro and Grok 3, no jailbreak was necessary. However, Claude 3.7 Sonnet and GPT-4.1 required a Best-of-$N$ (BoN) adversarial strategy, where the prompt is perturbed $N$ times (using typographical, formatting, and textual transformations) until a non-refusal output aligns with the target suffix at $s \geq 0.6$ based on normalized longest-common-substring similarity.

  Figure 1: Phase 1 initiates extraction by prompting the LLM to continue a brief ground-truth prefix; Gemini 2.5 Pro/Grok 3 respond directly, while Claude 3.7 Sonnet/GPT-4.1 require BoN jailbreaks.

• Phase 2: Upon Phase 1 success, the model is repeatedly queried for continuation of its previous output, assembling long-form generations. This process halts on refusal, a detected stop phrase, or an exhausted query budget.

  Figure 2: Phase 2 continues iterative generation, concatenating outputs until refusal (or stop phrase), culminating in long-form text compared blockwise to the reference book.

The evaluation of success centers on near-verbatim recall ($nv{$), computed via a greedy blockwise approximation of longest common substring, filtered to retain only blocks $\geq100$ words—ensuring claims are robust to coincidental overlap.

Empirical Results: Extent and Nature of Memorization

Robust extraction of memorized content was observed across all four LLMs, with significant inter-model and intra-model variance attributable to generation configuration and jailbreak strategy. Notably:

• For "Harry Potter and the Sorcerer's Stone", nearly the entire book ($95.8\%$) was extracted from jailbroken Claude 3.7 Sonnet.
• Gemini 2.5 Pro and Grok 3 yielded large swaths ($76.8\%$ and $70.3\%$, respectively) without jailbreaks, contradicting the expectation that safety-tuned production models refuse verbatim continuation requests.
• GPT-4.1 exhibited robust initial compliance under jailbreak ($4.0\%$ for single full-book seed), but output generation was truncated upon model refusal after chapter boundaries.

  Figure 3: Relative proportions of "Harry Potter and the Sorcerer's Stone" extracted from four LLMs, with Claude 3.7 Sonnet approaching full verbatim reproduction.

The paper extends the methodology across thirteen books (mostly in-copyright), demonstrating extraction failures are the minority and predominantly linked to conservative system-level filters or jailbreak limitations rather than inherent inability to memorize.

Figure 4: $nv{$ for twelve books across LLMs; Claude 3.7 Sonnet regularly attains $\geq94\%$ on multiple full works, others exhibit variability based on model and configuration.

Absolute extraction counts indicate thousands to tens of thousands of near-verbatim words extracted per run, even when percentage recall is modest due to lengthy reference texts.

Figure 5: Absolute word counts for "Harry Potter and the Sorcerer's Stone"—showcasing the large volume of verbatim material extracted even for LLMs with low $nv{$.

The methodology is validated with control runs on negative examples (recently published works absent from all LLM training sets), which consistently fail to achieve Phase 1 thresholds, supporting claims that generation is contingent on memorization from training corpora rather than surface-level prompt engineering.

Technical Analysis: Metrics and Procedure Robustness

Extraction measurement is intentionally conservative, employing stringent block formation and filtering to avoid over-attributing coincidental overlap. The two-pass merge-and-filter procedure depicted in Figure 6 ensures that only long, well-aligned contiguous matches between generation and reference are considered valid instances of memorization-derived reproduction.

Figure 6: Stepwise visualization of blockwise merging and filtering, yielding final set of validated long near-verbatim spans in the output.

The block-based approach monotonically aligns generated output with reference passages, discounting both duplicated extraction and out-of-order reproduction, thus underestimating total memorized content but ensuring precision in attribution of model-based copying.

Jailbreak sensitivity is model-specific. BoN works efficiently with Claude 3.7 Sonnet, but requires orders-of-magnitude larger $N$ for GPT-4.1, with rapid onset of refusals as output context increases.

Figure 7: The number of BoN attempts ($N$) needed for Claude 3.7 Sonnet to reach Phase 1 similarity thresholds, underscoring divergence in model-level alignment and refusal policies.

Practical and Legal Implications

The results assert that:

Extraction of in-copyright, memorized data remains feasible in production LLMs regardless of model- or system-level safeguards.

Experimental outcomes contravene prevailing industry assertions that mainstream LLM deployment mitigates training-data leakage through alignment, refusal mechanisms, or filtering.

Implications are twofold:

• Technical: Empirical evidence indicates that memorization of long-form copyrighted material is ubiquitous and that refusal policies are insufficiently robust, especially under adversarial prompting. The findings establish a loose lower bound on the quantity of memorized content accessible via generic or jailbreak-enhanced continuation prompts.
• Legal/Policy: The technical capacity for extraction of near-verbatim, in-copyright works directly informs ongoing litigation related to generative AI and copyright infringement. The capacity for adversarial users to recover entire works or substantial portions recontextualizes debates on fair use and transformativeness, and challenges the sufficiency of "reasonable best efforts" as a legal or compliance defense.

Methodological Limitations and Future Directions

While the study is statistically and procedurally conservative, limitations persist:

• Experimental scale is constrained by API costs; some full-book extractions cost \$100+ per run.
• Extraction success is configuration-dependent; slight changes in generation parameters or prompt engineering can affect outcomes.
• Cross-model comparisons are not normalized for generation conditions, so relative memorization risk cannot be robustly ranked.

Future developments in LLM deployment should focus on systematic hardening of both system- and model-level defenses against verbatim reproduction, periodic red-teaming with adaptive jailbreaks, and deeper understanding of the relationships between model scaling, alignment protocols, and memorization propensity. Further work is necessary to extend these extraction quantification techniques to multimodal models, non-book textual domains, and continual learning scenarios.

Conclusion

The paper rigorously demonstrates that production LLMs—despite safety measures and refusal protocols—memorize and can be induced to verbatimly reproduce substantial in-copyright training data, including entire books, under both direct and jailbreak-enhanced prompting. The two-phase block-based methodology provides actionable lower bounds on extractable memorized material. These results reveal technical failure modes in AI content moderation at scale, with direct implications for the adjudication of fair use and copyright infringement in AI training and deployment contexts. Long-term, addressing extraction risk will require multifaceted advancements in data governance, red-teaming, and technical policy alignment.