LLMSurgeon: Diagnosing Data Mixture of Large Language Models

Abstract: The pretraining data mixture of LLMs constitutes their "digital DNA", shaping model behaviors, capabilities, and failure modes. Yet this composition is rarely disclosed, making post-hoc auditing of data combination or provenance difficult. In this work, we formalize $\textbf{Data Mixture Surgery (DMS)}$: given only generated text from a target LLM, estimate the domain-level distribution of its pretraining corpus under a predefined taxonomy. We propose $\textbf{LLMSurgeon}$, a strong framework that casts DMS as an inverse problem under the label-shift assumption. Rather than directly aggregating classifier outputs, LLMSurgeon estimates a calibrated $\textit{soft}$ confusion matrix and solves a constrained inverse problem to correct systematic domain confusion and recover the latent mixture prior. To evaluate, we introduce $\textbf{LLMScan}$, a recipe-verifiable evaluation suite built from open-source LLMs with transparent pretraining mixtures. Across LLMScan, LLMSurgeon recovers domain mixtures with high fidelity under fixed protocols. Our work presents a practical, post-hoc approach for auditing the digital DNA of foundation models without access to their training data.

• The paper presents LLMSurgeon, a framework that estimates the domain-level composition of LLM pretraining data using a calibrated proxy classifier.
• It leverages inverse linear correction to overcome classifier confusion, achieving over 95% overlap accuracy in coarse-grained settings.
• The approach enables robust, post-hoc auditing for regulatory transparency, reproducibility, and risk management in LLM deployments.

Diagnosing LLM Pretraining Data Mixture with LLMSurgeon

Motivation and Problem Formulation

LLMs derive their generalization behavior, failure modes, and implicit biases from the composition of their pretraining corpora, which constitute the "digital DNA" of these models. Despite this, the actual domain-level proportions of the underlying data are often opaque—particularly in commercial or proprietary LLMs—obfuscating reproducibility, accountability, and risk management. Most prior post-hoc auditing work focuses on Membership Inference Attacks (MIAs), targeting instance-level exposure and failing to provide aggregate, distributional audits required for robust transparency (e.g., proportion of code, news, Wikipedia, etc.).

To address this, the paper formalizes the task of Data Mixture Surgery (DMS): estimating the semantic domain-level training data composition of an LLM given only its generated text, under a predefined taxonomy. The key statistical assumption is label shift, with domain priors changing across training and generation distributions while conditional domain-specific statistics remain stable.

Figure 1: Overview of Data Mixture Surgery problem and the LLMSurgeon framework for solving it.

The LLMSurgeon Framework

LLMSurgeon addresses DMS via an inverse problem grounded in label shift correction. The framework has three main phases:

1. Proxy Classifier Construction and Calibration: An external domain classifier is trained on reference data to output soft assignments. Its systematic semantic confusion is captured in a "soft confusion matrix" $C$, estimated using held-out reference samples.
2. Sampling and Classification: The target LLM is queried using neutral prompts designed to minimize prompt-style bias, generating a corpus of model outputs. The proxy classifier is used to obtain a noisy observation of the domain proportions from these outputs.
3. Inverse Linear Correction: Rather than naively aggregating classifier outputs—which would propagate systematic confusion and domain bias—LLMSurgeon solves a constrained least-squares inverse mapping using the confusion matrix to recover the latent mixture of the training domains.

   Figure 2: The LLMSurgeon framework pipeline for recovering pretraining data mixture from LLM-generated text.

A primary distinction from prior efforts is the domain-level focus, moving away from noisy pointwise MIA aggregation and targeting calibrated distributional recovery.

LLMScan Benchmark and Experimental Protocol

The authors introduce LLMScan, the first benchmark for DMS, leveraging open-source LLMs such as OLMo, LLaMA-1, Amber, Pythia, StarCoder, and GPT-Neo—spanning 1B to 65B parameters—with transparent, verifiable data mixture documentation. Evaluation spans coarse (6 classes), mid (17 classes), and fine (87 classes) granularities, capturing different levels of semantic overlap and domain confusion.

Key experimental setup choices include:

• Classifier architectures (DistilBERT variants significantly outperform simpler models).
• Per-model, per-domain sampling (neutral prompts to minimize style-induced bias).
• Multiple reconstruction metrics: Overlap Accuracy, Mean Absolute Error, $R^2$.
• Robust ablations: classifier capacity, prompt design, sample size, and calibration.

Empirical Results and Ablative Analysis

Across the LLMScan benchmark, LLMSurgeon consistently outperforms audit-by-aggregation baselines from the MIA literature by over 40% absolute accuracy in coarse-grained settings, achieving >95% overlap accuracy on general-purpose models. Even on fine-grained detection (87-language StarCoder taxonomy), LLMSurgeon exhibits superior robustness, although overall accuracy drops (about 30%), highlighting semantic indistinguishability as a hard limit.

Figure 3: Domain granularity strongly impacts mixture recovery precision; coarse domains are accurately recovered ($R^2=0.99$), while fine-grained categories with high similarity (C vs. C++) see sharply degraded correlation ($R^2=0.01$).

Figure 4: Throughout training, LLMSurgeon tracks domain mixing error, converging to ground-truth at final checkpoints for both Amber-13B and OLMo-1B.

Ablation studies underscore several findings:

• Classifier Quality vs. Mixture Fidelity: There is a strong positive correlation (Pearson $r > 0.9$) between classifier accuracy and mixture recovery.

  Figure 5: Final mixture estimation accuracy strongly correlates with proxy classifier discriminative performance.

• Necessity of Inverse Calibration: Simply aggregating classifier outputs, even with high-accuracy proxies, underperforms compared to LLMSurgeon’s explicit confusion-matrix inversion.
• Semantic Clustering: Attempting to distinguish merged/filtered subsets (e.g., C4 versus CommonCrawl) yields noisy, unstable estimates unless such sources are merged, supporting the formalization of domains as distinguishable classes.
• Sampling Robustness: Neutral prompt design is essential for stable, unbiased estimation on general-purpose models.

Visualization: Detection Reliability Across Models

LLMSurgeon’s domain recovery is visualized via predicted versus ground-truth data mixture bars for general-purpose models at coarse granularity, demonstrating the method’s stability across parameter scales and architectures.

Figure 6: Predicted domain compositions (bars) closely match ground-truth (hatched) across OLMo-1B, Amber-13B, LLaMA1-7B, and LLaMA1-65B, validating LLMSurgeon's effectiveness in real LLMs.

Implications, Practical Considerations, and Theoretical Constraints

LLMSurgeon establishes a practical, post-hoc, black-box-capable route to auditing the aggregate pretraining mixture of LLMs. This is relevant for regulatory transparency (bias, copyright risk, content moderation), reproducibility in empirical LLM research, and forensic analysis of commercial models.

Key theoretical limitations arise from the semantic separability of the target taxonomy, the validity of the label shift assumption (may break under aggressive RLHF/alignment), and closed-world constraints: the method can estimate only within the predefined classifier domain set.

For fine-grained or highly entangled domains, estimation suffers as the deconvolution problem becomes ill-conditioned. Hierarchical or non-linear mixture recovery (e.g., optimal transport) represent promising future avenues. Furthermore, robustness studies show that prompt-induced distribution shift or insufficient classifier data can degrade accuracy, mandating careful experimental controls.

Conclusion

LLMSurgeon (2605.30348) introduces the first methodologically rigorous approach for distributional auditing of LLMs, recovering semantic pretraining mixtures given only generated text and a domain proxy classifier. Its calibration-based inverse correction advances beyond prior MIA-style auditing, with strong numerical results at multiple taxonomic granularities and robust empirical validation. The implications extend to regulatory compliance, forensic auditability, and continuous model governance, although theoretical resolution limits and the label shift assumption bound its generality. Future developments will likely investigate open-domain, multilingual, and hierarchical extensions, as well as alignment-invariant auditing strategies.