Dataset Inference in Machine Learning

Updated 3 July 2026

Dataset Inference (DI) is a methodology that determines if and how a candidate dataset was used in training by aggregating per-sample membership signals.
It employs statistical hypothesis tests and mixture proportion estimation to robustly differentiate between data inclusion and exclusion in the training process.
DI integrates techniques like synthetic negatives and watermark-based methods to enhance detection accuracy and address compliance and ownership challenges.

Dataset Inference (DI) is the statistical and algorithmic methodology for determining, given a suspect machine learning model and a candidate dataset, whether and to what extent the dataset was used during model training. DI addresses the crucial problem of data usage auditing in the context of model ownership, copyright disputes, compliance, and accountability of large-scale machine learning systems. It generalizes Membership Inference (MI, single-sample inclusion) to questions about aggregate or partial usage at the dataset level, aiming to draw statistically robust conclusions about dataset inclusion—and, in advanced forms, to estimate the fraction of a dataset used for training.

1. Formalization and Statistical Frameworks

DI is commonly formulated as either a binary hypothesis test (“was dataset $\mathcal{D}$ used in model $M$ 's training?”) or a quantitative estimation problem (“what fraction $p$ of $\mathcal{D}$ was present in the training set?”).

The canonical DI setup defines two hypotheses for suspect model $f$ and candidate dataset $\mathcal{D}$ :

$H_0$ : $\mathcal{D}$ not used (non-member or independent)
$H_1$ : $\mathcal{D}$ (fully or partially) used in training (member)

In its most general form, DI proceeds as follows (Maini et al., 2021, Maini et al., 2024, Zhou et al., 4 Jul 2025, Łapacz et al., 24 Jun 2026):

Extract model-dependent sample-level “membership signals” $M$ 0 for each $M$ 1 using MIAs or related algorithms.
Aggregate these signals into dataset-level statistics (typically mean or sum, possibly after learned weighting).
Compare statistics from “suspect” samples against those from reference sets (public data, held-out, or synthetically constructed non-members).
Perform a test (e.g., one-sided $M$ 2-test, mixture proportion estimation, classifier-based test) to determine whether statistical evidence supports $M$ 3.
(For usage estimation) DI may be formalized via Mixture Proportion Estimation (MPE), where the observed distribution of membership scores in $M$ 4 is modeled as a convex mixture of member and non-member score distributions, and the mixing weight is estimated (interpreted as fraction used, $M$ 5).

Key statistical techniques include:

One-sided Welch’s $M$ 6-test or two-sample $M$ 7-test to determine mean (or higher-moment) deviation between suspect and non-member sets.
Empirical histograms and classifier-based methods for positive-unlabeled learning in MPE-based DUI (Łapacz et al., 24 Jun 2026).
Robust combination of $M$ 8-values across multiple random splits to mitigate partition sensitivity (Maini et al., 2024).

2. Signal Extraction and Feature Aggregation

Extraction of per-sample membership evidence is central to DI. The quality, diversity, and aggregation of these features critically determine the efficacy of the inference (Maini et al., 2024, Proboszcz et al., 10 Dec 2025, Łapacz et al., 24 Jun 2026). Approaches are model- and modality-specific:

LLMs: Up to 52 different MIAs may be aggregated (Maini et al., 2024), including:

Perplexity-based features (standard, min- $M$ 9, zlib compression ratio)
Loss-based statistics for class-conditionals vs. unconditionals
DetectGPT-style and perturbation scores
Baseline/reference model comparison features

Generative models (vision, audio):

Autoregressive models: Per-sample negative log-likelihood, top- $p$ 0 perplexity, compressor statistics (Proboszcz et al., 10 Dec 2025, Łapacz et al., 24 Jun 2026)
Diffusion models: Multi-timestep latent losses, secondary MI, CLiD, latent-space alignment errors

White-box neural models: Margin-based or influence-function-based embeddings (e.g., MinGD, BlindWalk) to estimate proximity to decision boundary (Maini et al., 2021).

Aggregators:

Linear regression (domain-specific weighting of MIAs)
Classifier-based integration (e.g., ensemble classifiers)
Feature normalization and outlier clipping are essential for test calibration.

Semantic fingerprinting: White-box extraction of correlation matrices (semantic correlation descriptors, SCDs) between internal representations and meaningful features (e.g., keywords × classes), followed by Pearson correlation or attribution analysis; this captures dataset-specific “semantic footprints” (Gobeaja et al., 28 May 2026).

3. Practical Methodologies and Reference Data Construction

Nearly all state-of-the-art DI pipelines must solve the problem of constructing a suitable reference for non-member statistics. This has led to several innovation areas:

Held-out or Reference Sets:

Classical DI assumes access to an IID held-out set known not to be in training, nearly identical in distribution to $p$ 1 (Maini et al., 2024, Zhao et al., 18 Jun 2025). In practice, such access is rare or infeasible.

Synthetic negatives / Automatic generation:

DUI for generative models (Łapacz et al., 24 Jun 2026) creates synthetic non-member reference data by cross-family paraphrasing (image-to-image+autoencoding, using a generator $p$ 2 from a distinct architectural family) and matching artifacts to avoid “fixed-point” confounds.
For LLMs, variants generate held-out texts via masked-suffix completion with fine-tuned generators (using data only from the suspect corpus but withheld from training proper), followed by post-hoc calibration to distinguish true membership effects from generation artifacts (Zhao et al., 18 Jun 2025).

Black-box DI:

Reference models (with and without fine-tuning on $p$ 3) are trained locally and used as behavioral baselines. The suspect model's outputs are compared via text similarity (e.g., BERTScore) to both member and non-member reference model outputs on “tainted” examples (where fine-tuning causes output shift) (Zhou et al., 4 Jul 2025).

Watermark-based DI for RAG:

In black-box retrieval-augmented generation (RAG), watermark-based dataset inference (WARD) embeds content-encoded bitstreams (e.g., “red–green” token colorings) in the data owner's documents prior to publication. Querying the RAG system and statistically counting retained watermarked tokens in responses allows detection via a simple binomial test (Jovanović et al., 2024).

4. Evaluation Protocols and Quantitative Results

DI methods are commonly evaluated on their accuracy, type I/II error control, sample efficiency, and robustness.

On LLMs, dataset-level tests systematically outperform single-sample MIAs: DI can decisively distinguish between train and validation splits (e.g., of The Pile), with $p$ 4-values often $p$ 5 and zero false positives/negatives across all tested domains and model scales (Maini et al., 2024).
In generative vision/audio, DUI closes most of the performance gap with “oracle” reference sets via synthetic negatives, delivering mean absolute errors (MAE) well below $p$ 6 for 1000-sample suspect sets (Łapacz et al., 24 Jun 2026). For strong diffusion models, a few hundred audio samples can suffice for confident detection ( $p$ 7) (Proboszcz et al., 10 Dec 2025).
Black-box DI for LLMs achieves 100% F1 under non-IID and near-perfect under challenging IID-matched splits, outperforming black-box MIA (Zhou et al., 4 Jul 2025).
Watermark-based RAG-DI achieves perfect type I error control and monotonic convergence, reliably detecting data leakage even under prompt defenses or aggressive $p$ 8-gram filtering, requiring only $p$ 9 queries (Jovanović et al., 2024).

5. Limitations, Robustness, and Failure Modes

Despite advances, DI methodologies operate under pronounced structural assumptions and exhibit characteristic limitations:

Robustness and False Positives:

The statistical gap exploited by DI can become indistinguishable for independent models trained on different partitions of the same underlying distribution, leading to high false positive rates as formalized via PAC-Bayes margin-theoretic analyses (Szyller et al., 2022, Maini et al., 2021).
Expressivity of the regressor/function used to aggregate features does not eliminate this fundamental indistinguishability in deployment scenarios where each party has access to indiscriminable draws from $\mathcal{D}$ 0.

Adversarial and Regularization Evasion: