Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data

Abstract: The rapid adoption of generative AI and LLMs has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. However, generating high-utility synthetic data often carries the risk of memorizing and regurgitating private information from the training corpus. In this work, we present a customizable empirical auditing framework designed to detect and explain such data disclosures. Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures''-where the system incidentally generates a user's data. By partitioning input data into training and holdout sets and applying rigorous statistical hypothesis testing, we determine if observed disclosures are consistent with strict privacy baselines, such as zero-learning or specific Differential Privacy (DP) bounds. Crucially, this approach requires no model access, no canary insertion, and no reference model training -only the synthetic output and a held-out control set. We demonstrate that this framework effectively functions as a membership inference attack, providing empirical lower bounds on privacy leakage that are tighter than prior data-based auditing methods. Our approach is model-agnostic, applies to any synthetic data generation mechanism, and requires orders of magnitude fewer computational resources than shadow-model or canary-based alternatives.

• The paper introduces a causal auditing framework that distinguishes true disclosures from phantom ones to quantify privacy leakage in synthetic data.
• The method employs statistical hypothesis testing and feature rarity analysis to provide type I error guarantees and supports membership inference attacks.
• Experimental results show that synthetic data generated with DP-SGD upholds privacy, while non-private methods exhibit significant disclosure rates.

A Causal Auditing Framework for Quantifying Privacy Risk in Synthetic Data

Problem Setting and Motivation

The increasing deployment of LLMs and generative models as sources of synthetic data accentuates privacy risks due to potential memorization and regurgitation of sensitive user data. Formal privacy guarantees such as differential privacy (DP) are context-dependent and often difficult to verify in practical deployments. This necessitates model-agnostic, data-driven empirical auditing frameworks to quantify privacy leakage in a deployment-agnostic manner. This paper develops a black-box, statistical hypothesis testing framework to audit and interpret privacy leakage in synthetic data, with formal type I error guarantees, even without access to models, canaries, or reference training.

Auditing Disclosures: True, Phantom, and Causality

The framework centers around the notion of disclosures: rare features or attributes present in the synthetic data that can be uniquely mapped to users in the source corpus. These are further categorized as:

• True Disclosures: Features from training data that reappear in the synthetic data and can be directly attributed to a specific user record.
• Phantom Disclosures: Features from holdout data (never seen by the model during training) that reappear in the synthetic data. These could arise from generalization, inductive bias, or model contamination, not privacy violations per se.

Distinguishing between these phenomena is foundational for causal inference of privacy breaches. The framework formalizes phantom disclosures as a baseline and only attributes excess in true disclosures (over phantoms) to privacy risk.

Figure 1: The impact of feature rarity ($k$) on audit lower bound $\hat{p}$, quantifying the tightness and sensitivity of statistical bounds as feature frequency varies.

Formal Audit Pipeline

The auditing process operates as follows:

1. Partition Data: The corpus is randomly split into training and holdout partitions.
2. Synthetic Generation: Synthetic data is generated using any mechanism on the training set.
3. Feature Extraction and Rareness: A feature extraction function identifies candidate disclosures (e.g., PII, n-grams, semantic embeddings) with controllable rarity criteria.
4. Disclosure Scoring and Matching: For each user, the intersection between rare features and synthetic data is computed for both training and holdout. Disclosure scores per user are defined (count-based for discrete features, similarity for embeddings).
5. Statistical Tests: Hypothesis testing machinery compares the distributions of disclosure scores between training and holdout. Two primary nulls are considered:
   • Zero-Learning Null: Synthetic data is independent of training data (ideal privacy).
   • DP-Bounded Null: Synthetic data is consistent with $\varepsilon$-DP for known $\varepsilon$.

Notably, the system does not access model internals, does not require synthetic canary design, and does not require retraining shadow/reference models.

Figure 2: Point estimates and empirical confidence intervals for varying feature rarity $k$, demonstrating the statistical gap between observed match rates and the conservative lower confidence bound $\hat{p}$.

Auditing Algorithms and Statistical Guarantees

Formally, the framework constructs test statistics quantifying the excess of disclosure scores in training relative to holdout. Under the zero-learning null, the audit lower bound $\hat{p}$ admits explicit concentration inequalities (Hoeffding-style), yielding sharp one-sided confidence intervals and empirical p-values. For the DP-bounded null, the audit yields empirical lower bounds on the smallest $\varepsilon$ consistent with the observed excess of true over phantom disclosures. The approach provably controls type I error at the user-selected significance level $\alpha$, ensuring that the probability of falsely flagging a privacy violation is bounded.

In addition, the auditing procedure supports direct construction of membership inference attacks (MIAs) by thresholding user disclosure scores, enabling standardized evaluation via AUC and comparison against model-based MIA and canary attacks.

Experimental Analysis

The authors empirically evaluate the framework on multiple datasets (including synthetic PII datasets and open corpora), multiple synthetic generation approaches (direct rewrites, SFT, and DP-SGD), and various feature types (PII match, n-gram, semantic embeddings). Key results include:

• Black-box efficacy: The framework detects significant privacy leakage in non-private synthetic datasets solely via data analysis—no model access is required.
• Phantom rates are non-trivial: Phantom disclosures account for more than 35% of apparent privacy violations in several benchmarks, underscoring the critical importance of differentiating between true and phantom leakage.
• DP efficacy: Synthetic data generated by DP-SGD passes audits, with true and phantom disclosure rates statistically indistinguishable, and lower bound $\varepsilon$ estimates reflect theoretical privacy protection.
• Semantic leakage: Embedding-based audits uncover semantic regurgitation overlooked by string- or n-gram-based audits and exceed the detection power of canary/reference model baselines.

  Figure 3: Two-sample tests comparing cosine similarities for training and holdout records to synthetic rewrites; power decays as rareness criteria are relaxed, highlighting increased difficulty distinguishing privacy leaks among frequent features.

  

  Figure 4: Two-sample test results for SFT-generated data, showing differences in distribution of embedding similarities between training/holdout and synthetic records as a function of feature rarity.

Theoretical and Practical Implications

The framework yields several critical implications:

• Deployment robustness: Auditing can be performed without assumptions on generative model architecture, data modality, or auditing expertise, making the approach robust to future advances in synthetic generation.
• Statistical interpretability: Disclosures detected by the system are directly interpretable, actionable, and align with real privacy risks, not just diagnostic canary leakage.
• Type I error calibration: The framework avoids the pitfall of false privacy alarms by statistically calibrating audit outcomes with respect to the rate of phantoms.
• Limitations: Empirical lower bounds for $\hat{p}$0 are necessarily conditioned on the chosen feature space; adversarial features or domain customizations may be required to maximize detection power.

Future Directions

This causal auditing formalism sets a precedent for robust, type I error-controlled privacy auditing of synthetic datasets. Promising follow-up directions include:

• Extension of the framework to unsupervised settings in which the audit partition is not under auditor control.
• Development of adaptive or learned user-oriented feature extractors to optimize power across diverse data domains.
• Expansion to new data modalities (e.g., structured, visual, temporal) and adversarial settings.

Conclusion

This work formalizes a data-driven, causal, type I error-calibrated framework for auditing privacy risks in synthetic data, quantifying and explaining privacy leakage as statistically significant excesses of true disclosures over phantoms. By providing both interpretability and statistical rigor, it advances the state of empirical privacy auditing, and clarifies the practical effect of differential privacy mechanisms in contrast to heuristic model restraints. This approach will likely remain a staple for model-agnostic privacy evaluations as generative data synthesis proliferates (2606.16952).