Data Approximation from Model Weights

• Data approximation from model weights is a technique that infers hidden training data by analyzing the differences between a base and a fine-tuned model.
• It employs gradient-based selection methods to score candidate data points from large public corpora based on their alignment with weight updates.
• Empirical results reveal improvements in accuracy and efficiency, underscoring its potential for model auditing, transparency, and privacy assessment.

Data approximation from model weights refers to the reconstruction, estimation, or selection of training data (or closely related data) using only the information encoded in a trained model’s weight parameters, particularly in contexts where the data itself is not accessible. This is of growing significance in modern machine learning, where open model weights are increasingly common but access to the proprietary, private, or censored training data is restricted. The task is relevant to issues in transparency, model auditing, privacy, and intellectual property.

1. Formalizing Data Approximation from Weights

The fundamental problem consists of, given access to a pair of models:

• The base model weights, $\theta_0$ (for example, a pre-trained LLM before supervised finetuning), and
• The finetuned model weights, $\theta_f$ (after supervised training on an unknown dataset $\mathcal{D}$),

and knowledge of the training procedure $\mathcal{T}$ (including optimizer, learning rates, number of epochs), approximate $\mathcal{D}$ as closely as possible by selecting a surrogate dataset $\mathcal{D}^*$—typically drawn from a very large, public source pool—such that training the base model on $\mathcal{D}^*$ brings it as close as possible to $\theta_f$. This is formalized in the optimization problem: $$\mathcal{D}^* = \arg\min_{\mathcal{D}'} \left\| \theta_f - \mathcal{T}(\mathcal{L}, \theta_0, \mathcal{D}') \right\|$$ where $\mathcal{L}$ is the (known) loss function.

Key properties:

• The problem is bi-level: the outer loop is over possible data subsets; the inner loop involves re-training the model.
• Direct optimization over text data is computationally prohibitive; thus, practical heuristics are required.
• Even with only two snapshots (base and fine-tuned), significant data recovery is possible if a large, general corpus is available (2506.15553).

2. Baselines and Metrics for Data Recovery

A variety of approaches can be considered:

• Random Selection: Randomly selecting items from the candidate pool. Provides a naïve lower bound.
• Label Recovery: Assigning labels to the candidate pool by feeding each candidate input $x$ through the finetuned model $\theta_f$ and taking $\hat{y} = \arg\max \ell(x; \theta_f)$.
• Gradient-based Selection: Select examples whose loss gradient (with respect to $\theta_0$) aligns maximally with the observed parameter difference $\Delta\theta = \theta_f - \theta_0$, i.e., maximizing $$s(x) = \nabla \ell(x; \theta_0) \cdot (\theta_f - \theta_0)$$.
• Greedy/Batch Submodular Maximization: Selecting batches of examples to maximize the cumulative alignment, with strategies to enforce class balance where appropriate.

Metrics for evaluation:

• Lexical Similarity: Overlap in vocabulary, e.g., Jaccard similarity.
• Semantic Proximity: Optimal Transport distance computed between embedding distributions of selected and original datasets.
• Downstream Task Performance: Classification accuracy, perplexity, or other direct model quality metrics after re-training on the recovered data.

3. Gradient-based Subset Selection (SELECT Algorithm)

The principal method introduced for data approximation is as follows:

• For each candidate $x$ in the pooled dataset, compute the gradient $\nabla \ell(x; \theta_0)$.
• Score each $x$ by $$s(x) = \nabla \ell(x; \theta_0) \cdot (\theta_f - \theta_0)$$ to measure how well $x$’s learning signal predicts the real parameter update.
• Select a subset $\mathcal{B}$ (of fixed budget) that maximizes the cumulative score

$$\mathcal{B}^* = \arg\max_{\mathcal{B} \subseteq \mathcal{D}} \sum_{x \in \mathcal{B}} \nabla \ell(x; \theta_0) \cdot (\theta_f - \theta_0)$$

• Optionally, distribute selection over multiple “synthetic checkpoints” interpolating between $\theta_0$ and $\theta_f$ to more closely mimic the training trajectory.
• Label recovery is performed by assigning each candidate the argmax output over $\theta_f$.

In practice, for computational tractability:

• Gradients are typically computed only for the last layer, and dimensionality reduction (e.g., via Johnson-Lindenstrauss projections) is employed.
• Submodular greedy selection is used for batched selection.

4. Empirical Results and Performance

Experiments on LLM datasets yield:

• Classification (AG News):
  • SELECT achieves 80% accuracy on AG News using only recovered data, compared to 65% for random selection and 88% for the expert (“oracle”) set, denoting the true training data.
• Supervised Finetuning (MSMARCO/LLAMA3):
  • SELECT reduces perplexity to 2.3, nearly closing the gap with the real LLAMA-trained model’s 2.0 and far below random (3.3).
• Semantic Similarity: SELECT-selected datasets have higher overlap with ground truth in both lexical and semantic metrics.
• Efficiency: Gradient-based selection methods are effective at realistic compute cost; greedy batch selection and batch gradient computation are tractable and scalable.

5. Theoretical and Methodological Implications

• The method exploits the fact that the model’s parameter update direction encodes directional information about the population gradient provided by the true training data. Samples whose gradients better align with the weight delta are, in expectation, more representative of the true dataset responsible for the observed finetuning.
• The approach provides a general-purpose instance of dataset distillation by selection (rather than synthetic input optimization), with the distinction that only the weight delta and optimizer details must be known.
• The method is robust to missing validation/test data and agnostic to label availability in the corpus, as pseudolabels can be reliably inferred from the final model.

6. Privacy, Transparency, and Model Security Considerations

• The demonstrated efficacy of data recovery from weights challenges common assumptions about training data privacy in open-weight model releases.
• This suggests that, even if the data itself is closed, its statistical “core” may be partially recoverable from the public weights alone, especially when those weights encode adaptation from widely available base models.
• A plausible implication is that organizations wishing to restrict exposure of proprietary or sensitive datasets may need to consider more sophisticated countermeasures than simply withholding training data.

7. Applications and Potential Impact

• Auditing and Transparency: Regulators or independent researchers can use these techniques to analyze open models’ likely training sources, supporting compliance or transparency goals.
• Model Forensics and Red-teaming: Security teams may assess risk of proprietary data leakage from released models.
• Synthetic Data Bootstrapping: A practical method to construct highly effective pseudo-training datasets, using only model weights and general corpora.

Method	Classification Accuracy (%) (AG News)	Perplexity (MSMARCO)
Random Selection	65.6	3.31
SELECT (Ours)	80.0	2.30
Expert Oracle	88.3	2.01

Note: All figures and metrics are taken from (2506.15553).

---

In summary, data approximation from model weights exploits the correspondence between weight updates and sample gradients to select or reconstruct datasets that, upon retraining, reproduce most of the empirical gains provided by the original finetuning data—even in the absence of direct access to that data. This raises significant questions about data privacy and sets a new frontier for transparency analysis and dataset distillation in modern large-scale machine learning.