---
title: Model Provenance via Model DNA
url: https://www.emergentmind.com/topics/model
type: topic
---

# Model Provenance via Model DNA

Searching arXiv for the target paper and closely related work on model provenance.
Model provenance is the problem of determining whether a source model serves as the provenance for a target model, specifically whether the target was initialized from the source and then fine-tuned rather than trained independently from scratch. “Model Provenance via Model DNA” formalizes this problem at the model level, introduces the notion of **Model DNA** as a compact representation of a model’s unique characteristics, and develops the **MGMP** framework for provenance identification across computer vision and natural language processing settings [2308.02121]. The work is motivated by intellectual property protection, model security, and auditability, and is positioned as complementary to data-provenance questions such as membership inference because it asks whether one model, rather than one datum, was the seed for another [2308.02121].

## 1. Problem formulation and scope

The paper defines Model Provenance (MP) using a source model \(M_s\) trained on dataset \(D_s\) and a candidate target model \(M_t\) trained on \(D_t\). The task is to decide whether \(M_t\) was initialized from \(M_s\), in which case the pair is described as “homologous,” or whether \(M_t\) was trained from scratch, in which case it is “non-homologous” [2308.02121]. Formally, MP seeks a function
$$
f(M_s, M_t) = 1 \text{ if } M_t \text{ is derived by fine-tuning from } M_s,\text{ and } 0 \text{ otherwise.}
$$

This formulation isolates lineage at the level of pretrained weights. The emphasis is not on whether the source and target solve the same downstream task, nor on whether they share data, but on whether the source model is the actual ancestor in a fine-tuning chain [2308.02121]. The paper’s current scope is the simplest fine-tuning scenario: source and target share identical architectures and differ only by weight initialization. More complex cases such as distillation, pruning, and cross-architecture transfer are explicitly left open [2308.02121].

The significance of the problem is framed in three ways. First, provenance identification supports intellectual property protection by allowing organizations to verify whether a privately trained model has been reused in downstream products. Second, it supports model security and audit in regulated or federated settings by making lineage visible. Third, it complements data-provenance methods by shifting the question from membership of examples in training data to ancestry of models in training pipelines [2308.02121].

## 2. Model DNA as a model fingerprint

The central concept is **Model DNA**, introduced as a representation of the unique characteristics of a machine learning model. Let \(D=\{x_i,y_i\}_{i=1}^n\) be the source training set for model \(M\). A Model DNA is defined as the set of \(N\) DNA fragments
$$
\mathcal{O} = \{o_1,o_2,\dots,o_n\},
$$
where each fragment is
$$
o_i = g(x_i;M),
$$
and the full construction is written as
$$
\mathcal{O} \leftarrow g(D,M)=\{g(x_i,M):x_i\in D\}.
$$
In the paper’s operational design, each fragment fuses a **data-driven** representation with a **model-driven** representation [2308.02121].

The data-driven part uses a learnable generator network \(g(\cdot)\), such as a small ResNet50 or a DistilBERT head, to map each input \(x_i\) to a latent code \(z_i=g(x_i)\). The model-driven part uses the target model’s prediction for the same input, represented as a logit or soft-probability vector \(y_i=M(x_i)\). These are assembled into a DNA fragment either by element-wise addition,
$$
o_i = z_i + y_i,
$$
or by concatenation,
$$
o_i = [z_i \| y_i].
$$
This design encodes both the structure of the source data and the behavioral response of the model to that data, which the paper treats as a compact and comprehensive fingerprint of model identity [2308.02121].

The formulation implies that provenance is not inferred from raw weights directly. Instead, it is inferred from a learned representational space in which homologous source-target pairs should induce nearby DNA fragments, while non-homologous pairs should be separated. This suggests that the method is behavioral and relational rather than purely parameter-comparative.

## 3. Learning objective and MGMP framework

The provenance-identification framework is called **MGMP** and consists of three stages: DNA generation, DNA similarity learning, and provenance classification [2308.02121].

In the DNA-generation stage, the framework uses homologous input pairs \((M_s,M_t)\) and non-homologous pairs \((M_s,\bar M_t)\). For each \(x_i\) in the source training set, it computes
- \(z_i=g(x_i)\),
- \(y_i^s=M_s(x_i)\),
- \(y_i^t=M_t(x_i)\),
- \(\bar y_i^t=\bar M_t(x_i)\),

and assembles the fragments
- \(o_i^s=z_i+ y_i^s\),
- \(o_i^t=z_i+ y_i^t\),
- \(\bar o_i^t=z_i+ \bar y_i^t\) [2308.02121].

The generator is trained with two contrastive components. Let cosine similarity be
$$
\mathrm{sim}(u,v)=\frac{u^T v}{\|u\|\|v\|},
$$
with temperature \(\tau\). The **cross-model contrastive loss** pulls homologous fragments together and pushes non-homologous fragments away:
$$
\mathcal{L}_s = - \sum_i \log
\frac{\exp(\mathrm{sim}(o_i^s,o_i^t)/\tau)}
{\sum_k \exp(\mathrm{sim}(o_i^s,\bar o_k^t)/\tau)}.
$$
The **within-model compactness loss** enforces clustering among fragments belonging to the same model:
$$
\mathcal{L}_i = -\sum_{i\neq k}\left[
\log \exp(\mathrm{sim}(o_i^s,o_k^s)/\tau)
+
\log \exp(\mathrm{sim}(o_i^t,o_k^t)/\tau)
+
\log \exp(\mathrm{sim}(\bar o_i^t,\bar o_k^t)/\tau)
\right].
$$
The full DNA similarity loss is
$$
\mathcal{L}_{DNA} = \mathcal{L}_s + \mathcal{L}_i + \lambda \|w_g\|_2^2,
$$
where \(w_g\) denotes the generator weights [2308.02121].

The third stage introduces a provenance classifier \(f_p(\cdot)\), implemented as a small MLP. It receives the concatenation \([o_i^s \| o_i^t]\) or \([o_i^s \| \bar o_i^t]\) and predicts a binary label \(h_i\in\{0,1\}\). Training uses binary cross-entropy,
$$
\mathcal{L}_{BCE} = -\frac{1}{N}\sum_i \left[h_i\log h_i' + (1-h_i)\log(1-h_i')\right],
$$
where \(h_i' = f_p(o_i^s \| o_i^t)\) [2308.02121].

At inference time, provenance can be assessed at fragment level and then aggregated to the model level. A fragment-level verdict is positive if \(f_p([o_i^s\|o_i^t])\ge 0.5\). The model-level decision averages these scores and compares them to a threshold \(\delta\):
$$
\frac{1}{N}\sum_i f_p([o_i^s\|o_i^t]) \ge \delta \Rightarrow \text{provenance}=1.
$$
The paper notes that \(\delta\) can be set, for example \(\delta=0.9\), to control the false-positive/negative trade-off [2308.02121].

## 4. Experimental design across vision and language

The empirical study spans both computer vision and natural language processing tasks. In computer vision, the source dataset is CIFAR-10 and the target domain is built from CIFAR-100 split into 10 disjoint 10-way subsets. The base models are ResNet18, AlexNet, and ViT-small, while the DNA generator backbone is ResNet50 [2308.02121].

In NLP, the evaluation uses text-classification benchmarks including AGNews, YahooQA, 20Newsgroups, DBPedia, Amazon Reviews, and Yelp. The main models are DistilBERT or BERT-base, and the DNA generator is DistilBERT [2308.02121].

The study constructs a model pool for each run. In the vision setting, 5 CIFAR100 subsets are used to train homologous target models and 5 other subsets to train non-homologous target models, with evaluation performed on held-out subsets. In the NLP setting, the split is analogous: one dataset is used for the source, three for the pool, and two for evaluation [2308.02121].

The principal reported metric is fragment-level accuracy averaged over homologous and non-homologous pairs. The experiments also include t-SNE visualizations, which show that DNA fragments from homologous and non-homologous models form well-separated clusters in computer vision and, in the appendix, in NLP as well [2308.02121]. This visualization evidence is consistent with the intended geometry induced by \(\mathcal{L}_{DNA}\).

## 5. Empirical performance and ablation results

The paper reports consistent gains from introducing the DNA generator. In computer vision, ResNet18 on CIFAR10\(\rightarrow\)CIFAR100 variants achieves **91.9%–96.1%** accuracy with the generator, compared with **75.3%–82.2%** without the generator. AlexNet and ViT-small show similar gains, with approximately **90%+** accuracy versus approximately **72%** without the generator [2308.02121].

In NLP, DistilBERT achieves **83.8%–93.9%** accuracy, compared with approximately **72%–84%** without the generator. BERT-base on AGNews reaches **86.6%**, compared with **70.7%** without the generator [2308.02121].

The main ablation baseline, labeled **“MGMP w/o DNA generator,”** uses only model outputs and omits the learned \(z_i\). Performance drops by **10–20%**, indicating that provenance identification is substantially weaker when it depends only on output behavior rather than on the joint data-driven and model-driven representation [2308.02121].

Additional ablations examine generator and assembly choices.

| Aspect | Variants | Reported outcome |
|---|---|---|
| Generator backbone | ResNet50, ViT-small, ViT-base | All work; ResNet50 slightly best |
| Assembly | Addition | Dimension \(=10\) |
| Assembly | Concatenation | Dimension \(=20, 60, 110\) |
| Accuracy trend | Larger concatenated codes | Improves gradually from \(92\%\rightarrow94\%\rightarrow95\%\) |

These results indicate that richer fragment encodings improve identification accuracy, at least within the tested settings [2308.02121]. A plausible implication is that provenance evidence is distributed across both latent input representations and output signatures, and benefits from preserving more of that joint information.

## 6. Limitations, open problems, and broader significance

The work states several limitations. The first is the restriction to the simplest fine-tuning scenario, where source and target share identical architectures and differ only in initialization. Distillation, pruning, and cross-architecture transfer remain outside the tested regime [2308.02121]. The second is the reliance on the original source training data \(D_s\) during identification; whether a small held-out probe set would suffice is left for future work. The third concerns scalability: very large source datasets and very deep DNA generators may introduce substantial computational cost. The fourth is adversarial robustness, since deliberate provenance masking through extra unrelated fine-tuning, as well as source-model modification through sparsification or quantization, is not resolved in the present study [2308.02121].

Within those constraints, the paper establishes a model-level provenance formalism, a concrete representational mechanism, and an end-to-end decision pipeline for distinguishing homologous from non-homologous models across vision and language tasks [2308.02121]. Its broader significance lies in treating model lineage as an inferential object in its own right. This shifts provenance analysis away from direct inspection of training logs or weight histories and toward post hoc identification from behavior anchored in source data. Such a perspective is technically important for security, licensing enforcement, and audit settings in which access to internal training artifacts may be incomplete or unavailable.

The paper’s terminology also clarifies a conceptual distinction that is often blurred in surrounding discussions: model provenance is not equivalent to model similarity. Two models may be behaviorally similar on a task without sharing an ancestry relation, and conversely a fine-tuned descendant may diverge substantially in downstream behavior while still retaining provenance-relevant traces in its Model DNA. This suggests that provenance, as formalized here, is a lineage question rather than merely a performance or representation similarity question [2308.02121].

Source: https://www.emergentmind.com/topics/model