Data Taggants: Dataset Ownership Verification via Harmless Targeted Data Poisoning (2410.09101v1)

Abstract: Dataset ownership verification, the process of determining if a dataset is used in a model's training data, is necessary for detecting unauthorized data usage and data contamination. Existing approaches, such as backdoor watermarking, rely on inducing a detectable behavior into the trained model on a part of the data distribution. However, these approaches have limitations, as they can be harmful to the model's performances or require unpractical access to the model's internals. Most importantly, previous approaches lack guarantee against false positives. This paper introduces data taggants, a novel non-backdoor dataset ownership verification technique. Our method uses pairs of out-of-distribution samples and random labels as secret keys, and leverages clean-label targeted data poisoning to subtly alter a dataset, so that models trained on it respond to the key samples with the corresponding key labels. The keys are built as to allow for statistical certificates with black-box access only to the model. We validate our approach through comprehensive and realistic experiments on ImageNet1k using ViT and ResNet models with state-of-the-art training recipes. Our findings demonstrate that data taggants can reliably make models trained on the protected dataset detectable with high confidence, without compromising validation accuracy, and demonstrates superiority over backdoor watermarking. Moreover, our method shows to be stealthy and robust against various defense mechanisms.

• The paper introduces data taggants, a clean-label targeted poisoning method that embeds secret keys to verify dataset ownership.
• It employs multiple secret keys and statistical tests via black-box access to reliably detect models trained on tagged datasets.
• Evaluations on ImageNet1k with ViT and ResNet models demonstrate high detection accuracy with minimal impact on model validation performance.

Data Taggants: Dataset Ownership Verification via Harmless Targeted Data Poisoning

The paper "Data Taggants: Dataset Ownership Verification via Harmless Targeted Data Poisoning" addresses the critical issue of dataset ownership verification in machine learning, particularly focused on image classification datasets. The authors present a novel technique that they term "data taggants," which enhances existing verification methodologies by mitigating some of their inherent limitations, such as model performance degradation and the requirement for intrusive model access.

Methodology

The core of the data taggants approach involves embedding secret keys within a dataset through clean-label targeted data poisoning. These keys are essentially pairs of out-of-distribution samples and randomly assigned labels, which are subtly integrated into the dataset as "signatures" designed to induce specific model behaviors, thereby marking models trained on the signed dataset.

A significant advancement here lies in the use of multiple keys with randomly chosen labels, which enables the application of statistical tests to verify model training on the tagged dataset using only black-box access, such as top-k predictions. This methodology circumvents the shortcomings of backdoor watermarking, which frequently depends on modifying model behavior and also requires internal model access, risking false positives.

Numerical Results

The empirical validation demonstrates that data taggants successfully achieve high-confidence detection of models trained on protected datasets without compromising validation accuracy. The implementation on ImageNet1k using ViT and ResNet models reveals a robust detection accuracy paired with reliable stealthiness. Additional robustness evaluations showcase their ability to withstand various defensive strategies, model architectures, and dataset adjustments.

Implications and Future Directions

From a practical standpoint, this research offers dataset owners a more reliable and non-intrusive option to verify unauthorized usage, which is becoming increasingly crucial given the widespread deployment of machine learning models with opaque training data. Its application is particularly relevant in safeguarding intellectual property rights in data, ensuring compliance with data governance policies, and preventing unforeseen use of open datasets.

Theoretically, the work expands the current understanding of data poisoning and ownership verification, contributing a statistical dimension to an area typically dominated by heuristic methods. This could foster further exploration into cryptographic techniques for data verification.

Looking ahead, potential developments could focus on enhancing the robustness of data taggants against more extensive model architecture variations and hybrid training datasets. Additionally, examining the method's efficacy across other data types beyond images could broaden its applicability. Overall, data taggants mark a significant step forward in the pursuit of safeguarding dataset integrity and verifying model training history.