Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples (2305.09241v5)

Published 16 May 2023 in cs.LG, cs.CR, and cs.CV

Abstract: Safeguarding data from unauthorized exploitation is vital for privacy and security, especially in recent rampant research in security breach such as adversarial/membership attacks. To this end, \textit{unlearnable examples} (UEs) have been recently proposed as a compelling protection, by adding imperceptible perturbation to data so that models trained on them cannot classify them accurately on original clean distribution. Unfortunately, we find UEs provide a false sense of security, because they cannot stop unauthorized users from utilizing other unprotected data to remove the protection, by turning unlearnable data into learnable again. Motivated by this observation, we formally define a new threat by introducing \textit{learnable unauthorized examples} (LEs) which are UEs with their protection removed. The core of this approach is a novel purification process that projects UEs onto the manifold of LEs. This is realized by a new joint-conditional diffusion model which denoises UEs conditioned on the pixel and perceptual similarity between UEs and LEs. Extensive experiments demonstrate that LE delivers state-of-the-art countering performance against both supervised UEs and unsupervised UEs in various scenarios, which is the first generalizable countermeasure to UEs across supervised learning and unsupervised learning. Our code is available at \url{https://github.com/jiangw-0/LE_JCDP}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Blended latent diffusion. arXiv preprint arXiv:2206.02779 (2022).
  2. Abeba Birhane and Vinay Uday Prabhu. 2021. Large image datasets: A pyrrhic win for computer vision?. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1536–1546.
  3. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
  4. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020).
  5. Terrance DeVries and Graham W Taylor. 2017. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017).
  6. Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems 34 (2021), 8780–8794.
  7. The Devil’s Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models. arXiv preprint arXiv:2303.08500 (2023).
  8. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? Advances in neural information processing systems 34 (2021), 21480–21492.
  9. Learning to confuse: generating training time adversarial data with auto-encoder. Advances in Neural Information Processing Systems 32.
  10. Adversarial examples make strong poisons. Advances in Neural Information Processing Systems 34 (2021), 30339–30351.
  11. Robust Unlearnable Examples: Protecting Data Privacy Against Adversarial Learning. In International Conference on Learning Representations. OpenReview.net.
  12. Shortcut learning in deep neural networks. Nature Machine Intelligence 2, 11 (2020), 665–673.
  13. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144.
  14. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems 33 (2020), 21271–21284.
  15. MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In European Conference on Computer Vision. Springer, 87–102.
  16. Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learning. In International Conference on Learning Representations.
  17. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  18. Unsupervised learning of 3d object categories from videos in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4700–4709.
  19. Kashmir Hill and Aaron Krolik. 2019. How photos of your kids are powering surveillance technology. The New York Times (2019).
  20. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems. 6840–6851.
  21. Unlearnable Examples: Making Personal Data Unexploitable. In International Conference on Learning Representations. OpenReview.net.
  22. Learning multiple layers of features from tiny images. (2009).
  23. Fast AutoAugment. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. 6665–6675.
  24. Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression. arXiv preprint arXiv:2301.13838 (2023).
  25. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations. OpenReview.net.
  26. Salient Conditional Diffusion for Defending Against Backdoor Attacks. arXiv preprint arXiv:2301.13862 (2023).
  27. Towards poisoning of deep learning algorithms with back-gradient optimization. In Proceedings of the 10th ACM workshop on artificial intelligence and security. 27–38.
  28. Reading digits in natural images with unsupervised feature learning. (2011).
  29. Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning. PMLR, 8162–8171.
  30. Diffusion Models for Adversarial Purification. In International Conference on Machine Learning. PMLR, 16805–16827.
  31. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition. IEEE, 3498–3505.
  32. Diffusion autoencoders: Toward a meaningful and decodable representation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10619–10629.
  33. Learning the Unlearnable: Adversarial Augmentations Suppress Unlearnable Example Attacks. arXiv preprint arXiv:2303.15127 (2023).
  34. Poisoning attacks and defenses on artificial intelligence: A survey. arXiv preprint arXiv:2202.10276 (2022).
  35. Transferable Unlearnable Examples. In International Conference on Learning Representations.
  36. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684–10695.
  37. Imagenet large scale visual recognition challenge. International journal of computer vision 115 (2015), 211–252.
  38. CUDA: Convolution-based Unlearnable Datasets. arXiv e-prints (2023), arXiv–2303.
  39. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning. PMLR, 2256–2265.
  40. Score-Based Generative Modeling through Stochastic Differential Equations. In 9th International Conference on Learning Representations. OpenReview.net.
  41. Guided diffusion model for adversarial purification. arXiv preprint arXiv:2205.14969 (2022).
  42. Transferring gans: generating images from limited data. In European Conference on Computer Vision. 218–234.
  43. Fooling adversarial training with inducing noise. arXiv preprint arXiv:2111.10130 (2021).
  44. Is Adversarial Training Really a Silver Bullet for Mitigating Data Poisoning. In International Conference on Learning Representations.
  45. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations. OpenReview.net.
  46. One-pixel shortcut: on the learning preference of deep neural networks. In International Conference on Learning Representations.
  47. Availability attacks create shortcuts. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2367–2376.
  48. Chia-Hung Yuan and Shan-Hung Wu. 2021. Neural tangent generalization attacks. In International Conference on Machine Learning. PMLR, 12230–12240.
  49. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision. 6023–6032.
  50. mixup: Beyond Empirical Risk Minimization. In International Conference on Learning Representations.
  51. Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples. arXiv preprint arXiv:2301.01217 (2022).
  52. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE conference on computer vision and pattern recognition. 586–595.
  53. A closer look at few-shot image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9140–9150.
Citations (12)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now