Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspective (2311.16646v2)

Published 28 Nov 2023 in cs.LG and cs.CR

Abstract: Dataset distillation offers a potential means to enhance data efficiency in deep learning. Recent studies have shown its ability to counteract backdoor risks present in original training samples. In this study, we delve into the theoretical aspects of backdoor attacks and dataset distillation based on kernel methods. We introduce two new theory-driven trigger pattern generation methods specialized for dataset distillation. Following a comprehensive set of analyses and experiments, we show that our optimization-based trigger design framework informs effective backdoor attacks on dataset distillation. Notably, datasets poisoned by our designed trigger prove resilient against conventional backdoor attack detection and mitigation methods. Our empirical results validate that the triggers developed using our approaches are proficient at executing resilient backdoor attacks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American mathematical society, 68(3):337–404, 1950.
  2. A new backdoor attack in cnns by training set corruption without label poisoning. In 2019 IEEE International Conference on Image Processing (ICIP), pp.  101–105. IEEE, 2019.
  3. Reproducing kernel Hilbert spaces in probability and statistics. Springer Science & Business Media, 2011.
  4. Detecting backdoor attacks on deep neural networks by activation clustering. In AAAI Artificial Intelligence Safety Workshop (SafeAI), 2018.
  5. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017.
  6. Privacy for free: How does dataset condensation help privacy? In International Conference on Machine Learning (ICML), 2022.
  7. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pp.  113–125, 2019.
  8. Reproducing kernel hilbert space, mercer’s theorem, eigenfunctions, nyström method, and use of kernels in machine learning: Tutorial and survey. arXiv preprint arXiv:2106.08443, 2021.
  9. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019.
  10. Bayesian deep ensembles via the neural tangent kernel. Advances in neural information processing systems, 33:1010–1022, 2020.
  11. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
  12. Some results on tchebycheffian spline functions. Journal of mathematical analysis and applications, 33(1):82–95, 1971.
  13. Dataset condensation with latent space knowledge factorization and sharing. arXiv preprint arXiv:2208.10494, 2022a.
  14. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019.
  15. Dataset condensation with contrastive signals. In International Conference on Machine Learning (ICML), 2022b.
  16. Anti-backdoor learning: Training clean models on poisoned data. Advances in Neural Information Processing Systems, 34:14900–14912, 2021a.
  17. Neural attention distillation: Erasing backdoor triggers from deep neural networks. In International Conference on Learning Representations (ICLR), 2021b.
  18. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  16463–16472, 2021c.
  19. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions and Defenses (RAID), 2018a.
  20. Dataset distillation via factorization. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  21. Copyright-certified distillation dataset: Distilling one million coins into one bitcoin with your private key. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023a.
  22. Dream: Efficient dataset distillation by representative matching. In International Conference on Computer Vision (ICCV), 2023b.
  23. Trojaning attack on neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018). Internet Soc, 2018b.
  24. Backdoor attacks against dataset distillation. Network and Distributed System Security (NDSS) Symposium, 2023c.
  25. Reflection backdoor: A natural backdoor attack on deep neural networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, pp.  182–199. Springer, 2020.
  26. Efficient dataset distillation using random feature approximation. Annual Conference on Neural Information Processing Systems (NeurIPS), 2022.
  27. Dataset distillation with convexified implicit gradients. International Conference on Machine Learning (ICML), 2023.
  28. Foundations of Machine Learning. The MIT Press, 2012. ISBN 026201825X.
  29. Dataset meta-learning from kernel ridge-regression. International Conference on Learning Representations (ICLR), 2021.
  30. Input-aware dynamic backdoor attack. Advances in Neural Information Processing Systems, 33:3454–3464, 2020.
  31. Wanet - imperceptible warping-based backdoor attack. In International Conference on Learning Representations (ICLR), 2021.
  32. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 2015.
  33. Revisiting the assumption of latent separability for backdoor defenses. In The eleventh international conference on learning representations, 2022.
  34. Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. Advances in Neural Information Processing Systems, 35:19165–19178, 2022.
  35. Demon in the variant: Statistical analysis of dnns for robust backdoor contamination detection. In USENIX Security Symposium, 2021.
  36. Spectral signatures in backdoor attacks. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.
  37. Label-consistent backdoor attacks. arXiv preprint arXiv:1912.02771, 2019.
  38. Cafe: Learning to condense dataset by aligning features. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  39. Dim: Distilling dataset into generative model. arXiv preprint arXiv:2303.04707, 2023.
  40. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018.
  41. Dataset distillation: A comprehensive review. arXiv preprint arXiv:2301.07014, 2023.
  42. Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp.  6514–6523, 2023.
  43. Dataset condensation with gradient matching. International Conference on Learning Representations (ICLR), 2021.
  44. Dataset distillation using neural feature regression. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now