Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Comprehensive Survey of Dataset Distillation (2301.05603v4)

Published 13 Jan 2023 in cs.LG

Abstract: Deep learning technology has developed unprecedentedly in the last decade and has become the primary choice in many application domains. This progress is mainly attributed to a systematic collaboration in which rapidly growing computing resources encourage advanced algorithms to deal with massive data. However, it has gradually become challenging to handle the unlimited growth of data with limited computing power. To this end, diverse approaches are proposed to improve data processing efficiency. Dataset distillation, a dataset reduction method, addresses this problem by synthesizing a small typical dataset from substantial data and has attracted much attention from the deep learning community. Existing dataset distillation methods can be taxonomized into meta-learning and data matching frameworks according to whether they explicitly mimic the performance of target data. Although dataset distillation has shown surprising performance in compressing datasets, there are still several limitations such as distilling high-resolution data or data with complex label spaces. This paper provides a holistic understanding of dataset distillation from multiple aspects, including distillation frameworks and algorithms, factorized dataset distillation, performance comparison, and applications. Finally, we discuss challenges and promising directions to further promote future studies on dataset distillation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (109)
  1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
  2. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  3. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  4. T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning.   PMLR, 2020, pp. 1597–1607.
  5. K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738.
  6. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
  7. C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1, 2017.
  8. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  9. S. Sagiroglu and D. Sinanc, “Big data: A review,” in 2013 international conference on collaboration technologies and systems (CTS).   IEEE, 2013, pp. 42–47.
  10. E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in NLP,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.   Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 3645–3650. [Online]. Available: https://aclanthology.org/P19-1355
  11. T. Wang, J.-Y. Zhu, A. Torralba, and A. A. Efros, “Dataset distillation,” arXiv preprint arXiv:1811.10959, 2018.
  12. J. Nalepa and M. Kawulok, “Selecting training sets for support vector machines: a review,” Artificial Intelligence Review, vol. 52, no. 2, pp. 857–900, 2019.
  13. B. Mirzasoleiman, J. Bilmes, and J. Leskovec, “Coresets for data-efficient training of machine learning models,” in International Conference on Machine Learning.   PMLR, 2020, pp. 6950–6960.
  14. C. Coleman, C. Yeh, S. Mussmann, B. Mirzasoleiman, P. Bailis, P. Liang, J. Leskovec, and M. Zaharia, “Selection via proxy: Efficient data selection for deep learning,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=HJg2b0VYDr
  15. O. Bohdal, Y. Yang, and T. Hospedales, “Flexible dataset distillation: Learn labels instead of images,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Workshop, 2020.
  16. I. Sucholutsky and M. Schonlau, “Soft-label dataset distillation and text dataset distillation,” in 2021 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2021, pp. 1–8.
  17. F. P. Such, A. Rawal, J. Lehman, K. Stanley, and J. Clune, “Generative teaching networks: Accelerating neural architecture search by learning to generate synthetic training data,” in International Conference on Machine Learning.   PMLR, 2020, pp. 9206–9216.
  18. Z. Deng and O. Russakovsky, “Remember the past: Distilling datasets into addressable memories for neural networks,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
  19. T. Nguyen, Z. Chen, and J. Lee, “Dataset meta-learning from kernel ridge-regression,” in Proceedings of the International Conference on Learning Representations (ICLR), 2021.
  20. T. Nguyen, R. Novak, L. Xiao, and J. Lee, “Dataset distillation with infinitely wide convolutional networks,” Advances in Neural Information Processing Systems, vol. 34, pp. 5186–5198, 2021.
  21. Y. Zhou, E. Nezhadarya, and J. Ba, “Dataset distillation using neural feature regression,” in Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. [Online]. Available: https://openreview.net/forum?id=2clwrA2tfik
  22. N. Loo, R. Hasani, A. Amini, and D. Rus, “Efficient dataset distillation using random feature approximation,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
  23. N. Loo, R. Hasani, M. Lechner, and D. Rus, “Dataset distillation with convexified implicit gradients,” in Proceedings of the International Conference on Machine Learning (ICML), 2023.
  24. B. Zhao, K. R. Mopuri, and H. Bilen, “Dataset condensation with gradient matching,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=mSAKhLYLSsl
  25. S. Lee, S. Chun, S. Jung, S. Yun, and S. Yoon, “Dataset condensation with contrastive signals,” in Proceedings of the International Conference on Machine Learning (ICML), 2022, pp. 12 352–12 364.
  26. J.-H. Kim, J. Kim, S. J. Oh, S. Yun, H. Song, J. Jeong, J.-W. Ha, and H. O. Song, “Dataset condensation via efficient synthetic-data parameterization,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, Eds., vol. 162.   PMLR, 17–23 Jul 2022, pp. 11 102–11 118. [Online]. Available: https://proceedings.mlr.press/v162/kim22c.html
  27. G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu, “Dataset distillation by matching training trajectories,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  28. J. Du, Y. Jiang, V. T. F. Tan, J. T. Zhou, and H. Li, “Minimizing the accumulated trajectory error to improve dataset distillation,” arXiv preprint arXiv:2211.11004, 2022.
  29. J. Cui, R. Wang, S. Si, and C.-J. Hsieh, “Scaling up dataset distillation to imagenet-1k with constant memory,” in Proceedings of the International Conference on Machine Learning (ICML), 2023.
  30. S. Liu, K. Wang, X. Yang, J. Ye, and X. Wang, “Dataset distillation via factorization,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
  31. B. Zhao and H. Bilen, “Dataset condensation with distribution matching,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023.
  32. K. Wang, B. Zhao, X. Peng, Z. Zhu, S. Yang, S. Wang, G. Huang, H. Bilen, X. Wang, and Y. You, “Cafe: Learning to condense dataset by aligning features,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 196–12 205.
  33. B. Zhao and H. Bilen, “Synthesizing informative training samples with gan,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Workshop, 2022.
  34. G. Zhao, G. Li, Y. Qin, and Y. Yu, “Improved distribution matching for dataset condensation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7856–7865.
  35. H. B. Lee, D. B. Lee, and S. J. Hwang, “Dataset condensation with latent space knowledge factorization and sharing,” arXiv preprint arXiv:2208.00719, 2022.
  36. D. Maclaurin, D. Duvenaud, and R. Adams, “Gradient-based hyperparameter optimization through reversible learning,” in International conference on machine learning.   PMLR, 2015, pp. 2113–2122.
  37. N. Sachdeva and J. McAuley, “Data distillation: A survey,” Transactions on Machine Learning Research, 2023, survey Certification. [Online]. Available: https://openreview.net/forum?id=lmXMXP74TO
  38. T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, “Meta-learning in neural networks: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 9, pp. 5149–5169, 2021.
  39. P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1550–1560, 1990.
  40. K. B. Petersen, M. S. Pedersen et al., “The matrix cookbook,” Technical University of Denmark, vol. 7, no. 15, p. 510, 2008.
  41. A. Jacot, F. Gabriel, and C. Hongler, “Neural tangent kernel: Convergence and generalization in neural networks,” Advances in neural information processing systems, vol. 31, 2018.
  42. J. Platt et al., “Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods,” Advances in large margin classifiers, vol. 10, no. 3, pp. 61–74, 1999.
  43. H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, “Visualizing the loss landscape of neural nets,” Advances in neural information processing systems, vol. 31, 2018.
  44. E. Golikov, E. Pokonechnyy, and V. Korviakov, “Neural tangent kernel: A survey,” arXiv preprint arXiv:2208.13614, 2022.
  45. A. Bietti and J. Mairal, “On the inductive bias of neural tangent kernels,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  46. L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research, vol. 9, no. 11, 2008.
  47. Z. Jiang, J. Gu, M. Liu, and D. Z. Pan, “Delving into effective gradient matching for dataset condensation,” arXiv preprint arXiv:2208.00311, 2022.
  48. B. Zhao and H. Bilen, “Dataset condensation with differentiable siamese augmentation,” in International Conference on Machine Learning.   PMLR, 2021, pp. 12 674–12 685.
  49. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large scale visual recognition challenge,” International journal of computer vision, vol. 115, pp. 211–252, 2015.
  50. J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision, vol. 129, no. 6, pp. 1789–1819, 2021.
  51. Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1798–1828, 2013.
  52. D. Zhang, J. Yin, X. Zhu, and C. Zhang, “Network representation learning: A survey,” IEEE transactions on Big Data, vol. 6, no. 1, pp. 3–28, 2018.
  53. G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu, “Generalizing dataset distillation via deep generative prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  54. D. J. Zhang, H. Wang, C. Xue, R. Yan, W. Zhang, S. Bai, and M. Z. Shou, “Dataset condensation via generative model,” arXiv preprint arXiv:2309.07698, 2023.
  55. Y. Liu, J. Gu, K. Wang, Z. Zhu, W. Jiang, and Y. You, “DREAM: Efficient dataset distillation by representative matching,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
  56. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  57. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011.
  58. A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009.
  59. Y. Le and X. Yang, “Tiny imagenet visual recognition challenge,” CS 231N, vol. 7, p. 7, 2015.
  60. J. Cui, R. Wang, S. Si, and C.-J. Hsieh, “DC-BENCH: Dataset condensation benchmark,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
  61. S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 2001–2010.
  62. S. Gidaris and N. Komodakis, “Dynamic few-shot visual learning without forgetting,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4367–4375.
  63. G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708.
  64. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=YicbFdNTTy
  65. W. Jin, L. Zhao, S. Zhang, Y. Liu, J. Tang, and N. Shah, “Graph condensation for graph neural networks,” in Proceedings of the International Conference on Learning Representations (ICLR), 2022.
  66. W. Jin, X. Tang, H. Jiang, Z. Li, D. Zhang, J. Tang, and B. Ying, “Condensing graphs via one-step gradient matching,” in Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022.
  67. Y. Li and W. Li, “Data distillation for text classification,” arXiv preprint arXiv:2104.08448, 2021.
  68. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).   Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  69. A. Maekawa, N. Kobayashi, K. Funakoshi, and M. Okumura, “Dataset distillation with attention labels for fine-tuning bert,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2023, pp. 119–127.
  70. F. M. Castro, M. J. Marín-Jiménez, N. Guil, C. Schmid, and K. Alahari, “End-to-end incremental learning,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 233–248.
  71. A. Carta, A. Cossu, V. Lomonaco, and D. Bacciu, “Distilled replay: Overcoming forgetting through synthetic samples,” in Continual Semi-Supervised Learning: First International Workshop, CSSL 2021, Virtual Event, August 19-20, 2021, Revised Selected Papers, vol. 13418.   Springer Nature, 2022, p. 104.
  72. F. Wiewel and B. Yang, “Condensed composite memory continual learning,” in 2021 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2021, pp. 1–8.
  73. M. Sangermano, A. Carta, A. Cossu, and D. Bacciu, “Sample condensation in online continual learning,” in 2022 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2022, pp. 01–08.
  74. J. Gu, K. Wang, W. Jiang, and Y. You, “Summarizing stream data for memory-restricted online continual learning,” https://arxiv.org/abs/2305.16645, 2023.
  75. W. Masarczyk and I. Tautkute, “Reducing catastrophic forgetting with learning on synthetic data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 252–253.
  76. V. P. C, C. White, P. Jain, S. Nayak, R. K. Iyer, and G. Ramakrishnan, “Speeding up NAS with adaptive subset selection,” in First Conference on Automated Machine Learning (Late-Breaking Workshop), 2022. [Online]. Available: https://openreview.net/forum?id=52OEvDa5xrj
  77. G. Li, G. Qian, I. C. Delgadillo, M. Muller, A. Thabet, and B. Ghanem, “Sgas: Sequential greedy architecture search,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1620–1630.
  78. Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu, “Federated learning,” Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 13, no. 3, pp. 1–207, 2019.
  79. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  80. J. Goetz and A. Tewari, “Federated learning via synthetic data,” arXiv preprint arXiv:2008.04489, 2020.
  81. H.-P. Wang, D. Chen, R. Kerkouche, and M. Fritz, “Fed-GLOSS-DP: Federated, global learning using synthetic sets with record level differential privacy,” arXiv preprint arXiv:2302.01068, 2023.
  82. S. Hu, J. Goetz, K. Malik, H. Zhan, Z. Liu, and Y. Liu, “Fedsynth: Gradient compression via synthetic data in federated learning,” arXiv preprint arXiv:2204.01273, 2022.
  83. Y. Zhou, G. Pu, X. Ma, X. Li, and D. Wu, “Distilled one-shot federated learning,” arXiv preprint arXiv:2009.07999, 2020.
  84. R. Song, D. Liu, D. Z. Chen, A. Festag, C. Trinitis, M. Schulz, and A. Knoll, “Federated learning via decentralized dataset distillation in resource-constrained edge environments,” arXiv preprint arXiv:2208.11311, 2022.
  85. Y. Xiong, R. Wang, M. Cheng, F. Yu, and C.-J. Hsieh, “FedDM: Iterative distribution matching for communication-efficient federated learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  86. P. Liu, X. Yu, and J. T. Zhou, “Meta knowledge condensation for federated learning,” arXiv preprint arXiv:2209.14851, 2022.
  87. R. Pi, W. Zhang, Y. Xie, J. Gao, X. Wang, S. Kim, and Q. Chen, “DYNAFED: Tackling client data heterogeneity with global dynamics,” arXiv preprint arXiv:2211.10878, 2022.
  88. G. Cazenavette, T. Wang, A. Torralba, A. A. Efros, and J.-Y. Zhu, “Wearable ImageNet: Synthesizing tileable textures via dataset distillation,” pp. 2278–2282, 2022.
  89. Y. Chen, Z. Wu, Z. Shen, and J. Jia, “Learning from designers: Fashion compatibility analysis via dataset distillation,” in Proceedings of the IEEE International Conference on Image Processing (ICIP), 2022, pp. 856–860.
  90. G. Li, R. Togo, T. Ogawa, and M. Haseyama, “Dataset distillation for medical dataset sharing,” pp. 1–6, 2023.
  91. ——, “Soft-label anonymous gastric x-ray image distillation,” in Proceedings of the IEEE International Conference on Image Processing (ICIP), 2020, pp. 305–309.
  92. ——, “Compressed gastric image generation based on soft-label dataset distillation for medical data sharing,” Computer Methods and Programs in Biomedicine, vol. 227, p. 107189, 2022.
  93. D. Medvedev and A. D’yakonov, “New properties of the data distillation method when working with tabular data,” in Proceedings of the International Conference on Analysis of Images, Social Networks and Texts (AIST), 2020, pp. 379–390.
  94. N. Sachdeva, M. Preet Dhaliwal, C.-J. Wu, and J. McAuley, “Infinite recommendation networks: A data-centric approach,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
  95. S. Herath, M. Harandi, and F. Porikli, “Going deeper into action recognition: A survey,” Image and vision computing, vol. 60, pp. 4–21, 2017.
  96. S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems, vol. 28, 2015.
  97. S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image segmentation using deep learning: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 7, pp. 3523–3542, 2021.
  98. K. Wang, J. Gu, D. Zhou, Z. Zhu, W. Jiang, and Y. You, “Dim: Distilling dataset into generative model,” arXiv preprint arXiv:2303.04707, 2023.
  99. L. Zhang, J. Zhang, B. Lei, S. Mukherjee, X. Pan, B. Zhao, C. Ding, Y. Li, and X. Dongkuan, “Accelerating dataset distillation via model augmentation,” arXiv preprint arXiv:2212.06152, 2022.
  100. T. Dong, B. Zhao, and L. Liu, “Privacy for free: How does dataset condensation help privacy?” in Proceedings of the International Conference on Machine Learning (ICML), 2022, pp. 5378–5396.
  101. T. Zheng and B. Li, “Differentially private dataset condensation,” 2023. [Online]. Available: https://openreview.net/forum?id=H8XpqEkbua_
  102. N. Carlini, V. Feldman, and M. Nasr, “No free lunch in” privacy for free: How does dataset condensation help privacy”,” arXiv preprint arXiv:2209.14987, 2022.
  103. D. Chen, R. Kerkouche, and M. Fritz, “Private set generation with discriminative information,” in Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2022.
  104. M. Vinaroz and M. J. Park, “Differentially private kernel inducing points (dp-kip) for privacy-preserving data distillation,” arXiv preprint arXiv:2301.13389, 2023.
  105. N. Tsilivis, J. Su, and J. Kempe, “Can we achieve robustness from data alone?” arXiv preprint arXiv:2207.11727, 2022.
  106. Y. Wu, X. Li, F. Kerschbaum, H. Huang, and H. Zhang, “Towards robust dataset learning,” arXiv preprint arXiv:2211.10752, 2022.
  107. Y. Liu, Z. Li, M. Backes, Y. Shen, and Y. Zhang, “Backdoor attacks against dataset distillation,” arXiv preprint arXiv:2301.01197, 2023.
  108. D. Zhu, B. Lei, J. Zhang, Y. Fang, R. Zhang, Y. Xie, and D. Xu, “Rethinking data distillation: Do not overlook calibration,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
  109. S. Ma, F. Zhu, Z. Cheng, and X.-Y. Zhang, “Towards trustworthy dataset distillation,” arXiv preprint arXiv:2307.09165, 2023.
Citations (65)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now