Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Vulnerabilities of Foundation Model Integrated Federated Learning Under Adversarial Threats (2401.10375v2)

Published 18 Jan 2024 in cs.CR, cs.DC, and cs.LG

Abstract: Federated Learning (FL) addresses critical issues in machine learning related to data privacy and security, yet suffering from data insufficiency and imbalance under certain circumstances. The emergence of foundation models (FMs) offers potential solutions to the limitations of existing FL frameworks, e.g., by generating synthetic data for model initialization. However, due to the inherent safety concerns of FMs, integrating FMs into FL could introduce new risks, which remains largely unexplored. To address this gap, we conduct the first investigation on the vulnerability of FM integrated FL (FM-FL) under adversarial threats. Based on a unified framework of FM-FL, we introduce a novel attack strategy that exploits safety issues of FM to compromise FL client models. Through extensive experiments with well-known models and benchmark datasets in both image and text domains, we reveal the high susceptibility of the FM-FL to this new threat under various FL configurations. Furthermore, we find that existing FL defense strategies offer limited protection against this novel attack approach. This research highlights the critical need for enhanced security measures in FL in the era of FMs.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. How to backdoor federated learning. In AISTATS, 2020.
  2. Machine learning with adversaries: Byzantine tolerant gradient descent. In NIPS, 2017.
  3. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  4. Language models are few-shot learners, 2020.
  5. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. arXiv:1712.05526, 2017.
  6. Feddat: An approach for foundation model finetuning in multi-modal heterogeneous federated learning. arXiv preprint arXiv:2308.12305, 2023.
  7. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872–138878, 2019.
  8. Mitigating data heterogeneity in federated learning with data augmentation. arXiv preprint arXiv:2206.09979, 2022.
  9. A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022.
  10. Local model poisoning attacks to {{\{{Byzantine-Robust}}\}} federated learning. In USENIX, 2020.
  11. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557, 2017.
  12. Badnets: Identifying vulnerabilities in the machine learning model supply chain. CoRR, abs/1708.06733, 2017.
  13. Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model. IEEE Transactions on Mobile Computing, 2023.
  14. Deep residual learning for image recognition, 2015.
  15. Distilling the knowledge in a neural network, 2015.
  16. Backdoor attacks for in-context learning with language models. CoRR, abs/2307.14692, 2023.
  17. Segment anything, 2023.
  18. Fedmd: Heterogenous federated learning via model distillation. CoRR, abs/1910.03581, 2019.
  19. Backdoor attacks on pre-trained models by layerwise weight poisoning. In EMNLP, 2021.
  20. Temporal-distributed backdoor attack against video based action recognition. CoRR, abs/2308.11070, 2023.
  21. Ensemble distillation for robust model fusion in federated learning. In NeurIPS, 2020.
  22. Defense against backdoor attack in federated learning. Comput. Secur., 121:102819, 2022.
  23. Communication-efficient learning of deep networks from decentralized data. In AISTATS, 2017.
  24. FLAME: taming backdoors in federated learning. In USENIX, 2022.
  25. Privacy risks of general-purpose language models. In SP, 2020.
  26. Federated self-supervised learning for video understanding, 2022.
  27. High-resolution image synthesis with latent diffusion models, 2022.
  28. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020.
  29. On the adversarial robustness of multi-modal foundation models. In ICCV, 2023.
  30. Badgpt: Exploring security vulnerabilities of chatgpt via backdoor attacks to instructgpt. CoRR, abs/2304.12298, 2023.
  31. Prompting gpt-3 to be reliable. arXiv preprint arXiv:2210.09150, 2022.
  32. Can you really backdoor federated learning? International Workshop on Federated Learning for Data Privacy and Confidentiality at NeurIPS 2019, 2019.
  33. Federated learning from pre-trained models: A contrastive learning approach. NeuIPS, 35:19332–19344, 2022.
  34. Data poisoning attacks against federated learning systems. In ESORICS, 2020.
  35. Llama: Open and efficient foundation language models, 2023.
  36. Attack of the tails: Yes, you really can backdoor federated learning. In NeurIPS, 2020.
  37. Towards federated covid-19 vaccine side effect prediction. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 437–452. Springer, 2022.
  38. Decodingtrust: A comprehensive assessment of trustworthiness in GPT models. CoRR, abs/2306.11698, 2023.
  39. Toward cleansing backdoored neural networks in federated learning. In ICDCS, 2022.
  40. A Backdoor Attack against 3D Point Cloud Classifiers. ICCV, 2021.
  41. DBA: distributed backdoor attacks against federated learning. In ICLR. OpenReview.net, 2020.
  42. CRFL: certifiably robust federated learning against backdoor attacks. In Marina Meila and Tong Zhang, editors, ICML, 2021.
  43. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models. CoRR, abs/2305.14710, 2023.
  44. Bridging the gap between foundation models and heterogeneous federated learning. arXiv preprint arXiv:2310.00247, 2023.
  45. Bayesian nonparametric federated learning of neural networks. In ICML, 2019.
  46. Character-level convolutional networks for text classification. In NeurIPS, pages 649–657, 2015.
  47. Personalized federated learning via variational bayesian inference. In International Conference on Machine Learning, pages 26293–26310. PMLR, 2022.
  48. Attack-sam: Towards evaluating adversarial robustness of segment anything model. arXiv preprint arXiv:2305.00866, 2023.
  49. GPT-FL: generative pre-trained model-assisted federated learning. CoRR, abs/2306.02210, 2023.
  50. When foundation model meets federated learning: Motivations, challenges, and future directions. CoRR, abs/2306.15546, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Chen Wu (169 papers)
  2. Xi Li (197 papers)
  3. Jiaqi Wang (218 papers)
Citations (2)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets