Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Kick Bad Guys Out! Conditionally Activated Anomaly Detection in Federated Learning with Zero-Knowledge Proof Verification (2310.04055v4)

Published 6 Oct 2023 in cs.CR and cs.AI

Abstract: Federated Learning (FL) systems are susceptible to adversarial attacks, where malicious clients submit poisoned models to disrupt the convergence or plant backdoors that cause the global model to misclassify some samples. Current defense methods are often impractical for real-world FL systems, as they either rely on unrealistic prior knowledge or cause accuracy loss even in the absence of attacks. Furthermore, these methods lack a protocol for verifying execution, leaving participants uncertain about the correct execution of the mechanism. To address these challenges, we propose a novel anomaly detection strategy that is designed for real-world FL systems. Our approach activates the defense only when potential attacks are detected, and enables the removal of malicious models without affecting the benign ones. Additionally, we incorporate zero-knowledge proofs to ensure the integrity of the proposed defense mechanism. Experimental results demonstrate the effectiveness of our approach in enhancing FL system security against a comprehensive set of adversarial attacks in various ML tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. T. Aoki. On the stability of the linear transformation in banach spaces. Journal of the Mathematical Society of Japan, 2(1-2):64–66, 1950.
  2. How to backdoor federated learning. In AISTATS, 2020a.
  3. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pp.  2938–2948. PMLR, 2020b.
  4. Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning, pp.  634–643. PMLR, 2019.
  5. Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurIPS, 2017.
  6. Differentially private secure multi-party computation for federated learning in financial applications. In Proceedings of the First ACM International Conference on AI in Finance, pp.  1–9, 2020.
  7. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018.
  8. Mpaf: Model poisoning attacks to federated learning based on fake clients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  3396–3404, 2022.
  9. Fltrust: Byzantine-robust federated learning via trust bootstrapping. arXiv preprint arXiv:2012.13995, 2020.
  10. Flcert: Provably secure federated learning against poisoning attacks. IEEE Transactions on Information Forensics and Security, 17:3691–3705, 2022.
  11. Federated learning of out-of-vocabulary words. arXiv preprint arXiv:1903.10635, 2019.
  12. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. ACM on Measurement and Analysis of Computing Systems, 1(2):1–25, 2017.
  13. A review of medical federated learning: Applications in oncology and cancer research. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, pp.  3–24. Springer, 2022.
  14. Local model poisoning attacks to Byzantine-robust federated learning. In USENIX Security, 2020.
  15. ZEN: An optimizing compiler for verifiable, zero-knowledge neural network inferences. 2021. Cryptology ePrint Archive.
  16. Grant S Fletcher. Clinical epidemiology: the essentials. Lippincott Williams & Wilkins, 2019.
  17. R. Freivalds. Probabilistic machines can use less running time. In IFIP Congress, 1977.
  18. Attack-resistant federated learning with residual-based reweighting. arXiv preprint arXiv:1912.11464, 2019.
  19. The limitations of federated learning in sybil settings. In RAID, pp.  301–316, 2020.
  20. The knowledge complexity of interactive proof systems. SIAM Jour. on Comp., 18(1):186–208, 1989.
  21. J. Groth. On the size of pairing-based non-interactive arguments. In Eurocrypt, 2016.
  22. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning, pp.  3521–3530. PMLR, 2018.
  23. An iterative scheme for leverage-based approximate aggregation. In IEEE ICDE, 2019.
  24. Fedmlsecurity: A benchmark for attacks and defenses in federated learning and federated llms. arXiv preprint arXiv:2306.04959, 2023.
  25. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604, 2018.
  26. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  27. Byzantine-robust decentralized learning via self-centered clipping. 2022. Available on arXiv:2202.01545.
  28. Cafe: Catastrophic data leakage in vertical federated learning. Advances in Neural Information Processing Systems, 34:994–1006, 2021.
  29. Byzantine-robust learning on heterogeneous datasets via bucketing. arXiv preprint arXiv:2006.09365, 2020.
  30. Cocktail party attack: Breaking aggregation-based privacy in federated learning using independent component analysis. In International Conference on Machine Learning, 2022. URL https://api.semanticscholar.org/CorpusID:252211968.
  31. Learning multiple layers of features from tiny images. 2009.
  32. Gradient disaggregation: Breaking privacy in federated learning by reconstructing the user participant matrix. In International Conference on Machine Learning, pp.  5959–5968. PMLR, 2021.
  33. vCNN: Verifiable convolutional neural network based on zk-snarks. 2020. Cryptology ePrint Archive.
  34. Federated learning for keyword spotting. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.  6341–6345, 2019.
  35. Learning to detect malicious clients for robust federated learning. arXiv preprint arXiv:2002.00211, 2020.
  36. ZkCNN: Zero knowledge proofs for convolutional neural network predictions and accuracy. In ACM CCS, 2021.
  37. A. Lyon. Why are normal distributions normal? The British Journal for the Philosophy of Science, 65(3):621–649, 2014.
  38. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp.  1273–1282. PMLR, 2017a.
  39. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp.  1273–1282. PMLR, 2017b.
  40. J. Osborne. Improving your data transformations: Applying the Box-Cox transformation. Practical Assessment, Research, and Evaluation, 15(1):12, 2010.
  41. Defending against backdoors in federated learning with robust learning rate. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  9268–9276, 2021.
  42. Robust aggregation for federated learning. IEEE Transactions on Signal Processing, 70:1142–1154, 2022.
  43. Federated learning for emoji prediction in a mobile keyboard. arXiv preprint arXiv:1906.04329, 2019.
  44. M. Rosenblatt. A central limit theorem and a strong mixing condition. National Academy of Sciences, 42(1):43–47, 1956.
  45. R. M. Sakia. The Box-Cox transformation technique: A review. Journal of the Royal Statistical Society: Series D, 41(2):169–178, 1992.
  46. Zerocash: Decentralized anonymous payments from bitcoin. In IEEE S&P, 2014.
  47. Fl-wbc: Enhancing robustness against model poisoning attacks in federated learning from a client perspective. Advances in Neural Information Processing Systems, 34:12613–12624, 2021.
  48. Can you really backdoor federated learning? arXiv preprint arXiv:1911.07963, 2019.
  49. Circom Contributors. Circom zkSNARK ecosystem, 2022. https://github.com/iden3/circom.
  50. Data poisoning attacks against federated learning systems. In European Symposium on Research in Computer Security, pp.  480–501. Springer, 2020.
  51. Model poisoning attacks against distributed machine learning systems. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, volume 11006, pp.  481–489. SPIE, 2019.
  52. Attack of the tails: Yes, you really can backdoor federated learning. In NeurIPS, 2020.
  53. Jianhua Wang. Pass: Parameters audit-based secure and fair federated learning scheme against free rider. arXiv preprint arXiv:2207.07292, 2022.
  54. S. Weisberg. Yeo-Johnson power transformations. 2001. Available at https://www.stat.umn.edu/arc/yjpower.pdf.
  55. SLSGD: Secure and Efficient Distributed On-device Machine Learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  213–228. Springer, 2020.
  56. Byzantine-resilient stochastic gradient descent for distributed learning: A Lipschitz-inspired coordinate-wise median approach. In IEEE CDC, 2019.
  57. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pp.  5650–5659. PMLR, 2018.
  58. Implementation of fldetector. https://github.com/zaixizhang/FLDetector, 2022a.
  59. Fldetector: Defending federated learning against model poisoning attacks via detecting malicious clients. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  2545–2555, 2022b.
  60. Neurotoxin: Durable backdoors in federated learning. In International Conference on Machine Learning, 2022c. URL https://api.semanticscholar.org/CorpusID:249889464.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Shanshan Han (18 papers)
  2. Wenxuan Wu (16 papers)
  3. Baturalp Buyukates (26 papers)
  4. Weizhao Jin (8 papers)
  5. Yuhang Yao (32 papers)
  6. Qifan Zhang (19 papers)
  7. Salman Avestimehr (116 papers)
  8. Chaoyang He (46 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now