Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Auditable Homomorphic-based Decentralized Collaborative AI with Attribute-based Differential Privacy (2403.00023v1)

Published 28 Feb 2024 in cs.CR, cs.AI, and cs.LG

Abstract: In recent years, the notion of federated learning (FL) has led to the new paradigm of distributed AI with privacy preservation. However, most current FL systems suffer from data privacy issues due to the requirement of a trusted third party. Although some previous works introduce differential privacy to protect the data, however, it may also significantly deteriorate the model performance. To address these issues, we propose a novel decentralized collaborative AI framework, named Auditable Homomorphic-based Decentralised Collaborative AI (AerisAI), to improve security with homomorphic encryption and fine-grained differential privacy. Our proposed AerisAI directly aggregates the encrypted parameters with a blockchain-based smart contract to get rid of the need of a trusted third party. We also propose a brand-new concept for eliminating the negative impacts of differential privacy for model performance. Moreover, the proposed AerisAI also provides the broadcast-aware group key management based on ciphertext-policy attribute-based encryption (CPABE) to achieve fine-grained access control based on different service-level agreements. We provide a formal theoretical analysis of the proposed AerisAI as well as the functionality comparison with the other baselines. We also conduct extensive experiments on real datasets to evaluate the proposed approach. The experimental results indicate that our proposed AerisAI significantly outperforms the other state-of-the-art baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  2. X. Zhang, M. Hu, J. Xia, T. Wei, M. Chen, and S. Hu, “Efficient federated learning for cloud-based aiot applications,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 40, no. 11, pp. 2211–2223, 2020.
  3. H. Baghban, A. Rezapour, C.-H. Hsu, S. Nuannimnoi, and C.-Y. Huang, “Edge-ai: Iot request service provisioning in federated edge computing using actor-critic reinforcement learning,” IEEE Transactions on Engineering Management, pp. 1–10, 2022.
  4. J. Leng, X. Zhu, Z. Huang, K. Xu, Z. Liu, Q. Liu, and X. Chen, “Manuchain ii: Blockchained smart contract system as the digital twin of decentralized autonomous manufacturing toward resilience in industry 5.0,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023.
  5. T. Wang, B. Sun, L. Wang, X. Zheng, and W. Jia, “Eidls: An edge-intelligence-based distributed learning system over internet of things,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023.
  6. Y. Chen, X. Qin, J. Wang, C. Yu, and W. Gao, “Fedhealth: A federated transfer learning framework for wearable healthcare,” IEEE Intelligent Systems, vol. 35, no. 4, pp. 83–93, 2020.
  7. Z. Yan, J. Wicaksana, Z. Wang, X. Yang, and K.-T. Cheng, “Variation-aware federated learning with multi-source decentralized medical image data,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 7, pp. 2615–2628, 2020.
  8. C. Zhao, X. Dai, Y. Lv, J. Niu, and Y. Lin, “Decentralized autonomous operations and organizations in transverse: Federated intelligence for smart mobility,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022.
  9. Y. Aono, T. Hayashi, L. Wang, S. Moriai et al., “Privacy-preserving deep learning via additively homomorphic encryption,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 5, pp. 1333–1345, 2017.
  10. L. Nagalapatti, R. S. Mittal, and R. Narayanam, “Is your data relevant?: Dynamic selection of relevant data for federated learning,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 7, pp. 7859–7867, Jun. 2022.
  11. L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” Advances in neural information processing systems, vol. 32, 2019.
  12. J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller, “Inverting gradients-how easy is it to break privacy in federated learning?” Advances in Neural Information Processing Systems, vol. 33, pp. 16 937–16 947, 2020.
  13. Y.-L. Huang, C.-Y. Shen, S. Shieh, H.-J. Wang, and C.-C. Lin, “Provable secure aka scheme with reliable key delegation in umts,” in 2009 Third IEEE International Conference on Secure Software Integration and Reliability Improvement.   IEEE, 2009, pp. 243–252.
  14. Y. Chen, J. Li, F. Wang, K. Yue, Y. Li, B. Xing, L. Zhang, and L. Chen, “Ds2pm: A data-sharing privacy protection model based on blockchain and federated learning,” IEEE Internet of Things Journal, vol. 10, no. 14, pp. 12 112–12 125, 2023.
  15. S. Guo, K. Zhang, B. Gong, L. Chen, Y. Ren, F. Qi, and X. Qiu, “Sandbox computing: A data privacy trusted sharing paradigm via blockchain and federated learning,” IEEE Transactions on Computers, vol. 72, no. 3, pp. 800–810, 2023.
  16. M. Xu, Z. Zou, Y. Cheng, Q. Hu, D. Yu, and X. Cheng, “Spdl: A blockchain-enabled secure and privacy-preserving decentralized learning system,” IEEE Transactions on Computers, 2022.
  17. A. P. Kalapaaking, I. Khalil, M. S. Rahman, M. Atiquzzaman, X. Yi, and M. Almashor, “Blockchain-based federated learning with secure aggregation in trusted execution environment for internet-of-things,” IEEE Transactions on Industrial Informatics, vol. 19, no. 2, pp. 1703–1714, 2023.
  18. C. Wang, C. Ma, M. Li, N. Gao, Y. Zhang, and Z. Shen, “Protecting data privacy in federated learning combining differential privacy and weak encryption,” in Science of Cyber Security: Third International Conference, SciSec 2021, Virtual Event, August 13–15, 2021, Revised Selected Papers 4.   Springer, 2021, pp. 95–109.
  19. S. Truex, N. Baracaldo, A. Anwar, T. Steinke, H. Ludwig, R. Zhang, and Y. Zhou, “A hybrid approach to privacy-preserving federated learning,” in Proceedings of the 12th ACM workshop on artificial intelligence and security, 2019, pp. 1–11.
  20. Y. Li, Y. Zhou, A. Jolfaei, D. Yu, G. Xu, and X. Zheng, “Privacy-preserving federated learning framework based on chained secure multiparty computing,” IEEE Internet of Things Journal, vol. 8, no. 8, pp. 6178–6186, 2020.
  21. C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography, S. Halevi and T. Rabin, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 265–284.
  22. J. Bethencourt, A. Sahai, and B. Waters, “Ciphertext-policy attribute-based encryption,” in 2007 IEEE Symposium on Security and Privacy (SP ’07), 2007, pp. 321–334.
  23. Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.
  24. R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” Advances in neural information processing systems, vol. 12, 1999.
  25. S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” Decentralized Business Review, p. 21260, 2008.
  26. P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” vol. 30, 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/file/f4b9ec30ad9f68f89b29639786cb62ef-Paper.pdf
  27. V. Costan and S. Devadas, “Intel sgx explained,” 2016, https://eprint.iacr.org/2016/086. [Online]. Available: https://eprint.iacr.org/2016/086
  28. C.-Y. Shen, D.-N. Yang, L.-H. Huang, W.-C. Lee, and M.-S. Chen, “Socio-spatial group queries for impromptu activity planning,” IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 1, pp. 196–210, 2015.
  29. C.-Y. Shen, L.-H. Huang, D.-N. Yang, H.-H. Shuai, W.-C. Lee, and M.-S. Chen, “On finding socially tenuous groups for online social networks,” in Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 2017, pp. 415–424.
  30. C.-Y. Shen, D.-N. Yang, W.-C. Lee, and M.-S. Chen, “Activity organization for friend-making optimization in online social networks,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 1, pp. 122–137, 2020.
  31. C.-Y. Shen, H.-H. Shuai, D.-N. Yang, G.-S. Lee, L.-H. Huang, W.-C. Lee, and M.-S. Chen, “On extracting socially tenuous groups for online social networks with k𝑘kitalic_k k-triangles,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 7, pp. 3431–3446, 2020.
  32. Y.-W. Chang, K.-P. Lin, and C.-Y. Shen, “Blockchain technology for e-marketplace,” in 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).   IEEE, 2019, pp. 429–430.
  33. Y.-L. Chen, D.-N. Yang, C.-Y. Shen, W.-C. Lee, and M.-S. Chen, “On efficient processing of group and subsequent queries for social activity planning,” IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 12, pp. 2364–2378, 2018.
  34. K.-P. Lin, Y.-W. Chang, Z.-H. Wei, C.-Y. Shen, and M.-Y. Chang, “A smart contract-based mobile ticketing system with multi-signature and blockchain,” in 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE).   IEEE, 2019, pp. 231–232.
  35. C.-H. Yang, H.-H. Shuai, C.-Y. Shen, and M.-S. Chen, “Learning to solve task-optimized group search for social internet of things,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 11, pp. 5429–5445, 2021.
  36. Y.-L. Chang, Y.-J. Chang, and C.-Y. Shen, “She is in a bad mood now: leveraging peers to increase data quantity via a chatbot-based esm,” in Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, 2019, pp. 1–6.
  37. L.-Y. Yeh, C.-Y. Shen, W.-C. Huang, W.-H. Hsu, and H.-C. Wu, “Gdpr-aware revocable p2p file-sharing system over consortium blockchain,” IEEE Systems Journal, vol. 16, no. 4, pp. 5234–5245, 2022.
  38. C.-C. Chang, M.-Y. Chang, J.-Y. Jhang, L.-Y. Yeh, and C.-Y. Shen, “Learning to extract expert teams in social networks,” IEEE Transactions on Computational Social Systems, vol. 9, no. 5, pp. 1552–1562, 2022.
  39. C.-H. Yang and C.-Y. Shen, “Enhancing machine learning approaches for graph optimization problems with diversifying graph augmentation,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 2191–2201.
  40. L. Nagalapatti and R. Narayanam, “Game of gradients: Mitigating irrelevant clients in federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 9046–9054.
  41. P. Mohassel and Y. Zhang, “Secureml: A system for scalable privacy-preserving machine learning,” in 2017 IEEE symposium on security and privacy (SP).   IEEE, 2017, pp. 19–38.
  42. E. Androulaki, A. Barger, V. Bortnikov, C. Cachin, K. Christidis, A. De Caro, D. Enyeart, C. Ferris, G. Laventman, Y. Manevich, S. Muralidharan, C. Murthy, B. Nguyen, M. Sethi, G. Singh, K. Smith, A. Sorniotti, C. Stathakopoulou, M. Vukolić, S. W. Cocco, and J. Yellick, “Hyperledger fabric: A distributed operating system for permissioned blockchains,” in Proceedings of the Thirteenth EuroSys Conference, ser. EuroSys ’18.   New York, NY, USA: Association for Computing Machinery, 2018. [Online]. Available: https://doi.org/10.1145/3190508.3190538
  43. X. Xu, C. Pautasso, L. Zhu, V. Gramoli, A. Ponomarev, A. B. Tran, and S. Chen, “The blockchain as a software connector,” in 2016 13th Working IEEE/IFIP Conference on Software Architecture (WICSA).   IEEE, 2016, pp. 182–191.
  44. H. Moudoud, S. Cherkaoui, and L. Khoukhi, “An iot blockchain architecture using oracles and smart contracts: the use-case of a food supply chain,” in 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).   IEEE, 2019, pp. 1–6.
  45. L.-Y. Yeh, N.-X. Shen, and R.-H. Hwang, “Blockchain-based privacy-preserving and sustainable data query service over 5g-vanets,” IEEE Transactions on Intelligent Transportation Systems, 2022.
  46. P. Zhao, Z. Cao, J. Jiang, and F. Gao, “Practical private aggregation in federated learning against inference attack,” IEEE Internet of Things Journal, 2022.
  47. A. Sahai and B. Waters, “Fuzzy identity-based encryption,” in Annual international conference on the theory and applications of cryptographic techniques.   Springer, 2005, pp. 457–473.
  48. A. Acar, H. Aksu, A. S. Uluagac, and M. Conti, “A survey on homomorphic encryption schemes: Theory and implementation,” ACM Computing Surveys (Csur), vol. 51, no. 4, pp. 1–35, 2018.
  49. P. Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” in International conference on the theory and applications of cryptographic techniques.   Springer, 1999, pp. 223–238.
  50. A. K. Lenstra, “Integer factoring,” Towards a quarter-century of public key cryptography, pp. 31–58, 2000.
  51. J. Sousa, A. Bessani, and M. Vukolic, “A byzantine fault-tolerant ordering service for the hyperledger fabric blockchain platform,” in 2018 48th annual IEEE/IFIP international conference on dependable systems and networks (DSN).   IEEE, 2018, pp. 51–58.
  52. A. Sharma, F. M. Schuhknecht, D. Agrawal, and J. Dittrich, “How to databasify a blockchain: the case of hyperledger fabric,” arXiv preprint arXiv:1810.13177, 2018.
  53. ——, “Blurring the lines between blockchains and database systems: the case of hyperledger fabric,” in Proceedings of the 2019 International Conference on Management of Data, 2019, pp. 105–122.
  54. L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012.
  55. A. Krizhevsky et al., “Learning multiple layers of features from tiny images,” 2009.
  56. F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,” The International Conference on Learning Representations (ICLR), 2017.
  57. fabric, “Hyperledger fabric documents,” https://hyperledger-fabric.readthedocs.io/en/latest/index.html, 2023.
  58. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

Summary

  • The paper introduces AerisAI, a novel framework that integrates homomorphic encryption and attribute-based differential privacy for decentralized AI.
  • It employs blockchain and CP-ABE to ensure transparent auditability and secure gradient aggregation without central authorities.
  • Empirical evaluations demonstrate that AerisAI maintains a strong balance between privacy preservation and model performance in federated learning.

Auditable Homomorphic-based Decentralized Collaborative AI with Attribute-based Differential Privacy

Introduction

The exponential growth in data and computational power has paved the way for advancements in AI, particularly in deep learning. However, the utility of such models often hinges on the quantity and quality of data, which, when sourced from multiple entities, brings forth significant privacy challenges. Federated Learning (FL) emerged as a paradigm to mitigate privacy concerns by decentralizing the training process, allowing data to remain at its source. Despite its advances, current FL systems exhibit critical limitations, including dependency on a trusted third party, susceptibility to gradient leakage, and the inherent tension between privacy and model performance when applying differential privacy techniques. Addressing these concerns, Lo-Yao Yeh et al. propose a novel framework named Auditable Homomorphic-based Decentralized Collaborative AI (AerisAI), integrating homomorphic encryption and attribute-based differential privacy within a blockchain-based architecture to enhance both privacy and auditability without compromising model accuracy.

System Model and Security Assumptions

AerisAI's system architecture comprises clients, a blockchain network, smart contracts, and an oracle. The decentralized approach eliminates reliance on a centralized server, thus mitigating single points of failure and enhancing system robustness. Through blockchain and smart contracts, AerisAI ensures transparency and auditability, bolstering system integrity and trust among participants. The framework employs homomorphic encryption and differential privacy techniques to secure gradient information, ensuring privacy preservation even in a decentralized setup. Furthermore, the system adopts group key management based on ciphertext-policy attribute-based encryption (CP-ABE), facilitating efficient and scalable distribution of encrypted artifacts.

The Proposed Scheme

AerisAI's operational workflow encapsulates several steps: local model training by clients, noise addition for differential privacy, encryption of perturbed gradients and noise using different keys for clients and the oracle, respectively, and aggregation of encrypted data via smart contracts. Crucially, AerisAI distinguishes itself by aggregating encrypted noise and leveraging group key management for its efficient distribution, addressing the scalability issues earlier schemes faced. The intricate balance of encryption and differential privacy underpins the scheme's capability to preserve data privacy without undermining model performance significantly.

Security Analysis and Evaluation

The formal security analysis substantiates the robustness of AerisAI against gradient leakage and unauthorized access to sensitive data. Leveraging the trusted properties of homomorphic encryption, differential privacy, and CP-ABE, AerisAI ensures that gradient information and noise remain confidential, accessible only to authorized parties. Empirical evaluations further demonstrate AerisAI's superiority in model performance across diverse datasets and configurations, underscoring its potential for practical deployment in privacy-sensitive collaborative AI tasks.

Conclusion and Future Directions

AerisAI emerges as a comprehensive solution to the intricate challenges of privacy preservation in decentralized collaborative AI. By harmoniously integrating homomorphic encryption, differential privacy, and blockchain technology, AerisAI not only advances the state of federated learning but also opens novel avenues for research in secure, decentralized AI systems. Future work could explore the optimization of cryptographic primitives and blockchain protocols to further enhance the scalability and efficiency of such frameworks, catering to the ever-growing demands of AI-driven industries.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets