Local Differential Privacy based Federated Learning for Internet of Things (2004.08856v2)

Abstract: Internet of Vehicles (IoV) is a promising branch of the Internet of Things. IoV simulates a large variety of crowdsourcing applications such as Waze, Uber, and Amazon Mechanical Turk, etc. Users of these applications report the real-time traffic information to the cloud server which trains a machine learning model based on traffic information reported by users for intelligent traffic management. However, crowdsourcing application owners can easily infer users' location information, which raises severe location privacy concerns of the users. In addition, as the number of vehicles increases, the frequent communication between vehicles and the cloud server incurs unexpected amount of communication cost. To avoid the privacy threat and reduce the communication cost, in this paper, we propose to integrate federated learning and local differential privacy (LDP) to facilitate the crowdsourcing applications to achieve the machine learning model. Specifically, we propose four LDP mechanisms to perturb gradients generated by vehicles. The Three-Outputs mechanism is proposed which introduces three different output possibilities to deliver a high accuracy when the privacy budget is small. The output possibilities of Three-Outputs can be encoded with two bits to reduce the communication cost. Besides, to maximize the performance when the privacy budget is large, an optimal piecewise mechanism (PM-OPT) is proposed. We further propose a suboptimal mechanism (PM-SUB) with a simple formula and comparable utility to PM-OPT. Then, we build a novel hybrid mechanism by combining Three-Outputs and PM-SUB.

• The paper introduces four novel Local Differential Privacy (LDP) mechanisms (Three-Outputs, PM-OPT, PM-SUB, and HM-TP) specifically designed to perturb gradients for privacy in Federated Learning for IoT.
• The LDP-FedSGD algorithm is presented to enable privacy-preserving collaborative learning in IoT environments, such as Internet of Vehicles, by uploading perturbed gradients instead of raw data.
• Experimental validation confirms that the proposed LDP mechanisms integrated into FL achieve higher accuracy in tasks like mean frequency estimation and regression compared to existing privacy solutions.

Local Differential Privacy-Based Federated Learning for the Internet of Things

This paper introduces a comprehensive approach to enhancing federated learning (FL) in the Internet of Things (IoT) domain by integrating local differential privacy (LDP). The research specifically addresses the growing concerns related to privacy and communication costs in the Internet of Vehicles (IoV), leveraging IoT infrastructure for intelligent traffic management.

Core Contributions

1. LDP Mechanisms:
   • The paper proposes four distinct LDP mechanisms: Three-Outputs, PM-OPT, PM-SUB, and HM-TP. These mechanisms are designed to perturb gradients from vehicles, each offering benefits tailored to different privacy budgets.
   • The Three-Outputs mechanism features three possible outputs, designed to optimize accuracy in high privacy regimes with minimal communication cost since each output can be encoded with just two bits.
   • PM-OPT is proposed for scenarios with larger privacy budgets, providing superior utility through an optimal piecewise mechanism framework.
   • PM-SUB offers a simplified alternative to PM-OPT, maintaining comparable utility with less complexity.
   • HM-TP represents a hybrid approach, combining Three-Outputs and PM-SUB to optimize across a range of privacy budgets.
2. Federated Learning Integration:
   • The LDP-FedSGD algorithm is introduced to allow vehicles to collaborate with a cloud server, preserving privacy by uploading perturbed gradients rather than raw data.
   • This integration helps mitigate inference attacks, addressing potential privacy threats that could arise from analyzing gradients individually.
3. Theoretical Underpinnings and Optimization:
   • The paper explores the mathematical foundations of the proposed mechanisms, providing rigorous proofs of privacy and utility guarantees.
   • Extensive analysis determines the optimal mechanism under specified conditions, further bolstering claims with experimental validation on real-world datasets.
4. Experimental Validation:
   • Thorough experimentation across multiple datasets, including synthetic and real-world IoT data, demonstrates the performance improvements of the proposed mechanisms over existing solutions.
   • The research confirms that the new mechanisms achieve higher accuracy in estimating mean frequencies of numeric data and in conducting machine learning tasks such as logistic and linear regression, as well as SVM classification.

Implications for Research and Practice

The integration of LDP into FL frameworks, as proposed, represents a significant advancement in secure and efficient IoT applications. The mechanisms facilitate robust privacy preservation without sacrificing the utility necessary for real-time decision-making in systems like IoV. This research has the potential to be expanded to broader IoT applications involving various data types and higher-dimensional datasets.

Future Directions

Moving forward, the insights and methodologies presented in this paper can provide a basis for enhancing other FL mechanisms or developing new AI models that require strong privacy guarantees. The scalability of these solutions to more complex IoT environments remains an exciting avenue for future exploration.

In conclusion, this paper offers a robust framework of enhanced privacy-preserving techniques for distributed learning in IoT applications. Its rigorous analytical foundation, coupled with practical experimental results, suggests that its integration into IoV could result in significant advancements in both privacy and computational efficiency.