You Still See Me: How Data Protection Supports the Architecture of AI Surveillance (2402.06609v3)
Abstract: Data forms the backbone of AI. Privacy and data protection laws thus have strong bearing on AI systems. Shielded by the rhetoric of compliance with data protection and privacy regulations, privacy-preserving techniques have enabled the extraction of more and new forms of data. We illustrate how the application of privacy-preserving techniques in the development of AI systems--from private set intersection as part of dataset curation to homomorphic encryption and federated learning as part of model computation--can further support surveillance infrastructure under the guise of regulatory permissibility. Finally, we propose technology and policy strategies to evaluate privacy-preserving techniques in light of the protections they actually confer. We conclude by highlighting the role that technologists could play in devising policies that combat surveillance AI technologies.
- Exploring deep federated learning for the internet of things: A gdpr-compliant architecture. IEEE Access, 2023.
- A survey on homomorphic encryption schemes: Theory and implementation. ACM Computing Surveys (Csur), 51(4):1–35, 2018.
- Amazon. Enable fully homomorphic encryption with amazon sagemaker endpoints for secure, real-time inferencing. https://aws.amazon.com/blogs/machine-learning/enable-fully-homomorphic-encryption-with-amazon-sagemaker-endpoints-for-secure-real-time-inferencing/, 2023.
- Jack M Balkin. Information fiduciaries and the first amendment. UCDL Rev., 49:1183, 2015.
- Big data’s end run around anonymity and consent. Privacy, big data, and the public good: Frameworks for engagement, 1:44–75, 2014.
- CCD-MPC, 2019.
- When the curious abandon honesty: Federated learning is not private. In 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P), pages 175–199. IEEE, 2023.
- What does it mean for a language model to preserve privacy? In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2280–2292, 2022.
- Dan L Burk. Perverse innovation. Wm. & Mary L. Rev., 58:1, 2016.
- California State Legislature. 1.81.5. california consumer privacy act of 2018 [1798.100 - 1798.199.100], 2018.
- Subscriptions and external links help drive resentful users to alternative and extremist youtube channels. Science Advances, 9(35):eadd8080, 2023.
- Bryan H Choi. The grokster dead-end. Harv. JL & Tech., 19:393, 2005.
- Danielle Citron. The Fight for Privacy: Protecting Dignity, Identity, and Love in the Digital Age. Random House, 2022.
- Julie E Cohen. Surveillance vs. privacy: effects and implications. Cambridge Handbook of Surveillance Law, eds. David Gray & Stephen E. Henderson (New York: Cambridge University Press, 2017), pages 455–69, 2017.
- The role of differential privacy in gdpr compliance. In FAT’18: Proceedings of the Conference on Fairness, Accountability, and Transparency, page 20, 2018.
- A Pragmatic Introduction to Secure Multi-Party Computation. NOW Publishers, December 2018.
- CoVault: A Secure Analytics Platform. arXiv preprint arXiv:2208.03784, 2022.
- Privacy side channels in machine learning systems. arXiv preprint arXiv:2309.05610, 2023.
- Delaware State Legislature. Delaware personal data privacy act. https://legiscan.com/DE/text/HB154/id/2807502/Delaware-2023-HB154-Draft.html, 2024.
- Value-laden disciplinary shifts in machine learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 294–294, 2020.
- Cynthia Dwork. A firm foundation for private data analysis. Communications of the ACM, 54(1):86–95, 2011.
- Generalization in adaptive data analysis and holdout reuse. Advances in Neural Information Processing Systems, 28, 2015.
- EU Legislature. General data protection regulation, 2018.
- European Union Legislature. Eu ai act. https://artificialintelligenceact.eu/the-act/, 2021.
- The principle of purpose limitation and big data. New technology, big data and the law, pages 17–42, 2017.
- On aereo and’avoision’. 2014.
- Andrew S Gold. The fiduciary duty of loyalty. The Oxford Handbook of Fiduciary Law (New York: Oxford University Press, 2019), 2019.
- The case for establishing a collective perspective to address the harms of platform personalization. In Proceedings of the 2022 Symposium on Computer Science and Law, pages 119–130, 2022.
- Graham Greenleaf. California’s ccpa 2.0: Does the us finally have a data privacy act? 2020.
- Privacy nicks: How the law normalizes surveillance. 2023.
- Self-destructing models: Increasing the costs of harmful dual uses of foundation models. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 287–296, 2023.
- Illinois State Legislature. (740 ilcs 14/) biometric information privacy act., 2008.
- Having your privacy cake and eating it too: Platform-supported auditing of social media algorithms for public interest. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1):1–33, 2023.
- James Reyes. Building the next generation of digital advertising in MPC, April 2022.
- A gdpr-compliant ecosystem for speech recognition with transfer, federated, and evolutionary learning. ACM Transactions on Intelligent Systems and Technology (TIST), 12(3):1–19, 2021.
- Nonrivalry and the economics of data. American Economic Review, 110(9):2819–2858, 2020.
- Comprehension from Chaos: Towards Informed Consent for Private Computation. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 210–224, Copenhagen Denmark, November 2023. ACM.
- Differentially private two-party set operations. In 2020 IEEE European Symposium on Security and Privacy (EuroS&P), pages 390–404. IEEE, 2020.
- The surveillance ai pipeline, 2023.
- A new way to protect privacy in large-scale genome-wide association studies. Bioinformatics, 29(7):886–893, 2013.
- The ethical algorithm: The science of socially aware algorithm design. Oxford University Press, 2019.
- A skeptical view of information fiduciaries. Harvard Law Review, 133(2):497–541, 2019.
- Privacy-preserving set operations. In Annual International Cryptology Conference, pages 241–257. Springer, 2005.
- Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527, 2016.
- Bert-Jaap Koops. Some reflections on profiling, power shifts, and protection paradigms. PROFILING THE EUROPEAN CITIZEN, Hildebrandt & Gutwirth, eds., Springer, 2008.
- Equalizing credit opportunity in algorithms: Aligning algorithmic fairness research with us fair lending regulation. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 357–368, 2022.
- Lawrence Lessig. Code is law. Harvard magazine, 1:2000, 2000.
- We need to focus on how our data is used, not just how it is shared. Communications of the ACM, 66(9):32–34, 2023.
- Vertical federated learning: Concepts, advances and challenges, 2023.
- David Lyon. Surveillance as social sorting: Privacy, risk, and digital discrimination. Psychology Press, 2003.
- David Lyon. Surveillance studies: An overview. 2007.
- Evan Malmgren. Resisting “big other”: What will it take to defeat surveillance capitalism? In New Labor Forum, volume 28, pages 42–50. SAGE Publications Sage CA: Los Angeles, CA, 2019.
- Google and Mastercard Cut a Secret Ad Deal to Track Retail Sales. Bloomberg, August 2018.
- Microsoft. Federated learning with azure machine learning: Powering privacy-preserving innovation in ai. https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/federated-learning-with-azure-machine-learning-powering-privacy/ba-p/3824720, 2023.
- Paul B Miller. The identification of fiduciary relationships. The Oxford Handbook of Fiduciary Law (New York: Oxford University Press, 2019), 2019.
- New Jersey State Legislature. Sb332. https://www.njleg.state.nj.us/bill-search/2022/S332, 2023.
- Helen Nissenbaum. Accountability in a computerized society. Science and engineering ethics, 2:25–42, 1996.
- Helen Nissenbaum. Privacy as contextual integrity. Wash. L. Rev., 79:119, 2004.
- Sok: Security and privacy in machine learning. In 2018 IEEE European Symposium on Security and Privacy (EuroS&P), pages 399–414. IEEE, 2018.
- How to deploy machine learning with differential privacy, 2021.
- Valuing social data. U of Colorado Law Legal Studies Research Paper, (23-16), 2023.
- Radia Perlman. The ephemerizer: Making data disappear, 2005.
- How to dp-fy ml: A practical guide to machine learning with differential privacy. Journal of Artificial Intelligence Research, 77:1113–1201, 2023.
- David E Pozen. Privacy-privacy tradeoffs. The University of Chicago Law Review, pages 221–247, 2016.
- A duty of loyalty for privacy law. Wash. UL Rev., 99:961, 2021.
- Neil M Richards. The dangers of surveillance. Harv. L. Rev., 126:1934, 2012.
- Vanish: increasing data privacy with self-destructing data. In Proceedings of the 18th USENIX Security Symposium, Montreal, Canada, August 2009. USENIX Association.
- Confidential machine learning on untrusted platforms: a survey. Cybersecurity, 4(1):1–19, 2021.
- Bruce Schneier. A Hacker’s Mind: How the Powerful Bend Society’s Rules, and how to Bend Them Back. WW Norton & Company, 2023.
- The pii problem: Privacy and a new concept of personally identifiable information. NYUL rev., 86:1814, 2011.
- Deconstructing design decisions: Why courts must interrogate machine learning and other technologies. Ohio State Law Journal, pages 23–22, 2024.
- Seny Kamara. Crypto for the People, August 2020.
- Daniel J Solove. Data is what data does: Regulating use, harm, and risk instead of sensitive data. 118 Northwestern University Law Review (Forthcoming), 2023.
- Personal data and encryption in the european general data protection regulation. J. Intell. Prop. Info. Tech. & Elec. Com. L., 7:163, 2016.
- Beyond memorization: Violating privacy via inference with large language models. arXiv preprint arXiv:2310.07298, 2023.
- Applied federated learning: Architectural design for robust and efficient learning in privacy aware settings. arXiv preprint arXiv:2206.00807, 2022.
- A survey of collaborative filtering techniques. Advances in artificial intelligence, 2009, 2009.
- Texas State Legislature. Texas data privacy and security act. https://capitol.texas.gov/BillLookup/Text.aspx?LegSess=88R&Bill=HB4, 2023.
- Data protection law and multi-party computation: Applications to information exchange between law enforcement agencies. In Proceedings of the 21st Workshop on Privacy in the Electronic Society, pages 69–82, 2022.
- United States Congress. S.919 - data care act of 2021. https://www.congress.gov/bill/117th-congress/senate-bill/919, 2021.
- U.S. Legislature. H.r.8152 - american data privacy and protection act, 2022.
- Rory Van Loo. Regulatory monitors. Columbia Law Review, 119(2):369–444, 2019.
- Rory Van Loo. Privacy pretexts. Cornell L. Rev., 108:1, 2022.
- Michael Veale. Rights for those who unwillingly, unknowingly and unidentifiably compute! 2023.
- The Verge. 23andme admits hackers accessed 6.9 million users’ dna relatives data. https://www.theverge.com/2023/12/4/23988050/23andme-hackers-accessed-user-data-confirmed, 2023.
- Salome Viljoen. A relational theory of data governance. Yale LJ, 131:573, 2021.
- Visa. Secure collaborative machine learning. https://usa.visa.com/dam/VCOM/regional/na/us/about-visa/research/documents/secure-collaborative-machine-learning.pdf, 2022.
- Conclave: secure multi-party computation on big data. In Proceedings of the Fourteenth EuroSys Conference 2019, pages 1–18, Dresden Germany, March 2019. ACM.
- Multi-regulation computing: Examining the legal and policy questions that arise from secure multiparty computation. In Proceedings of the 2022 Symposium on Computer Science and Law, pages 53–65, 2022.
- Riverbed: Enforcing user-defined privacy constraints in distributed web services. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19), pages 615–630, 2019.
- White House. Blueprint for an AI Bill of Rights. https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf.
- Tim Wu. When code isn’t law. Virginia Law Review, pages 679–751, 2003.
- Federated synthetic data generation with differential privacy. Neurocomputing, 468:1–10, 2022.
- Privacy-preserving machine learning: Methods, challenges and directions. arXiv preprint arXiv:2108.04417, 2021.
- Regulating facial processing technologies: Tensions between legal and technical considerations in the application of illinois bipa. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1017–1027, 2022.
- Tal Z Zarsky. Privacy and manipulation in the digital age. Theoretical Inquiries in Law, 20(1):157–188, 2019.
- Safevanish: An improved data self-destruction for protecting data privacy. In 2010 IEEE Second International Conference on Cloud Computing Technology and Science, pages 521–528. IEEE, 2010.
- A survey on federated learning. Knowledge-Based Systems, 216:106775, 2021.