Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
88 tokens/sec
Gemini 2.5 Pro Premium
39 tokens/sec
GPT-5 Medium
25 tokens/sec
GPT-5 High Premium
22 tokens/sec
GPT-4o
88 tokens/sec
DeepSeek R1 via Azure Premium
95 tokens/sec
GPT OSS 120B via Groq Premium
457 tokens/sec
Kimi K2 via Groq Premium
252 tokens/sec
2000 character limit reached

Locally Differentially Private Document Generation Using Zero Shot Prompting (2310.16111v2)

Published 24 Oct 2023 in cs.CL, cs.CR, and cs.LG

Abstract: Numerous studies have highlighted the privacy risks associated with pretrained LLMs. In contrast, our research offers a unique perspective by demonstrating that pretrained LLMs can effectively contribute to privacy preservation. We propose a locally differentially private mechanism called DP-Prompt, which leverages the power of pretrained LLMs and zero-shot prompting to counter author de-anonymization attacks while minimizing the impact on downstream utility. When DP-Prompt is used with a powerful LLM like ChatGPT (gpt-3.5), we observe a notable reduction in the success rate of de-anonymization attacks, showing that it surpasses existing approaches by a considerable margin despite its simpler design. For instance, in the case of the IMDB dataset, DP-Prompt (with ChatGPT) perfectly recovers the clean sentiment F1 score while achieving a 46\% reduction in author identification F1 score against static attackers and a 26\% reduction against adaptive attackers. We conduct extensive experiments across six open-source LLMs, ranging up to 7 billion parameters, to analyze various effects of the privacy-utility tradeoff.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. John M Abowd. 2018. The US census bureau adopts differential privacy. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2867–2867.
  2. Geo-indistinguishability: Differential privacy for location-based systems. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security, pages 901–914.
  3. Large-scale differentially private bert. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6481–6491.
  4. D Apple. 2017. Learning with privacy at scale. Apple Machine Learning Journal, 1(8).
  5. Lamp: Extracting text from gradients with language model priors. In Advances in Neural Information Processing Systems.
  6. Michael Barbaro and Tom Zeller Jr. 2006. A face is exposed for aol searcher no. 4417749. New York Times.
  7. Heuristic authorship obfuscation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1098–1108.
  8. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745.
  9. A critical review on the use (and misuse) of differential privacy in machine learning. ACM Computing Surveys, 55(8):1–16.
  10. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  11. The secret sharer: Evaluating and testing unintended memorization in neural networks. In USENIX Security Symposium, volume 267.
  12. Extracting training data from large language models. In USENIX Security Symposium, volume 6.
  13. Tem: high utility metric differential privacy on text. arXiv preprint arXiv:2107.07928.
  14. Broadening the scope of differential privacy using metrics. In Privacy Enhancing Technologies: 13th International Symposium, PETS 2013, Bloomington, IN, USA, July 10-12, 2013. Proceedings 13, pages 82–102. Springer.
  15. A customized text sanitization mechanism with differential privacy. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5747–5758.
  16. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  17. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  18. Tag: Gradient attack on transformer-based language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3600–3610.
  19. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  20. Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pages 429–438. IEEE.
  21. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, pages 265–284. Springer.
  22. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407.
  23. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, pages 1054–1067.
  24. Privacy-and utility-preserving textual analysis via calibrated multivariate perturbations. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 178–186.
  25. Ran Gilad-Bachrach and Chris J.C. Burges. 2012. The median hypothesis. Technical Report MSR-TR-2012-56.
  26. User review sites as a resource for large-scale sociolinguistic studies. In Proceedings of the 24th international conference on World Wide Web, pages 452–461.
  27. Timour Igamberdiev and Ivan Habernal. 2023. DP-BART for privatized text rewriting under local differential privacy. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13914–13934, Toronto, Canada. Association for Computational Linguistics.
  28. Bargav Jayaraman and David Evans. 2019. Evaluating differentially private machine learning in practice. In USENIX Security Symposium.
  29. Mistral 7b. arXiv preprint arXiv:2310.06825.
  30. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025.
  31. I know what you did last summer: query logs and user privacy. In Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, pages 909–914.
  32. What can we learn privately? SIAM Journal on Computing, 40(3):793–826.
  33. Differentially private language models benefit from public pre-training. In Proceedings of the Second Workshop on Privacy in NLP, pages 39–45.
  34. N-gram-based author profiles for authorship attribution. In Proceedings of the conference pacific association for computational linguistics, PACLING, volume 3, pages 255–264.
  35. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems.
  36. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880.
  37. Tao Li and Chris Clifton. 2021. Differentially private imaging via latent space manipulation. arXiv preprint arXiv:2103.05472.
  38. Large language models can be strong differentially private learners. In International Conference on Learning Representations.
  39. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463.
  40. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  41. Differentially private language models for secure data sharing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4860–4873.
  42. The limits of word level differential privacy. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 867–881.
  43. Frank McSherry and Kunal Talwar. 2007. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pages 94–103. IEEE.
  44. Sentence-level privacy for document embeddings. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3367–3380.
  45. Joe Near. 2018. Differential privacy at scale: Uber and Berkeley collaboration. In Enigma 2018 (Enigma 2018).
  46. OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  47. A picture of search. In Proceedings of the 1st international conference on Scalable information systems, pages 1–es.
  48. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  49. On the challenges of using black-box apis for toxicity evaluation in research. arXiv preprint arXiv:2304.12397.
  50. An analysis of the user occupational class through twitter content. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1754–1764.
  51. Misleading authorship attribution of source code using adversarial learning. In 28th USENIX Security Symposium (USENIX Security 19), pages 479–496.
  52. Language models are unsupervised multitask learners.
  53. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
  54. Can pseudonymity really guarantee privacy? In USENIX Security Symposium, pages 85–96.
  55. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.
  56. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100.
  57. {{\{{A4NT}}\}}: Author attribute anonymity by adversarial training of neural machine translation. In 27th USENIX Security Symposium (USENIX Security 18), pages 1633–1650.
  58. Convolutional neural networks for authorship attribution of short texts. In Proceedings of the 15th conference of the European chapter of the association for computational linguistics: Volume 2, short papers, pages 669–674.
  59. Mpnet: Masked and permuted pre-training for language understanding. Advances in Neural Information Processing Systems, 33:16857–16867.
  60. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  61. John W Tukey. 1975. Mathematics and the picturing of data. In Proceedings of the International Congress of Mathematicians, Vancouver, 1975, volume 2, pages 523–531.
  62. Upton: Unattributable authorship text via data poisoning. arXiv preprint arXiv:2211.09717.
  63. Finetuned language models are zero-shot learners. In International Conference on Learning Representations.
  64. Sam Witteveen and Martin Andrews. 2019. Paraphrasing with large language models. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 215–220.
  65. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  66. A comprehensive survey on local differential privacy. Security and Communication Networks, 2020:1–29.
  67. A differentially private text perturbation method using regularized mahalanobis metric. In Proceedings of the Second Workshop on Privacy in NLP, pages 7–17.
  68. Differentially private fine-tuning of language models. In International Conference on Learning Representations.
  69. Paws: Paraphrase adversaries from word scrambling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298–1308.
Citations (23)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube