Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer (2312.03724v2)

Published 27 Nov 2023 in cs.CL and cs.AI

Abstract: LLMs have emerged as dominant tools for various tasks, particularly when tailored for a specific target by prompt tuning. Nevertheless, concerns surrounding data privacy present obstacles due to the tuned prompts' dependency on sensitive private information. A practical solution is to host a local LLM and optimize a soft prompt privately using data. Yet, hosting a local model becomes problematic when model ownership is protected. Alternative methods, like sending data to the model's provider for training, intensify these privacy issues facing an untrusted provider. In this paper, we present a novel solution called Differentially-Private Offsite Prompt Tuning (DP-OPT) to address this challenge. Our approach involves tuning a discrete prompt on the client side and then applying it to the desired cloud models. We demonstrate that prompts suggested by LLMs themselves can be transferred without compromising performance significantly. To ensure that the prompts do not leak private information, we introduce the first private prompt generation mechanism, by a differentially-private (DP) ensemble of in-context learning with private demonstrations. With DP-OPT, generating privacy-preserving prompts by Vicuna-7b can yield competitive performance compared to non-private in-context learning on GPT3.5 or local private prompt tuning. Codes are available at https://github.com/VITA-Group/DP-OPT .

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Deep Learning with Differential Privacy. In CCS: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, pp.  308–318, New York, NY, USA, 2016. ACM. ISBN 978-1-4503-4139-4. doi: 10.1145/2976749.2978318.
  2. Learning to few-shot learn across diverse natural language classification tasks. arXiv preprint arXiv:1911.03863, 2019.
  3. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  4. Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds. In Martin Hirt and Adam Smith (eds.), Theory of Cryptography, volume 9985, pp.  635–658. Springer Berlin Heidelberg, Berlin, Heidelberg, 2016. ISBN 978-3-662-53640-7 978-3-662-53641-4. doi: 10.1007/978-3-662-53641-4_24.
  5. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633–2650, 2021.
  6. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1897–1914. IEEE, 2022a.
  7. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646, 2022b.
  8. Tem: High utility metric differential privacy on text. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pp.  883–890. SIAM, 2023.
  9. Adaprompt: Adaptive model training for prompt-based nlp. arXiv preprint arXiv:2202.04824, 2022.
  10. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https://lmsys.org/blog/2023-03-30-vicuna/.
  11. Unlocking high-accuracy differentially private image classification through scale. arXiv preprint arXiv:2204.13650, 2022.
  12. Rlprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205.12548, 2022.
  13. Sanitizing sentence embeddings (and labels) for local differential privacy. In Proceedings of the ACM Web Conference 2023, pp. 2349–2359, 2023.
  14. Flocks of stochastic parrots: Differentially private prompt learning for large language models. Conference on Neural Information Processing Systems, 2023a.
  15. On the privacy risk of in-context learning. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023b.
  16. Practical differentially private top-k selection with pay-what-you-get composition. Advances in Neural Information Processing Systems, 32, 2019.
  17. Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pp.  1–12. Springer, 2006.
  18. Calibrating Noise to Sensitivity in Private Data Analysis. In Shai Halevi and Tal Rabin (eds.), Theory of Cryptography, Lecture Notes in Computer Science, pp.  265–284. Springer Berlin Heidelberg, 2006. ISBN 978-3-540-32732-5.
  19. Boosting and differential privacy. In 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pp.  51–60. IEEE, 2010.
  20. Privacy-and utility-preserving textual analysis via calibrated multivariate perturbations. In Proceedings of the 13th international conference on web search and data mining, pp.  178–186, 2020.
  21. GDPR. Gdpr, 2016. URL https://gdpr-info.eu/.
  22. Ensembles and cocktails: Robust finetuning for natural language generation. 2021.
  23. Promptboosting: Black-box text classification with ten forward passes. In International Conference on Machine Learning, pp. 13309–13324. PMLR, 2023.
  24. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pp. 2790–2799. PMLR, 2019.
  25. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2019.
  26. A deep ensemble model with slot alignment for sequence-to-sequence natural language generation. arXiv preprint arXiv:1805.06553, 2018.
  27. What can we learn privately? SIAM Journal on Computing, 40(3):793–826, 2011.
  28. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.
  29. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.
  30. What makes good in-context examples for gpt-3333? arXiv preprint arXiv:2101.06804, 2021.
  31. Kevin Liu. The entire prompt of microsoft bing chat?! (hi, sydney.), 2023. URL https://twitter.com/kliu128/status/1623472922374574080.
  32. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786, 2021.
  33. Analyzing leakage of personally identifiable information in language models. arXiv preprint arXiv:2302.00539, 2023.
  34. The limits of word level differential privacy. arXiv preprint arXiv:2205.02130, 2022.
  35. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pp.  94–103. IEEE, 2007.
  36. Quantifying privacy risks of masked language models using membership inference attacks. arXiv preprint arXiv:2203.03929, 2022.
  37. Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF), pp.  263–275. IEEE, 2017.
  38. Smooth sensitivity and sampling in private data analysis. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing, pp.  75–84, 2007.
  39. R OpenAI. Gpt-4 technical report. arXiv, pp.  2303–08774, 2023.
  40. Prompting ai art: An investigation into the creative skill of prompt engineering. arXiv preprint arXiv:2303.13534, 2023.
  41. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  42. Differentially private in-context learning. arXiv preprint arXiv:2305.01639, 2023.
  43. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019.
  44. Boosted prompt ensembles for large language models. arXiv preprint arXiv:2304.05970, 2023.
  45. Grips: Gradient-free, edit-based instruction search for prompting large language models. arXiv preprint arXiv:2203.07281, 2022.
  46. Improving language understanding by generative pre-training. 2018.
  47. Exploring the limits of transfer learning with a unified text-to-text transformer. 2019.
  48. Tan without a burn: Scaling laws of dp-sgd. In International Conference on Machine Learning, pp. 29937–29949. PMLR, 2023.
  49. Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too? arXiv preprint arXiv:2212.10539, 2022.
  50. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020.
  51. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 3–18. IEEE, 2017.
  52. Deep language networks: Joint prompt training of stacked llms using variational inference. arXiv preprint arXiv:2306.12509, 2023.
  53. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  54. Legal prompt engineering for multilingual legal judgement prediction. arXiv preprint arXiv:2212.02199, 2022.
  55. Locally differentially private document generation using zero shot prompting. arXiv preprint arXiv:2310.16111, 2023.
  56. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018.
  57. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. arXiv preprint arXiv:2306.11698, 2023a.
  58. Prompt engineering for healthcare: Methodologies and applications. arXiv preprint arXiv:2304.14670, 2023b.
  59. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Conference on Neural Information Processing Systems, 2023.
  60. dp-transformers: Training transformer models with differential privacy. https://www.microsoft.com/en-us/research/project/dp-transformers, August 2022.
  61. Offsite-tuning: Transfer learning without full model. arXiv preprint arXiv:2302.04870, 2023.
  62. A differentially private text perturbation method using a regularized mahalanobis metric. arXiv preprint arXiv:2010.11947, 2020.
  63. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.
  64. Large language models are human-level prompt engineers. International Conference on Learning Representations, 2022.
  65. Adaptive private-k-selection with adaptive k and application to multi-label pate. In International Conference on Artificial Intelligence and Statistics, pp.  5622–5635. PMLR, 2022.
Citations (20)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - VITA-Group/DP-OPT: Differentially Private Offsite Prompt Tuning (43 stars)