Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners (2310.02469v3)

Published 3 Oct 2023 in cs.CL, cs.AI, and cs.LG

Abstract: The proliferation of LLMs has driven considerable interest in fine-tuning them with domain-specific data to create specialized LLMs. Nevertheless, such domain-specific fine-tuning data often contains contextually sensitive personally identifiable information (PII). Direct fine-tuning of LLMs on this data without privacy protection poses a risk of data leakage of sensitive PII during inference time. To address this challenge, we introduce Contextual Privacy Protection LLMs (PrivacyMind), a novel paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding inference-time data privacy. Our work offers a theoretical analysis for model design and benchmarks various techniques such as corpus curation, penalty-based unlikelihood in training loss, instruction-based tuning, etc. Extensive experiments across diverse datasets and scenarios demonstrate the effectiveness of our approaches. In particular, instruction tuning with both positive and negative examples stands out as a promising method, effectively protecting private data while enhancing the model's knowledge. Our work underscores the potential for LLMs as robust contextual privacy protection learners. The complete code and data for the work can be found at https://github.com/Yijia-Xiao/PrivacyMind.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Large-scale differentially private BERT. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.  6481–6491, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-emnlp.484. URL https://aclanthology.org/2022.findings-emnlp.484.
  2. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, abs/2204.05862, 2022. URL https://api.semanticscholar.org/CorpusID:248118878.
  3. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity, 2023. URL http://arxiv.org/abs/2302.04023.
  4. Extracting training data from large language models. In USENIX Security Symposium, 2020. URL https://api.semanticscholar.org/CorpusID:229156229.
  5. Quantifying memorization across neural language models. ArXiv, abs/2202.07646, 2022. URL https://api.semanticscholar.org/CorpusID:246863735.
  6. Chatgpt evaluation on sentence level relations: A focus on temporal, causal, and discourse relations. arXiv:2304.14827, 2023.
  7. Semi-offline reinforcement learning for optimized text generation. In ICML, 2023.
  8. Privacy-preserving neural representations of text, 2018.
  9. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  320–335, 2022.
  10. Helixfold-single: Msa-free protein structure prediction by using protein language model as an alternative. ArXiv, abs/2207.13921, 2022. URL https://api.semanticscholar.org/CorpusID:251135365.
  11. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. In Findings, 2020. URL https://api.semanticscholar.org/CorpusID:221878771.
  12. Google. Bard, 2023. URL https://bard.google.com/.
  13. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247, 2023.
  14. Using self-supervised learning can improve model robustness and uncertainty. ArXiv, abs/1906.12340, 2019. URL https://api.semanticscholar.org/CorpusID:195750576.
  15. Training compute-optimal large language models. ArXiv, abs/2203.15556, 2022a. URL https://api.semanticscholar.org/CorpusID:247778764.
  16. An empirical analysis of compute-optimal large language model training. In Neural Information Processing Systems, 2022b. URL https://api.semanticscholar.org/CorpusID:258509679.
  17. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
  18. Prototypical fine-tuning: Towards robust performance under varying data sizes. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  19. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103:102274, 2023.
  20. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL, pp.  4171–4186, 2019.
  21. Pretraining language models with human preferences. ArXiv, abs/2302.08582, 2023. URL https://api.semanticscholar.org/CorpusID:257020046.
  22. Large language models can be strong differentially private learners, 2022.
  23. Privacy-preserving prompt tuning for large language model services, 2023.
  24. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp.  74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics.
  25. Truthfulqa: Measuring how models mimic human falsehoods. In Annual Meeting of the Association for Computational Linguistics, 2021. URL https://api.semanticscholar.org/CorpusID:237532606.
  26. Privaterec: Differentially private training and serving for federated news recommendation. 2023.
  27. Tuning language models as training data generators for augmentation-enhanced few-shot learning. In International Conference on Machine Learning, 2023.
  28. Teaching language models to support answers with verified quotes. ArXiv, abs/2203.11147, 2022. URL https://api.semanticscholar.org/CorpusID:247594830.
  29. OpenAI. Gpt-4 technical report. Arxiv Preprint, arXiv:2303.08774, 2023. URL https://arxiv.org/abs/2303.08774.
  30. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155, 2022. URL https://api.semanticscholar.org/CorpusID:246426909.
  31. Pytorch: An imperative style, high-performance deep learning library, 2019.
  32. Rwkv: Reinventing rnns for the transformer era, 2023.
  33. Direct preference optimization: Your language model is secretly a reward model, 2023.
  34. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations, 2022. URL https://api.semanticscholar.org/CorpusID:251648120.
  35. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences of the United States of America, 118, 2019. URL https://api.semanticscholar.org/CorpusID:155162335.
  36. Selective differential privacy for language modeling. arXiv preprint arXiv:2108.12944, 2021.
  37. Large language models encode clinical knowledge. Nature, 620(7972):172–180, August 2023a. ISSN 1476-4687. doi: 10.1038/s41586-023-06291-2. URL https://doi.org/10.1038/s41586-023-06291-2.
  38. Towards expert-level medical question answering with large language models. arXiv:2305.09617, 2023b.
  39. Process for adapting language models to society (palms) with values-targeted datasets. ArXiv, abs/2106.10328, 2021. URL https://api.semanticscholar.org/CorpusID:235489789.
  40. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023.
  41. Llama: Open and efficient foundation language models, 2023.
  42. Will we run out of data? an analysis of the limits of scaling datasets in machine learning. ArXiv, abs/2211.04325, 2022. URL https://api.semanticscholar.org/CorpusID:253397775.
  43. Overcoming catastrophic forgetting in zero-shot cross-lingual generation. In Conference on Empirical Methods in Natural Language Processing, 2022. URL https://api.semanticscholar.org/CorpusID:249062610.
  44. Poisoning language models during instruction tuning. In International Conference on Machine Learning, 2023.
  45. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, 2023a.
  46. Self-instruct: Aligning language model with self generated instructions, 2022.
  47. Aligning large language models with human: A survey. arXiv preprint arXiv:2307.12966, 2023b.
  48. Challenges in detoxifying language models. ArXiv, abs/2109.07445, 2021. URL https://api.semanticscholar.org/CorpusID:237513578.
  49. Bloomberggpt: A large language model for finance, 2023.
  50. Recipes for safety in open-domain chatbots. ArXiv, abs/2010.07079, 2020. URL https://api.semanticscholar.org/CorpusID:222341902.
  51. Fingpt: Open-source financial large language models. ArXiv, abs/2306.06031, 2023. URL https://api.semanticscholar.org/CorpusID:259129734.
  52. Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records. ArXiv, abs/2203.03540, 2022. URL https://api.semanticscholar.org/CorpusID:247157824.
  53. Differentially private fine-tuning of language models. In International Conference on Learning Representations (ICLR), 2022.
  54. G22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPTuardfl: Safeguarding federated learning against backdoor attacks through attributed client graph clustering, 2023a.
  55. Bag of tricks for training data extraction from language models. In ICML, 2023b.
  56. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020.
  57. Enhancing small medical learners with privacy-preserving contextual prompting, 2023.
  58. Fldp: Flexible strategy for local differential privacy. In ICASSP, pp.  2974–2978. IEEE, 2022.
  59. Controlled text generation with natural language instructions. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pp.  42602–42613. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/zhou23g.html.
  60. Fine-tuning language models from human preferences, 2020.
  61. Adversarial training for high-stakes reliability. ArXiv, abs/2205.01663, 2022. URL https://api.semanticscholar.org/CorpusID:248506146.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (13)
  1. Yijia Xiao (19 papers)
  2. Yiqiao Jin (27 papers)
  3. Yushi Bai (31 papers)
  4. Yue Wu (338 papers)
  5. Xianjun Yang (37 papers)
  6. Xiao Luo (111 papers)
  7. Wenchao Yu (23 papers)
  8. Xujiang Zhao (26 papers)
  9. Yanchi Liu (41 papers)
  10. Haifeng Chen (99 papers)
  11. Wei Wang (1793 papers)
  12. Wei Cheng (175 papers)
  13. Quanquan Gu (198 papers)
Citations (17)
View on Semantic Scholar