Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DP-TabICL: In-Context Learning with Differentially Private Tabular Data (2403.05681v1)

Published 8 Mar 2024 in cs.CR, cs.AI, and cs.LG

Abstract: In-context learning (ICL) enables LLMs to adapt to new tasks by conditioning on demonstrations of question-answer pairs and it has been shown to have comparable performance to costly model retraining and fine-tuning. Recently, ICL has been extended to allow tabular data to be used as demonstration examples by serializing individual records into natural language formats. However, it has been shown that LLMs can leak information contained in prompts, and since tabular data often contain sensitive information, understanding how to protect the underlying tabular data used in ICL is a critical area of research. This work serves as an initial investigation into how to use differential privacy (DP) -- the long-established gold standard for data privacy and anonymization -- to protect tabular data used in ICL. Specifically, we investigate the application of DP mechanisms for private tabular ICL via data privatization prior to serialization and prompting. We formulate two private ICL frameworks with provable privacy guarantees in both the local (LDP-TabICL) and global (GDP-TabICL) DP scenarios via injecting noise into individual records or group statistics, respectively. We evaluate our DP-based frameworks on eight real-world tabular datasets and across multiple ICL and DP settings. Our evaluations show that DP-based ICL can protect the privacy of the underlying tabular data while achieving comparable performance to non-LLM baselines, especially under high privacy regimes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  2. H. Duan, A. Dziedzic, M. Yaghini, N. Papernot, and F. Boenisch, “On the privacy risk of in-context learning,” in The 61st Annual Meeting Of The Association For Computational Linguistics, 2023.
  3. Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, “A survey for in-context learning,” arXiv preprint arXiv:2301.00234, 2022.
  4. H. Zhou, A. Nova, H. Larochelle, A. Courville, B. Neyshabur, and H. Sedghi, “Teaching algorithmic reasoning via in-context learning,” 2022.
  5. V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci, “Deep neural networks and tabular data: A survey,” arXiv preprint arXiv:2110.01889, 2021.
  6. S. Hegselmann, A. Buendia, H. Lang, M. Agrawal, X. Jiang, and D. Sontag, “Tabllm: Few-shot classification of tabular data with large language models,” in International Conference on Artificial Intelligence and Statistics, pp. 5549–5581, PMLR, 2023.
  7. H. Duan, A. Dziedzic, N. Papernot, and F. Boenisch, “Flocks of stochastic parrots: Differentially private prompt learning for large language models,” arXiv preprint arXiv:2305.15594, 2023.
  8. X. Tang, R. Shin, H. A. Inan, A. Manoel, F. Mireshghallah, Z. Lin, S. Gopi, J. Kulkarni, and R. Sim, “Privacy-preserving in-context learning with differentially private few-shot generation,” arXiv preprint arXiv:2309.11765, 2023.
  9. C. Dwork, A. Roth, et al., “The algorithmic foundations of differential privacy,” Foundations and Trends® in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014.
  10. D. Yu, S. Naik, A. Backurs, S. Gopi, H. A. Inan, G. Kamath, J. Kulkarni, Y. T. Lee, A. Manoel, L. Wutschitz, S. Yekhanin, and H. Zhang, “Differentially private fine-tuning of language models,” in The Tenth International Conference on Learning Representations, ICLR, OpenReview.net, 2022.
  11. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318, 2016.
  12. J. Hong, J. T. Wang, C. Zhang, Z. Li, B. Li, and Z. Wang, “Dp-opt: Make large language model your privacy-preserving prompt engineer,” arXiv preprint arXiv:2312.03724, 2023.
  13. S. L. Warner, “Randomized response: A survey technique for eliminating evasive answer bias,” JASA, vol. 60, no. 309, pp. 63–69, 1965.
  14. T. Wang, M. Lopuhaä-Zwakenberg, Z. Li, B. Skoric, and N. Li, “Locally differentially private frequency estimation with consistency,” in 27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, February 23-26, 2020, The Internet Society, 2020.
  15. Y. Wang, X. Wu, and D. Hu, “Using randomized response for differential privacy preserving data collection.,” in EDBT/ICDT, vol. 1558, pp. 0090–6778, 2016.
  16. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023.
  17. P. Yin, G. Neubig, W.-t. Yih, and S. Riedel, “Tabert: Pretraining for joint understanding of textual and tabular data,” arXiv preprint arXiv:2005.08314, 2020.
  18. Y. Li, J. Li, Y. Suhara, A. Doan, and W.-C. Tan, “Deep entity matching with pre-trained language models,” arXiv preprint arXiv:2004.00584, 2020.
  19. A. Harari and G. Katz, “Few-shot tabular data enrichment using fine-tuned transformer architectures,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1577–1591, 2022.
  20. D. Bertsimas, K. V. Carballo, Y. Ma, L. Na, L. Boussioux, C. Zeng, L. R. Soenksen, and I. Fuentes, “Tabtext: a systematic approach to aggregate knowledge across tabular data structures,” arXiv preprint arXiv:2206.10381, 2022.
  21. S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer, “Rethinking the role of demonstrations: What makes in-context learning work?,” arXiv preprint arXiv:2202.12837, 2022.
  22. V. Borisov, K. Seßler, T. Leemann, M. Pawelczyk, and G. Kasneci, “Language models are realistic tabular data generators,” arXiv preprint arXiv:2210.06280, 2022.
  23. T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J.-y. Sohn, D. Papailiopoulos, and K. Lee, “Lift: Language-interfaced fine-tuning for non-language machine learning tasks,” Advances in Neural Information Processing Systems, vol. 35, pp. 11763–11784, 2022.
  24. H. Zhang, X. Wen, S. Zheng, W. Xu, and J. Bian, “Towards foundation models for learning on tabular data,” arXiv preprint arXiv:2310.07338, 2023.
  25. D. Slack and S. Singh, “Tablet: Learning from instructions for tabular data,” arXiv preprint arXiv:2304.13188, 2023.
  26. N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang, “Quantifying memorization across neural language models,” arXiv preprint arXiv:2202.07646, 2022.
  27. D. Ippolito, F. Tramèr, M. Nasr, C. Zhang, M. Jagielski, K. Lee, C. A. Choquette-Choo, and N. Carlini, “Preventing verbatim memorization in language models gives a false sense of privacy,” arXiv preprint arXiv:2210.17546, 2022.
  28. E. Kharitonov, M. Baroni, and D. Hupkes, “How bpe affects memorization in transformers,” arXiv preprint arXiv:2110.02782, 2021.
  29. R. T. McCoy, P. Smolensky, T. Linzen, J. Gao, and A. Celikyilmaz, “How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven,” Transactions of the Association for Computational Linguistics, vol. 11, pp. 652–670, 2023.
  30. B. Wang, W. Chen, H. Pei, C. Xie, M. Kang, C. Zhang, C. Xu, Z. Xiong, R. Dutta, R. Schaeffer, S. T. Truong, S. Arora, M. Mazeika, D. Hendrycks, Z. Lin, Y. Cheng, S. Koyejo, D. Song, and B. Li, “Decodingtrust: A comprehensive assessment of trustworthiness in gpt models,” arXiv:2306.11698, 2024.
  31. F. Mireshghallah, A. Uniyal, T. Wang, D. Evans, and T. Berg-Kirkpatrick, “Memorization in nlp fine-tuning methods,” arXiv preprint arXiv:2205.12506, 2022.
  32. A. Priyanshu, S. Vijay, A. Kumar, R. Naidu, and F. Mireshghallah, “Are chatbots ready for privacy-sensitive applications? an investigation into input regurgitation and prompt-induced sanitization,” arXiv:2305.15008, 2023.
  33. R. Anil, B. Ghazi, V. Gupta, R. Kumar, and P. Manurangsi, “Large-scale differentially private bert,” arXiv preprint arXiv:2108.01624, 2021.
  34. S. Hoory, A. Feder, A. Tendler, S. Erell, A. Peled-Cohen, I. Laish, H. Nakhost, U. Stemmer, A. Benjamini, A. Hassidim, et al., “Learning and evaluating a differentially private pre-trained language model,” in Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 1178–1189, 2021.
  35. X. Li, F. Tramer, P. Liang, and T. Hashimoto, “Large language models can be strong differentially private learners,” arXiv preprint arXiv:2110.05679, 2021.
  36. Z. Bu, Y.-X. Wang, S. Zha, and G. Karypis, “Differentially private bias-term only fine-tuning of foundation models,” arXiv preprint arXiv:2210.00036, 2022.
  37. T. Wu, A. Panda, J. T. Wang, and P. Mittal, “Privacy-preserving in-context learning for large language models,” arXiv e-prints, pp. arXiv–2305, 2023.
  38. N. Papernot, S. Song, I. Mironov, A. Raghunathan, K. Talwar, and Ú. Erlingsson, “Scalable private learning with pate,” arXiv preprint arXiv:1802.08908, 2018.
  39. A. Sordoni, X. Yuan, M.-A. Côté, M. Pereira, A. Trischler, Z. Xiao, A. Hosseini, F. Niedtner, and N. L. Roux, “Deep language networks: Joint prompt training of stacked llms using variational inference,” arXiv preprint arXiv:2306.12509, 2023.
  40. B. Balle, G. Barthe, and M. Gaboardi, “Privacy amplification by subsampling: Tight analyses via couplings and divergences,” Advances in neural information processing systems, vol. 31, 2018.
  41. N. Li, W. Qardaji, and D. Su, “On sampling, anonymization, and differential privacy or, k-anonymization meets differential privacy,” in Proceedings of the 7th ACM Symposium on Information, Computer and Communications Security, pp. 32–33, 2012.
  42. N. Holohan, S. Braghin, P. Mac Aonghusa, and K. Levacher, “Diffprivlib: the IBM differential privacy library,” ArXiv e-prints, vol. 1907.02444 [cs.CR], July 2019.
  43. A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres, “Quantifying the carbon emissions of machine learning,” arXiv preprint arXiv:1910.09700, 2019.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now