Papers
Topics
Authors
Recent
2000 character limit reached

Securing Reliability: A Brief Overview on Enhancing In-Context Learning for Foundation Models

Published 27 Feb 2024 in cs.LG | (2402.17671v1)

Abstract: As foundation models (FMs) continue to shape the landscape of AI, the in-context learning (ICL) paradigm thrives but also encounters issues such as toxicity, hallucination, disparity, adversarial vulnerability, and inconsistency. Ensuring the reliability and responsibility of FMs is crucial for the sustainable development of the AI ecosystem. In this concise overview, we investigate recent advancements in enhancing the reliability and trustworthiness of FMs within ICL frameworks, focusing on four key methodologies, each with its corresponding subgoals. We sincerely hope this paper can provide valuable insights for researchers and practitioners endeavoring to build safe and dependable FMs and foster a stable and consistent ICL environment, thereby unlocking their vast potential.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (110)
  1. Foundation models can robustify themselves, for free. In R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023. URL https://openreview.net/forum?id=XoacWibt7b.
  2. Mistral AI. Mixtral of Experts: A High-Quality Sparse Mixture-of-Experts. https://mistral.ai/news/mixtral-of-experts/, 2023.
  3. Instructeval: Systematic evaluation of instruction selection methods, 2023.
  4. Alex Albert. Jailbreak chat: the largest collection of chatgpt jailbreaks on the internet. https://www.jailbreakchat.com/, 2023.
  5. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132, 2023.
  6. A gentle introduction to conformal prediction and distribution-free uncertainty quantification, 2022.
  7. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023.
  8. Anonymous. Don’t trust: Verify – grounding LLM quantitative reasoning with autoformalization. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=V5tdi14ple.
  9. Anthropic. Model card and evaluations for claude models. https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf, July 2023.
  10. Promptsource: An integrated development environment and repository for natural language prompts, 2022.
  11. Detecting concept drift with neural network model uncertainty. arXiv preprint arXiv:2107.01873, 2021.
  12. Graph of thoughts: Solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687, 2023.
  13. Re-contextualizing fairness in nlp: The case of india, 2022.
  14. On the opportunities and risks of foundation models, 2022.
  15. Icl markup: Structuring in-context learning using soft-token tags, 2023.
  16. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023.
  17. Are aligned neural networks adversarially aligned? arXiv preprint arXiv:2306.15447, 2023.
  18. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055, 2017.
  19. Jailbreaking black box large language models in twenty queries, 2023.
  20. Envisioning legal mitigations for llm-based intentional and unintentional harms. Administrative Law Journal, 2022.
  21. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https://lmsys.org/blog/2023-03-30-vicuna/.
  22. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  23. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In proceedings of the Conference on Fairness, Accountability, and Transparency, pp.  120–128, 2019.
  24. Detecting concept drift in data streams using model explanation. Expert Systems with Applications, 92:546–559, 2018.
  25. Rlprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205.12548, 2022.
  26. Building socio-culturally inclusive stereotype resources with community engagement, 2023.
  27. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  28. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp.  67–73, 2018.
  29. A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022.
  30. How robust is google’s bard to adversarial image attacks?, 2023.
  31. Estimating uncertainty in multimodal foundation models using public internet data, 2023.
  32. Mitigating label biases for in-context learning. arXiv preprint arXiv:2305.19148, 2023.
  33. Evaluating superhuman models with consistency checks, 2023.
  34. Flexible visual prompts for in-context learning in computer vision, 2023.
  35. Flexible text generation for counterfactual fairness probing, 2022.
  36. Counterfactual fairness in text classification through robustness. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp.  219–226, 2019.
  37. Google Gemini Team. Gemini: A family of highly capable multimodal models, Dec 2023. URL https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf.
  38. Pathfinder: Guided search over multi-step reasoning paths, 2023.
  39. Demystifying prompts in language models via perplexity estimation, 2022.
  40. Deep learning, volume 1. MIT Press, 2016.
  41. Google. Bard. https://bard.google.com/, 2023.
  42. Advancing transformer architecture in long-context large language models: A comprehensive survey, 2023.
  43. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023.
  44. Seegull: A stereotype benchmark with broad geo-cultural coverage leveraging generative models, 2023.
  45. Mistral 7B, 2023.
  46. A universal prompt generator for large language models. In R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023. URL https://openreview.net/forum?id=HAqPAqztEU.
  47. Challenges and applications of large language models. arXiv preprint arXiv:2307.10169, 2023.
  48. Born-again decision boundary: Unsupervised concept drift detection by inspector neural network. In 2021 International Joint Conference on Neural Networks (IJCNN), pp.  1–8. IEEE, 2021.
  49. Albert: A lite bert for self-supervised learning of language representations, 2020.
  50. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021.
  51. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.
  52. Jailbreaking chatgpt via prompt engineering: An empirical study, 2023.
  53. Dr.icl: Demonstration-retrieved in-context learning, 2023.
  54. Automix: Mixing models with few-shot self and meta verification. In R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023. URL https://openreview.net/forum?id=FJo2lroF7R.
  55. It’s all in the name: Mitigating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871, 2019.
  56. The lean 4 theorem prover and programming language. In Automated Deduction–CADE 28: 28th International Conference on Automated Deduction, Virtual Event, July 12–15, 2021, Proceedings 28, pp.  625–635. Springer, 2021.
  57. Isabelle/HOL: a proof assistant for higher-order logic. Springer, 2002.
  58. Risk assessment and statistical significance in the age of foundation models, 2024.
  59. Adversarial robustness of prompt-based few-shot learning for natural language understanding, 2023.
  60. OpenAI. Openai: Introducing chatgpt, 2022. URL https://openai.com/blog/chatgpt.
  61. OpenAI. Planning for agi and beyond. https://openai.com/blog/planning-for-agi-and-beyond, February 2023a.
  62. OpenAI. Dall-e 3. https://openai.com/dall-e-3, 2023b.
  63. OpenAI. Gpt-4 technical report, 2023.
  64. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
  65. Prompt Library. Prompt library. https://promptlibrary.org/, 2024.
  66. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  67. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.  8748–8763. PMLR, 2021.
  68. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023.
  69. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020.
  70. Cross-modal attribute insertions for assessing the robustness of vision-and-language learning, 2023.
  71. Tricking llms into disobedience: Understanding, analyzing, and preventing jailbreaks, 2023.
  72. Weakly supervised detection of hallucinations in llm activations, 2023.
  73. Smoothllm: Defending large language models against jailbreaking attacks, 2023.
  74. Scalable and transferable black-box jailbreaks for language models via persona modulation, 2023.
  75. Sincode AI. Prompt library. https://www.sincode.ai/prompt-library, 2024.
  76. Break it, imitate it, fix it: Robustness by generating human-like attacks, 2023.
  77. Med-mmhl: A multi-modal dataset for detecting human-and llm-generated misinformation in the medical domain. arXiv preprint arXiv:2306.08871, 2023.
  78. Quantifying uncertainty in natural language explanations of large language models, 2023.
  79. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022.
  80. THUDM. ChatGLM3. https://github.com/THUDM/ChatGLM3, 2023.
  81. Interpretable stereotype identification through reasoning, 2023a.
  82. Efficient evaluation of bias in large language models through prompt tuning. In Socially Responsible Language Modelling Research, 2023b. URL https://openreview.net/forum?id=v1WL01lgp8.
  83. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  84. Tensor trust: Interpretable prompt injection attacks from an online game, 2023.
  85. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  86. Overcoming language disparity in online content classification with multimodal learning. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pp.  1040–1051, 2022.
  87. Are large language models really robust to word-level perturbations?, 2023.
  88. Measuring and reducing gendered correlations in pre-trained models, 2021.
  89. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483, 2023a.
  90. Chain-of-thought prompting elicits reasoning in large language models, 2023b.
  91. An empirical study on challenging math problem solving with gpt-4. arXiv preprint arXiv:2306.01337, 2023.
  92. Chia Jeng Yang. Mega prompts: Turning expertise into code. https://medium.com/messy-problems-original-concepts/, 2023.
  93. LeanDojo: Theorem proving with retrieval-augmented language models. In Neural Information Processing Systems (NeurIPS), 2023.
  94. Tree of thoughts: Deliberate problem solving with large language models, may 2023. arXiv preprint arXiv:2305.10601, 14, 2023.
  95. Finpt: Financial risk prediction with profile tuning on pretrained foundation models. arXiv preprint arXiv:2308.00065, 2023.
  96. Automatic hallucination assessment for aligned large language models via transferable adversarial attacks, 2023.
  97. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414, 2022.
  98. Promptgen: Automatically generate prompts using generative models. In Findings of the Association for Computational Linguistics: NAACL 2022, pp.  30–37, 2022.
  99. Defending large language models against jailbreaking attacks through goal prioritization. arXiv preprint arXiv:2311.09096, 2023.
  100. Gender bias in coreference resolution: Evaluation and debiasing methods. arXiv preprint arXiv:1804.06876, 2018.
  101. Group preference optimization: Few-shot alignment of large language models, 2023.
  102. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pp.  12697–12706. PMLR, 2021.
  103. Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models. arXiv preprint arXiv:2310.16436, 2023a.
  104. Gpt-fathom: Benchmarking large language models to decipher the evolutionary path towards gpt-4 and beyond, 2023b.
  105. Robust prompt optimization for defending language models against jailbreaking attacks. arXiv preprint arXiv:2401.17263, 2024a.
  106. Batch calibration: Rethinking calibration for in-context learning and prompt engineering, 2024b.
  107. Large language models are human-level prompt engineers, 2023.
  108. Autodan: Interpretable gradient-based adversarial attacks on large language models, 2023.
  109. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. arXiv preprint arXiv:1906.04571, 2019.
  110. Universal and transferable adversarial attacks on aligned language models, 2023.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up