Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Knowledge Editing for Large Language Models: A Survey (2310.16218v4)

Published 24 Oct 2023 in cs.CL and cs.AI
Knowledge Editing for Large Language Models: A Survey

Abstract: LLMs have recently transformed both the academic and industrial landscapes due to their remarkable capacity to understand, analyze, and generate texts based on their vast knowledge and reasoning ability. Nevertheless, one major drawback of LLMs is their substantial computational cost for pre-training due to their unprecedented amounts of parameters. The disadvantage is exacerbated when new knowledge frequently needs to be introduced into the pre-trained model. Therefore, it is imperative to develop effective and efficient techniques to update pre-trained LLMs. Traditional methods encode new knowledge in pre-trained LLMs through direct fine-tuning. However, naively re-training LLMs can be computationally intensive and risks degenerating valuable pre-trained knowledge irrelevant to the update in the model. Recently, Knowledge-based Model Editing (KME) has attracted increasing attention, which aims to precisely modify the LLMs to incorporate specific knowledge, without negatively influencing other irrelevant knowledge. In this survey, we aim to provide a comprehensive and in-depth overview of recent advances in the field of KME. We first introduce a general formulation of KME to encompass different KME strategies. Afterward, we provide an innovative taxonomy of KME techniques based on how the new knowledge is introduced into pre-trained LLMs, and investigate existing KME strategies while analyzing key insights, advantages, and limitations of methods from each category. Moreover, representative metrics, datasets, and applications of KME are introduced accordingly. Finally, we provide an in-depth analysis regarding the practicality and remaining challenges of KME and suggest promising research directions for further advancement in this field.

PDF HTML Abstract

Essay on "Knowledge Editing for LLMs: A Survey"

The survey "Knowledge Editing for LLMs: A Survey" addresses a critical challenge in the field of NLP: the ability to efficiently and precisely update LLMs with new knowledge. LLMs are transformative in the field of NLP, notable for their capacity to analyze and generate text akin to human experts. Their pre-training, however, is resource-intensive, and updating these models to encode new or corrected information frequently poses additional computational burdens. The survey presents a comprehensive overview of Knowledge-based Model Editing (KME) as a solution to efficiently introduce specific updates to LLMs while maintaining the integrity of existing knowledge.

Main Contributions

The survey systematically categorizes existing KME strategies into three primary methods based on how the knowledge is introduced into the LLMs: External Memorization, Global Optimization, and Local Modification.

  1. External Memorization leverages additional parameters or memory to store new knowledge without altering the model's pre-trained weights. This method achieves high scalability and minimal disruption to existing model knowledge, as additional memories or parameters can be easily adjusted to store an extensive range of edits. However, the challenge arises in memory management and ensuring efficient retrieval of stored knowledge.
  2. Global Optimization involves updating all model parameters through fine-tuning strategies that are constrained to minimize their impact on non-target knowledge. Though these methods can improve the generality and incorporate new knowledge effectively, they often suffer from computational inefficiencies due to the need to optimize numerous parameters. The survey notes that these methods focus on enhancing the generality of new knowledge incorporation.
  3. Local Modification seeks to precisely identify and update the specific parameters of the model that encode the knowledge needing change. This strategy can effectively maintain the locality of the model, as it ensures the preservation of original knowledge not pertinent to the update. However, this precision sometimes comes at the cost of reduced scalability.

Evaluation and Future Prospects

The survey delineates standard metrics for evaluating KME strategies: accuracy, locality, generality, retainability, and scalability. Emphasizing the need for a nuanced evaluation, these metrics help determine each method’s ability to incorporate new knowledge without compromising the existing information encoded in the model.

The implications of KME in practical and theoretical realms are substantial. Practically, KME offers a pathway to efficiently update LLMs, ensuring their relevance in dynamic real-world scenarios. Theoretically, KME introduces new paradigms for model adaptability and transfer learning. Future research directions suggested in the survey include balancing the trade-off between locality and generality, enhancing the theoretical understanding of KME processes, and developing methods that support large-scale, continuous edits.

The survey effectively captures the current landscape of KME strategies, providing a structured analysis of methodologies and future challenges. By doing so, it not only sheds light on the necessity of efficient model updating strategies but also sets the stage for innovations that could significantly reduce computational overheads while maintaining model accuracy—a vital advancement in the ongoing evolution of NLP technologies.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (155)
  1. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 298–306.
  2. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In ACL.
  3. James A Anderson. 1972. A simple neural network generating an interactive memory. Mathematical biosciences 14, 3-4 (1972), 197–220.
  4. Vqa: Visual question answering. In ICCV.
  5. FRUIT: Faithfully Reflecting Updated Information in Text. arXiv:2112.08634 [cs.CL]
  6. Large language models and the perils of their hallucinations. Critical Care 27, 1 (2023), 1–2.
  7. Fine-tuning language models to find agreement among humans with diverse preferences. In NeurIPS.
  8. Rewriting a deep generative model. In ECCV.
  9. LEACE: Perfect linear concept erasure in closed form. arXiv preprint arXiv:2306.03819 (2023).
  10. SciBERT: A Pretrained Language Model for Scientific Text. In EMNLP-IJCNLP.
  11. Language models are few-shot learners. In NeurIPS, Vol. 33. 1877–1901.
  12. FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing. In ACL.
  13. A survey on evaluation of large language models. arXiv preprint arXiv:2307.03109 (2023).
  14. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021).
  15. Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting. In EMNLP.
  16. Microsoft coco captions: Data collection and evaluation server. arXiv 2015. arXiv preprint arXiv:1504.00325 (2015).
  17. Can We Edit Multimodal Large Language Models?. In EMNLP.
  18. Editing Language Model-based Knowledge Graph Embeddings. arXiv preprint arXiv:2301.10405 (2023).
  19. Cheng-Han Chiang and Hung-yi Lee. 2023. Can Large Language Models Be an Alternative to Human Evaluations? arXiv preprint arXiv:2305.01937 (2023).
  20. Alebachew Chiche and Betselot Yitagesu. 2022. Part of speech tagging: a systematic review of deep learning and machine learning approaches. Journal of Big Data 9, 1 (2022), 1–25.
  21. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 (2022).
  22. Knowledge Neurons in Pretrained Transformers. In ACL.
  23. Neural knowledge bank for pretrained transformers. In CCF International Conference on Natural Language Processing and Chinese Computing.
  24. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In FAccT.
  25. Editing Factual Knowledge in Language Models. In EMNLP.
  26. Imagenet: A large-scale hierarchical image database. In CVPR. Ieee, 248–255.
  27. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  28. Unified language model pre-training for natural language understanding and generation. NeurIPS (2019).
  29. Calibrating Factual Knowledge in Pretrained Language Models. In EMNLP.
  30. A survey for in-context learning. arXiv preprint arXiv:2301.00234 (2022).
  31. Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint arXiv:2305.14387 (2023).
  32. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics (2021).
  33. T-rex: A large scale alignment of natural language with knowledge base triples. In LREC.
  34. Recommender systems in the era of large language models (llms). arXiv preprint arXiv:2307.02046 (2023).
  35. Enriching contextualized language model from knowledge graph for biomedical information extraction. Briefings in Bioinformatics (2021).
  36. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML.
  37. Boris A Galitsky. 2023. Truth-O-Meter: Collaborating with LLM in Fighting its Hallucinations. (2023).
  38. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858 (2022).
  39. Audio set: An ontology and human-labeled dataset for audio events. In ICASSP. IEEE, 776–780.
  40. Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space. In EMNLP.
  41. Transformer Feed-Forward Layers Are Key-Value Memories. In EMNLP. 5484–5495.
  42. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375 (2022).
  43. Generative adversarial networks. Commun. ACM (2020).
  44. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In CVPR.
  45. Editing Commonsense Knowledge in GPT. arXiv preprint arXiv:2305.14956 (2023).
  46. HyperNetworks. arXiv:1609.09106 [cs.LG]
  47. Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors. In NeurIPS Workshop on Robustness in Sequence Modeling.
  48. Methods for measuring, updating, and visualizing factual beliefs in language models. In EACL.
  49. Deep residual learning for image recognition. In CVPR. 770–778.
  50. Mark Heitmann. 2020. More than a feeling: Benchmarks for sentiment analysis accuracy. In More than a Feeling: Benchmarks for Sentiment Analysis Accuracy: Heitmann, Mark.
  51. Image captioning: Transforming objects into words. NeurIPS (2019).
  52. Measuring and manipulating knowledge representations in language models. arXiv preprint arXiv:2304.00740 (2023).
  53. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.
  54. Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor. arXiv preprint arXiv:2212.09689 (2022).
  55. Meta-learning in neural networks: A survey. IEEE TPAMI (2021).
  56. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 [cs.CL]
  57. A Survey of Knowledge Enhanced Pre-Trained Language Models. IEEE TKDE (2023).
  58. LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models. arXiv preprint arXiv:2304.01933 (2023).
  59. Transformer-Patcher: One Mistake Worth One Neuron. In ICLR.
  60. A survey of deep meta-learning. Artificial Intelligence Review (2021).
  61. Editing models with task arithmetic. In ICLR.
  62. How can we know what language models know? Transactions of the Association for Computational Linguistics (2020).
  63. AMMU: a survey of transformer-based biomedical pretrained language models. Journal of Biomedical Informatics 126 (2022), 103982.
  64. Atoosa Kasirzadeh and Iason Gabriel. 2023. In conversation with Artificial Intelligence: aligning language models with human values. Philosophy & Technology (2023).
  65. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences 103 (2023), 102274.
  66. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL.
  67. Teuvo Kohonen. 1972. Correlation matrix memories. IEEE Trans. Comput. (1972).
  68. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics (2019).
  69. Illustrating Reinforcement Learning from Human Feedback (RLHF). Hugging Face Blog (2022). https://huggingface.co/blog/rlhf.
  70. Plug-and-Play Adaptation for Continuously-updated QA. arXiv:2204.12785
  71. Zero-Shot Relation Extraction via Reading Comprehension. In CoNLL 2017.
  72. Large Language Models with Controllable Working Memory. In ACL.
  73. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597 (2023).
  74. PMET: Precise Model Editing in a Transformer. arXiv preprint arXiv:2308.08742 (2023).
  75. Xiaonan Li and Xipeng Qiu. 2023. Finding supporting examples for in-context learning. arXiv preprint arXiv:2302.13539 (2023).
  76. Parameter-efficient sparsity for large language models fine-tuning. arXiv preprint arXiv:2205.11005 (2022).
  77. Q Vera Liao and Jennifer Wortman Vaughan. 2023. AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. arXiv preprint arXiv:2306.01941 (2023).
  78. Chain of hindsight aligns language models with feedback. arXiv preprint arXiv:2302.02676 3 (2023).
  79. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. In NeurIPS.
  80. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
  81. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747 (2023).
  82. Untying the Reversal Curse via Bidirectional Language Model Editing. arXiv preprint arXiv:2310.10322 (2023).
  83. Yuxuan Ma. 2021. distilgpt2-finetuned-wikitext2. https://huggingface.co/MYX4567/distilgpt2-finetuned-wikitext2
  84. Memory-assisted prompt editing to improve GPT-3 after deployment. CoRR (2022).
  85. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896 (2023).
  86. Locating and editing factual associations in GPT. In NeurIPS, Vol. 35. 17359–17372.
  87. Mass-Editing Memory in a Transformer. In The Eleventh International Conference on Learning Representations.
  88. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147 (2022).
  89. Recent advances in natural language processing via large pre-trained language models: A survey. Comput. Surveys 56, 2 (2023), 1–40.
  90. Fast Model Editing at Scale. In ICLR.
  91. Memory-Based Model Editing at Scale. In ICML.
  92. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786 (2022).
  93. Fixing Model Bugs with Natural Language Patches. In EMNLP.
  94. Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models. arXiv preprint arXiv:2311.08011 (2023).
  95. Entity Cloze By Date: What LMs Know About Unseen Entities. In Findings of NAACL.
  96. Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge. In ACL.
  97. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]
  98. Hariom A. Pandya and Brijesh S. Bhatt. 2021. Question Answering Survey: Directions, Challenges, Datasets, Evaluation Matrices. arXiv:2112.03572 [cs.CL]
  99. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813 (2023).
  100. Instruction tuning with GPT-4. arXiv preprint arXiv:2304.03277 (2023).
  101. Red Teaming Language Models with Language Models. In EMNLP.
  102. KILT: a Benchmark for Knowledge Intensive Language Tasks. In ACL.
  103. Language Models as Knowledge Bases?. In EMNLP-IJCNLP.
  104. Yuval Pinter and Michael Elhadad. 2023. Emptying the Ocean with a Spoon: Should We Edit Models?. In EMNLP.
  105. Exploring Universal Intrinsic Task Subspace via Prompt Tuning. arXiv:2110.07867 [cs.CL]
  106. Improving language understanding by generative pre-training. (2018).
  107. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research (2020).
  108. Sachin Ravi and Hugo Larochelle. 2016. Optimization as a model for few-shot learning. In International conference on learning representations.
  109. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In EMNLP-IJCNLP.
  110. EHUD REITER and ROBERT DALE. 1997. Building applied natural language generation systems. Natural Language Engineering (1997).
  111. Marco Tulio Ribeiro and Scott Lundberg. 2022. Adaptive testing and debugging of NLP models. In ACL.
  112. Recipes for Building an Open-Domain Chatbot. In EACL.
  113. Editing a classifier by rewriting its prediction rules. In NeurIPS.
  114. LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs. In NeurIPS Workshop Datacentric AI.
  115. Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence. In NAACL.
  116. Fine-tuned language models are continual learners. In EMNLP.
  117. An exploratory study of COVID-19 misinformation on Twitter. Online social networks and media 22 (2021), 100104.
  118. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In EMNLP.
  119. Editable Neural Networks. In ICLR.
  120. ConPET: Continual Parameter-Efficient Tuning for Large Language Models. arXiv preprint arXiv:2309.14763 (2023).
  121. Preference Ranking Optimization for Human Alignment. arXiv preprint arXiv:2306.17492 (2023).
  122. Felix Stahlberg. 2020. Neural machine translation: A review. Journal of Artificial Intelligence Research (2020).
  123. Selective annotation makes language models better few-shot learners. arXiv preprint arXiv:2209.01975 (2022).
  124. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937 (2018).
  125. Repairing Neural Networks by Leaving the Right Past Behind. In NeurIPS.
  126. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca.
  127. Large language models in medicine. Nature medicine 29, 8 (2023), 1930–1940.
  128. FEVER: a Large-scale Dataset for Fact Extraction and VERification. In ACL.
  129. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).
  130. Joaquin Vanschoren. 2018. Meta-learning: A survey. arXiv preprint arXiv:1810.03548 (2018).
  131. Attention is all you need. NeurIPS 30 (2017).
  132. Continual learning with hypernetworks. arXiv:1906.00695 [cs.LG]
  133. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM (2014).
  134. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926 (2023).
  135. EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models. arXiv preprint arXiv:2308.07269 (2023).
  136. K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv preprint arXiv:2002.01808 (2020).
  137. Self-Instruct: Aligning Language Model with Self Generated Instructions. arXiv preprint arXiv:2212.10560 (2022).
  138. Adamix: Mixture-of-adapter for parameter-efficient tuning of large language models. arXiv preprint arXiv:2205.12410 (2022).
  139. Aligning large language models with human: A survey. arXiv preprint arXiv:2307.12966 (2023).
  140. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review (2022).
  141. Finetuned Language Models are Zero-Shot Learners.
  142. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, Vol. 35. 24824–24837.
  143. Robust fine-tuning of zero-shot models. In CVPR.
  144. DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models. In EMNLP.
  145. Editing Large Language Models: Problems, Methods, and Opportunities. arXiv preprint arXiv:2305.13172 (2023).
  146. Unlearning bias in language models by partitioning gradients. In ACL Findings.
  147. BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. In ACL.
  148. Michael Zhang and Eunsol Choi. 2021. SituatedQA: Incorporating Extra-Linguistic Contexts into QA. In EMNLP.
  149. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).
  150. Can We Edit Factual Knowledge by In-Context Learning? arXiv:2305.12740 [cs.CL]
  151. MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions. CoRR abs/2305.14795 (2023). arXiv:2305.14795
  152. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In CHI. 1–20.
  153. Modifying Memories in Transformer Models. arXiv:2012.00363 [cs.CL]
  154. A comprehensive survey on transfer learning. Proc. IEEE (2020).
  155. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Song Wang (313 papers)
  2. Yaochen Zhu (23 papers)
  3. Haochen Liu (40 papers)
  4. Zaiyi Zheng (6 papers)
  5. Chen Chen (752 papers)
  6. Jundong Li (126 papers)
Citations (89)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube