Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models (2309.05605v3)

Published 11 Sep 2023 in cs.CL, cs.AI, and cs.LG

Abstract: Answering multi-hop reasoning questions requires retrieving and synthesizing information from diverse sources. LLMs struggle to perform such reasoning consistently. Here we propose an approach to pinpoint and rectify multi-hop reasoning failures through targeted memory injections on LLM attention heads. First, we analyze the per-layer activations of GPT-2 models in response to single and multi-hop prompts. We then propose a mechanism that allows users to inject pertinent prompt-specific information, which we refer to as "memories," at critical LLM locations during inference. By thus enabling the LLM to incorporate additional relevant information during inference, we enhance the quality of multi-hop prompt completions. We show empirically that a simple, efficient, and targeted memory injection into a key attention layer can often increase the probability of the desired next token in multi-hop tasks, by up to 424%.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Konstantine Arkoudas. 2023. GPT-4 can’t reason. arXiv preprint arXiv:2308.03762.
  2. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08112.
  3. Graph of thoughts: Solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687.
  4. Can GPT-3 perform statutory reasoning? arXiv preprint arXiv:2302.06100.
  5. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901.
  6. Evaluating the ripple effects of knowledge editing in language models. arXiv preprint arXiv:2307.12976.
  7. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493–8502.
  8. Neural knowledge bank for pretrained transformers. arXiv preprint arXiv:2208.00399.
  9. Analyzing transformers in embedding space. arXiv preprint arXiv:2209.02535.
  10. Mark Davies. 2010. The Corpus of Contemporary American English as the first reliable monitor corpus of English. Literary and Linguistic Computing, 25(4):447–464.
  11. Mark Davies. 2011. Word frequency data from the Corpus of Contemporary American English (COCA).
  12. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6491–6506.
  13. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186.
  14. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257–273.
  15. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031.
  16. A mathematical framework for transformer circuits.
  17. Dissecting recall of factual associations in auto-regressive language models. arXiv preprint arXiv:2304.14767.
  18. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30–45, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  19. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  20. What indeed can GPT models do in chemistry? a comprehensive benchmark on eight tasks. arXiv preprint arXiv:2305.18365.
  21. Does localization inform editing? Surprising differences in causality-based localization vs. knowledge editing in language models. arXiv preprint arXiv:2301.04213.
  22. Do language models have beliefs? Methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.13654.
  23. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  24. Transformer-patcher: One mistake worth one neuron. In The Eleventh International Conference on Learning Representations.
  25. Towards continual knowledge learning of language models. In International Conference on Learning Representations.
  26. UniKGQA: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. In The Eleventh International Conference on Learning Representations.
  27. PMET: Precise model editing in a transformer. arXiv preprint arXiv:2308.08742.
  28. Jieyi Long. 2023. Large language model guided tree-of-thought. arXiv preprint arXiv:2305.08291.
  29. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 35:17359–17372.
  30. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229.
  31. Fast model editing at scale. In International Conference on Learning Representations.
  32. Memory-based model editing at scale. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 15817–15831. PMLR.
  33. Neel Nanda and Joseph Bloom. 2022. TransformerLens.
  34. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations.
  35. nostalgebraist. 2021. Logit Lens on non-GPT2 models + extensions.
  36. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  37. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4231–4242, Brussels, Belgium. Association for Computational Linguistics.
  38. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. arXiv preprint arXiv:2305.04388.
  39. Attention is all you need. Advances in Neural Information Processing Systems, 30.
  40. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  41. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  42. Decomposition enhances reasoning via self-evaluation guided decoding. arXiv preprint arXiv:2305.00633.
  43. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601.
  44. Subgraph retrieval enhanced model for multi-hop knowledge base question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5773–5784, Dublin, Ireland. Association for Computational Linguistics.
  45. MQuAKE: Assessing knowledge editing in language models via multi-hop questions. arXiv preprint arXiv:2305.14795.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Mansi Sakarvadia (10 papers)
  2. Aswathy Ajith (8 papers)
  3. Arham Khan (7 papers)
  4. Daniel Grzenda (5 papers)
  5. Nathaniel Hudson (16 papers)
  6. André Bauer (11 papers)
  7. Kyle Chard (87 papers)
  8. Ian Foster (138 papers)
Citations (12)
View on Semantic Scholar
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube