Efficiently Updating Knowledge in Large Language Models

Develop scalable, accurate, and cost-effective methods to update factual knowledge encoded in large language models trained via probabilistic language modeling, while preserving existing knowledge and enabling the application of updated facts in complex reasoning tasks.

Background

The paper frames knowledge updating as a central limitation of today’s LLM knowledge paradigm, where facts are implicitly encoded in model parameters via probabilistic language modeling. The authors emphasize that effective updating should satisfy three aims: accuracy on new facts, retention of prior knowledge without catastrophic forgetting, and the ability to use updated facts in multi-hop reasoning. Existing knowledge editing methods struggle especially with generalization to reasoning tasks, motivating this open problem.