Objective for trading off parametric and non-parametric memory in language models

Develop a principled objective or decision criterion to determine the tradeoff between parametric memory (knowledge encoded in model weights) and non-parametric memory (e.g., long-context conditioning and retrieval augmentation) in language models, including when and how to allocate reliance on each memory type.

Background

The paper emphasizes the complementary roles of parametric memory, which compresses and integrates knowledge but is lossy, and non-parametric memory, which is lossless and attributable. It demonstrates scenarios where parametric memory enables complex reasoning beyond what current long-context or retrieval-augmented LLMs can achieve.

The authors explicitly identify the lack of a principled framework for determining how much to rely on parametric versus non-parametric memory for a given task as an open problem.

References

How to decide the tradeoff between parametric and non-parametric memory (or, how to define the objective for such a tradeoff) is another interesting open problem for future work.

— Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization  (2405.15071 - Wang et al., 2024) in Section 6: Related Work

A substituted sentence checked against the entity's attested true value has never populated on this corpus, zero of 295, 54 and 125 built items across three option counts, leaving open whether parametric belief enters here.

— Does a model's stated reason for rejecting a candidate do any work?  (2609.30151 - Rastogi, 24 Sep 2026) in Section 2.3, “The repair edit and the five conditions”

Only 2 memory answers were observed, too few to bound how often that case occurs or is caught (a denominator of 2 puts the one-sided 95\% ceiling at 77.6\%); it stays open.

— Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction  (2608.28439 - Ye et al., 28 Aug 2026) in Section 5.4, paragraph “What the rules miss” (Section~\ref{sec:res-silent})