Recognition Memory in LLMs: Performance & Retention

• Recognition memory is the ability to distinguish previously seen stimuli from novel ones, operationalized via 2-AFC tasks in LLMs.
• LLMs rapidly achieve near-perfect recognition accuracy after minimal exposures, outperforming human benchmarks by 10–20 percentage points.
• Retention exhibits a biphasic forgetting curve, with initial rapid decline followed by stabilization, highlighting robustness to interference compared to recall.

Recognition memory denotes the capability to distinguish previously encountered stimuli from novel ones. In experimental paradigms applied to LLMs, recognition memory is rigorously quantified using two-alternative forced-choice (2-AFC) tasks, wherein a studied item and an unseen foil are presented, and the model's selection of the familiar item (by aggregate next-token loss) operationalizes successful recognition. Analysis of this mechanism reveals critical insights into the dynamics of memory encoding, retention, forgetting, and susceptibility to interference in over-parameterized neural networks compared to human memory systems (Orhan, 2023).

1. Formal Definition and Experimental Paradigm

Recognition memory in LLMs is measured post–study phase, which consists of brief exposure to a small set of novel sentences. Each recognition trial involves presenting one studied sentence ($x_s$) and one foil ($x_x$), with both inputted to the model in isolation. For each, the total next-token loss (negative log-likelihood over tokens) is computed; the item with lower loss is inferred to be more "familiar." The recognition criterion is $loss(x_s) < loss(x_x)$. Aggregate recognition accuracy is defined as the proportion of trials where the studied item is correctly identified, using held-out study/foil pairs. This protocol relies exclusively on two-choice accuracy, without direct computation of hit/false-alarm rates; however, it maps straightforwardly to signal detection theory via the relationship $d' \approx \sqrt{2} \cdot z(p)$, where $p$ is forced-choice accuracy.

2. Recognition Accuracy Across Model Scales and Exposures

Empirical results indicate rapid acquisition and robust recognition capacity:

• Single exposure: Small models (125M–350M parameters) achieve $\approx$80–90% accuracy; medium models (1B–2.7B) $\approx$90–95%; large models (6B–13B) $\approx$95–98%.
• Two exposures: All tested models cluster at $\approx$97–100% accuracy.
• Three exposures: All models approach effectively perfect (100%) recognition performance.

The study phase (1–3 passes over 600 sentences) and subsequent recognition testing reveal that recognition memory in LLMs saturates after very few exposures, with minimal variance across multiple experimental replications.

Model Scale (parameters)	1 Exposure (%)	2 Exposures (%)	3 Exposures (%)
Small (125M–350M)	80–90	97–100	100
Medium (1B–2.7B)	90–95	97–100	100
Large (6B–13B)	95–98	97–100	100

3. Comparison to Human Recognition Benchmarks

Historical experiments by Shepard (1967) provide a human baseline under parallel conditions (600 study/foil sentences, 2-AFC task), with reported single-exposure human accuracy in the 70–80% range. The LLM results substantially exceed this: even the smallest models tested outperform humans; medium and large LLMs outstrip the human benchmark by 10–20 percentage points. The superior recognition is attributed to the efficiency of gradient descent as a "one-shot" learning mechanism in over-parameterized neural nets, whereas human memory formation in analogous paradigms exhibits greater variability and noise.

4. Retention and Forgetting Dynamics Under Continued Training

Retention of recognition memory exhibits a biphasic curve: after the initial three-exposure study phase, further training of the best-performing model (gpt-j-6B) on 224K novel sentences shows a minor decline in recognition within ~10 updates (from 100% to 98%), followed by slow, asymptotic decay (to $\approx$95% after 100K updates). Fitted parameters for power-law or exponential forgetting are not provided, but the empirical curve closely parallels human "permastore" findings (Bahrick 1984), wherein a rapid initial loss precedes stabilization at a nonzero asymptote. This pattern suggests a generic feature of large, incremental memory systems.

5. Interference Effects: Recognition Versus Recall

Recognition memory in LLMs is notably robust to interference from continued training, in contrast with recall memory (measured by prompt plus greedy completion), which degrades precipitously to chance within ~10 updates. Recognition sustains $\approx$98% accuracy under equivalent conditions. The resilience is hypothesized to stem from the coarse resolution of recognition (binary choice) and reliance on relative likelihoods, not exact reproduction. However, in tasks distinguishing random-word or random-string sequences from natural sentences, continued training reverses familiarity rankings—recognition for structureless stimuli drops steeply because new training augments the likelihood of natural stimuli.

6. Broader Implications and Theoretical Insights

LLMs manifest extraordinarily rapid few-shot learning for recognition memory—one to two gradient updates suffice for accurate encoding. Their vulnerability lies in retention; gradient descent affords little inherent protection against catastrophic interference. Techniques such as replay, elastic weight consolidation, or continual-learning strategies may ameliorate retention deficits, possibly reducing phenomena like hallucinations and model instability. The observed two-phase forgetting curve suggests structural parallels with human long-term memory. Recognition experiments in LLMs offer scalable testbeds for probing fundamental memory theories and for designing rehearsal schedules to optimize long-term retention (Orhan, 2023).